All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH net v2 0/3] igb: fix runtime PM resume issues
@ 2026-11-16 15:30 Antonio Quartulli
  2026-11-16 15:30 ` [PATCH net v2 1/3] igb: fix RTNL assertion when resuming from runtime suspend Antonio Quartulli
                   ` (2 more replies)
  0 siblings, 3 replies; 5+ messages in thread
From: Antonio Quartulli @ 2026-11-16 15:30 UTC (permalink / raw)
  To: intel-wired-lan
  Cc: Antonio Quartulli, Tony Nguyen, Przemek Kitszel, Andrew Lunn,
	David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
	Heiner Kallweit, Kurt Kanzenbach, Joe Damato, Yan, Zheng, netdev,
	linux-kernel

igb can be runtime suspended when the link goes down (with
power/control set to auto). Resuming it calls functions that need RTNL
without holding it (patches 1 and 2), and devices that can't signal a
wake-up are suspended anyway, so their port stays down when the link
comes back (patch 3).

Tested with the 82576 emulated by QEMU.

Changes in v2:
- patch 1: keep setting the queue counts in __igb_open(), only skipping
  them on runtime resume (Sashiko)
- mention the tested hardware in the commit messages

v1: https://lore.kernel.org/netdev/20261113105835.3821433-1-antonio@mandelbit.com/

Antonio Quartulli (3):
  igb: fix RTNL assertion when resuming from runtime suspend
  igb: link queues to NAPIs under RTNL after runtime resume
  igb: only allow runtime PM if the device can wake up

 drivers/net/ethernet/intel/igb/igb.h      |  1 +
 drivers/net/ethernet/intel/igb/igb_main.c | 69 ++++++++++++++++++-----
 2 files changed, 57 insertions(+), 13 deletions(-)

-- 
2.56.0


^ permalink raw reply	[flat|nested] 5+ messages in thread

* [PATCH net v2 1/3] igb: fix RTNL assertion when resuming from runtime suspend
  2026-11-16 15:30 [PATCH net v2 0/3] igb: fix runtime PM resume issues Antonio Quartulli
@ 2026-11-16 15:30 ` Antonio Quartulli
  2026-11-17 15:31   ` sashiko-bot
  2026-11-16 15:30 ` [PATCH net v2 2/3] igb: link queues to NAPIs under RTNL after runtime resume Antonio Quartulli
  2026-11-16 15:30 ` [PATCH net v2 3/3] igb: only allow runtime PM if the device can wake up Antonio Quartulli
  2 siblings, 1 reply; 5+ messages in thread
From: Antonio Quartulli @ 2026-11-16 15:30 UTC (permalink / raw)
  To: intel-wired-lan
  Cc: Antonio Quartulli, Tony Nguyen, Przemek Kitszel, Andrew Lunn,
	David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
	Heiner Kallweit, netdev, linux-kernel, stable

Since commit ac8c58f5b535 ("igb: fix deadlock caused by taking RTNL in
RPM resume path") __igb_resume() doesn't take RTNL for runtime resume,
but __igb_open() still calls netif_set_real_num_{tx,rx}_queues(), which
require it. An asynchronous runtime resume (e.g. after the link comes
back) then triggers "RTNL: assertion failed".

Skip them when resuming. The counts are set again on every regular open,
which happens under RTNL; the only case left out is a fallback from
MSI-X to MSI during a resume, which leaves the stack with more queues
than the device uses until the next open.

Tested with the 82576 emulated by QEMU.

Fixes: ac8c58f5b535 ("igb: fix deadlock caused by taking RTNL in RPM resume path")
Cc: stable@vger.kernel.org
Closes: https://bugzilla.ipfire.org/show_bug.cgi?id=13562
Signed-off-by: Antonio Quartulli <antonio@mandelbit.com>
---
v2:
- keep setting the queue counts in __igb_open() after igb_request_irq(),
  only skipping them on runtime resume: setting them in igb_open()
  before the device was resumed broke the MSI-X to MSI fallback and XDP
  setup on a runtime suspended device (Sashiko)
- drop the reference to igc, which has the same ordering problem

Note: igc sets the queue counts in igc_open() before igc_request_irq(),
which can fall back to MSI, since 6f31d6b643a3 ("igc: Refactor runtime
power management flow"). We didn't have igc hardware to test a fix with,
so no igc patch is sent for now.

 drivers/net/ethernet/intel/igb/igb_main.c | 22 +++++++++++++---------
 1 file changed, 13 insertions(+), 9 deletions(-)

diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index 78f0786..4447546 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -4178,16 +4178,20 @@ static int __igb_open(struct net_device *netdev, bool resuming)
 	if (err)
 		goto err_req_irq;
 
-	/* Notify the stack of the actual queue counts. */
-	err = netif_set_real_num_tx_queues(adapter->netdev,
-					   adapter->num_tx_queues);
-	if (err)
-		goto err_set_queues;
+	/* Notify the stack of the actual queue counts. This needs RTNL,
+	 * which isn't held when resuming from runtime suspend.
+	 */
+	if (!resuming) {
+		err = netif_set_real_num_tx_queues(adapter->netdev,
+						   adapter->num_tx_queues);
+		if (err)
+			goto err_set_queues;
 
-	err = netif_set_real_num_rx_queues(adapter->netdev,
-					   adapter->num_rx_queues);
-	if (err)
-		goto err_set_queues;
+		err = netif_set_real_num_rx_queues(adapter->netdev,
+						   adapter->num_rx_queues);
+		if (err)
+			goto err_set_queues;
+	}
 
 	/* From here on the code is the same as igb_up() */
 	clear_bit(__IGB_DOWN, &adapter->state);
-- 
2.56.0


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH net v2 2/3] igb: link queues to NAPIs under RTNL after runtime resume
  2026-11-16 15:30 [PATCH net v2 0/3] igb: fix runtime PM resume issues Antonio Quartulli
  2026-11-16 15:30 ` [PATCH net v2 1/3] igb: fix RTNL assertion when resuming from runtime suspend Antonio Quartulli
@ 2026-11-16 15:30 ` Antonio Quartulli
  2026-11-16 15:30 ` [PATCH net v2 3/3] igb: only allow runtime PM if the device can wake up Antonio Quartulli
  2 siblings, 0 replies; 5+ messages in thread
From: Antonio Quartulli @ 2026-11-16 15:30 UTC (permalink / raw)
  To: intel-wired-lan
  Cc: Antonio Quartulli, Tony Nguyen, Przemek Kitszel, Andrew Lunn,
	David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
	Kurt Kanzenbach, Joe Damato, netdev, linux-kernel, stable

__igb_open() links queues to NAPIs with netif_queue_set_napi(), which
requires RTNL, but RTNL isn't held when resuming from runtime suspend:
this triggers "RTNL: assertion failed".

Don't link them in __igb_open() when resuming: do it right away on
system resume (RTNL held) and from a work item taking RTNL on runtime
resume. The links can't be kept across a suspend, because
__igb_shutdown() frees the q_vectors. These are only freed under RTNL
with __IGB_DOWN set, so the work item checks __IGB_DOWN under RTNL.

Tested with the 82576 emulated by QEMU.

Fixes: b75a1dea500f ("igb: Link queues to NAPI instances")
Cc: stable@vger.kernel.org # 6.16+
Closes: https://bugzilla.ipfire.org/show_bug.cgi?id=13562
Signed-off-by: Antonio Quartulli <antonio@mandelbit.com>
---
v2: no code changes; mention the tested hardware.

Note: igc may have the same issue: __igc_open() also links queues to NAPIs
(igc_set_queue_napi()) on the RTNL-less runtime resume path. We didn't
have igc hardware to test a fix with, so no igc patch is sent for now.

 drivers/net/ethernet/intel/igb/igb.h      |  1 +
 drivers/net/ethernet/intel/igb/igb_main.c | 38 +++++++++++++++++++++--
 2 files changed, 37 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/intel/igb/igb.h b/drivers/net/ethernet/intel/igb/igb.h
index 0fff1df..ea2d739 100644
--- a/drivers/net/ethernet/intel/igb/igb.h
+++ b/drivers/net/ethernet/intel/igb/igb.h
@@ -585,6 +585,7 @@ struct igb_adapter {
 
 	struct work_struct reset_task;
 	struct work_struct watchdog_task;
+	struct work_struct queue_napi_task;
 	bool fc_autoneg;
 	u8  tx_timeout_factor;
 	struct timer_list blink_timer;
diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index 4447546..894eedb 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -2115,6 +2115,30 @@ void igb_set_queue_napi(struct igb_adapter *adapter, int vector,
 				     NETDEV_QUEUE_TYPE_TX, napi);
 }
 
+static void igb_link_queue_napis(struct igb_adapter *adapter)
+{
+	int i;
+
+	for (i = 0; i < adapter->num_q_vectors; i++)
+		igb_set_queue_napi(adapter, i, &adapter->q_vector[i]->napi);
+}
+
+/* Linking queues to NAPIs requires RTNL, which isn't held when resuming
+ * from runtime suspend: do it from here instead. The q_vectors are only
+ * freed under RTNL with __IGB_DOWN set, so checking it under RTNL is safe.
+ */
+static void igb_queue_napi_task(struct work_struct *work)
+{
+	struct igb_adapter *adapter = container_of(work, struct igb_adapter,
+						   queue_napi_task);
+
+	rtnl_lock();
+	if (!test_bit(__IGB_DOWN, &adapter->state) &&
+	    netif_running(adapter->netdev))
+		igb_link_queue_napis(adapter);
+	rtnl_unlock();
+}
+
 /**
  *  igb_up - Open the interface and prepare it to handle traffic
  *  @adapter: board private structure
@@ -3426,6 +3450,7 @@ static int igb_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 
 	INIT_WORK(&adapter->reset_task, igb_reset_task);
 	INIT_WORK(&adapter->watchdog_task, igb_watchdog_task);
+	INIT_WORK(&adapter->queue_napi_task, igb_queue_napi_task);
 
 	/* Initialize link properties that are user-changeable */
 	adapter->fc_autoneg = true;
@@ -3890,6 +3915,7 @@ static void igb_remove(struct pci_dev *pdev)
 
 	cancel_work_sync(&adapter->reset_task);
 	cancel_work_sync(&adapter->watchdog_task);
+	cancel_work_sync(&adapter->queue_napi_task);
 
 #ifdef CONFIG_IGB_DCA
 	if (adapter->flags & IGB_FLAG_DCA_ENABLED) {
@@ -4199,7 +4225,8 @@ static int __igb_open(struct net_device *netdev, bool resuming)
 	for (i = 0; i < adapter->num_q_vectors; i++) {
 		napi = &adapter->q_vector[i]->napi;
 		napi_enable(napi);
-		igb_set_queue_napi(adapter, i, napi);
+		if (!resuming)
+			igb_set_queue_napi(adapter, i, napi);
 	}
 
 	/* Clear any pending interrupts. */
@@ -9637,8 +9664,15 @@ static int __igb_resume(struct device *dev, bool rpm)
 
 	if (!rpm)
 		rtnl_lock();
-	if (!err && netif_running(netdev))
+	if (!err && netif_running(netdev)) {
 		err = __igb_open(netdev, true);
+		if (!err) {
+			if (rpm)
+				schedule_work(&adapter->queue_napi_task);
+			else
+				igb_link_queue_napis(adapter);
+		}
+	}
 
 	if (!err)
 		netif_device_attach(netdev);
-- 
2.56.0


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* [PATCH net v2 3/3] igb: only allow runtime PM if the device can wake up
  2026-11-16 15:30 [PATCH net v2 0/3] igb: fix runtime PM resume issues Antonio Quartulli
  2026-11-16 15:30 ` [PATCH net v2 1/3] igb: fix RTNL assertion when resuming from runtime suspend Antonio Quartulli
  2026-11-16 15:30 ` [PATCH net v2 2/3] igb: link queues to NAPIs under RTNL after runtime resume Antonio Quartulli
@ 2026-11-16 15:30 ` Antonio Quartulli
  2 siblings, 0 replies; 5+ messages in thread
From: Antonio Quartulli @ 2026-11-16 15:30 UTC (permalink / raw)
  To: intel-wired-lan
  Cc: Antonio Quartulli, Tony Nguyen, Przemek Kitszel, Andrew Lunn,
	David S. Miller, Eric Dumazet, Jakub Kicinski, Paolo Abeni,
	Yan, Zheng, netdev, linux-kernel, stable

igb lets the device be runtime suspended even if it can't signal a
wake-up from D3. Such a device doesn't notice the link coming back, so
the port stays down until it is resumed from user space.

Allow runtime PM only if pci_dev_run_wake() is true, as e1000e and
r8169 do. Devices that can't wake up no longer runtime suspend, even
with power/control set to auto.

Tested with the 82576 emulated by QEMU.

Fixes: 749ab2cd1270 ("igb: add basic runtime PM support")
Cc: stable@vger.kernel.org
Closes: https://bugzilla.ipfire.org/show_bug.cgi?id=13562
Signed-off-by: Antonio Quartulli <antonio@mandelbit.com>
---
v2: no code changes; mention the tested hardware.

Note: igc may have the same issue: igc_probe() also allows runtime PM
regardless of pci_dev_run_wake(). We didn't have igc hardware to test a
fix with, so no igc patch is sent for now.

 drivers/net/ethernet/intel/igb/igb_main.c | 9 +++++++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
index 894eedb..9046611 100644
--- a/drivers/net/ethernet/intel/igb/igb_main.c
+++ b/drivers/net/ethernet/intel/igb/igb_main.c
@@ -3677,7 +3677,11 @@ static int igb_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
 
 	dev_pm_set_driver_flags(&pdev->dev, DPM_FLAG_NO_DIRECT_COMPLETE);
 
-	pm_runtime_put_noidle(&pdev->dev);
+	/* A device that can't wake up wouldn't notice the link coming back
+	 * while runtime suspended
+	 */
+	if (pci_dev_run_wake(pdev))
+		pm_runtime_put_noidle(&pdev->dev);
 	return 0;
 
 err_register:
@@ -3900,7 +3904,8 @@ static void igb_remove(struct pci_dev *pdev)
 	struct igb_adapter *adapter = netdev_priv(netdev);
 	struct e1000_hw *hw = &adapter->hw;
 
-	pm_runtime_get_noresume(&pdev->dev);
+	if (pci_dev_run_wake(pdev))
+		pm_runtime_get_noresume(&pdev->dev);
 #ifdef CONFIG_IGB_HWMON
 	igb_sysfs_exit(adapter);
 #endif
-- 
2.56.0


^ permalink raw reply related	[flat|nested] 5+ messages in thread

* Re: [PATCH net v2 1/3] igb: fix RTNL assertion when resuming from runtime suspend
  2026-11-16 15:30 ` [PATCH net v2 1/3] igb: fix RTNL assertion when resuming from runtime suspend Antonio Quartulli
@ 2026-11-17 15:31   ` sashiko-bot
  0 siblings, 0 replies; 5+ messages in thread
From: sashiko-bot @ 2026-11-17 15:31 UTC (permalink / raw)
  To: Antonio Quartulli, intel-wired-lan

Thank you for your contribution! Sashiko AI review found 1 potential
issue to consider:
- [Medium] igb: stale real queue counts after resume MSI fallback in
  __igb_open()
--

commit 6c8e67bb8e01233489112f0bf629f1764d87d2c6
Author: Antonio Quartulli <antonio@mandelbit.com>

igb: fix RTNL assertion when resuming from runtime suspend

This commit skips calling netif_set_real_num_tx_queues() and
netif_set_real_num_rx_queues() in __igb_open() when resuming is true to
avoid triggering an RTNL assertion during runtime resume.

> diff --git a/drivers/net/ethernet/intel/igb/igb_main.c b/drivers/net/ethernet/intel/igb/igb_main.c
> index 131df3e09ba14..bf8acac72b324 100644
> --- a/drivers/net/ethernet/intel/igb/igb_main.c
> +++ b/drivers/net/ethernet/intel/igb/igb_main.c
> @@ -4178,16 +4178,20 @@ static int __igb_open(struct net_device *netdev, bool resuming)
>  	if (err)
>  		goto err_req_irq;
>  
> -	/* Notify the stack of the actual queue counts. */
> -	err = netif_set_real_num_tx_queues(adapter->netdev,
> -					   adapter->num_tx_queues);
> -	if (err)
> -		goto err_set_queues;
> +	/* Notify the stack of the actual queue counts. This needs RTNL,
> +	 * which isn't held when resuming from runtime suspend.
> +	 */
> +	if (!resuming) {
> +		err = netif_set_real_num_tx_queues(adapter->netdev,
> +						   adapter->num_tx_queues);
> +		if (err)
> +			goto err_set_queues;
>  
> -	err = netif_set_real_num_rx_queues(adapter->netdev,
> -					   adapter->num_rx_queues);
> -	if (err)
> -		goto err_set_queues;
> +		err = netif_set_real_num_rx_queues(adapter->netdev,
> +						   adapter->num_rx_queues);
> +		if (err)
> +			goto err_set_queues;
> +	}
>  
>  	/* From here on the code is the same as igb_up() */
>  	clear_bit(__IGB_DOWN, &adapter->state);

[Severity: Medium]
Can leaving netdev->real_num_tx_queues and netdev->real_num_rx_queues stale
when MSI-X falls back to MSI during resume cause concurrent unlocked TX on
adapter->tx_ring[0] and a NULL pointer dereference in igb_xsk_pool_disable()?

During suspend, __igb_shutdown() -> igb_clear_interrupt_scheme() frees all
q_vectors and clears adapter->tx_ring[] and adapter->rx_ring[].  If MSI-X
allocation fails on resume (including system resume via igb_resume() ->
__igb_resume(dev, false), which already holds rtnl_lock() but still passes
resuming = true), the driver falls back to MSI with adapter->num_tx_queues
and adapter->num_rx_queues set to 1, allocating only index 0 while
netdev->real_num_tx_queues and netdev->real_num_rx_queues remain at N > 1.

In the TX path, the stack selects subqueues across 0 .. real_num_tx_queues - 1,
each serialized only by its own netdev->_tx[q]._xmit_lock, but
igb_tx_queue_mapping() folds all of them onto adapter->tx_ring[0]:

drivers/net/ethernet/intel/igb/igb_main.c:igb_tx_queue_mapping() {
	unsigned int r_idx = skb->queue_mapping;

	if (r_idx >= adapter->num_tx_queues)
		r_idx = r_idx % adapter->num_tx_queues;

	return adapter->tx_ring[r_idx];
}

CPU0 (subqueue 0)                 CPU1 (subqueue 1)
netdev_start_xmit()               netdev_start_xmit()
  HARD_TX_LOCK(dev, txq0)           HARD_TX_LOCK(dev, txq1)
  igb_xmit_frame()                  igb_xmit_frame()
    igb_tx_queue_mapping()            igb_tx_queue_mapping()
      -> adapter->tx_ring[0]            -> adapter->tx_ring[0]
    igb_xmit_frame_ring(tx_ring[0])   igb_xmit_frame_ring(tx_ring[0])
      first = &tx_ring->                first = &tx_ring->
        tx_buffer_info[next_to_use]       tx_buffer_info[next_to_use]
                                          <-- concurrent ring corruption

Multiple CPUs transmitting on different subqueues concurrently run
igb_xmit_frame_ring() and igb_tx_map() on adapter->tx_ring[0], corrupting
tx_ring->next_to_use, tx_buffer_info, hardware descriptors, and BQL state.

In addition, igb_xsk_pool_disable() checks xsk_get_pool_from_qid(), which
validates qid against real_num_rx_queues and real_num_tx_queues; disabling an
AF_XDP pool bound to qid >= 1 after a resume fallback passes qid to
igb_txrx_ring_disable(), dereferencing NULL adapter->tx_ring[qid] and
adapter->rx_ring[qid].

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20261116153041.3002666-1-antonio@mandelbit.com?part=1

^ permalink raw reply	[flat|nested] 5+ messages in thread

end of thread, other threads:[~2026-11-17 15:31 UTC | newest]

Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-16 15:30 [PATCH net v2 0/3] igb: fix runtime PM resume issues Antonio Quartulli
2026-11-16 15:30 ` [PATCH net v2 1/3] igb: fix RTNL assertion when resuming from runtime suspend Antonio Quartulli
2026-11-17 15:31   ` sashiko-bot
2026-11-16 15:30 ` [PATCH net v2 2/3] igb: link queues to NAPIs under RTNL after runtime resume Antonio Quartulli
2026-11-16 15:30 ` [PATCH net v2 3/3] igb: only allow runtime PM if the device can wake up Antonio Quartulli

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.