From: Sasha Levin <sashal@kernel.org>
To: patches@lists.linux.dev, stable@vger.kernel.org
Cc: Junnan Wu <junnan01.wu@samsung.com>,
Ying Xu <ying123.xu@samsung.com>,
"Michael S. Tsirkin" <mst@redhat.com>,
Sasha Levin <sashal@kernel.org>,
jasowang@redhat.com, andrew+netdev@lunn.ch, davem@davemloft.net,
edumazet@google.com, kuba@kernel.org, pabeni@redhat.com,
virtualization@lists.linux.dev, netdev@vger.kernel.org
Subject: [PATCH AUTOSEL 6.16] virtio_net: adjust the execution order of function `virtnet_close` during freeze
Date: Tue, 2 Sep 2025 08:08:28 -0400 [thread overview]
Message-ID: <20250902120833.1342615-17-sashal@kernel.org> (raw)
In-Reply-To: <20250902120833.1342615-1-sashal@kernel.org>
From: Junnan Wu <junnan01.wu@samsung.com>
[ Upstream commit 45d8ef6322b8a828d3b1e2cfb8893e2ff882cb23 ]
"Use after free" issue appears in suspend once race occurs when
napi poll scheduls after `netif_device_detach` and before napi disables.
For details, during suspend flow of virtio-net,
the tx queue state is set to "__QUEUE_STATE_DRV_XOFF" by CPU-A.
And at some coincidental times, if a TCP connection is still working,
CPU-B does `virtnet_poll` before napi disable.
In this flow, the state "__QUEUE_STATE_DRV_XOFF"
of tx queue will be cleared. This is not the normal process it expects.
After that, CPU-A continues to close driver then virtqueue is removed.
Sequence likes below:
--------------------------------------------------------------------------
CPU-A CPU-B
----- -----
suspend is called A TCP based on
virtio-net still work
virtnet_freeze
|- virtnet_freeze_down
| |- netif_device_detach
| | |- netif_tx_stop_all_queues
| | |- netif_tx_stop_queue
| | |- set_bit
| | (__QUEUE_STATE_DRV_XOFF,...)
| | softirq rasied
| | |- net_rx_action
| | |- napi_poll
| | |- virtnet_poll
| | |- virtnet_poll_cleantx
| | |- netif_tx_wake_queue
| | |- test_and_clear_bit
| | (__QUEUE_STATE_DRV_XOFF,...)
| |- virtnet_close
| |- virtnet_disable_queue_pair
| |- virtnet_napi_tx_disable
|- remove_vq_common
--------------------------------------------------------------------------
When TCP delayack timer is up, a cpu gets softirq and irq handler
`tcp_delack_timer_handler` will be called, which will finally call
`start_xmit` in virtio net driver.
Then the access to tx virtq will cause panic.
The root cause of this issue is that napi tx
is not disable before `netif_tx_stop_queue`,
once `virnet_poll` schedules in such coincidental time,
the tx queue state will be cleared.
To solve this issue, adjusts the order of
function `virtnet_close` in `virtnet_freeze_down`.
Co-developed-by: Ying Xu <ying123.xu@samsung.com>
Signed-off-by: Ying Xu <ying123.xu@samsung.com>
Signed-off-by: Junnan Wu <junnan01.wu@samsung.com>
Message-Id: <20250812090817.3463403-1-junnan01.wu@samsung.com>
Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
Signed-off-by: Sasha Levin <sashal@kernel.org>
---
LLM Generated explanations, may be completely bogus:
**Backport Status: YES**
This commit should be backported to stable kernel trees. Here's my
extensive analysis:
## Critical Bug Fix
This commit fixes a **use-after-free vulnerability** in the virtio_net
driver that can cause kernel panics during system suspend. The issue is
a race condition between the suspend process and active network traffic,
specifically:
1. **Race Condition Details** (from commit message):
- CPU-A calls `netif_device_detach()` which sets
`__QUEUE_STATE_DRV_XOFF` on TX queues
- CPU-B running `virtnet_poll()` can clear this flag via
`netif_tx_wake_queue()`
- CPU-A then calls `virtnet_close()` which disables NAPI and removes
virtqueues
- Later, when TCP delayed ACK timer fires, accessing the removed TX
virtqueue causes a panic
## Code Analysis
The fix is **minimal and contained** - it simply reorders two operations
in `virtnet_freeze_down()`:
**Before (buggy order):**
```c
netif_tx_lock_bh(vi->dev);
netif_device_detach(vi->dev); // Sets __QUEUE_STATE_DRV_XOFF
netif_tx_unlock_bh(vi->dev);
if (netif_running(vi->dev)) {
virtnet_close(vi->dev); // Disables NAPI - TOO LATE!
}
```
**After (fixed order):**
```c
if (netif_running(vi->dev)) {
virtnet_close(vi->dev); // Disables NAPI FIRST
}
netif_tx_lock_bh(vi->dev);
netif_device_detach(vi->dev); // Then detach device
netif_tx_unlock_bh(vi->dev);
```
## Backport Suitability Criteria
1. **Fixes a real bug**: YES - Use-after-free causing kernel panics
2. **Small and contained**: YES - Only 8 lines moved, no logic changes
3. **No side effects**: YES - Just reordering existing operations
4. **No architectural changes**: YES - Simple ordering fix
5. **Critical subsystem**: YES - Network driver suspend/resume path
6. **Clear bug fix**: YES - Not a feature addition
7. **Minimal regression risk**: YES - The new order is logically correct
(disable NAPI before marking queues stopped)
## Historical Context
From git blame analysis:
- The problematic code structure has existed since 2017 (commit
9fe7bfce8b3e11)
- Recent changes in 2025 (commit e7231f49d52682) modified the locking
but didn't fix the race
- This indicates the bug has been latent for **years** and affects many
kernel versions
## Impact Assessment
- **Severity**: HIGH - Causes kernel panics during suspend with active
network traffic
- **Likelihood**: MODERATE - Requires specific timing but can happen
with normal TCP traffic
- **Affected systems**: All systems using virtio_net driver (VMs,
containers)
The commit already has "[ Upstream commit
45d8ef6322b8a828d3b1e2cfb8893e2ff882cb23 ]" indicating it's been
identified for stable backporting, which further confirms its
importance.
drivers/net/virtio_net.c | 7 ++++---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/drivers/net/virtio_net.c b/drivers/net/virtio_net.c
index 82b4a2a2b8c42..c9b18b13cd940 100644
--- a/drivers/net/virtio_net.c
+++ b/drivers/net/virtio_net.c
@@ -5728,14 +5728,15 @@ static void virtnet_freeze_down(struct virtio_device *vdev)
disable_rx_mode_work(vi);
flush_work(&vi->rx_mode_work);
- netif_tx_lock_bh(vi->dev);
- netif_device_detach(vi->dev);
- netif_tx_unlock_bh(vi->dev);
if (netif_running(vi->dev)) {
rtnl_lock();
virtnet_close(vi->dev);
rtnl_unlock();
}
+
+ netif_tx_lock_bh(vi->dev);
+ netif_device_detach(vi->dev);
+ netif_tx_unlock_bh(vi->dev);
}
static int init_vqs(struct virtnet_info *vi);
--
2.50.1
prev parent reply other threads:[~2025-09-02 12:08 UTC|newest]
Thread overview: 17+ messages / expand[flat|nested] mbox.gz Atom feed top
2025-09-02 12:08 [PATCH AUTOSEL 6.16] gpiolib: acpi: Add quirk for ASUS ProArt PX13 Sasha Levin
2025-09-02 12:08 ` [PATCH AUTOSEL 6.16] pinctrl: meson: Fix typo in device table macro Sasha Levin
2025-09-02 12:08 ` [PATCH AUTOSEL 6.16-5.15] HID: intel-ish-hid: Increase ISHTP resume ack timeout to 300ms Sasha Levin
2025-09-02 12:08 ` [PATCH AUTOSEL 6.16-5.4] gpio: timberdale: fix off-by-one in IRQ type boundary check Sasha Levin
2025-09-02 12:08 ` [PATCH AUTOSEL 6.16-6.12] drm/msm: Fix a7xx debugbus read Sasha Levin
2025-09-02 12:08 ` [PATCH AUTOSEL 6.16-6.1] ata: ahci_xgene: Use int type for 'rc' to store error codes Sasha Levin
2025-09-02 12:08 ` [PATCH AUTOSEL 6.16] HID: elecom: add support for ELECOM M-DT2DRBK Sasha Levin
2025-09-02 12:08 ` [PATCH AUTOSEL 6.16-6.12] hid: fix I2C read buffer overflow in raw_event() for mcp2221 Sasha Levin
2025-09-02 12:08 ` [PATCH AUTOSEL 6.16-6.12] drm/msm: Fix a7xx TPL1 cluster snapshot Sasha Levin
2025-09-02 12:08 ` [PATCH AUTOSEL 6.16-6.1] virtio_input: Improve freeze handling Sasha Levin
2025-09-02 12:08 ` [PATCH AUTOSEL 6.16-6.12] regulator: pm8008: fix probe failure due to negative voltage selector Sasha Levin
2025-09-02 12:08 ` [PATCH AUTOSEL 6.16-6.12] drm/msm: Fix debugbus snapshot Sasha Levin
2025-09-02 12:08 ` [PATCH AUTOSEL 6.16-6.6] HID: quirks: add support for Legion Go dual dinput modes Sasha Levin
2025-09-02 12:08 ` [PATCH AUTOSEL 6.16-6.6] HID: logitech: Add ids for G PRO 2 LIGHTSPEED Sasha Levin
2025-09-02 12:08 ` [PATCH AUTOSEL 6.16-6.12] drm/msm: Fix order of selector programming in cluster snapshot Sasha Levin
2025-09-02 12:08 ` [PATCH AUTOSEL 6.16-5.4] HID: hid-ntrig: fix unable to handle page fault in ntrig_report_version() Sasha Levin
2025-09-02 12:08 ` Sasha Levin [this message]
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20250902120833.1342615-17-sashal@kernel.org \
--to=sashal@kernel.org \
--cc=andrew+netdev@lunn.ch \
--cc=davem@davemloft.net \
--cc=edumazet@google.com \
--cc=jasowang@redhat.com \
--cc=junnan01.wu@samsung.com \
--cc=kuba@kernel.org \
--cc=mst@redhat.com \
--cc=netdev@vger.kernel.org \
--cc=pabeni@redhat.com \
--cc=patches@lists.linux.dev \
--cc=stable@vger.kernel.org \
--cc=virtualization@lists.linux.dev \
--cc=ying123.xu@samsung.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox