Re: [PATCH v2] virtio_net: fix infinite loop in virtnet_poll_cleantx when device is broken

[Jinqian Yang <yangjinqian1@huawei.com>]

Hi,

On 2026/7/16 13:14, Michael S. Tsirkin wrote:
> On Thu, Jul 16, 2026 at 11:52:01AM +0800, Jinqian Yang wrote:
>> virtnet_poll_cleantx() contains a do-while loop that cleans up
>> transmitted TX buffers and calls virtqueue_enable_cb_delayed() to check
>> whether more buffers need processing. When the virtio backend stops
>> responding during guest reboot, used->idx is never updated, so
>> virtqueue_enable_cb_delayed() always returns false and the loop never
>> terminates. Then it will block reboot process, and the guest will hang.
>>
>> The problem occurs during guest reboot under network traffic:
>>
>>    1. kernel_restart() -> device_shutdown() traverses the device list
>>    2. virtio_dev_shutdown() calls virtio_break_device() which sets
>>       vq->broken = true
>>    3. virtio_dev_shutdown() then calls virtio_synchronize_cbs() to wait
>>       for in-flight callbacks to complete
>>    4. A virtio interrupt fires, softirq is deferred to ksoftirqd which
>>       calls net_rx_action() -> virtnet_poll() -> virtnet_poll_cleantx()
>>    5. virtnet_poll_cleantx() enters the do-while loop and never exits
>>       because the QEMU backend has stopped updating used->idx, despite
>>       vq->broken having been set to true in step 2.
>>
>> Since the loop runs inside ksoftirqd (a SCHED_OTHER kthread), it is
>> visible to the scheduler and does not trigger a hard lockup. However,
>> the kthread never leaves the loop, so RCU detects it as a CPU stall
>> and reports it periodically. Meanwhile, the reboot process remains
>> blocked in device_shutdown() because virtio_dev_shutdown() cannot
>> complete its synchronization step, and the guest hangs permanently.
>>
>> This can be reproduced on a guest with a virtio-net device: run iperf3
>> traffic in the guest, then trigger reboot. The reboot occasionally hangs
>> permanently with RCU stall on ksoftirqd.
>>
>> Observed on ARM64 KVM guest:
>>
>>    CPU#1 RCU stall (ksoftirqd/1), repeated periodically:
>>      virtqueue_enable_cb_delayed_split <- virtnet_poll <- __napi_poll <-
>>      net_rx_action <- handle_softirqs <- run_ksoftirqd <-
>>      smpboot_thread_fn <- kthread
>>
>> Fix by adding a virtqueue_is_broken() check to the loop condition, so
>> that the loop exits immediately when the device is broken, allowing
>> the device shutdown to proceed.
>>
>> Signed-off-by: Jinqian Yang <yangjinqian1@huawei.com>
> 
> 
> Good thanks! Just the subject needs change so it's clear
> we are changing virtio core not virtio net.
> 

Thanks for the catch. Will change the subject prefix to "virtio_ring:"
and send v3 shortly.

Thanks,
Jinqian

>> ---
>> Changes in v2:
>>    - Moved vq->broken check to virtqueue_enable_cb_delayed().
>>
>> v1: https://lore.kernel.org/lkml/20260713132025.703147-1-yangjinqian1@huawei.com/
>> ---
>>   drivers/virtio/virtio_ring.c | 8 ++++++++
>>   1 file changed, 8 insertions(+)
>>
>> diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
>> index b438dc2ce1b8..5c169fbb418a 100644
>> --- a/drivers/virtio/virtio_ring.c
>> +++ b/drivers/virtio/virtio_ring.c
>> @@ -3233,6 +3233,14 @@ bool virtqueue_enable_cb_delayed(struct virtqueue *_vq)
>>   {
>>   	struct vring_virtqueue *vq = to_vvq(_vq);
>>   
>> +	/*
>> +	 * When the device is broken there is no point in polling used->idx,
>> +	 * the backend will never update it. Return true to let callers
>> +	 * exit their cleanup loops instead of spinning forever.
>> +	 */
>> +	if (unlikely(vq->broken))
>> +		return true;
>> +
>>   	if (vq->event_triggered)
>>   		data_race(vq->event_triggered = false);
>>   
>> -- 
>> 2.33.0
> 
>