The Linux Kernel Mailing List
 help / color / mirror / Atom feed
* [PATCH 0/2] virtio_balloon: quiesce balloon work on device shutdown
@ 2026-06-22 13:37 Denis V. Lunev
  2026-06-22 13:37 ` [PATCH 1/2] virtio_balloon: factor out virtballoon_quiesce() Denis V. Lunev
                   ` (2 more replies)
  0 siblings, 3 replies; 9+ messages in thread
From: Denis V. Lunev @ 2026-06-22 13:37 UTC (permalink / raw)
  To: mst, david; +Cc: virtualization, linux-kernel, Denis V. Lunev

Since commit 8bd2fa086a04 ("virtio: break and reset virtio devices on
device_shutdown()") the virtio bus breaks and resets every virtio device
during device_shutdown(), i.e. on reboot and kexec. virtio_balloon has no
.shutdown of its own, so that generic path runs while the balloon's
asynchronous work is still armed: the free page reporting worker, the
inflate/deflate and stats workers, the OOM notifier and the free page
shrinker.

Once the device has been broken, virtqueue_add_inbuf() in
virtballoon_free_page_report() returns -EIO and trips its WARN_ON_ONCE().
On a kernel booted with panic_on_warn that turns an ordinary reboot into a
fatal panic in the middle of device_shutdown(), so the machine never
reaches the new kernel. The inflate/deflate and OOM paths do not warn but
are no better off: they call wait_event(vb->acked, ...) and would block
forever on a queue that can no longer complete.

This was hit in the field as an intermittent failure of a virtualization
cluster upgrade: guest storage nodes were rebooted via kexec into the new
kernel, and the ones whose free page reporting happened to run during
device_shutdown() panicked (the guests run with panic_on_warn) and never
came back, stalling the rolling upgrade. The crash dump showed the WARN at
virtio_balloon.c:216 in a page_reporting kworker, with all the balloon
virtqueues already broken.

Patch 1 factors the teardown out of virtballoon_remove() into a
virtballoon_quiesce() helper (no functional change). Patch 2 adds a
virtio_balloon .shutdown handler that quiesces via that helper while the
device is still alive, then breaks and resets it the way the generic
virtio_dev_shutdown() would.

Relaxing the single WARN_ON_ONCE() instead was considered and rejected: it
would silence the panic but leave the inflate/deflate and OOM paths
hanging on the broken device. The device has to be quiesced, not just kept
quiet.

Validated by churning balloon inflate/deflate from the host while
kexec-rebooting the guest in a loop under panic_on_warn: the unpatched
module reproduces the WARN within a couple of cycles, while the patched
module survives many consecutive kexec cycles cleanly (12/12 in the final
run, 0 WARNs). checkpatch is clean on both patches.

Denis V. Lunev (2):
  virtio_balloon: factor out virtballoon_quiesce()
  virtio_balloon: quiesce balloon work before device shutdown

 drivers/virtio/virtio_balloon.c | 37 ++++++++++++++++++++++++++-------
 1 file changed, 30 insertions(+), 7 deletions(-)

-- 
2.53.0


^ permalink raw reply	[flat|nested] 9+ messages in thread

end of thread, other threads:[~2026-06-22 14:59 UTC | newest]

Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-06-22 13:37 [PATCH 0/2] virtio_balloon: quiesce balloon work on device shutdown Denis V. Lunev
2026-06-22 13:37 ` [PATCH 1/2] virtio_balloon: factor out virtballoon_quiesce() Denis V. Lunev
2026-06-22 14:46   ` David Hildenbrand (Arm)
2026-06-22 14:59     ` Michael S. Tsirkin
2026-06-22 13:37 ` [PATCH 2/2] virtio_balloon: quiesce balloon work before device shutdown Denis V. Lunev
2026-06-22 14:38   ` David Hildenbrand (Arm)
2026-06-22 14:58     ` Michael S. Tsirkin
2026-06-22 14:29 ` [PATCH 0/2] virtio_balloon: quiesce balloon work on " David Hildenbrand (Arm)
2026-06-22 14:33   ` Denis V. Lunev

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox