From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id D0D7E35DA43 for ; Mon, 13 Jul 2026 20:37:22 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783975044; cv=none; b=S0B7i+t3rllVTXydUqRB57pGug04EqnN8CoqeCgHt0Tr6uISH+D5QPInRTXeUVLYebAmZHXCCK+31IfKFm3UCKhXjdlCV8fafLnLWszWESzEhhgXrzV7DpCwEBVJGgqGoyeWPgVgc43Q+JmCXIFRxkSXURPZvX3qBO5oaKXHYf0= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1783975044; c=relaxed/simple; bh=oUwoqKe1I/6hnoW/yPmZKqCl+Dd44tauTgfIsfpIof0=; h=Date:From:To:Cc:Subject:Message-ID:References:MIME-Version: Content-Type:Content-Disposition:In-Reply-To; b=NFZ5rb0cQfqrTCHDEML7tyUQu/Y+6Edama/lD0aRAlbfWTJIND722Z/RVrp/V7RnH6kBgLu2uJdUjr/rw5pXtUT6jvAHSGKK1JC4xz/vr9Q+/VoR8JSkjbzq13ZbN5P2InBQ7VX1cQ8Zlf4DXagXXgTK2KL8Om4fAo9MMKNq1po= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=ZDxDGcrj; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b=osgMG7xe; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=quarantine dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="ZDxDGcrj"; dkim=pass (2048-bit key) header.d=redhat.com header.i=@redhat.com header.b="osgMG7xe" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1783975042; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=93agWWxYqJptOVx1oz7T8JnMRAoNjNWehGTGOvfl/80=; b=ZDxDGcrjNschN4kl4an9bas49DFeZtjxzzT7RFv79LuHbE8C2g4y1dLI5EXLF1hqKSjYZg SdLRDMCjrgA5IPnJORQz6/m51ud7HO6GkgbnJF3nRox6WDeZ5k95a5AqBjv4zqs/7Ny+Qw DOuGQjryJAog/nn1sJ0f29POHmouwxE= Received: from mail-wr1-f69.google.com (mail-wr1-f69.google.com [209.85.221.69]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-73-xOe80doxPBWN-rWruz6d0Q-1; Mon, 13 Jul 2026 16:37:20 -0400 X-MC-Unique: xOe80doxPBWN-rWruz6d0Q-1 X-Mimecast-MFC-AGG-ID: xOe80doxPBWN-rWruz6d0Q_1783975039 Received: by mail-wr1-f69.google.com with SMTP id ffacd0b85a97d-4767036f8faso2134180f8f.3 for ; Mon, 13 Jul 2026 13:37:20 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=google; t=1783975039; x=1784579839; darn=vger.kernel.org; h=in-reply-to:content-transfer-encoding:content-disposition :content-type:mime-version:references:message-id:subject:cc:to:from :date:from:to:cc:subject:date:message-id:reply-to:content-type; bh=93agWWxYqJptOVx1oz7T8JnMRAoNjNWehGTGOvfl/80=; b=osgMG7xeuMGTPdR9GpmNH8R4j8HD0SQBVZJjlCdJAq3RnsYkqjeyf7UT4UDTARhb8a ezXI9SKkjwGGo0hBKsMLwIlDVXZ1Fvv7kUrQyNKsGMdxto/YOKPmGjOfvUPaL1phI893 DKbd41X591y2gcaZNu1L5QB5BLBWdn85ZeGwTlUBbCzT1+wewyaeUV0UAYfi2kK3+qYD QX3eTAH4fwTkqgO2AGW+nQl459PvCIEemj7IG0jAEt1XH/+DZnr4NsZCNLdy2FSXV9Z6 e/7GdMNE5/M2YhQ1csPRpb9g89SxuU7Hx1uvY2BCb/aVEqsgJr9bQPdSOOpgs2kKfMeq 2v1w== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20251104; t=1783975039; x=1784579839; h=in-reply-to:content-transfer-encoding:content-disposition :content-type:mime-version:references:message-id:subject:cc:to:from :date:x-gm-gg:x-gm-message-state:from:to:cc:subject:date:message-id :reply-to:content-type; bh=93agWWxYqJptOVx1oz7T8JnMRAoNjNWehGTGOvfl/80=; b=froiWngcvFHC6GH1u+owBMlX2c/L1SYM5Oyf7WRUFOpG30ju6+ZCIA7c76nsdgJ2hn GBj3hM5NAI3h/jN33VC2c7JsUvuKZQ1lHuNY553iYJzrk8yZOzRreAYeEKMq7uq39gSr BmVpf8tL/p9b9Zk1WNxASryb9DDoDIdJ2TLuo9J0b8lqvtSsIvL01u0xIqdEGO7JQf6Y c+q4fvJj8LFpzgJkgm7dF5GzhuggsigBS/YAKXQqGCsya20lMoyVn1XZly7rC64oG4Qq WalXvOT1QFMq3Gkc6KfxO1XomOPqbifE+0dPezF+Ja0RExjRGD/8/6ftB+nJXRy3zSGR 2RYw== X-Forwarded-Encrypted: i=1; AHgh+RrBjyH9r9EeEWrEwWtmeKctKuqAOqyQuvj8mtoPlHoJ2WYHGGXABXRh7RkVXDfwxZ63TEFDy7xSyARm3JY=@vger.kernel.org X-Gm-Message-State: AOJu0YyEMB4tLDPKf5EpwkVDgOnVJ3TrExN1pRZ3wM9wlN6fmLlPhq+8 mCqKqBfwnAPeusyNwhwhH4D0cYPJyTifZoqxKtw2uodsoIxTOb34HzQKqSLOdBFjb6DaKBrYYlD ItQ9fMy9bwz3XVbOnS5N7uY6r57kRe8eMZfJ0FEveCXOzHVixAplYFDISCb9DjTD0qA== X-Gm-Gg: AfdE7cnlvbzW0yfuoVnA7PWta3KMgtblZ4ROpeKf1rMZ5aaddj3NmlUSzIsovIuHZkb yt++nPiE2vihGg09KQgFW80bfDdN9J/KSbBNCq3fe6M/z4hcmZk5T57qgcwpuI/mKJlidaacm2n 7v6Sfs473QZJzwBr4s8uC4C7k5hA7a5Pe44cT6PbAfI513FpLZrSNXKDamO8DLpVxRUZIaAVIJQ K5qqWOlQEoEvyXdoaiEWOtrbcDmZ/zsauq26CwHg8wdn/5a5n0DE2L4GtZ/BsqOMXoU1H7xl1O1 VBxzXA5gdZRq2SkBwc5IVqtypHReGQhxfzboTrkspwU+xyl2SgLce9jzwgxVc1iCDh81RRywPzK 1kIc5xDw7twE5hptVvz+of2rN692x+RSh5WM= X-Received: by 2002:a05:6000:430a:b0:475:94e1:29d2 with SMTP id ffacd0b85a97d-47f2dca8a2amr12617045f8f.2.1783975039020; Mon, 13 Jul 2026 13:37:19 -0700 (PDT) X-Received: by 2002:a05:6000:430a:b0:475:94e1:29d2 with SMTP id ffacd0b85a97d-47f2dca8a2amr12617028f8f.2.1783975038543; Mon, 13 Jul 2026 13:37:18 -0700 (PDT) Received: from redhat.com (IGLD-80-230-24-117.inter.net.il. [80.230.24.117]) by smtp.gmail.com with ESMTPSA id ffacd0b85a97d-47f464a9884sm2463389f8f.23.2026.07.13.13.37.17 (version=TLS1_3 cipher=TLS_AES_256_GCM_SHA384 bits=256/256); Mon, 13 Jul 2026 13:37:18 -0700 (PDT) Date: Mon, 13 Jul 2026 16:37:15 -0400 From: "Michael S. Tsirkin" To: "Denis V. Lunev" Cc: "David Hildenbrand (Arm)" , "Denis V. Lunev" , virtualization@lists.linux.dev, linux-kernel@vger.kernel.org Subject: Re: [PATCH v2 0/4] virtio_balloon: quiesce balloon work on device shutdown Message-ID: <20260713163611-mutt-send-email-mst@kernel.org> References: <20260624140846.2616797-1-den@openvz.org> <00ebb86d-f7b8-4ddc-a17c-c4700cab73a9@kernel.org> <20260702183236-mutt-send-email-mst@kernel.org> <6697d41e-a2fd-4207-982d-999cafd3cd56@virtuozzo.com> Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Disposition: inline Content-Transfer-Encoding: 8bit In-Reply-To: <6697d41e-a2fd-4207-982d-999cafd3cd56@virtuozzo.com> On Mon, Jul 13, 2026 at 10:32:29PM +0200, Denis V. Lunev wrote: > On 7/3/26 00:33, Michael S. Tsirkin wrote: > > On Thu, Jul 02, 2026 at 09:30:26PM +0200, David Hildenbrand (Arm) wrote: > >> On 7/2/26 19:50, Denis V. Lunev wrote: > >>> On 6/24/26 16:08, Denis V. Lunev wrote: > >>>> This email originated from an IP that might not be authorized by the domain it was sent from. > >>>> Do not click links or open attachments unless it is an email you expected to receive. > >>>> Since commit 8bd2fa086a04 ("virtio: break and reset virtio devices on > >>>> device_shutdown()") the virtio bus breaks and resets every virtio device > >>>> during device_shutdown(), i.e. on reboot and kexec. virtio_balloon has no > >>>> .shutdown of its own, so that generic path runs while the balloon's > >>>> asynchronous work is still armed: the free page reporting worker, the > >>>> inflate/deflate and stats workers, the OOM notifier and the free page > >>>> shrinker. > >>>> > >>>> Once the device has been broken, virtqueue_add_inbuf() in > >>>> virtballoon_free_page_report() returns -EIO and trips its WARN_ON_ONCE(). > >>>> On a kernel booted with panic_on_warn that turns an ordinary reboot into a > >>>> fatal panic in the middle of device_shutdown(), so the machine never > >>>> reaches the new kernel. The inflate/deflate and OOM paths do not warn but > >>>> are no better off: they call wait_event(vb->acked, ...) and would block > >>>> forever on a queue that can no longer complete. > >>>> > >>>> This was hit in the field as an intermittent failure of a virtualization > >>>> cluster upgrade: guest storage nodes were rebooted via kexec into the new > >>>> kernel, and the ones whose free page reporting happened to run during > >>>> device_shutdown() panicked (the guests run with panic_on_warn) and never > >>>> came back, stalling the rolling upgrade. The crash dump showed the WARN at > >>>> virtio_balloon.c:216 in a page_reporting kworker, with all the balloon > >>>> virtqueues already broken. > >>>> > >>>> Validated by churning balloon inflate/deflate from the host while > >>>> kexec-rebooting the guest in a loop under panic_on_warn: the unpatched > >>>> kernel reproduces the WARN within a couple of cycles, while the patched > >>>> kernel survives many consecutive kexec cycles cleanly (12/12 in the final > >>>> run, 0 WARNs). checkpatch is clean across the series. > >>>> > >>>> Changes in v2: > >>>> - Add a virtio_device_shutdown() core helper and call it from the balloon > >>>> .shutdown handler instead of open-coding break + synchronize_cbs + reset > >>>> (David Hildenbrand). > >>>> - New patch: make tell_host() warn and bail instead of hanging if a buffer > >>>> add ever fails (David Hildenbrand); kept as a separate patch > >>>> (Michael S. Tsirkin). > >>>> > >>>> v1: https://lore.kernel.org/all/20260622133715.3707707-1-den@openvz.org > >>>> > >>>> Denis V. Lunev (4): > >>>> virtio: add virtio_device_shutdown() helper > >>>> virtio_balloon: factor out virtballoon_quiesce() > >>>> virtio_balloon: quiesce balloon work before device shutdown > >>>> virtio_balloon: warn on failed buffer add in tell_host() > >>>> > >>>> drivers/virtio/virtio.c | 41 ++++++++++++++++++++++----------- > >>>> drivers/virtio/virtio_balloon.c | 40 ++++++++++++++++++++++++-------- > >>>> include/linux/virtio.h | 1 + > >>>> 3 files changed, 59 insertions(+), 23 deletions(-) > >>>> > >>> Hi, David! > >> Hi! :) > >> > >>> Is this good to go in? I have not seen the confirmation that the > >>> series is taken into your tree. > >> I don't have a tree (yet), and once I have one it will likely be more mm focused :) > >> > >> @MST, I think this is good to go! > >> > >> -- > >> Cheers, > >> > >> David > > Indeed, it's just a bugfix and I'm trying to get features into qemu > > now before their freeze. I'll work on linux end of next week. > > > Hi, Michael! > > Have you had a chance to take series into your tree, i.e. > should I continue to track this submission? > > Thank you in advance, >     Den Hi! It's on my queue and will be in the next Linux release, don't worry. -- MST