* Re: [PATCH 7.2 v10 0/2] skip redundant sync IPIs when TLB flush sent them
From: David Hildenbrand (Arm) @ 2026-04-24 14:20 UTC (permalink / raw)
To: Andrew Morton, Pasha Tatashin
Cc: Lance Yang, peterz, dave.hansen, dave.hansen, ypodemsk, hughd,
will, aneesh.kumar, npiggin, tglx, mingo, bp, x86, hpa, arnd, ljs,
ziy, baolin.wang, Liam.Howlett, npache, ryan.roberts, dev.jain,
baohua, shy828301, riel, jannh, jgross, seanjc, pbonzini,
boris.ostrovsky, virtualization, kvm, linux-arch, linux-mm,
linux-kernel, ioworker0, roman.gushchin
In-Reply-To: <20260424071533.d28ce90126f05e1c6fc1b740@linux-foundation.org>
On 4/24/26 16:15, Andrew Morton wrote:
> On Fri, 24 Apr 2026 13:37:04 +0000 Pasha Tatashin <pasha.tatashin@soleen.com> wrote:
>
>> On 04-24 06:30, Andrew Morton wrote:
>>>
>>>
>>> Sashiko questions:
>>> https://sashiko.dev/#/patchset/20260424062528.71951-1-lance.yang@linux.dev
>>>
>>> (I never know if my Sashiko emails are welcome/useful. Maybe Sashiko
>>> said the same stuff about v9 and it's all wrong. But better safe than
>>> sorry!)
>>
>> These emails are helpful; but, I do not believe you should have to
>> manually follow up with a link to every new patch series.
>>
>> Perhaps Sashiko could automatically send a summary email in response to
>> the cover letter, or provide a link once the reviews are complete. For
>> the kexec ML, we opted-in with Roman to receive automated emails from
>> sashiko.
>
> Yep. I'd be OK with an automatic reply-to-all. Maybe some won't like
> that.
>
> An alternative I've discussed with Roman is an automated
> reply-to-author with a cc to a dedicated list (we could use mm-commits
> for now).
>
> Preferences?
Reply-to-author would likely be better as a first step.
--
Cheers,
David
^ permalink raw reply
* Re: [PATCH 7.2 v10 0/2] skip redundant sync IPIs when TLB flush sent them
From: Andrew Morton @ 2026-04-24 14:15 UTC (permalink / raw)
To: Pasha Tatashin
Cc: Lance Yang, peterz, david, dave.hansen, dave.hansen, ypodemsk,
hughd, will, aneesh.kumar, npiggin, tglx, mingo, bp, x86, hpa,
arnd, ljs, ziy, baolin.wang, Liam.Howlett, npache, ryan.roberts,
dev.jain, baohua, shy828301, riel, jannh, jgross, seanjc,
pbonzini, boris.ostrovsky, virtualization, kvm, linux-arch,
linux-mm, linux-kernel, ioworker0, roman.gushchin
In-Reply-To: <aetw4aoABYj3keMh@plex>
On Fri, 24 Apr 2026 13:37:04 +0000 Pasha Tatashin <pasha.tatashin@soleen.com> wrote:
> On 04-24 06:30, Andrew Morton wrote:
> > On Fri, 24 Apr 2026 14:25:26 +0800 Lance Yang <lance.yang@linux.dev> wrote:
> >
> > > When page table operations require synchronization with software/lockless
> > > walkers, they call tlb_remove_table_sync_{one,rcu}() after flushing the
> > > TLB (tlb->freed_tables or tlb->unshared_tables).
> > >
> > > On architectures where the TLB flush already sends IPIs to all target CPUs,
> > > the subsequent sync IPI broadcast is redundant. This is not only costly on
> > > large systems where it disrupts all CPUs even for single-process page table
> > > operations, but has also been reported to hurt RT workloads[1].
> > >
> > > This series introduces tlb_table_flush_implies_ipi_broadcast() to check if
> > > the prior TLB flush already provided the necessary synchronization. When
> > > true, the sync calls can early-return.
> > >
> > > A few cases rely on this synchronization:
> > >
> > > 1) hugetlb PMD unshare[2]: The problem is not the freeing but the reuse
> > > of the PMD table for other purposes in the last remaining user after
> > > unsharing.
> > >
> > > 2) khugepaged collapse[3]: Ensure no concurrent GUP-fast before collapsing
> > > and (possibly) freeing the page table / re-depositing it.
> >
> > Sashiko questions:
> > https://sashiko.dev/#/patchset/20260424062528.71951-1-lance.yang@linux.dev
> >
> > (I never know if my Sashiko emails are welcome/useful. Maybe Sashiko
> > said the same stuff about v9 and it's all wrong. But better safe than
> > sorry!)
>
> These emails are helpful; but, I do not believe you should have to
> manually follow up with a link to every new patch series.
>
> Perhaps Sashiko could automatically send a summary email in response to
> the cover letter, or provide a link once the reviews are complete. For
> the kexec ML, we opted-in with Roman to receive automated emails from
> sashiko.
Yep. I'd be OK with an automatic reply-to-all. Maybe some won't like
that.
An alternative I've discussed with Roman is an automated
reply-to-author with a cc to a dedicated list (we could use mm-commits
for now).
Preferences?
(I'd suggest automated-reply-to-all, see who complains, then figure out
why they can't figure out mail filtering ;))
^ permalink raw reply
* [PATCH] drm/bochs: Drop manual put on probe error path
From: Myeonghun Pak @ 2026-04-24 13:54 UTC (permalink / raw)
To: Gerd Hoffmann, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter
Cc: Myeonghun Pak, virtualization, dri-devel, stable, linux-kernel
bochs_pci_probe() allocates the DRM device with devm_drm_dev_alloc(),
which registers a devres action to drop the initial DRM device reference
on driver detach or probe failure.
The error path currently calls drm_dev_put() manually. If probe then
returns an error, devres will run the registered release action and put
the same device again, after the first put may already have released it.
Return the probe error directly and let devres own the final put.
Fixes: 04826f588682 ("drm/bochs: Allocate DRM device in struct bochs_device")
Cc: stable@vger.kernel.org
Signed-off-by: Myeonghun Pak <mhun512@gmail.com>
---
drivers/gpu/drm/tiny/bochs.c | 10 +++-------
1 file changed, 3 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/drm/tiny/bochs.c b/drivers/gpu/drm/tiny/bochs.c
index 222e4ae1ab..5d8dc5efec 100644
--- a/drivers/gpu/drm/tiny/bochs.c
+++ b/drivers/gpu/drm/tiny/bochs.c
@@ -761,25 +761,21 @@ static int bochs_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent
ret = pcim_enable_device(pdev);
if (ret)
- goto err_free_dev;
+ return ret;
pci_set_drvdata(pdev, dev);
ret = bochs_load(bochs);
if (ret)
- goto err_free_dev;
+ return ret;
ret = drm_dev_register(dev, 0);
if (ret)
- goto err_free_dev;
+ return ret;
drm_client_setup(dev, NULL);
return ret;
-
-err_free_dev:
- drm_dev_put(dev);
- return ret;
}
static void bochs_pci_remove(struct pci_dev *pdev)
--
2.39.5
^ permalink raw reply related
* Re: [PATCH v2] virtio: rtc: tear down old virtqueues before restore
From: Peter Hilber @ 2026-04-24 13:54 UTC (permalink / raw)
To: JiaJia; +Cc: mst, jasowang, xuanzhuo, eperezma, virtualization, linux-kernel
In-Reply-To: <69eaeca0.050a0220.408e2.027b@mx.google.com>
On Fri, Apr 24, 2026 at 11:14:53AM +0800, JiaJia wrote:
> virtio_device_restore() resets the device and restores the negotiated
> features before calling ->restore(). viortc_freeze() intentionally
> leaves the existing virtqueues in place so the alarm queue can still
> wake the system, but viortc_restore() immediately calls
> viortc_init_vqs() without first deleting those old queues.
>
> If virtqueue reinitialization fails on virtio-pci, the transport error
> path can run vp_del_vqs() against a newly allocated vp_dev->vqs array
> while vdev->vqs still contains the old virtqueues. vp_del_vqs() then
> looks up queue state through the new array and can dereference a NULL
> info pointer in vp_del_vq(), crashing the guest kernel during restore.
>
> Delete the stale virtqueues before rebuilding them. If restore fails
> before virtio_device_ready(), reuse the remove path to stop the device.
> Once the device is ready, return errors directly instead of deleting the
> virtqueues again.
>
> Signed-off-by: JiaJia <physicalmtea@gmail.com>
Reviewed-by: Peter Hilber <peter.hilber@oss.qualcomm.com>
What should still be added is the Fixes tag:
Fixes: 0623c7592768 ("virtio_rtc: Add module and driver core")
Maybe the commit message could also mention that this can happen during
a non-faulty reinitialization, when one of the vp_find_vqs_msix()
attempts is unsuccessful before a later attempt would succeed (as far as
I understand).
Thanks!
Peter
^ permalink raw reply
* Re: [PATCH] virtio: Add ID for virtio media
From: Albert Esteve @ 2026-04-24 13:37 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: Jason Wang, Xuan Zhuo, Eugenio Pérez, virtualization,
linux-kernel, gnurou
In-Reply-To: <20260424090616-mutt-send-email-mst@kernel.org>
On Fri, Apr 24, 2026 at 3:07 PM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> On Thu, Apr 09, 2026 at 06:53:41PM +0200, Albert Esteve wrote:
> > Hi,
> >
> >
> > On Tue, Mar 10, 2026 at 9:41 AM Albert Esteve <aesteve@redhat.com> wrote:
> > >
> > > Add VIRTIO_ID_MEDIA definition for virtio-media.
> > >
> > > Signed-off-by: Albert Esteve <aesteve@redhat.com>
> >
> > Are there any thoughts on this patch, or any actions I need to take?
> >
> > Thanks,
> > Albert
>
> Create a github issue and request a vote as documented in README.md
The inclusion of this ID was already voted and approved. The device
already appears in the VirtIO v1.4 specification, with the same ID (as
linked below). This patch is only meant to translate the decision to
the kernel header.
>
> > > ---
> > > While the Linux driver for virtio media is currently under active
> > > development and review [1], the VirtIO v1.4 specification has
> > > already been published and oficially defines the media device type [2].
> > >
> > > Adding the device ID now allows other components and tools to reference it
> > > before the driver lands, and ensures the kernel headers stay aligned
> > > with the specification.
> > >
> > > [1] https://lore.kernel.org/linux-media/20250412-virtio-media-v3-1-97dc94c18398@gmail.com/
> > > [2] https://docs.oasis-open.org/virtio/virtio/v1.4/virtio-v1.4.html#x1-82200022
> > > ---
> > > include/uapi/linux/virtio_ids.h | 1 +
> > > 1 file changed, 1 insertion(+)
> > >
> > > diff --git a/include/uapi/linux/virtio_ids.h b/include/uapi/linux/virtio_ids.h
> > > index 6c12db16faa3a..f9056af0c6223 100644
> > > --- a/include/uapi/linux/virtio_ids.h
> > > +++ b/include/uapi/linux/virtio_ids.h
> > > @@ -69,6 +69,7 @@
> > > #define VIRTIO_ID_BT 40 /* virtio bluetooth */
> > > #define VIRTIO_ID_GPIO 41 /* virtio gpio */
> > > #define VIRTIO_ID_SPI 45 /* virtio spi */
> > > +#define VIRTIO_ID_MEDIA 48 /* virtio media */
> > >
> > > /*
> > > * Virtio Transitional IDs
> > >
> > > ---
> > > base-commit: 6de23f81a5e08be8fbf5e8d7e9febc72a5b5f27f
> > > change-id: 20260310-virtio-media-id-b31179fcdc44
> > >
> > > Best regards,
> > > --
> > > Albert Esteve <aesteve@redhat.com>
> > >
>
^ permalink raw reply
* Re: [PATCH 7.2 v10 0/2] skip redundant sync IPIs when TLB flush sent them
From: Pasha Tatashin @ 2026-04-24 13:37 UTC (permalink / raw)
To: Andrew Morton
Cc: Lance Yang, peterz, david, dave.hansen, dave.hansen, ypodemsk,
hughd, will, aneesh.kumar, npiggin, tglx, mingo, bp, x86, hpa,
arnd, ljs, ziy, baolin.wang, Liam.Howlett, npache, ryan.roberts,
dev.jain, baohua, shy828301, riel, jannh, jgross, seanjc,
pbonzini, boris.ostrovsky, virtualization, kvm, linux-arch,
linux-mm, linux-kernel, ioworker0, roman.gushchin
In-Reply-To: <20260424063024.ce42ee6a5546e4d9337dd007@linux-foundation.org>
On 04-24 06:30, Andrew Morton wrote:
> On Fri, 24 Apr 2026 14:25:26 +0800 Lance Yang <lance.yang@linux.dev> wrote:
>
> > When page table operations require synchronization with software/lockless
> > walkers, they call tlb_remove_table_sync_{one,rcu}() after flushing the
> > TLB (tlb->freed_tables or tlb->unshared_tables).
> >
> > On architectures where the TLB flush already sends IPIs to all target CPUs,
> > the subsequent sync IPI broadcast is redundant. This is not only costly on
> > large systems where it disrupts all CPUs even for single-process page table
> > operations, but has also been reported to hurt RT workloads[1].
> >
> > This series introduces tlb_table_flush_implies_ipi_broadcast() to check if
> > the prior TLB flush already provided the necessary synchronization. When
> > true, the sync calls can early-return.
> >
> > A few cases rely on this synchronization:
> >
> > 1) hugetlb PMD unshare[2]: The problem is not the freeing but the reuse
> > of the PMD table for other purposes in the last remaining user after
> > unsharing.
> >
> > 2) khugepaged collapse[3]: Ensure no concurrent GUP-fast before collapsing
> > and (possibly) freeing the page table / re-depositing it.
>
> Sashiko questions:
> https://sashiko.dev/#/patchset/20260424062528.71951-1-lance.yang@linux.dev
>
> (I never know if my Sashiko emails are welcome/useful. Maybe Sashiko
> said the same stuff about v9 and it's all wrong. But better safe than
> sorry!)
These emails are helpful; but, I do not believe you should have to
manually follow up with a link to every new patch series.
Perhaps Sashiko could automatically send a summary email in response to
the cover letter, or provide a link once the reviews are complete. For
the kexec ML, we opted-in with Roman to receive automated emails from
sashiko.
+Cc: Roman.
>
>
^ permalink raw reply
* Re: [PATCH 7.2 v10 0/2] skip redundant sync IPIs when TLB flush sent them
From: Andrew Morton @ 2026-04-24 13:30 UTC (permalink / raw)
To: Lance Yang
Cc: peterz, david, dave.hansen, dave.hansen, ypodemsk, hughd, will,
aneesh.kumar, npiggin, tglx, mingo, bp, x86, hpa, arnd, ljs, ziy,
baolin.wang, Liam.Howlett, npache, ryan.roberts, dev.jain, baohua,
shy828301, riel, jannh, jgross, seanjc, pbonzini, boris.ostrovsky,
virtualization, kvm, linux-arch, linux-mm, linux-kernel,
ioworker0
In-Reply-To: <20260424062528.71951-1-lance.yang@linux.dev>
On Fri, 24 Apr 2026 14:25:26 +0800 Lance Yang <lance.yang@linux.dev> wrote:
> When page table operations require synchronization with software/lockless
> walkers, they call tlb_remove_table_sync_{one,rcu}() after flushing the
> TLB (tlb->freed_tables or tlb->unshared_tables).
>
> On architectures where the TLB flush already sends IPIs to all target CPUs,
> the subsequent sync IPI broadcast is redundant. This is not only costly on
> large systems where it disrupts all CPUs even for single-process page table
> operations, but has also been reported to hurt RT workloads[1].
>
> This series introduces tlb_table_flush_implies_ipi_broadcast() to check if
> the prior TLB flush already provided the necessary synchronization. When
> true, the sync calls can early-return.
>
> A few cases rely on this synchronization:
>
> 1) hugetlb PMD unshare[2]: The problem is not the freeing but the reuse
> of the PMD table for other purposes in the last remaining user after
> unsharing.
>
> 2) khugepaged collapse[3]: Ensure no concurrent GUP-fast before collapsing
> and (possibly) freeing the page table / re-depositing it.
Sashiko questions:
https://sashiko.dev/#/patchset/20260424062528.71951-1-lance.yang@linux.dev
(I never know if my Sashiko emails are welcome/useful. Maybe Sashiko
said the same stuff about v9 and it's all wrong. But better safe than
sorry!)
^ permalink raw reply
* Re: [PATCH] drm/bochs: Drop manual put on probe error path
From: Thomas Zimmermann @ 2026-04-24 13:25 UTC (permalink / raw)
To: Myeonghun Pak, Gerd Hoffmann, Maarten Lankhorst, Maxime Ripard,
David Airlie, Simona Vetter
Cc: virtualization, dri-devel, linux-kernel
In-Reply-To: <20260424123506.32275-1-mhun512@gmail.com>
Hi,
please add fixes tags to all these patches you're sending. You also need
to CC stable so that they can be backported easily. Also list the AI
you're using to find and create these patches.
Best regards
Thomas
Am 24.04.26 um 14:34 schrieb Myeonghun Pak:
> bochs_pci_probe() allocates the DRM device with devm_drm_dev_alloc(),
> which registers a devres action to drop the initial DRM device reference
> on driver detach or probe failure.
>
> The error path currently calls drm_dev_put() manually. If probe then
> returns an error, devres will run the registered release action and put
> the same device again, after the first put may already have released it.
>
> Return the probe error directly and let devres own the final put.
>
> Signed-off-by: Myeonghun Pak <mhun512@gmail.com>
> ---
> drivers/gpu/drm/tiny/bochs.c | 10 +++-------
> 1 file changed, 3 insertions(+), 7 deletions(-)
>
> diff --git a/drivers/gpu/drm/tiny/bochs.c b/drivers/gpu/drm/tiny/bochs.c
> index 222e4ae1ab..5d8dc5efec 100644
> --- a/drivers/gpu/drm/tiny/bochs.c
> +++ b/drivers/gpu/drm/tiny/bochs.c
> @@ -761,25 +761,21 @@ static int bochs_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent
>
> ret = pcim_enable_device(pdev);
> if (ret)
> - goto err_free_dev;
> + return ret;
>
> pci_set_drvdata(pdev, dev);
>
> ret = bochs_load(bochs);
> if (ret)
> - goto err_free_dev;
> + return ret;
>
> ret = drm_dev_register(dev, 0);
> if (ret)
> - goto err_free_dev;
> + return ret;
>
> drm_client_setup(dev, NULL);
>
> return ret;
> -
> -err_free_dev:
> - drm_dev_put(dev);
> - return ret;
> }
>
> static void bochs_pci_remove(struct pci_dev *pdev)
--
--
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Frankenstr. 146, 90461 Nürnberg, Germany, www.suse.com
GF: Jochen Jaser, Andrew McDonald, Werner Knoblich, (HRB 36809, AG Nürnberg)
^ permalink raw reply
* Re: [PATCH] virtio: Add ID for virtio media
From: Michael S. Tsirkin @ 2026-04-24 13:06 UTC (permalink / raw)
To: Albert Esteve
Cc: Jason Wang, Xuan Zhuo, Eugenio Pérez, virtualization,
linux-kernel, gnurou
In-Reply-To: <CADSE00+VyDf6hMAB79wP2koSQ3FznO_OJuGCZ6zN8zuCrwtZMg@mail.gmail.com>
On Thu, Apr 09, 2026 at 06:53:41PM +0200, Albert Esteve wrote:
> Hi,
>
>
> On Tue, Mar 10, 2026 at 9:41 AM Albert Esteve <aesteve@redhat.com> wrote:
> >
> > Add VIRTIO_ID_MEDIA definition for virtio-media.
> >
> > Signed-off-by: Albert Esteve <aesteve@redhat.com>
>
> Are there any thoughts on this patch, or any actions I need to take?
>
> Thanks,
> Albert
Create a github issue and request a vote as documented in README.md
> > ---
> > While the Linux driver for virtio media is currently under active
> > development and review [1], the VirtIO v1.4 specification has
> > already been published and oficially defines the media device type [2].
> >
> > Adding the device ID now allows other components and tools to reference it
> > before the driver lands, and ensures the kernel headers stay aligned
> > with the specification.
> >
> > [1] https://lore.kernel.org/linux-media/20250412-virtio-media-v3-1-97dc94c18398@gmail.com/
> > [2] https://docs.oasis-open.org/virtio/virtio/v1.4/virtio-v1.4.html#x1-82200022
> > ---
> > include/uapi/linux/virtio_ids.h | 1 +
> > 1 file changed, 1 insertion(+)
> >
> > diff --git a/include/uapi/linux/virtio_ids.h b/include/uapi/linux/virtio_ids.h
> > index 6c12db16faa3a..f9056af0c6223 100644
> > --- a/include/uapi/linux/virtio_ids.h
> > +++ b/include/uapi/linux/virtio_ids.h
> > @@ -69,6 +69,7 @@
> > #define VIRTIO_ID_BT 40 /* virtio bluetooth */
> > #define VIRTIO_ID_GPIO 41 /* virtio gpio */
> > #define VIRTIO_ID_SPI 45 /* virtio spi */
> > +#define VIRTIO_ID_MEDIA 48 /* virtio media */
> >
> > /*
> > * Virtio Transitional IDs
> >
> > ---
> > base-commit: 6de23f81a5e08be8fbf5e8d7e9febc72a5b5f27f
> > change-id: 20260310-virtio-media-id-b31179fcdc44
> >
> > Best regards,
> > --
> > Albert Esteve <aesteve@redhat.com>
> >
^ permalink raw reply
* [PATCH] drm/bochs: Drop manual put on probe error path
From: Myeonghun Pak @ 2026-04-24 12:34 UTC (permalink / raw)
To: Gerd Hoffmann, Maarten Lankhorst, Maxime Ripard,
Thomas Zimmermann, David Airlie, Simona Vetter
Cc: Myeonghun Pak, virtualization, dri-devel, linux-kernel
bochs_pci_probe() allocates the DRM device with devm_drm_dev_alloc(),
which registers a devres action to drop the initial DRM device reference
on driver detach or probe failure.
The error path currently calls drm_dev_put() manually. If probe then
returns an error, devres will run the registered release action and put
the same device again, after the first put may already have released it.
Return the probe error directly and let devres own the final put.
Signed-off-by: Myeonghun Pak <mhun512@gmail.com>
---
drivers/gpu/drm/tiny/bochs.c | 10 +++-------
1 file changed, 3 insertions(+), 7 deletions(-)
diff --git a/drivers/gpu/drm/tiny/bochs.c b/drivers/gpu/drm/tiny/bochs.c
index 222e4ae1ab..5d8dc5efec 100644
--- a/drivers/gpu/drm/tiny/bochs.c
+++ b/drivers/gpu/drm/tiny/bochs.c
@@ -761,25 +761,21 @@ static int bochs_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent
ret = pcim_enable_device(pdev);
if (ret)
- goto err_free_dev;
+ return ret;
pci_set_drvdata(pdev, dev);
ret = bochs_load(bochs);
if (ret)
- goto err_free_dev;
+ return ret;
ret = drm_dev_register(dev, 0);
if (ret)
- goto err_free_dev;
+ return ret;
drm_client_setup(dev, NULL);
return ret;
-
-err_free_dev:
- drm_dev_put(dev);
- return ret;
}
static void bochs_pci_remove(struct pci_dev *pdev)
--
2.39.5
^ permalink raw reply related
* Re: [PATCH RFC v3 01/19] mm: thread user_addr through page allocator for cache-friendly zeroing
From: Michael S. Tsirkin @ 2026-04-24 12:33 UTC (permalink / raw)
To: Gregory Price
Cc: David Hildenbrand (Arm), linux-kernel, Andrew Morton,
Vlastimil Babka, Brendan Jackman, Michal Hocko,
Suren Baghdasaryan, Jason Wang, Andrea Arcangeli, linux-mm,
virtualization, Johannes Weiner, Zi Yan, Lorenzo Stoakes,
Liam R. Howlett, Mike Rapoport, Matthew Wilcox (Oracle),
Muchun Song, Oscar Salvador, Baolin Wang, Nico Pache,
Ryan Roberts, Dev Jain, Barry Song, Lance Yang, Matthew Brost,
Joshua Hahn, Rakie Kim, Byungchul Park, Ying Huang,
Alistair Popple, Hugh Dickins, Christoph Lameter, David Rientjes,
Roman Gushchin, Harry Yoo, Chris Li, Kairui Song, Kemeng Shi,
Nhat Pham, Baoquan He, linux-fsdevel
In-Reply-To: <aeticBh5XlAlD1vn@gourry-fedora-PF4VCD3F>
On Fri, Apr 24, 2026 at 08:30:40AM -0400, Gregory Price wrote:
> On Fri, Apr 24, 2026 at 09:28:36AM +0200, David Hildenbrand (Arm) wrote:
> > On 4/23/26 16:57, Gregory Price wrote:
> > > On Thu, Apr 23, 2026 at 04:13:50PM +0200, David Hildenbrand (Arm) wrote:
> > >> On 4/23/26 15:42, Gregory Price wrote:
> > >>
> > >> Maybe we could forward the vma+addr here and call a vma_alloc_froze_folio() if
> > >> we have a VMA+addr to have a clean interface.
> > >>
> > >> But really, that hugetlb code is rather messy. I'd vote for leaving hugetlb
> > >> alone on a v1, and focusing on non-hugetlb first.
> > >>
> > >
> > > If we're ok increasing the buddy surface this way, then I'd vote for
> > > only updating the exact interfaces that MST needs to update for his use
> > > case in a base set of patches, and then have each additional updated
> > > location (or logical set of locations) updated in follow-ups.
> > >
> > > My initial go around with this - the patch was hard to read at best.
> > >
> > > But I also think we should also seriously consider not increasing the
> > > surface of the buddy.
> >
> > Exactly, that's why I am saying that vma_alloc_folio() is the only external
> > interface people should be using with a user address. all other _noprof helpers
> > are supposed to be internal.
> >
> > For hugetlb, we might need another interface for frozen folios later, which is
> > why I suggest to defer that.
> >
>
> Yeah I follow now - we're of the same mind on all this then.
>
> ~Gregory
OK v4 incoming, thanks.
^ permalink raw reply
* Re: [PATCH RFC v3 01/19] mm: thread user_addr through page allocator for cache-friendly zeroing
From: Gregory Price @ 2026-04-24 12:30 UTC (permalink / raw)
To: David Hildenbrand (Arm)
Cc: Michael S. Tsirkin, linux-kernel, Andrew Morton, Vlastimil Babka,
Brendan Jackman, Michal Hocko, Suren Baghdasaryan, Jason Wang,
Andrea Arcangeli, linux-mm, virtualization, Johannes Weiner,
Zi Yan, Lorenzo Stoakes, Liam R. Howlett, Mike Rapoport,
Matthew Wilcox (Oracle), Muchun Song, Oscar Salvador, Baolin Wang,
Nico Pache, Ryan Roberts, Dev Jain, Barry Song, Lance Yang,
Matthew Brost, Joshua Hahn, Rakie Kim, Byungchul Park, Ying Huang,
Alistair Popple, Hugh Dickins, Christoph Lameter, David Rientjes,
Roman Gushchin, Harry Yoo, Chris Li, Kairui Song, Kemeng Shi,
Nhat Pham, Baoquan He, linux-fsdevel
In-Reply-To: <016cc5e5-044c-46c6-a668-200f90a64d85@kernel.org>
On Fri, Apr 24, 2026 at 09:28:36AM +0200, David Hildenbrand (Arm) wrote:
> On 4/23/26 16:57, Gregory Price wrote:
> > On Thu, Apr 23, 2026 at 04:13:50PM +0200, David Hildenbrand (Arm) wrote:
> >> On 4/23/26 15:42, Gregory Price wrote:
> >>
> >> Maybe we could forward the vma+addr here and call a vma_alloc_froze_folio() if
> >> we have a VMA+addr to have a clean interface.
> >>
> >> But really, that hugetlb code is rather messy. I'd vote for leaving hugetlb
> >> alone on a v1, and focusing on non-hugetlb first.
> >>
> >
> > If we're ok increasing the buddy surface this way, then I'd vote for
> > only updating the exact interfaces that MST needs to update for his use
> > case in a base set of patches, and then have each additional updated
> > location (or logical set of locations) updated in follow-ups.
> >
> > My initial go around with this - the patch was hard to read at best.
> >
> > But I also think we should also seriously consider not increasing the
> > surface of the buddy.
>
> Exactly, that's why I am saying that vma_alloc_folio() is the only external
> interface people should be using with a user address. all other _noprof helpers
> are supposed to be internal.
>
> For hugetlb, we might need another interface for frozen folios later, which is
> why I suggest to defer that.
>
Yeah I follow now - we're of the same mind on all this then.
~Gregory
^ permalink raw reply
* [RFC PATCH v5 4/4] virtio-mmio: wire up noirq system sleep PM callbacks
From: Sungho Bae @ 2026-04-24 12:29 UTC (permalink / raw)
To: mst, jasowang
Cc: xuanzhuo, eperezma, virtualization, linux-kernel, Sungho Bae
In-Reply-To: <20260424122954.273-1-baver.bae@gmail.com>
From: Sungho Bae <baver.bae@lge.com>
Add noirq system-sleep PM support to the virtio-mmio transport.
This change wires noirq freeze/restore callbacks into virtio-mmio and
hooks queue reset/reactivation into the transport config ops so virtqueues
can be reinitialized and reused across suspend/resume.
For legacy (v1) devices, keep GUEST_PAGE_SIZE programming aligned with the
noirq restore path while avoiding duplicate programming in normal restore.
This enables virtio-mmio based devices to participate safely in the noirq
PM phase, which is required for early-restore users.
Signed-off-by: Sungho Bae <baver.bae@lge.com>
---
drivers/virtio/virtio_mmio.c | 134 ++++++++++++++++++++++++-----------
1 file changed, 94 insertions(+), 40 deletions(-)
diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
index 595c2274fbb5..1cd262f9f8b6 100644
--- a/drivers/virtio/virtio_mmio.c
+++ b/drivers/virtio/virtio_mmio.c
@@ -336,6 +336,75 @@ static void vm_del_vqs(struct virtio_device *vdev)
free_irq(platform_get_irq(vm_dev->pdev, 0), vm_dev);
}
+static int vm_active_vq(struct virtio_device *vdev, struct virtqueue *vq)
+{
+ struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vdev);
+ int q_num = virtqueue_get_vring_size(vq);
+
+ writel(q_num, vm_dev->base + VIRTIO_MMIO_QUEUE_NUM);
+ if (vm_dev->version == 1) {
+ u64 q_pfn = virtqueue_get_desc_addr(vq) >> PAGE_SHIFT;
+
+ /*
+ * virtio-mmio v1 uses a 32bit QUEUE PFN. If we have something
+ * that doesn't fit in 32bit, fail the setup rather than
+ * pretending to be successful.
+ */
+ if (q_pfn >> 32) {
+ dev_err(&vdev->dev,
+ "platform bug: legacy virtio-mmio must not be used with RAM above 0x%llxGB\n",
+ 0x1ULL << (32 + PAGE_SHIFT - 30));
+ return -E2BIG;
+ }
+
+ writel(PAGE_SIZE, vm_dev->base + VIRTIO_MMIO_QUEUE_ALIGN);
+ writel(q_pfn, vm_dev->base + VIRTIO_MMIO_QUEUE_PFN);
+ } else {
+ u64 addr;
+
+ addr = virtqueue_get_desc_addr(vq);
+ writel((u32)addr, vm_dev->base + VIRTIO_MMIO_QUEUE_DESC_LOW);
+ writel((u32)(addr >> 32),
+ vm_dev->base + VIRTIO_MMIO_QUEUE_DESC_HIGH);
+
+ addr = virtqueue_get_avail_addr(vq);
+ writel((u32)addr, vm_dev->base + VIRTIO_MMIO_QUEUE_AVAIL_LOW);
+ writel((u32)(addr >> 32),
+ vm_dev->base + VIRTIO_MMIO_QUEUE_AVAIL_HIGH);
+
+ addr = virtqueue_get_used_addr(vq);
+ writel((u32)addr, vm_dev->base + VIRTIO_MMIO_QUEUE_USED_LOW);
+ writel((u32)(addr >> 32),
+ vm_dev->base + VIRTIO_MMIO_QUEUE_USED_HIGH);
+
+ writel(1, vm_dev->base + VIRTIO_MMIO_QUEUE_READY);
+ }
+
+ return 0;
+}
+
+static int vm_reset_vqs(struct virtio_device *vdev)
+{
+ struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vdev);
+ struct virtqueue *vq;
+ int err;
+
+ virtio_device_for_each_vq(vdev, vq) {
+ /* Re-initialize vring state */
+ virtqueue_reinit_vring(vq);
+
+ /* Select the queue we're interested in */
+ writel(vq->index, vm_dev->base + VIRTIO_MMIO_QUEUE_SEL);
+
+ /* Activate the queue */
+ err = vm_active_vq(vdev, vq);
+ if (err < 0)
+ return err;
+ }
+
+ return 0;
+}
+
static void vm_synchronize_cbs(struct virtio_device *vdev)
{
struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vdev);
@@ -388,45 +457,9 @@ static struct virtqueue *vm_setup_vq(struct virtio_device *vdev, unsigned int in
vq->num_max = num;
/* Activate the queue */
- writel(virtqueue_get_vring_size(vq), vm_dev->base + VIRTIO_MMIO_QUEUE_NUM);
- if (vm_dev->version == 1) {
- u64 q_pfn = virtqueue_get_desc_addr(vq) >> PAGE_SHIFT;
-
- /*
- * virtio-mmio v1 uses a 32bit QUEUE PFN. If we have something
- * that doesn't fit in 32bit, fail the setup rather than
- * pretending to be successful.
- */
- if (q_pfn >> 32) {
- dev_err(&vdev->dev,
- "platform bug: legacy virtio-mmio must not be used with RAM above 0x%llxGB\n",
- 0x1ULL << (32 + PAGE_SHIFT - 30));
- err = -E2BIG;
- goto error_bad_pfn;
- }
-
- writel(PAGE_SIZE, vm_dev->base + VIRTIO_MMIO_QUEUE_ALIGN);
- writel(q_pfn, vm_dev->base + VIRTIO_MMIO_QUEUE_PFN);
- } else {
- u64 addr;
-
- addr = virtqueue_get_desc_addr(vq);
- writel((u32)addr, vm_dev->base + VIRTIO_MMIO_QUEUE_DESC_LOW);
- writel((u32)(addr >> 32),
- vm_dev->base + VIRTIO_MMIO_QUEUE_DESC_HIGH);
-
- addr = virtqueue_get_avail_addr(vq);
- writel((u32)addr, vm_dev->base + VIRTIO_MMIO_QUEUE_AVAIL_LOW);
- writel((u32)(addr >> 32),
- vm_dev->base + VIRTIO_MMIO_QUEUE_AVAIL_HIGH);
-
- addr = virtqueue_get_used_addr(vq);
- writel((u32)addr, vm_dev->base + VIRTIO_MMIO_QUEUE_USED_LOW);
- writel((u32)(addr >> 32),
- vm_dev->base + VIRTIO_MMIO_QUEUE_USED_HIGH);
-
- writel(1, vm_dev->base + VIRTIO_MMIO_QUEUE_READY);
- }
+ err = vm_active_vq(vdev, vq);
+ if (err < 0)
+ goto error_bad_pfn;
return vq;
@@ -528,11 +561,13 @@ static const struct virtio_config_ops virtio_mmio_config_ops = {
.reset = vm_reset,
.find_vqs = vm_find_vqs,
.del_vqs = vm_del_vqs,
+ .reset_vqs = vm_reset_vqs,
.get_features = vm_get_features,
.finalize_features = vm_finalize_features,
.bus_name = vm_bus_name,
.get_shm_region = vm_get_shm_region,
.synchronize_cbs = vm_synchronize_cbs,
+ .noirq_safe = true,
};
#ifdef CONFIG_PM_SLEEP
@@ -547,14 +582,33 @@ static int virtio_mmio_restore(struct device *dev)
{
struct virtio_mmio_device *vm_dev = dev_get_drvdata(dev);
- if (vm_dev->version == 1)
+ if (vm_dev->version == 1 && !vm_dev->vdev.noirq_restore_done)
writel(PAGE_SIZE, vm_dev->base + VIRTIO_MMIO_GUEST_PAGE_SIZE);
return virtio_device_restore(&vm_dev->vdev);
}
+static int virtio_mmio_freeze_noirq(struct device *dev)
+{
+ struct virtio_mmio_device *vm_dev = dev_get_drvdata(dev);
+
+ return virtio_device_freeze_noirq(&vm_dev->vdev);
+}
+
+static int virtio_mmio_restore_noirq(struct device *dev)
+{
+ struct virtio_mmio_device *vm_dev = dev_get_drvdata(dev);
+
+ if (vm_dev->version == 1)
+ writel(PAGE_SIZE, vm_dev->base + VIRTIO_MMIO_GUEST_PAGE_SIZE);
+
+ return virtio_device_restore_noirq(&vm_dev->vdev);
+}
+
static const struct dev_pm_ops virtio_mmio_pm_ops = {
SET_SYSTEM_SLEEP_PM_OPS(virtio_mmio_freeze, virtio_mmio_restore)
+ SET_NOIRQ_SYSTEM_SLEEP_PM_OPS(virtio_mmio_freeze_noirq,
+ virtio_mmio_restore_noirq)
};
#endif
--
2.43.0
^ permalink raw reply related
* [RFC PATCH v5 3/4] virtio: add noirq system sleep PM infrastructure
From: Sungho Bae @ 2026-04-24 12:29 UTC (permalink / raw)
To: mst, jasowang
Cc: xuanzhuo, eperezma, virtualization, linux-kernel, Sungho Bae
In-Reply-To: <20260424122954.273-1-baver.bae@gmail.com>
From: Sungho Bae <baver.bae@lge.com>
Some virtio-mmio devices, such as virtio-clock or virtio-regulator,
must become operational before the regular PM restore callback runs
because other devices may depend on them.
Add the core infrastructure needed to support noirq system-sleep PM
callbacks for virtio transports:
- virtio_add_status_noirq(): status helper without might_sleep().
- virtio_features_ok_noirq(): feature negotiation without might_sleep().
- virtio_reset_device_noirq(): device reset that skips
virtio_synchronize_cbs() (IRQ handlers are already quiesced in the
noirq phase).
- virtio_device_reinit_noirq(): full noirq bring-up sequence using the
above helpers.
- virtio_config_core_enable_noirq(): config enable with irqsave
locking.
- virtio_device_ready_noirq(): marks DRIVER_OK without
virtio_synchronize_cbs().
Not all transports can safely call reset, get_status, set_status, or
finalize_features during the noirq phase: transports like virtio-ccw
issue channel commands and wait for a completion interrupt, which will
never be delivered because device interrupts are masked at the interrupt
controller during noirq suspend/resume. To address this, introduce a
boolean field noirq_safe in struct virtio_config_ops. Transports that
implement the above operations via simple MMIO reads/writes (e.g.
virtio-mmio) set this flag; all others leave it at the default false.
The noirq helpers assert noirq_safe via WARN_ON at runtime.
virtio_device_freeze_noirq() enforces the contract at freeze time,
returning -EOPNOTSUPP early if the driver provides restore_noirq but
the transport does not meet the requirements, to prevent a deadlock on
resume. virtio_device_restore_noirq() performs a second check as a
safety net in case freeze_noirq was not called.
Add freeze_noirq/restore_noirq callbacks to struct virtio_driver and
provide matching helper wrappers in the virtio core:
- virtio_device_freeze_noirq(): validates noirq_safe and reset_vqs
requirements, then forwards to drv->freeze_noirq().
- virtio_device_restore_noirq(): guards against unsafe transports,
runs the noirq bring-up sequence, resets existing vrings via the
new config_ops->reset_vqs() hook, then calls drv->restore_noirq().
Modify virtio_device_restore() so that when a driver provides
restore_noirq, the normal-phase restore skips the re-initialization
that was already done in the noirq phase.
Signed-off-by: Sungho Bae <baver.bae@lge.com>
---
drivers/virtio/virtio.c | 239 +++++++++++++++++++++++++++++++++-
include/linux/virtio.h | 17 +++
include/linux/virtio_config.h | 39 ++++++
3 files changed, 289 insertions(+), 6 deletions(-)
diff --git a/drivers/virtio/virtio.c b/drivers/virtio/virtio.c
index 98f1875f8df1..b1d2cba7d59a 100644
--- a/drivers/virtio/virtio.c
+++ b/drivers/virtio/virtio.c
@@ -193,6 +193,17 @@ static void virtio_config_core_enable(struct virtio_device *dev)
spin_unlock_irq(&dev->config_lock);
}
+static void virtio_config_core_enable_noirq(struct virtio_device *dev)
+{
+ unsigned long flags;
+
+ spin_lock_irqsave(&dev->config_lock, flags);
+ dev->config_core_enabled = true;
+ if (dev->config_change_pending)
+ __virtio_config_changed(dev);
+ spin_unlock_irqrestore(&dev->config_lock, flags);
+}
+
void virtio_add_status(struct virtio_device *dev, unsigned int status)
{
might_sleep();
@@ -200,6 +211,21 @@ void virtio_add_status(struct virtio_device *dev, unsigned int status)
}
EXPORT_SYMBOL_GPL(virtio_add_status);
+/*
+ * Same as virtio_add_status() but without the might_sleep() assertion,
+ * so it is safe to call from noirq context.
+ *
+ * Requires the transport to have set config_ops->noirq_safe, which declares
+ * that reset, get_status, and set_status do not wait for a completion
+ * interrupt and are therefore safe during the noirq PM phase.
+ */
+void virtio_add_status_noirq(struct virtio_device *dev, unsigned int status)
+{
+ WARN_ON(!dev->config->noirq_safe);
+ dev->config->set_status(dev, dev->config->get_status(dev) | status);
+}
+EXPORT_SYMBOL_GPL(virtio_add_status_noirq);
+
/* Do some validation, then set FEATURES_OK */
static int virtio_features_ok(struct virtio_device *dev)
{
@@ -234,6 +260,38 @@ static int virtio_features_ok(struct virtio_device *dev)
return 0;
}
+/* noirq-safe variant: no might_sleep(), uses virtio_add_status_noirq() */
+static int virtio_features_ok_noirq(struct virtio_device *dev)
+{
+ unsigned int status;
+
+ if (virtio_check_mem_acc_cb(dev)) {
+ if (!virtio_has_feature(dev, VIRTIO_F_VERSION_1)) {
+ dev_warn(&dev->dev,
+ "device must provide VIRTIO_F_VERSION_1\n");
+ return -ENODEV;
+ }
+
+ if (!virtio_has_feature(dev, VIRTIO_F_ACCESS_PLATFORM)) {
+ dev_warn(&dev->dev,
+ "device must provide VIRTIO_F_ACCESS_PLATFORM\n");
+ return -ENODEV;
+ }
+ }
+
+ if (!virtio_has_feature(dev, VIRTIO_F_VERSION_1))
+ return 0;
+
+ virtio_add_status_noirq(dev, VIRTIO_CONFIG_S_FEATURES_OK);
+ status = dev->config->get_status(dev);
+ if (!(status & VIRTIO_CONFIG_S_FEATURES_OK)) {
+ dev_err(&dev->dev, "virtio: device refuses features: %x\n",
+ status);
+ return -ENODEV;
+ }
+ return 0;
+}
+
/**
* virtio_reset_device - quiesce device for removal
* @dev: the device to reset
@@ -267,6 +325,28 @@ void virtio_reset_device(struct virtio_device *dev)
}
EXPORT_SYMBOL_GPL(virtio_reset_device);
+/**
+ * virtio_reset_device_noirq - noirq-safe variant of virtio_reset_device()
+ * @dev: the device to reset
+ *
+ * Requires the transport to have set config_ops->noirq_safe.
+ */
+void virtio_reset_device_noirq(struct virtio_device *dev)
+{
+ WARN_ON(!dev->config->noirq_safe);
+
+#ifdef CONFIG_VIRTIO_HARDEN_NOTIFICATION
+ /*
+ * The noirq stage runs with device IRQ handlers disabled, so
+ * virtio_synchronize_cbs() must not be called here.
+ */
+ virtio_break_device(dev);
+#endif
+
+ dev->config->reset(dev);
+}
+EXPORT_SYMBOL_GPL(virtio_reset_device_noirq);
+
static int virtio_dev_probe(struct device *_d)
{
int err, i;
@@ -539,6 +619,7 @@ int register_virtio_device(struct virtio_device *dev)
dev->config_driver_disabled = false;
dev->config_core_enabled = false;
dev->config_change_pending = false;
+ dev->noirq_restore_done = false;
INIT_LIST_HEAD(&dev->vqs);
spin_lock_init(&dev->vqs_list_lock);
@@ -618,6 +699,47 @@ static int virtio_device_reinit(struct virtio_device *dev)
return virtio_features_ok(dev);
}
+/*
+ * noirq-safe variant of virtio_device_reinit().
+ *
+ * Requires the transport to declare config_ops->noirq_safe, which means
+ * reset, get_status, set_status, and finalize_features are safe to call
+ * during the noirq PM phase.
+ */
+static int virtio_device_reinit_noirq(struct virtio_device *dev)
+{
+ struct virtio_driver *drv = drv_to_virtio(dev->dev.driver);
+ int ret;
+
+ /*
+ * We always start by resetting the device, in case a previous
+ * driver messed it up.
+ */
+ virtio_reset_device_noirq(dev);
+
+ /* Acknowledge that we've seen the device. */
+ virtio_add_status_noirq(dev, VIRTIO_CONFIG_S_ACKNOWLEDGE);
+
+ /*
+ * Maybe driver failed before freeze.
+ * Restore the failed status, for debugging.
+ */
+ if (dev->failed)
+ virtio_add_status_noirq(dev, VIRTIO_CONFIG_S_FAILED);
+
+ if (!drv)
+ return 0;
+
+ /* We have a driver! */
+ virtio_add_status_noirq(dev, VIRTIO_CONFIG_S_DRIVER);
+
+ ret = dev->config->finalize_features(dev);
+ if (ret)
+ return ret;
+
+ return virtio_features_ok_noirq(dev);
+}
+
#ifdef CONFIG_PM_SLEEP
int virtio_device_freeze(struct virtio_device *dev)
{
@@ -627,6 +749,7 @@ int virtio_device_freeze(struct virtio_device *dev)
virtio_config_core_disable(dev);
dev->failed = dev->config->get_status(dev) & VIRTIO_CONFIG_S_FAILED;
+ dev->noirq_restore_done = false;
if (drv && drv->freeze) {
ret = drv->freeze(dev);
@@ -645,12 +768,22 @@ int virtio_device_restore(struct virtio_device *dev)
struct virtio_driver *drv = drv_to_virtio(dev->dev.driver);
int ret;
- ret = virtio_device_reinit(dev);
- if (ret)
- goto err;
-
- if (!drv)
- return 0;
+ /*
+ * If this device was already brought up in the noirq phase,
+ * skip the re-initialization here.
+ *
+ * Note: this normal restore path does not call reset_vqs().
+ * Drivers that implement restore_noirq and preserve virtqueues
+ * must make queue/device state consistent in ->restore() when
+ * noirq restore did not complete.
+ */
+ if (!drv || !dev->noirq_restore_done) {
+ ret = virtio_device_reinit(dev);
+ if (ret)
+ goto err;
+ if (!drv)
+ return 0;
+ }
if (drv->restore) {
ret = drv->restore(dev);
@@ -671,6 +804,100 @@ int virtio_device_restore(struct virtio_device *dev)
return ret;
}
EXPORT_SYMBOL_GPL(virtio_device_restore);
+
+int virtio_device_freeze_noirq(struct virtio_device *dev)
+{
+ struct virtio_driver *drv = drv_to_virtio(dev->dev.driver);
+
+ if (!drv)
+ return 0;
+
+ /*
+ * restore_noirq requires that the transport's config ops
+ * (reset, get_status, set_status) are safe to call during the noirq
+ * PM phase. Catch the mismatch early at freeze time so the PM core
+ * can abort cleanly rather than deadlocking on resume.
+ */
+ if (drv->restore_noirq && !dev->config->noirq_safe) {
+ dev_warn(&dev->dev,
+ "transport does not support noirq PM\n");
+ return -EOPNOTSUPP;
+ }
+
+ /*
+ * If the driver provides restore_noirq and has active vqs,
+ * the transport must support reset_vqs to restore them.
+ * Fail here so the PM core can abort the transition gracefully,
+ * rather than hitting -EOPNOTSUPP on resume.
+ */
+ if (drv->restore_noirq && !list_empty(&dev->vqs) &&
+ !dev->config->reset_vqs) {
+ dev_warn(&dev->dev,
+ "transport does not support noirq PM restore with active vqs (missing reset_vqs)\n");
+ return -EOPNOTSUPP;
+ }
+
+ if (drv->freeze_noirq)
+ return drv->freeze_noirq(dev);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(virtio_device_freeze_noirq);
+
+int virtio_device_restore_noirq(struct virtio_device *dev)
+{
+ struct virtio_driver *drv = drv_to_virtio(dev->dev.driver);
+ int ret;
+
+ if (!drv || !drv->restore_noirq)
+ return 0;
+
+ /*
+ * All transport ops called below (reset, get_status, set_status) must
+ * be noirq-safe. Return early if not - this should normally have
+ * been caught at freeze_noirq time.
+ */
+ if (!dev->config->noirq_safe) {
+ dev_warn(&dev->dev,
+ "transport does not support noirq PM; skipping restore\n");
+ return -EOPNOTSUPP;
+ }
+
+ ret = virtio_device_reinit_noirq(dev);
+ if (ret)
+ goto err;
+
+ if (!list_empty(&dev->vqs)) {
+ if (!dev->config->reset_vqs) {
+ ret = -EOPNOTSUPP;
+ goto err;
+ }
+
+ ret = dev->config->reset_vqs(dev);
+ if (ret)
+ goto err;
+ }
+
+ ret = drv->restore_noirq(dev);
+ if (ret)
+ goto err;
+
+ /* Mark that noirq restore has completed. */
+ dev->noirq_restore_done = true;
+
+ /* If restore_noirq set DRIVER_OK, enable config now. */
+ if (dev->config->get_status(dev) & VIRTIO_CONFIG_S_DRIVER_OK)
+ virtio_config_core_enable_noirq(dev);
+
+ return 0;
+
+err:
+ /* Record that noirq restore failed so FAILED status persists across reset/reinit. */
+ dev->failed = true;
+ virtio_add_status_noirq(dev, VIRTIO_CONFIG_S_FAILED);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(virtio_device_restore_noirq);
#endif
int virtio_device_reset_prepare(struct virtio_device *dev)
diff --git a/include/linux/virtio.h b/include/linux/virtio.h
index 3bbc4cb6a672..c4f85f1ebffa 100644
--- a/include/linux/virtio.h
+++ b/include/linux/virtio.h
@@ -151,6 +151,7 @@ struct virtio_admin_cmd {
* @config_driver_disabled: configuration change reporting disabled by
* a driver
* @config_change_pending: configuration change reported while disabled
+ * @noirq_restore_done: set if the noirq restore phase completed successfully
* @config_lock: protects configuration change reporting
* @vqs_list_lock: protects @vqs.
* @dev: underlying device.
@@ -171,6 +172,7 @@ struct virtio_device {
bool config_core_enabled;
bool config_driver_disabled;
bool config_change_pending;
+ bool noirq_restore_done;
spinlock_t config_lock;
spinlock_t vqs_list_lock;
struct device dev;
@@ -209,8 +211,12 @@ void virtio_config_driver_enable(struct virtio_device *dev);
#ifdef CONFIG_PM_SLEEP
int virtio_device_freeze(struct virtio_device *dev);
int virtio_device_restore(struct virtio_device *dev);
+int virtio_device_freeze_noirq(struct virtio_device *dev);
+int virtio_device_restore_noirq(struct virtio_device *dev);
#endif
void virtio_reset_device(struct virtio_device *dev);
+void virtio_reset_device_noirq(struct virtio_device *dev);
+void virtio_add_status_noirq(struct virtio_device *dev, unsigned int status);
int virtio_device_reset_prepare(struct virtio_device *dev);
int virtio_device_reset_done(struct virtio_device *dev);
@@ -237,6 +243,15 @@ size_t virtio_max_dma_size(const struct virtio_device *vdev);
* changes; may be called in interrupt context.
* @freeze: optional function to call during suspend/hibernation.
* @restore: optional function to call on resume.
+ * If noirq resume was skipped or failed, core may have reset the device
+ * before calling this callback. Drivers that preserve virtqueues for
+ * @restore_noirq must make queue/device state consistent here before
+ * returning success.
+ * @freeze_noirq: optional function to call during noirq suspend/hibernation.
+ * @restore_noirq: optional function to call on noirq resume.
+ * If this callback fails, PM core may fall back to @restore. The fallback
+ * normal resume path does not implicitly perform transport queue reset for
+ * preserved virtqueues; drivers must handle that in @restore if needed.
* @reset_prepare: optional function to call when a transport specific reset
* occurs.
* @reset_done: optional function to call after transport specific reset
@@ -258,6 +273,8 @@ struct virtio_driver {
void (*config_changed)(struct virtio_device *dev);
int (*freeze)(struct virtio_device *dev);
int (*restore)(struct virtio_device *dev);
+ int (*freeze_noirq)(struct virtio_device *dev);
+ int (*restore_noirq)(struct virtio_device *dev);
int (*reset_prepare)(struct virtio_device *dev);
int (*reset_done)(struct virtio_device *dev);
void (*shutdown)(struct virtio_device *dev);
diff --git a/include/linux/virtio_config.h b/include/linux/virtio_config.h
index 69f84ea85d71..81af2ad6a7c3 100644
--- a/include/linux/virtio_config.h
+++ b/include/linux/virtio_config.h
@@ -70,6 +70,9 @@ struct virtqueue_info {
* vqs_info: array of virtqueue info structures
* Returns 0 on success or error status
* @del_vqs: free virtqueues found by find_vqs().
+ * @reset_vqs: reinitialize existing virtqueues without allocating or
+ * freeing them (optional). Used during noirq restore.
+ * Returns 0 on success or error status.
* @synchronize_cbs: synchronize with the virtqueue callbacks (optional)
* The function guarantees that all memory operations on the
* queue before it are visible to the vring_interrupt() that is
@@ -108,6 +111,14 @@ struct virtqueue_info {
* Returns 0 on success or error status
* If disable_vq_and_reset is set, then enable_vq_after_reset must also be
* set.
+ * @noirq_safe: set to true if @reset, @get_status, @set_status, and
+ * @finalize_features are safe to call during the noirq phase of system
+ * suspend/resume. Transports that implement these operations via simple
+ * MMIO reads/writes (e.g. virtio-mmio) can set this flag. Transports
+ * that issue channel commands and wait for a completion interrupt (e.g.
+ * virtio-ccw) must NOT set it, because device interrupts are masked at
+ * the interrupt controller during the noirq phase, which would cause the
+ * wait to hang.
*/
struct virtio_config_ops {
void (*get)(struct virtio_device *vdev, unsigned offset,
@@ -123,6 +134,7 @@ struct virtio_config_ops {
struct virtqueue_info vqs_info[],
struct irq_affinity *desc);
void (*del_vqs)(struct virtio_device *);
+ int (*reset_vqs)(struct virtio_device *vdev);
void (*synchronize_cbs)(struct virtio_device *);
u64 (*get_features)(struct virtio_device *vdev);
void (*get_extended_features)(struct virtio_device *vdev,
@@ -137,6 +149,7 @@ struct virtio_config_ops {
struct virtio_shm_region *region, u8 id);
int (*disable_vq_and_reset)(struct virtqueue *vq);
int (*enable_vq_after_reset)(struct virtqueue *vq);
+ bool noirq_safe;
};
/**
@@ -371,6 +384,32 @@ void virtio_device_ready(struct virtio_device *dev)
dev->config->set_status(dev, status | VIRTIO_CONFIG_S_DRIVER_OK);
}
+/**
+ * virtio_device_ready_noirq - noirq-safe variant of virtio_device_ready()
+ * @dev: the virtio device
+ *
+ * Requires the transport to have set config_ops->noirq_safe, which declares
+ * that get_status and set_status do not wait for a completion interrupt.
+ */
+static inline
+void virtio_device_ready_noirq(struct virtio_device *dev)
+{
+ unsigned int status = dev->config->get_status(dev);
+
+ WARN_ON(!dev->config->noirq_safe);
+ WARN_ON(status & VIRTIO_CONFIG_S_DRIVER_OK);
+
+#ifdef CONFIG_VIRTIO_HARDEN_NOTIFICATION
+ /*
+ * The noirq stage runs with device IRQ handlers disabled, so
+ * virtio_synchronize_cbs() must not be called here.
+ */
+ __virtio_unbreak_device(dev);
+#endif
+
+ dev->config->set_status(dev, status | VIRTIO_CONFIG_S_DRIVER_OK);
+}
+
static inline
const char *virtio_bus_name(struct virtio_device *vdev)
{
--
2.43.0
^ permalink raw reply related
* [RFC PATCH v5 2/4] virtio_ring: export virtqueue_reinit_vring() for noirq restore
From: Sungho Bae @ 2026-04-24 12:29 UTC (permalink / raw)
To: mst, jasowang
Cc: xuanzhuo, eperezma, virtualization, linux-kernel, Sungho Bae
In-Reply-To: <20260424122954.273-1-baver.bae@gmail.com>
From: Sungho Bae <baver.bae@lge.com>
After a device reset in noirq context the existing vrings must be
re-initialized without any memory allocation, because GFP_KERNEL is
not available.
The internal helpers virtqueue_reset_split() and
virtqueue_reset_packed() already reset vring indices and descriptor
state in place. Add a thin exported wrapper, virtqueue_reinit_vring(),
that dispatches to the appropriate helper based on the ring layout.
This will be used by a subsequent patch that adds noirq system-sleep
PM callbacks for virtio-mmio.
Signed-off-by: Sungho Bae <baver.bae@lge.com>
---
drivers/virtio/virtio_ring.c | 51 ++++++++++++++++++++++++++++++++++++
include/linux/virtio_ring.h | 3 +++
2 files changed, 54 insertions(+)
diff --git a/drivers/virtio/virtio_ring.c b/drivers/virtio/virtio_ring.c
index fbca7ce1c6bf..6631c30cb706 100644
--- a/drivers/virtio/virtio_ring.c
+++ b/drivers/virtio/virtio_ring.c
@@ -506,6 +506,15 @@ static void virtqueue_init(struct vring_virtqueue *vq, u32 num)
vq->event_triggered = false;
vq->num_added = 0;
+ /*
+ * Keep IN_ORDER state aligned with a freshly initialized/reset queue.
+ * For packed IN_ORDER, free_head is unused but harmlessly reset.
+ */
+ if (virtqueue_is_in_order(vq)) {
+ vq->free_head = 0;
+ vq->batch_last.id = UINT_MAX;
+ }
+
#ifdef DEBUG
vq->in_use = false;
vq->last_add_time_valid = false;
@@ -3936,5 +3945,47 @@ void virtqueue_map_sync_single_range_for_device(const struct virtqueue *_vq,
}
EXPORT_SYMBOL_GPL(virtqueue_map_sync_single_range_for_device);
+/**
+ * virtqueue_reinit_vring - reinitialize vring state without reallocation
+ * @_vq: the virtqueue
+ *
+ * Reset the avail/used indices and descriptor state of an existing
+ * virtqueue so it can be reused after a device reset. No memory is
+ * allocated or freed, making this safe for use in noirq context.
+ *
+ * Preconditions for callers:
+ * 1) The vq must be fully quiesced (no concurrent add/get/kick/IRQ callback).
+ * 2) Transport/device side must already have stopped/reset this queue.
+ * 3) All in-flight buffers must already be completed or detached.
+ *
+ * If called with outstanding descriptors, free-list state can be corrupted:
+ * num_free is restored to full capacity while desc_extra next-chain/free_head
+ * may still represent a partially consumed list.
+ */
+void virtqueue_reinit_vring(struct virtqueue *_vq)
+{
+ struct vring_virtqueue *vq = to_vvq(_vq);
+ unsigned int num = virtqueue_is_packed(vq) ?
+ vq->packed.vring.num : vq->split.vring.num;
+
+ /* All in-flight descriptors must be completed or detached */
+ WARN_ON(vq->vq.num_free != num);
+
+ if (virtqueue_is_packed(vq)) {
+ virtqueue_reset_packed(vq);
+ } else {
+ /*
+ * Split queue shadow index should match the visible avail
+ * index when the queue is fully quiesced.
+ */
+ WARN_ON(vq->split.avail_idx_shadow !=
+ virtio16_to_cpu(vq->vq.vdev,
+ vq->split.vring.avail->idx));
+
+ virtqueue_reset_split(vq);
+ }
+}
+EXPORT_SYMBOL_GPL(virtqueue_reinit_vring);
+
MODULE_DESCRIPTION("Virtio ring implementation");
MODULE_LICENSE("GPL");
diff --git a/include/linux/virtio_ring.h b/include/linux/virtio_ring.h
index c97a12c1cda3..26c7c9d0a151 100644
--- a/include/linux/virtio_ring.h
+++ b/include/linux/virtio_ring.h
@@ -118,6 +118,9 @@ void vring_del_virtqueue(struct virtqueue *vq);
/* Filter out transport-specific feature bits. */
void vring_transport_features(struct virtio_device *vdev);
+/* Reinitialize a virtqueue without reallocation (safe in noirq context) */
+void virtqueue_reinit_vring(struct virtqueue *_vq);
+
irqreturn_t vring_interrupt(int irq, void *_vq);
u32 vring_notification_data(struct virtqueue *_vq);
--
2.43.0
^ permalink raw reply related
* [RFC PATCH v5 1/4] virtio: separate PM restore and reset_done paths
From: Sungho Bae @ 2026-04-24 12:29 UTC (permalink / raw)
To: mst, jasowang
Cc: xuanzhuo, eperezma, virtualization, linux-kernel, Sungho Bae
In-Reply-To: <20260424122954.273-1-baver.bae@gmail.com>
From: Sungho Bae <baver.bae@lge.com>
Refactor virtio_device_restore_priv() by extracting the common device
re-initialization sequence into virtio_device_reinit(). This helper
performs the full bring-up sequence: reset, status acknowledgment,
feature finalization, and feature negotiation.
virtio_device_restore() and virtio_device_reset_done() now each call
virtio_device_reinit() directly instead of going through a boolean-
dispatched wrapper. This makes each path independently readable and
extensible without further complicating the dispatch logic.
A follow-up series will add noirq PM callbacks that only affect the
restore path; having the two paths separated avoids adding more
conditionals to a shared function.
No functional change.
Signed-off-by: Sungho Bae <baver.bae@lge.com>
---
drivers/virtio/virtio.c | 81 +++++++++++++++++++++++++----------------
1 file changed, 50 insertions(+), 31 deletions(-)
diff --git a/drivers/virtio/virtio.c b/drivers/virtio/virtio.c
index 5bdc6b82b30b..98f1875f8df1 100644
--- a/drivers/virtio/virtio.c
+++ b/drivers/virtio/virtio.c
@@ -588,7 +588,7 @@ void unregister_virtio_device(struct virtio_device *dev)
}
EXPORT_SYMBOL_GPL(unregister_virtio_device);
-static int virtio_device_restore_priv(struct virtio_device *dev, bool restore)
+static int virtio_device_reinit(struct virtio_device *dev)
{
struct virtio_driver *drv = drv_to_virtio(dev->dev.driver);
int ret;
@@ -613,35 +613,9 @@ static int virtio_device_restore_priv(struct virtio_device *dev, bool restore)
ret = dev->config->finalize_features(dev);
if (ret)
- goto err;
-
- ret = virtio_features_ok(dev);
- if (ret)
- goto err;
-
- if (restore) {
- if (drv->restore) {
- ret = drv->restore(dev);
- if (ret)
- goto err;
- }
- } else {
- ret = drv->reset_done(dev);
- if (ret)
- goto err;
- }
-
- /* If restore didn't do it, mark device DRIVER_OK ourselves. */
- if (!(dev->config->get_status(dev) & VIRTIO_CONFIG_S_DRIVER_OK))
- virtio_device_ready(dev);
-
- virtio_config_core_enable(dev);
-
- return 0;
+ return ret;
-err:
- virtio_add_status(dev, VIRTIO_CONFIG_S_FAILED);
- return ret;
+ return virtio_features_ok(dev);
}
#ifdef CONFIG_PM_SLEEP
@@ -668,7 +642,33 @@ EXPORT_SYMBOL_GPL(virtio_device_freeze);
int virtio_device_restore(struct virtio_device *dev)
{
- return virtio_device_restore_priv(dev, true);
+ struct virtio_driver *drv = drv_to_virtio(dev->dev.driver);
+ int ret;
+
+ ret = virtio_device_reinit(dev);
+ if (ret)
+ goto err;
+
+ if (!drv)
+ return 0;
+
+ if (drv->restore) {
+ ret = drv->restore(dev);
+ if (ret)
+ goto err;
+ }
+
+ /* If restore didn't do it, mark device DRIVER_OK ourselves. */
+ if (!(dev->config->get_status(dev) & VIRTIO_CONFIG_S_DRIVER_OK))
+ virtio_device_ready(dev);
+
+ virtio_config_core_enable(dev);
+
+ return 0;
+
+err:
+ virtio_add_status(dev, VIRTIO_CONFIG_S_FAILED);
+ return ret;
}
EXPORT_SYMBOL_GPL(virtio_device_restore);
#endif
@@ -698,11 +698,30 @@ EXPORT_SYMBOL_GPL(virtio_device_reset_prepare);
int virtio_device_reset_done(struct virtio_device *dev)
{
struct virtio_driver *drv = drv_to_virtio(dev->dev.driver);
+ int ret;
if (!drv || !drv->reset_done)
return -EOPNOTSUPP;
- return virtio_device_restore_priv(dev, false);
+ ret = virtio_device_reinit(dev);
+ if (ret)
+ goto err;
+
+ ret = drv->reset_done(dev);
+ if (ret)
+ goto err;
+
+ /* If reset_done didn't do it, mark device DRIVER_OK ourselves. */
+ if (!(dev->config->get_status(dev) & VIRTIO_CONFIG_S_DRIVER_OK))
+ virtio_device_ready(dev);
+
+ virtio_config_core_enable(dev);
+
+ return 0;
+
+err:
+ virtio_add_status(dev, VIRTIO_CONFIG_S_FAILED);
+ return ret;
}
EXPORT_SYMBOL_GPL(virtio_device_reset_done);
--
2.43.0
^ permalink raw reply related
* [RFC PATCH v5 0/4] virtio: add noirq system sleep PM callbacks for virtio-mmio
From: Sungho Bae @ 2026-04-24 12:29 UTC (permalink / raw)
To: mst, jasowang
Cc: xuanzhuo, eperezma, virtualization, linux-kernel, Sungho Bae
From: Sungho Bae <baver.bae@lge.com>
Hi all,
Some virtio-mmio based devices, such as virtio-clock or virtio-regulator,
must become operational before other devices have their regular PM restore
callbacks invoked, because those other devices depend on them.
Generally, PM framework provides the three phases (freeze, freeze_late,
freeze_noirq) for the system sleep sequence, and the corresponding resume
phases. But, virtio core only supports the normal freeze/restore phase,
so virtio drivers have no way to participate in the noirq phase, which runs
with IRQs disabled and is guaranteed to run before any normal-phase restore
callbacks.
This series adds the infrastructure and the virtio-mmio transport
wiring so that virtio drivers can implement freeze_noirq/restore_noirq
callbacks.
Design overview
===============
The noirq phase runs with device IRQ handlers disabled and must avoid
sleepable operations. The main constraints addressed are:
- might_sleep() in virtio_add_status() and virtio_features_ok().
- virtio_synchronize_cbs() in virtio_reset_device() (IRQs are already
quiesced).
- spin_lock_irq() in virtio_config_core_enable() (not safe to call
with interrupts already disabled).
- Memory allocation during vq setup (virtqueue_reinit_vring() reuses
existing buffers instead).
The series provides noirq-safe variants for each of these, plus a new
config_ops->reset_vqs() callback that lets the transport reprogram
queue registers without freeing/reallocating vring memory.
Not all transports can safely perform these operations in the noirq phase.
Transports like virtio-ccw issue channel commands and wait for a completion
interrupt, which will never arrive while device interrupts are masked at
the interrupt controller. A new boolean field config_ops->noirq_safe marks
transports that implement reset/status operations via simple MMIO
reads/writes and are therefore safe to use in noirq context. The noirq
helpers assert this flag at runtime, and virtio_device_freeze_noirq()
enforces it at freeze time, returning -EOPNOTSUPP early to prevent
a deadlock on resume.
When a driver implements restore_noirq, the device bring-up (reset ->
ACKNOWLEDGE -> DRIVER -> finalize_features -> FEATURES_OK) happens in
the noirq phase. The subsequent normal-phase virtio_device_restore()
detects this and skips the redundant re-initialization.
Patch breakdown
===============
Patch 1 is a preparatory refactoring with no functional change.
Patches 2-3 add the core infrastructure. Patch 4 wires it up for
virtio-mmio.
1. virtio: separate PM restore and reset_done paths
Splits virtio_device_restore_priv() into independent
virtio_device_restore() and virtio_device_reset_done() paths,
using a shared virtio_device_reinit() helper. This is a pure
refactoring to make the restore path independently extensible
without complicating the boolean dispatch.
2. virtio_ring: export virtqueue_reinit_vring() for noirq restore
Adds virtqueue_reinit_vring(), an exported wrapper that resets
vring indices and descriptor state in place without any memory
allocation, making it safe to call from noirq context. Also
resets IN_ORDER-specific state (free_head, batch_last.id) in
virtqueue_init() to keep the ring consistent after reinit, and
adds runtime WARN_ON checks for unexpected in-flight descriptor
state and split-ring index consistency.
3. virtio: add noirq system sleep PM infrastructure
Adds noirq-safe helpers (virtio_add_status_noirq,
virtio_features_ok_noirq, virtio_reset_device_noirq,
virtio_config_core_enable_noirq, virtio_device_ready_noirq) and
the freeze_noirq/restore_noirq driver callbacks plus the
config_ops->reset_vqs() transport hook. Introduces
config_ops->noirq_safe to mark transports whose reset/status
operations are safe in noirq context (e.g. simple MMIO), and
enforces early -EOPNOTSUPP in virtio_device_freeze_noirq() when
the transport does not meet noirq requirements. Modifies
virtio_device_restore() to skip bring-up when restore_noirq
already ran.
4. virtio-mmio: wire up noirq system sleep PM callbacks
Implements vm_reset_vqs() which iterates existing virtqueues,
reinitializes the vring state via virtqueue_reinit_vring(), and
reprograms the MMIO queue registers. Adds
virtio_mmio_freeze_noirq/virtio_mmio_restore_noirq and registers
them via SET_NOIRQ_SYSTEM_SLEEP_PM_OPS(). Sets .noirq_safe = true
in virtio_mmio_config_ops to declare that MMIO-based status and
reset operations are safe during the noirq PM phase.
Testing
=======
Build-tested with arm64 cross-compilation.
(make ARCH=arm64 M=drivers/virtio)
Runtime-tested on an internal virtio-mmio platform with virtio-clock,
confirming that the clock device works well before other devices' normal
restore() callbacks run.
Changes
=======
v5:
virtio: add noirq system sleep PM infrastructure
- Preserve FAILED across restore_noirq() failure by recording the
failure in dev->failed before falling back to the normal restore path.
- Document the restore/restore_noirq fallback contract more clearly,
especially for drivers that preserve virtqueues across suspend.
v4:
virtio_ring: export virtqueue_reinit_vring() for noirq restore
- Reinit safety was tightened by resetting IN_ORDER-specific state.
- Added extra split-ring consistency WARN_ON checks.
- Clarified caller preconditions for noirq-safe vring reinit.
virtio: add noirq system sleep PM infrastructure
- Added config_ops->noirq_safe to explicitly mark transports that are
safe in noirq PM phase.
- Enforced early -EOPNOTSUPP checks in freeze_noirq for unsupported
transport combinations (noirq_safe/reset_vqs requirements).
- Added defensive runtime guards/warnings in noirq helper and restore
paths.
- Discussed the freeze-before-freeze_noirq abort scenario raised in
review and concluded that the fallback restore path is intentionally
handled by regular restore after core reinit/reset, so reset_vqs is
kept limited to the noirq restore flow.
virtio-mmio: wire up noirq system sleep PM callbacks
- Marked virtio-mmio transport as noirq-capable by setting .noirq_safe
in virtio_mmio_config_ops.
v3:
virtio: separate PM restore and reset_done paths
- Refined restore flow to explicitly handle the no-driver case after
reinit, improving clarity and avoiding unnecessary driver-path
assumptions.
virtio_ring: export virtqueue_reinit_vring() for noirq restore
- Hardened virtqueue_reinit_vring() with stronger safety notes and
a runtime WARN_ON check to catch reinit with unexpected free-list
state.
virtio: add noirq system sleep PM infrastructure
- Added explicit noirq restore completion tracking noirq_restore_done
and updated PM sequencing to use it, plus early freeze_noirq
validation for missing reset_vqs support.
virtio-mmio: wire up noirq system sleep PM callbacks
- Updated virtio-mmio restore path to skip legacy GUEST_PAGE_SIZE
rewrite when noirq restore already completed.
v2:
virtio-mmio: wire up noirq system sleep PM callbacks
- The code that was duplicated in vm_setup_vq() and vm_reset_vqs() has
been moved to vm_active_vq() function.
Sungho Bae (4):
virtio: separate PM restore and reset_done paths
virtio_ring: export virtqueue_reinit_vring() for noirq restore
virtio: add noirq system sleep PM infrastructure
virtio-mmio: wire up noirq system sleep PM callbacks
drivers/virtio/virtio.c | 298 +++++++++++++++++++++++++++++++---
drivers/virtio/virtio_mmio.c | 134 ++++++++++-----
drivers/virtio/virtio_ring.c | 51 ++++++
include/linux/virtio.h | 17 ++
include/linux/virtio_config.h | 39 +++++
include/linux/virtio_ring.h | 3 +
6 files changed, 476 insertions(+), 66 deletions(-)
--
2.43.0
^ permalink raw reply
* [PATCH] drm/qxl: Fix missing KMS poll cleanup
From: Myeonghun Pak @ 2026-04-24 11:25 UTC (permalink / raw)
To: Dave Airlie, Gerd Hoffmann
Cc: Maarten Lankhorst, Maxime Ripard, Thomas Zimmermann, David Airlie,
Simona Vetter, virtualization, spice-devel, dri-devel,
linux-kernel, Myeonghun Pak
drm_kms_helper_poll_init() initializes the output polling work and
enables polling for the DRM device. qxl enables polling before calling
drm_dev_register(), but the drm_dev_register() failure path tears down
the modeset and device state without disabling the polling helper.
The remove path also unregisters and shuts down the DRM device without
first disabling the polling helper. Add matching drm_kms_helper_poll_fini()
calls in both paths so the delayed polling work is cancelled before qxl
tears down the associated modeset/device state.
Signed-off-by: Myeonghun Pak <mhun512@gmail.com>
---
drivers/gpu/drm/qxl/qxl_drv.c | 6 ++++--
1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/drivers/gpu/drm/qxl/qxl_drv.c b/drivers/gpu/drm/qxl/qxl_drv.c
index 2bbb1168a3..1e6a2392d7 100644
--- a/drivers/gpu/drm/qxl/qxl_drv.c
+++ b/drivers/gpu/drm/qxl/qxl_drv.c
@@ -118,12 +118,13 @@ qxl_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent)
/* Complete initialization. */
ret = drm_dev_register(&qdev->ddev, ent->driver_data);
if (ret)
- goto modeset_cleanup;
+ goto poll_fini;
drm_client_setup(&qdev->ddev, NULL);
return 0;
-modeset_cleanup:
+poll_fini:
+ drm_kms_helper_poll_fini(&qdev->ddev);
qxl_modeset_fini(qdev);
unload:
qxl_device_fini(qdev);
@@ -154,6 +155,7 @@ qxl_pci_remove(struct pci_dev *pdev)
{
struct drm_device *dev = pci_get_drvdata(pdev);
+ drm_kms_helper_poll_fini(dev);
drm_dev_unregister(dev);
drm_atomic_helper_shutdown(dev);
if (pci_is_vga(pdev) && pdev->revision < 5)
--
2.50.1
^ permalink raw reply related
* [PATCH] virtio-mmio: fix device release warning on module unload
From: Johan Hovold @ 2026-04-24 10:48 UTC (permalink / raw)
To: Michael S . Tsirkin, Jason Wang
Cc: Xuan Zhuo, Eugenio Pérez, Greg Kroah-Hartman, virtualization,
linux-kernel, Johan Hovold, stable, Pawel Moll
Driver core expects devices to be allocated dynamically and complains
loudly when a device that lacks a release function is freed.
Use __root_device_register() to allocate and register the root device
instead of open coding using a static device.
Note that root_device_register(), which also creates a link to the
module, cannot be used as the device is registered when parsing the
module parameters which happens before the module kobject has been set
up.
Fixes: 81a054ce0b46 ("virtio-mmio: Devices parameter parsing")
Cc: stable@vger.kernel.org # 3.5
Cc: Pawel Moll <pawel.moll@arm.com>
Signed-off-by: Johan Hovold <johan@kernel.org>
---
drivers/virtio/virtio_mmio.c | 20 ++++++++------------
1 file changed, 8 insertions(+), 12 deletions(-)
diff --git a/drivers/virtio/virtio_mmio.c b/drivers/virtio/virtio_mmio.c
index 595c2274fbb5..1b580de81e82 100644
--- a/drivers/virtio/virtio_mmio.c
+++ b/drivers/virtio/virtio_mmio.c
@@ -662,9 +662,7 @@ static void virtio_mmio_remove(struct platform_device *pdev)
#if defined(CONFIG_VIRTIO_MMIO_CMDLINE_DEVICES)
-static struct device vm_cmdline_parent = {
- .init_name = "virtio-mmio-cmdline",
-};
+static struct device *vm_cmdline_parent;
static int vm_cmdline_parent_registered;
static int vm_cmdline_id;
@@ -672,7 +670,6 @@ static int vm_cmdline_id;
static int vm_cmdline_set(const char *device,
const struct kernel_param *kp)
{
- int err;
struct resource resources[2] = {};
char *str;
long long base, size;
@@ -704,11 +701,10 @@ static int vm_cmdline_set(const char *device,
resources[1].start = resources[1].end = irq;
if (!vm_cmdline_parent_registered) {
- err = device_register(&vm_cmdline_parent);
- if (err) {
- put_device(&vm_cmdline_parent);
+ vm_cmdline_parent = __root_device_register("virtio-mmio-cmdline", NULL);
+ if (IS_ERR(vm_cmdline_parent)) {
pr_err("Failed to register parent device!\n");
- return err;
+ return PTR_ERR(vm_cmdline_parent);
}
vm_cmdline_parent_registered = 1;
}
@@ -719,7 +715,7 @@ static int vm_cmdline_set(const char *device,
(unsigned long long)resources[0].end,
(int)resources[1].start);
- pdev = platform_device_register_resndata(&vm_cmdline_parent,
+ pdev = platform_device_register_resndata(vm_cmdline_parent,
"virtio-mmio", vm_cmdline_id++,
resources, ARRAY_SIZE(resources), NULL, 0);
@@ -743,7 +739,7 @@ static int vm_cmdline_get_device(struct device *dev, void *data)
static int vm_cmdline_get(char *buffer, const struct kernel_param *kp)
{
buffer[0] = '\0';
- device_for_each_child(&vm_cmdline_parent, buffer,
+ device_for_each_child(vm_cmdline_parent, buffer,
vm_cmdline_get_device);
return strlen(buffer) + 1;
}
@@ -766,9 +762,9 @@ static int vm_unregister_cmdline_device(struct device *dev,
static void vm_unregister_cmdline_devices(void)
{
if (vm_cmdline_parent_registered) {
- device_for_each_child(&vm_cmdline_parent, NULL,
+ device_for_each_child(vm_cmdline_parent, NULL,
vm_unregister_cmdline_device);
- device_unregister(&vm_cmdline_parent);
+ root_device_unregister(vm_cmdline_parent);
vm_cmdline_parent_registered = 0;
}
}
--
2.53.0
^ permalink raw reply related
* [PATCH 1/2] vdpa_sim_blk: switch to dynamic root device
From: Johan Hovold @ 2026-04-24 10:47 UTC (permalink / raw)
To: Michael S . Tsirkin, Jason Wang
Cc: Xuan Zhuo, Eugenio Pérez, virtualization, linux-kernel,
Johan Hovold
In-Reply-To: <20260424104703.2619093-1-johan@kernel.org>
Driver core expects devices to be dynamically allocated and will, for
example, complain loudly when no release function has been provided.
Use root_device_register() to allocate and register the root device
instead of open coding using a static device.
Signed-off-by: Johan Hovold <johan@kernel.org>
---
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 24 ++++++++----------------
1 file changed, 8 insertions(+), 16 deletions(-)
diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c b/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c
index b137f3679343..f70f454dde8e 100644
--- a/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c
+++ b/drivers/vdpa/vdpa_sim/vdpa_sim_blk.c
@@ -397,14 +397,7 @@ static void vdpasim_blk_free(struct vdpasim *vdpasim)
kvfree(blk->buffer);
}
-static void vdpasim_blk_mgmtdev_release(struct device *dev)
-{
-}
-
-static struct device vdpasim_blk_mgmtdev = {
- .init_name = "vdpasim_blk",
- .release = vdpasim_blk_mgmtdev_release,
-};
+static struct device *vdpasim_blk_mgmtdev;
static int vdpasim_blk_dev_add(struct vdpa_mgmt_dev *mdev, const char *name,
const struct vdpa_dev_set_config *config)
@@ -475,7 +468,6 @@ static struct virtio_device_id id_table[] = {
};
static struct vdpa_mgmt_dev mgmt_dev = {
- .device = &vdpasim_blk_mgmtdev,
.id_table = id_table,
.ops = &vdpasim_blk_mgmtdev_ops,
};
@@ -484,12 +476,11 @@ static int __init vdpasim_blk_init(void)
{
int ret;
- ret = device_register(&vdpasim_blk_mgmtdev);
- if (ret) {
- put_device(&vdpasim_blk_mgmtdev);
- return ret;
- }
+ vdpasim_blk_mgmtdev = root_device_register("vdpasim_blk");
+ if (IS_ERR(vdpasim_blk_mgmtdev))
+ return PTR_ERR(vdpasim_blk_mgmtdev);
+ mgmt_dev.device = vdpasim_blk_mgmtdev;
ret = vdpa_mgmtdev_register(&mgmt_dev);
if (ret)
goto parent_err;
@@ -507,7 +498,8 @@ static int __init vdpasim_blk_init(void)
mgmt_dev_err:
vdpa_mgmtdev_unregister(&mgmt_dev);
parent_err:
- device_unregister(&vdpasim_blk_mgmtdev);
+ root_device_unregister(vdpasim_blk_mgmtdev);
+
return ret;
}
@@ -515,7 +507,7 @@ static void __exit vdpasim_blk_exit(void)
{
kvfree(shared_buffer);
vdpa_mgmtdev_unregister(&mgmt_dev);
- device_unregister(&vdpasim_blk_mgmtdev);
+ root_device_unregister(vdpasim_blk_mgmtdev);
}
module_init(vdpasim_blk_init)
--
2.53.0
^ permalink raw reply related
* [PATCH 0/2] vdpa_sim: switch to dynamic root device
From: Johan Hovold @ 2026-04-24 10:47 UTC (permalink / raw)
To: Michael S . Tsirkin, Jason Wang
Cc: Xuan Zhuo, Eugenio Pérez, virtualization, linux-kernel,
Johan Hovold
This series replaces the statically allocated VDPA SIM root devices.
Johan
Johan Hovold (2):
vdpa_sim_blk: switch to dynamic root device
vdpa_sim_net: switch to dynamic root device
drivers/vdpa/vdpa_sim/vdpa_sim_blk.c | 24 ++++++++----------------
drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 23 +++++++----------------
2 files changed, 15 insertions(+), 32 deletions(-)
--
2.53.0
^ permalink raw reply
* [PATCH 2/2] vdpa_sim_net: switch to dynamic root device
From: Johan Hovold @ 2026-04-24 10:47 UTC (permalink / raw)
To: Michael S . Tsirkin, Jason Wang
Cc: Xuan Zhuo, Eugenio Pérez, virtualization, linux-kernel,
Johan Hovold
In-Reply-To: <20260424104703.2619093-1-johan@kernel.org>
Driver core expects devices to be dynamically allocated and will, for
example, complain loudly when no release function has been provided.
Use root_device_register() to allocate and register the root device
instead of open coding using a static device.
Signed-off-by: Johan Hovold <johan@kernel.org>
---
drivers/vdpa/vdpa_sim/vdpa_sim_net.c | 23 +++++++----------------
1 file changed, 7 insertions(+), 16 deletions(-)
diff --git a/drivers/vdpa/vdpa_sim/vdpa_sim_net.c b/drivers/vdpa/vdpa_sim/vdpa_sim_net.c
index 6caf09a1907b..29fd14ce5860 100644
--- a/drivers/vdpa/vdpa_sim/vdpa_sim_net.c
+++ b/drivers/vdpa/vdpa_sim/vdpa_sim_net.c
@@ -453,14 +453,7 @@ static void vdpasim_net_free(struct vdpasim *vdpasim)
kvfree(net->buffer);
}
-static void vdpasim_net_mgmtdev_release(struct device *dev)
-{
-}
-
-static struct device vdpasim_net_mgmtdev = {
- .init_name = "vdpasim_net",
- .release = vdpasim_net_mgmtdev_release,
-};
+static struct device *vdpasim_net_mgmtdev;
static int vdpasim_net_dev_add(struct vdpa_mgmt_dev *mdev, const char *name,
const struct vdpa_dev_set_config *config)
@@ -538,7 +531,6 @@ static struct virtio_device_id id_table[] = {
};
static struct vdpa_mgmt_dev mgmt_dev = {
- .device = &vdpasim_net_mgmtdev,
.id_table = id_table,
.ops = &vdpasim_net_mgmtdev_ops,
.config_attr_mask = (1 << VDPA_ATTR_DEV_NET_CFG_MACADDR |
@@ -552,26 +544,25 @@ static int __init vdpasim_net_init(void)
{
int ret;
- ret = device_register(&vdpasim_net_mgmtdev);
- if (ret) {
- put_device(&vdpasim_net_mgmtdev);
- return ret;
- }
+ vdpasim_net_mgmtdev = root_device_register("vdpasim_net");
+ if (IS_ERR(vdpasim_net_mgmtdev))
+ return PTR_ERR(vdpasim_net_mgmtdev);
+ mgmt_dev.device = vdpasim_net_mgmtdev;
ret = vdpa_mgmtdev_register(&mgmt_dev);
if (ret)
goto parent_err;
return 0;
parent_err:
- device_unregister(&vdpasim_net_mgmtdev);
+ root_device_unregister(vdpasim_net_mgmtdev);
return ret;
}
static void __exit vdpasim_net_exit(void)
{
vdpa_mgmtdev_unregister(&mgmt_dev);
- device_unregister(&vdpasim_net_mgmtdev);
+ root_device_unregister(vdpasim_net_mgmtdev);
}
module_init(vdpasim_net_init);
--
2.53.0
^ permalink raw reply related
* Re: [PATCH RFC v3 01/19] mm: thread user_addr through page allocator for cache-friendly zeroing
From: David Hildenbrand (Arm) @ 2026-04-24 7:28 UTC (permalink / raw)
To: Gregory Price
Cc: Michael S. Tsirkin, linux-kernel, Andrew Morton, Vlastimil Babka,
Brendan Jackman, Michal Hocko, Suren Baghdasaryan, Jason Wang,
Andrea Arcangeli, linux-mm, virtualization, Johannes Weiner,
Zi Yan, Lorenzo Stoakes, Liam R. Howlett, Mike Rapoport,
Matthew Wilcox (Oracle), Muchun Song, Oscar Salvador, Baolin Wang,
Nico Pache, Ryan Roberts, Dev Jain, Barry Song, Lance Yang,
Matthew Brost, Joshua Hahn, Rakie Kim, Byungchul Park, Ying Huang,
Alistair Popple, Hugh Dickins, Christoph Lameter, David Rientjes,
Roman Gushchin, Harry Yoo, Chris Li, Kairui Song, Kemeng Shi,
Nhat Pham, Baoquan He, linux-fsdevel
In-Reply-To: <aeozRZ9AvW2jTQkX@gourry-fedora-PF4VCD3F>
On 4/23/26 16:57, Gregory Price wrote:
> On Thu, Apr 23, 2026 at 04:13:50PM +0200, David Hildenbrand (Arm) wrote:
>> On 4/23/26 15:42, Gregory Price wrote:
>>
>> Maybe we could forward the vma+addr here and call a vma_alloc_froze_folio() if
>> we have a VMA+addr to have a clean interface.
>>
>> But really, that hugetlb code is rather messy. I'd vote for leaving hugetlb
>> alone on a v1, and focusing on non-hugetlb first.
>>
>
> If we're ok increasing the buddy surface this way, then I'd vote for
> only updating the exact interfaces that MST needs to update for his use
> case in a base set of patches, and then have each additional updated
> location (or logical set of locations) updated in follow-ups.
>
> My initial go around with this - the patch was hard to read at best.
>
> But I also think we should also seriously consider not increasing the
> surface of the buddy.
Exactly, that's why I am saying that vma_alloc_folio() is the only external
interface people should be using with a user address. all other _noprof helpers
are supposed to be internal.
For hugetlb, we might need another interface for frozen folios later, which is
why I suggest to defer that.
--
Cheers,
David
^ permalink raw reply
* Re: [PATCH RFC v3 10/19] mm: remove arch vma_alloc_zeroed_movable_folio overrides
From: Magnus Lindholm @ 2026-04-24 7:16 UTC (permalink / raw)
To: Michael S. Tsirkin
Cc: linux-kernel, Andrew Morton, David Hildenbrand, Vlastimil Babka,
Brendan Jackman, Michal Hocko, Suren Baghdasaryan, Jason Wang,
Andrea Arcangeli, Gregory Price, linux-mm, virtualization,
Richard Henderson, Matt Turner, Greg Ungerer, Geert Uytterhoeven,
Heiko Carstens, Vasily Gorbik, Alexander Gordeev,
Christian Borntraeger, Sven Schnelle, Thomas Gleixner,
Ingo Molnar, Borislav Petkov, Dave Hansen, x86, H. Peter Anvin,
linux-alpha, linux-m68k, linux-s390
In-Reply-To: <006f9142e591ba8c340c3b354aee76aec5c285b9.1776808210.git.mst@redhat.com>
On Wed, Apr 22, 2026 at 12:01 AM Michael S. Tsirkin <mst@redhat.com> wrote:
>
> Now that the generic vma_alloc_zeroed_movable_folio() uses
> __GFP_ZERO, the arch-specific macros on alpha, m68k, s390, and
> x86 that did the same thing are redundant. Remove them.
>
> arm64 is not affected: it has a real function override that
> handles MTE tag zeroing, not just __GFP_ZERO.
>
> Suggested-by: David Hildenbrand <david@kernel.org>
> Signed-off-by: Michael S. Tsirkin <mst@redhat.com>
> ---
> arch/alpha/include/asm/page.h | 3 ---
> arch/m68k/include/asm/page_no.h | 3 ---
> arch/s390/include/asm/page.h | 3 ---
> arch/x86/include/asm/page.h | 3 ---
> 4 files changed, 12 deletions(-)
>
> diff --git a/arch/alpha/include/asm/page.h b/arch/alpha/include/asm/page.h
> index 59d01f9b77f6..4327029cd660 100644
> --- a/arch/alpha/include/asm/page.h
> +++ b/arch/alpha/include/asm/page.h
> @@ -12,9 +12,6 @@
>
> extern void clear_page(void *page);
>
> -#define vma_alloc_zeroed_movable_folio(vma, vaddr) \
> - vma_alloc_folio(GFP_HIGHUSER_MOVABLE | __GFP_ZERO, 0, vma, vaddr)
> -
> extern void copy_page(void * _to, void * _from);
> #define copy_user_page(to, from, vaddr, pg) copy_page(to, from)
>
From an alpha perspective, this looks good to me.
Acked-by: Magnus Lindholm <linmag7@gmail.com>
^ permalink raw reply
* [PATCH 7.2 v10 2/2] x86/tlb: skip redundant sync IPIs for native TLB flush
From: Lance Yang @ 2026-04-24 6:25 UTC (permalink / raw)
To: akpm
Cc: peterz, david, dave.hansen, dave.hansen, ypodemsk, hughd, will,
aneesh.kumar, npiggin, tglx, mingo, bp, x86, hpa, arnd, ljs, ziy,
baolin.wang, Liam.Howlett, npache, ryan.roberts, dev.jain, baohua,
shy828301, riel, jannh, jgross, seanjc, pbonzini, boris.ostrovsky,
virtualization, kvm, linux-arch, linux-mm, linux-kernel,
ioworker0, Lance Yang
In-Reply-To: <20260424062528.71951-1-lance.yang@linux.dev>
From: Lance Yang <lance.yang@linux.dev>
Some page table operations need to synchronize with software/lockless
walkers after a TLB flush by calling tlb_remove_table_sync_{one,rcu}().
On x86, that extra synchronization is redundant when the preceding TLB
flush already broadcast IPIs to all relevant CPUs.
native_pv_tlb_init() checks whether native_flush_tlb_multi() is in use.
On CONFIG_PARAVIRT systems, it checks pv_ops; on non-PARAVIRT, native
flush is always in use.
It decides once at boot whether to enable the optimization: if using
native TLB flush and INVLPGB is not supported, we know IPIs were sent
and can skip the redundant sync. The decision is fixed via a static
key as Peter suggested[1].
PV backends (KVM, Xen, Hyper-V) typically have their own implementations
and don't call native_flush_tlb_multi() directly, so they cannot be trusted
to provide the IPI guarantees we need.
Also rename the x86 flush_tlb_info bit from freed_tables to wake_lazy_cpus,
as Dave suggested[2], to match the behavior it controls: whether the remote
flush may skip CPUs in lazy TLB mode. Both freed_tables and unshared_tables
set it, because lazy-TLB CPUs must receive IPIs before page tables can be
freed or reused. With that guarantee in place,
tlb_table_flush_implies_ipi_broadcast() can safely skip the later sync IPI.
Two-step plan as David suggested[3]:
Step 1 (this patch): Skip redundant sync when we're 100% certain the TLB
flush sent IPIs. INVLPGB is excluded because when supported, we cannot
guarantee IPIs were sent, keeping it clean and simple.
Step 2 (future work): Send targeted IPIs only to CPUs actually doing
software/lockless page table walks, benefiting all architectures.
Regarding Step 2, it obviously only applies to setups where Step 1 does
not apply: like x86 with INVLPGB or arm64.
[1] https://lore.kernel.org/linux-mm/20260302145652.GH1395266@noisy.programming.kicks-ass.net/
[2] https://lore.kernel.org/linux-mm/f856051b-10c7-4d65-9dbe-6b1677af74bd@intel.com/
[3] https://lore.kernel.org/linux-mm/bbfdf226-4660-4949-b17b-0d209ee4ef8c@kernel.org/
Suggested-by: Dave Hansen <dave.hansen@intel.com>
Suggested-by: Peter Zijlstra <peterz@infradead.org>
Suggested-by: David Hildenbrand (Arm) <david@kernel.org>
Signed-off-by: Lance Yang <lance.yang@linux.dev>
---
arch/x86/hyperv/mmu.c | 4 ++--
arch/x86/include/asm/tlb.h | 19 +++++++++++++++-
arch/x86/include/asm/tlbflush.h | 6 +++--
arch/x86/kernel/smpboot.c | 1 +
arch/x86/mm/tlb.c | 39 +++++++++++++++++++++++----------
5 files changed, 52 insertions(+), 17 deletions(-)
diff --git a/arch/x86/hyperv/mmu.c b/arch/x86/hyperv/mmu.c
index cfcb60468b01..2cf1eeaffd6f 100644
--- a/arch/x86/hyperv/mmu.c
+++ b/arch/x86/hyperv/mmu.c
@@ -63,7 +63,7 @@ static void hyperv_flush_tlb_multi(const struct cpumask *cpus,
struct hv_tlb_flush *flush;
u64 status;
unsigned long flags;
- bool do_lazy = !info->freed_tables;
+ bool do_lazy = !info->wake_lazy_cpus;
trace_hyperv_mmu_flush_tlb_multi(cpus, info);
@@ -198,7 +198,7 @@ static u64 hyperv_flush_tlb_others_ex(const struct cpumask *cpus,
flush->hv_vp_set.format = HV_GENERIC_SET_SPARSE_4K;
nr_bank = cpumask_to_vpset_skip(&flush->hv_vp_set, cpus,
- info->freed_tables ? NULL : cpu_is_lazy);
+ info->wake_lazy_cpus ? NULL : cpu_is_lazy);
if (nr_bank < 0)
return HV_STATUS_INVALID_PARAMETER;
diff --git a/arch/x86/include/asm/tlb.h b/arch/x86/include/asm/tlb.h
index 866ea78ba156..fb256fd95f95 100644
--- a/arch/x86/include/asm/tlb.h
+++ b/arch/x86/include/asm/tlb.h
@@ -5,22 +5,39 @@
#define tlb_flush tlb_flush
static inline void tlb_flush(struct mmu_gather *tlb);
+#define tlb_table_flush_implies_ipi_broadcast tlb_table_flush_implies_ipi_broadcast
+static inline bool tlb_table_flush_implies_ipi_broadcast(void);
+
#include <asm-generic/tlb.h>
#include <linux/kernel.h>
#include <vdso/bits.h>
#include <vdso/page.h>
+DECLARE_STATIC_KEY_FALSE(tlb_ipi_broadcast_key);
+
+static inline bool tlb_table_flush_implies_ipi_broadcast(void)
+{
+ return static_branch_likely(&tlb_ipi_broadcast_key);
+}
+
static inline void tlb_flush(struct mmu_gather *tlb)
{
unsigned long start = 0UL, end = TLB_FLUSH_ALL;
unsigned int stride_shift = tlb_get_unmap_shift(tlb);
+ /*
+ * Both freed_tables and unshared_tables must wake lazy-TLB CPUs, so
+ * they receive IPIs before reusing or freeing page tables, allowing
+ * us to safely implement tlb_table_flush_implies_ipi_broadcast().
+ */
+ bool wake_lazy_cpus = tlb->freed_tables || tlb->unshared_tables;
+
if (!tlb->fullmm && !tlb->need_flush_all) {
start = tlb->start;
end = tlb->end;
}
- flush_tlb_mm_range(tlb->mm, start, end, stride_shift, tlb->freed_tables);
+ flush_tlb_mm_range(tlb->mm, start, end, stride_shift, wake_lazy_cpus);
}
static inline void invlpg(unsigned long addr)
diff --git a/arch/x86/include/asm/tlbflush.h b/arch/x86/include/asm/tlbflush.h
index 5a3cdc439e38..39b9454781c3 100644
--- a/arch/x86/include/asm/tlbflush.h
+++ b/arch/x86/include/asm/tlbflush.h
@@ -18,6 +18,8 @@
DECLARE_PER_CPU(u64, tlbstate_untag_mask);
+void __init native_pv_tlb_init(void);
+
void __flush_tlb_all(void);
#define TLB_FLUSH_ALL -1UL
@@ -225,7 +227,7 @@ struct flush_tlb_info {
u64 new_tlb_gen;
unsigned int initiating_cpu;
u8 stride_shift;
- u8 freed_tables;
+ u8 wake_lazy_cpus;
u8 trim_cpumask;
};
@@ -315,7 +317,7 @@ static inline bool mm_in_asid_transition(struct mm_struct *mm) { return false; }
extern void flush_tlb_all(void);
extern void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start,
unsigned long end, unsigned int stride_shift,
- bool freed_tables);
+ bool wake_lazy_cpus);
extern void flush_tlb_kernel_range(unsigned long start, unsigned long end);
static inline void flush_tlb_page(struct vm_area_struct *vma, unsigned long a)
diff --git a/arch/x86/kernel/smpboot.c b/arch/x86/kernel/smpboot.c
index 294a8ea60298..df776b645a9c 100644
--- a/arch/x86/kernel/smpboot.c
+++ b/arch/x86/kernel/smpboot.c
@@ -1256,6 +1256,7 @@ void __init native_smp_prepare_boot_cpu(void)
switch_gdt_and_percpu_base(me);
native_pv_lock_init();
+ native_pv_tlb_init();
}
void __init native_smp_cpus_done(unsigned int max_cpus)
diff --git a/arch/x86/mm/tlb.c b/arch/x86/mm/tlb.c
index 621e09d049cb..3ce254a3982c 100644
--- a/arch/x86/mm/tlb.c
+++ b/arch/x86/mm/tlb.c
@@ -26,6 +26,8 @@
#include "mm_internal.h"
+DEFINE_STATIC_KEY_FALSE(tlb_ipi_broadcast_key);
+
#ifdef CONFIG_PARAVIRT
# define STATIC_NOPV
#else
@@ -1360,16 +1362,16 @@ STATIC_NOPV void native_flush_tlb_multi(const struct cpumask *cpumask,
(info->end - info->start) >> PAGE_SHIFT);
/*
- * If no page tables were freed, we can skip sending IPIs to
- * CPUs in lazy TLB mode. They will flush the CPU themselves
- * at the next context switch.
+ * If lazy-TLB CPUs do not need to be woken, we can skip sending
+ * IPIs to them. They will flush themselves at the next context
+ * switch.
*
- * However, if page tables are getting freed, we need to send the
- * IPI everywhere, to prevent CPUs in lazy TLB mode from tripping
- * up on the new contents of what used to be page tables, while
- * doing a speculative memory access.
+ * However, if page tables are getting freed or unshared, we need
+ * to send the IPI everywhere, to prevent CPUs in lazy TLB mode
+ * from tripping up on the new contents of what used to be page
+ * tables, while doing a speculative memory access.
*/
- if (info->freed_tables || mm_in_asid_transition(info->mm))
+ if (info->wake_lazy_cpus || mm_in_asid_transition(info->mm))
on_each_cpu_mask(cpumask, flush_tlb_func, (void *)info, true);
else
on_each_cpu_cond_mask(should_flush_tlb, flush_tlb_func,
@@ -1402,7 +1404,7 @@ static DEFINE_PER_CPU(unsigned int, flush_tlb_info_idx);
static struct flush_tlb_info *get_flush_tlb_info(struct mm_struct *mm,
unsigned long start, unsigned long end,
- unsigned int stride_shift, bool freed_tables,
+ unsigned int stride_shift, bool wake_lazy_cpus,
u64 new_tlb_gen)
{
struct flush_tlb_info *info = this_cpu_ptr(&flush_tlb_info);
@@ -1429,7 +1431,7 @@ static struct flush_tlb_info *get_flush_tlb_info(struct mm_struct *mm,
info->end = end;
info->mm = mm;
info->stride_shift = stride_shift;
- info->freed_tables = freed_tables;
+ info->wake_lazy_cpus = wake_lazy_cpus;
info->new_tlb_gen = new_tlb_gen;
info->initiating_cpu = smp_processor_id();
info->trim_cpumask = 0;
@@ -1448,7 +1450,7 @@ static void put_flush_tlb_info(void)
void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start,
unsigned long end, unsigned int stride_shift,
- bool freed_tables)
+ bool wake_lazy_cpus)
{
struct flush_tlb_info *info;
int cpu = get_cpu();
@@ -1457,7 +1459,7 @@ void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start,
/* This is also a barrier that synchronizes with switch_mm(). */
new_tlb_gen = inc_mm_tlb_gen(mm);
- info = get_flush_tlb_info(mm, start, end, stride_shift, freed_tables,
+ info = get_flush_tlb_info(mm, start, end, stride_shift, wake_lazy_cpus,
new_tlb_gen);
/*
@@ -1834,3 +1836,16 @@ static int __init create_tlb_single_page_flush_ceiling(void)
return 0;
}
late_initcall(create_tlb_single_page_flush_ceiling);
+
+void __init native_pv_tlb_init(void)
+{
+#ifdef CONFIG_PARAVIRT
+ if (pv_ops.mmu.flush_tlb_multi != native_flush_tlb_multi)
+ return;
+#endif
+
+ if (cpu_feature_enabled(X86_FEATURE_INVLPGB))
+ return;
+
+ static_branch_enable(&tlb_ipi_broadcast_key);
+}
--
2.49.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox