* [PATCH v1 0/1] Don't split flush for amd_iommu_domain_flush_all() @ 2026-04-14 21:06 Weinan Liu 2026-04-14 21:06 ` [PATCH v1 1/1] iommu/amd: " Weinan Liu 0 siblings, 1 reply; 8+ messages in thread From: Weinan Liu @ 2026-04-14 21:06 UTC (permalink / raw) To: iommu, jgg Cc: joro, patches, robin.murphy, suravee.suthikulpanit, wei.w.wang, will, kpsingh, josef, Weinan Liu This patch is a respin of Josef Bacik's original work[1] to fix the performance issues and soft lockups in the AMD IOMMU driver when running within a VM. I am taking over the respin as Josef currently has limited time for this [1]:https://lore.kernel.org/linux-iommu/ad8652c5e9f8aeee05e2103f4987589cdd4a3fd0.1772659768.git.josef@toxicpanda.com Weinan Liu (1): iommu/amd: Don't split flush for amd_iommu_domain_flush_all() drivers/iommu/amd/iommu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) -- 2.54.0.rc0.605.g598a273b03-goog ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v1 1/1] iommu/amd: Don't split flush for amd_iommu_domain_flush_all() 2026-04-14 21:06 [PATCH v1 0/1] Don't split flush for amd_iommu_domain_flush_all() Weinan Liu @ 2026-04-14 21:06 ` Weinan Liu 2026-04-14 23:36 ` Jason Gunthorpe 2026-04-14 23:55 ` Samiullah Khawaja 0 siblings, 2 replies; 8+ messages in thread From: Weinan Liu @ 2026-04-14 21:06 UTC (permalink / raw) To: iommu, jgg Cc: joro, patches, robin.murphy, suravee.suthikulpanit, wei.w.wang, will, kpsingh, josef, Weinan Liu We have observed multiple full invalidations occurring during device detach when we are done using the vfio-device. blocked_domain_attach_device() -> detach_device() -> amd_iommu_domain_flush_all() -> amd_iommu_domain_flush_pages(..., CMD_INV_IOMMU_ALL_PAGES_ADDRESS) while (size != 0) { -> __domain_flush_pages( flush_size /* power of 2 flush_size */) -> domain_flush_pages_v1() -> build_inv_iommu_pages() -> build_inv_address() } build_inv_address() will trigger a full invalidation if the chunk size > (1 << 51). Consequently, the guest will issue multiple full invalidations for a single call to amd_iommu_domain_flush_all() Without this patch, we will see 10 time instead of 1 time full invalidations for every amd_iommu_domain_flush_all(). Fixes: a270be1b3fdf ("iommu/amd: Use only natural aligned flushes in a VM") Suggested-by: Josef Bacik <josef@toxicpanda.com> Suggested-by: Jason Gunthorpe <jgg@nvidia.com> Signed-off-by: Weinan Liu <wnliu@google.com> --- drivers/iommu/amd/iommu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c index 760d5f4623b5..bcec8721d228 100644 --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -1769,7 +1769,8 @@ void amd_iommu_domain_flush_pages(struct protection_domain *domain, { lockdep_assert_held(&domain->lock); - if (likely(!amd_iommu_np_cache)) { + if (likely(!amd_iommu_np_cache) || + size == CMD_INV_IOMMU_ALL_PAGES_ADDRESS) { __domain_flush_pages(domain, address, size); /* Wait until IOMMU TLB and all device IOTLB flushes are complete */ -- 2.54.0.rc0.605.g598a273b03-goog ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH v1 1/1] iommu/amd: Don't split flush for amd_iommu_domain_flush_all() 2026-04-14 21:06 ` [PATCH v1 1/1] iommu/amd: " Weinan Liu @ 2026-04-14 23:36 ` Jason Gunthorpe 2026-04-15 0:30 ` Weinan Liu 2026-04-14 23:55 ` Samiullah Khawaja 1 sibling, 1 reply; 8+ messages in thread From: Jason Gunthorpe @ 2026-04-14 23:36 UTC (permalink / raw) To: Weinan Liu Cc: iommu, joro, patches, robin.murphy, suravee.suthikulpanit, wei.w.wang, will, kpsingh, josef On Tue, Apr 14, 2026 at 09:06:26PM +0000, Weinan Liu wrote: > diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c > index 760d5f4623b5..bcec8721d228 100644 > --- a/drivers/iommu/amd/iommu.c > +++ b/drivers/iommu/amd/iommu.c > @@ -1769,7 +1769,8 @@ void amd_iommu_domain_flush_pages(struct protection_domain *domain, > { > lockdep_assert_held(&domain->lock); > > - if (likely(!amd_iommu_np_cache)) { > + if (likely(!amd_iommu_np_cache) || > + size == CMD_INV_IOMMU_ALL_PAGES_ADDRESS) { This is better as size >= (1ULL<<52) Which is the break point where it switches over to full invalidation anyhow, there is no sense then in doing the special np flow.. Otherwise Reviewed-by: Jason Gunthorpe <jgg@nvidia.com> Jason ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v1 1/1] iommu/amd: Don't split flush for amd_iommu_domain_flush_all() 2026-04-14 23:36 ` Jason Gunthorpe @ 2026-04-15 0:30 ` Weinan Liu 2026-04-17 11:57 ` Jason Gunthorpe 2026-04-21 5:02 ` Wei Wang 0 siblings, 2 replies; 8+ messages in thread From: Weinan Liu @ 2026-04-15 0:30 UTC (permalink / raw) To: jgg Cc: iommu, joro, josef, kpsingh, patches, robin.murphy, suravee.suthikulpanit, wei.w.wang, will, wnliu We have observed multiple full invalidations occurring during device detach when we are done using the vfio-device. blocked_domain_attach_device() -> detach_device() -> amd_iommu_domain_flush_all() -> amd_iommu_domain_flush_pages(..., CMD_INV_IOMMU_ALL_PAGES_ADDRESS) while (size != 0) { -> __domain_flush_pages( flush_size /* power of 2 flush_size */) -> domain_flush_pages_v1() -> build_inv_iommu_pages() -> build_inv_address() } build_inv_address() will trigger a full invalidation if the chunk size > (1 << 51). Consequently, the guest will issue multiple full invalidations for a single call to amd_iommu_domain_flush_all() Without this patch, we will see 10 time instead of 1 time full invalidations for every amd_iommu_domain_flush_all(). Fixes: a270be1b3fdf ("iommu/amd: Use only natural aligned flushes in a VM") Suggested-by: Josef Bacik <josef@toxicpanda.com> Suggested-by: Jason Gunthorpe <jgg@nvidia.com> Signed-off-by: Weinan Liu <wnliu@google.com> --- drivers/iommu/amd/iommu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c index 760d5f4623b5..59650a44ab50 100644 --- a/drivers/iommu/amd/iommu.c +++ b/drivers/iommu/amd/iommu.c @@ -1769,7 +1769,8 @@ void amd_iommu_domain_flush_pages(struct protection_domain *domain, { lockdep_assert_held(&domain->lock); - if (likely(!amd_iommu_np_cache)) { + if (likely(!amd_iommu_np_cache) || + size >= (1ULL<<52)) { __domain_flush_pages(domain, address, size); /* Wait until IOMMU TLB and all device IOTLB flushes are complete */ -- 2.54.0.rc0.605.g598a273b03-goog ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [PATCH v1 1/1] iommu/amd: Don't split flush for amd_iommu_domain_flush_all() 2026-04-15 0:30 ` Weinan Liu @ 2026-04-17 11:57 ` Jason Gunthorpe 2026-04-21 5:02 ` Wei Wang 1 sibling, 0 replies; 8+ messages in thread From: Jason Gunthorpe @ 2026-04-17 11:57 UTC (permalink / raw) To: Weinan Liu Cc: iommu, joro, josef, kpsingh, patches, robin.murphy, suravee.suthikulpanit, wei.w.wang, will On Wed, Apr 15, 2026 at 12:30:46AM +0000, Weinan Liu wrote: > We have observed multiple full invalidations occurring during device > detach when we are done using the vfio-device. > > blocked_domain_attach_device() > -> detach_device() > -> amd_iommu_domain_flush_all() > -> amd_iommu_domain_flush_pages(..., CMD_INV_IOMMU_ALL_PAGES_ADDRESS) > > while (size != 0) { > > -> __domain_flush_pages( flush_size /* power of 2 flush_size */) > -> domain_flush_pages_v1() > -> build_inv_iommu_pages() > -> build_inv_address() > > } > > build_inv_address() will trigger a full invalidation if the chunk > size > (1 << 51). Consequently, the guest will issue multiple full > invalidations for a single call to amd_iommu_domain_flush_all() > > Without this patch, we will see 10 time instead of 1 time full > invalidations for every amd_iommu_domain_flush_all(). > > Fixes: a270be1b3fdf ("iommu/amd: Use only natural aligned flushes in a VM") > > Suggested-by: Josef Bacik <josef@toxicpanda.com> > Suggested-by: Jason Gunthorpe <jgg@nvidia.com> > Signed-off-by: Weinan Liu <wnliu@google.com> > --- > drivers/iommu/amd/iommu.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) Reviewed-by: Jason Gunthorpe <jgg@nvidia.com> Jason ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v1 1/1] iommu/amd: Don't split flush for amd_iommu_domain_flush_all() 2026-04-15 0:30 ` Weinan Liu 2026-04-17 11:57 ` Jason Gunthorpe @ 2026-04-21 5:02 ` Wei Wang 1 sibling, 0 replies; 8+ messages in thread From: Wei Wang @ 2026-04-21 5:02 UTC (permalink / raw) To: Weinan Liu, jgg Cc: iommu, joro, josef, kpsingh, patches, robin.murphy, suravee.suthikulpanit, will On 4/15/26 8:30 AM, Weinan Liu wrote: > We have observed multiple full invalidations occurring during device > detach when we are done using the vfio-device. > > blocked_domain_attach_device() > -> detach_device() > -> amd_iommu_domain_flush_all() > -> amd_iommu_domain_flush_pages(..., CMD_INV_IOMMU_ALL_PAGES_ADDRESS) > > while (size != 0) { > > -> __domain_flush_pages( flush_size /* power of 2 flush_size */) > -> domain_flush_pages_v1() > -> build_inv_iommu_pages() > -> build_inv_address() > > } > > build_inv_address() will trigger a full invalidation if the chunk > size > (1 << 51). Consequently, the guest will issue multiple full > invalidations for a single call to amd_iommu_domain_flush_all() > > Without this patch, we will see 10 time instead of 1 time full > invalidations for every amd_iommu_domain_flush_all(). > > Fixes: a270be1b3fdf ("iommu/amd: Use only natural aligned flushes in a VM") > > Suggested-by: Josef Bacik <josef@toxicpanda.com> > Suggested-by: Jason Gunthorpe <jgg@nvidia.com> > Signed-off-by: Weinan Liu <wnliu@google.com> > --- > drivers/iommu/amd/iommu.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > > diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c > index 760d5f4623b5..59650a44ab50 100644 > --- a/drivers/iommu/amd/iommu.c > +++ b/drivers/iommu/amd/iommu.c > @@ -1769,7 +1769,8 @@ void amd_iommu_domain_flush_pages(struct protection_domain *domain, > { > lockdep_assert_held(&domain->lock); > > - if (likely(!amd_iommu_np_cache)) { > + if (likely(!amd_iommu_np_cache) || > + size >= (1ULL<<52)) { > __domain_flush_pages(domain, address, size); > > /* Wait until IOMMU TLB and all device IOTLB flushes are complete */ Reviewed-by: Wei Wang <wei.w.wang@hotmail.com> ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v1 1/1] iommu/amd: Don't split flush for amd_iommu_domain_flush_all() 2026-04-14 21:06 ` [PATCH v1 1/1] iommu/amd: " Weinan Liu 2026-04-14 23:36 ` Jason Gunthorpe @ 2026-04-14 23:55 ` Samiullah Khawaja 2026-05-14 5:07 ` Suthikulpanit, Suravee 1 sibling, 1 reply; 8+ messages in thread From: Samiullah Khawaja @ 2026-04-14 23:55 UTC (permalink / raw) To: Weinan Liu Cc: iommu, jgg, joro, patches, robin.murphy, suravee.suthikulpanit, wei.w.wang, will, kpsingh, josef On Tue, Apr 14, 2026 at 09:06:26PM +0000, Weinan Liu wrote: >We have observed multiple full invalidations occurring during device >detach when we are done using the vfio-device. > >blocked_domain_attach_device() > -> detach_device() > -> amd_iommu_domain_flush_all() > -> amd_iommu_domain_flush_pages(..., CMD_INV_IOMMU_ALL_PAGES_ADDRESS) > > while (size != 0) { > > -> __domain_flush_pages( flush_size /* power of 2 flush_size */) > -> domain_flush_pages_v1() > -> build_inv_iommu_pages() > -> build_inv_address() > > } > >build_inv_address() will trigger a full invalidation if the chunk >size > (1 << 51). Consequently, the guest will issue multiple full >invalidations for a single call to amd_iommu_domain_flush_all() > >Without this patch, we will see 10 time instead of 1 time full >invalidations for every amd_iommu_domain_flush_all(). > >Fixes: a270be1b3fdf ("iommu/amd: Use only natural aligned flushes in a VM") > >Suggested-by: Josef Bacik <josef@toxicpanda.com> >Suggested-by: Jason Gunthorpe <jgg@nvidia.com> >Signed-off-by: Weinan Liu <wnliu@google.com> >--- > drivers/iommu/amd/iommu.c | 3 ++- > 1 file changed, 2 insertions(+), 1 deletion(-) > >diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c >index 760d5f4623b5..bcec8721d228 100644 >--- a/drivers/iommu/amd/iommu.c >+++ b/drivers/iommu/amd/iommu.c >@@ -1769,7 +1769,8 @@ void amd_iommu_domain_flush_pages(struct protection_domain *domain, > { > lockdep_assert_held(&domain->lock); > >- if (likely(!amd_iommu_np_cache)) { >+ if (likely(!amd_iommu_np_cache) || >+ size == CMD_INV_IOMMU_ALL_PAGES_ADDRESS) { > __domain_flush_pages(domain, address, size); > > /* Wait until IOMMU TLB and all device IOTLB flushes are complete */ >-- >2.54.0.rc0.605.g598a273b03-goog > > Reviewed-by: Samiullah Khawaja <skhawaja@google.com> ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [PATCH v1 1/1] iommu/amd: Don't split flush for amd_iommu_domain_flush_all() 2026-04-14 23:55 ` Samiullah Khawaja @ 2026-05-14 5:07 ` Suthikulpanit, Suravee 0 siblings, 0 replies; 8+ messages in thread From: Suthikulpanit, Suravee @ 2026-05-14 5:07 UTC (permalink / raw) To: Samiullah Khawaja, Weinan Liu Cc: iommu, jgg, joro, patches, robin.murphy, wei.w.wang, will, kpsingh, josef, Hegde, Vasant On 4/15/2026 6:55 AM, Samiullah Khawaja wrote: > On Tue, Apr 14, 2026 at 09:06:26PM +0000, Weinan Liu wrote: >> We have observed multiple full invalidations occurring during device >> detach when we are done using the vfio-device. >> >> blocked_domain_attach_device() >> -> detach_device() >> -> amd_iommu_domain_flush_all() >> -> amd_iommu_domain_flush_pages(..., >> CMD_INV_IOMMU_ALL_PAGES_ADDRESS) >> >> while (size != 0) { >> >> -> __domain_flush_pages( flush_size /* power of 2 flush_size */) >> -> domain_flush_pages_v1() >> -> build_inv_iommu_pages() >> -> build_inv_address() >> >> } >> >> build_inv_address() will trigger a full invalidation if the chunk >> size > (1 << 51). Consequently, the guest will issue multiple full >> invalidations for a single call to amd_iommu_domain_flush_all() >> >> Without this patch, we will see 10 time instead of 1 time full >> invalidations for every amd_iommu_domain_flush_all(). >> >> Fixes: a270be1b3fdf ("iommu/amd: Use only natural aligned flushes in a >> VM") >> >> Suggested-by: Josef Bacik <josef@toxicpanda.com> >> Suggested-by: Jason Gunthorpe <jgg@nvidia.com> >> Signed-off-by: Weinan Liu <wnliu@google.com> >> --- >> drivers/iommu/amd/iommu.c | 3 ++- >> 1 file changed, 2 insertions(+), 1 deletion(-) >> >> diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c >> index 760d5f4623b5..bcec8721d228 100644 >> --- a/drivers/iommu/amd/iommu.c >> +++ b/drivers/iommu/amd/iommu.c >> @@ -1769,7 +1769,8 @@ void amd_iommu_domain_flush_pages(struct >> protection_domain *domain, >> { >> lockdep_assert_held(&domain->lock); >> >> - if (likely(!amd_iommu_np_cache)) { >> + if (likely(!amd_iommu_np_cache) || >> + size == CMD_INV_IOMMU_ALL_PAGES_ADDRESS) { >> __domain_flush_pages(domain, address, size); >> >> /* Wait until IOMMU TLB and all device IOTLB flushes are >> complete */ >> -- >> 2.54.0.rc0.605.g598a273b03-goog >> >> > > Reviewed-by: Samiullah Khawaja <skhawaja@google.com> Thanks, Reviewed-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com> ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2026-05-14 5:07 UTC | newest] Thread overview: 8+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-04-14 21:06 [PATCH v1 0/1] Don't split flush for amd_iommu_domain_flush_all() Weinan Liu 2026-04-14 21:06 ` [PATCH v1 1/1] iommu/amd: " Weinan Liu 2026-04-14 23:36 ` Jason Gunthorpe 2026-04-15 0:30 ` Weinan Liu 2026-04-17 11:57 ` Jason Gunthorpe 2026-04-21 5:02 ` Wei Wang 2026-04-14 23:55 ` Samiullah Khawaja 2026-05-14 5:07 ` Suthikulpanit, Suravee
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.