[PATCH] iommu/amd: Fix domain flush size when syncing iotlb

[PATCH] iommu/amd: Fix domain flush size when syncing iotlb

[Jon Pan-Doh]

When running on an AMD vIOMMU, we observed multiple invalidations (of
decreasing power of 2 aligned sizes) when unmapping a single page.

Domain flush takes gather bounds (end-start) as size param. However,
gather->end is defined as the last inclusive address (start + size - 1).
This leads to an off by 1 error.

With this patch, verified that 1 invalidation occurs when unmapping a
single page.

Fixes: a270be1b3fdf ("iommu/amd: Use only natural aligned flushes in a VM")
Signed-off-by: Jon Pan-Doh <pandoh@google.com>
Tested-by: Sudheer Dantuluri <dantuluris@google.com>
Suggested-by: Gary Zibrat <gzibrat@google.com>
---
 drivers/iommu/amd/iommu.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
index 5a505ba5467e..da45b1ab042d 100644
--- a/drivers/iommu/amd/iommu.c
+++ b/drivers/iommu/amd/iommu.c
@@ -2378,7 +2378,7 @@ static void amd_iommu_iotlb_sync(struct iommu_domain *domain,
 	unsigned long flags;
 
 	spin_lock_irqsave(&dom->lock, flags);
-	domain_flush_pages(dom, gather->start, gather->end - gather->start, 1);
+	domain_flush_pages(dom, gather->start, gather->end - gather->start + 1, 1);
 	amd_iommu_domain_flush_complete(dom);
 	spin_unlock_irqrestore(&dom->lock, flags);
 }
-- 
2.40.0.634.g4ca3ef3211-goog

Re: [PATCH] iommu/amd: Fix domain flush size when syncing iotlb

[Nadav Amit]

> On Apr 26, 2023, at 1:32 PM, Jon Pan-Doh <pandoh@google.com> wrote:
> 
> When running on an AMD vIOMMU, we observed multiple invalidations (of
> decreasing power of 2 aligned sizes) when unmapping a single page.
> 
> Domain flush takes gather bounds (end-start) as size param. However,
> gather->end is defined as the last inclusive address (start + size - 1).
> This leads to an off by 1 error.
> 
> With this patch, verified that 1 invalidation occurs when unmapping a
> single page.
> 
> Fixes: a270be1b3fdf ("iommu/amd: Use only natural aligned flushes in a VM")
> Signed-off-by: Jon Pan-Doh <pandoh@google.com>
> Tested-by: Sudheer Dantuluri <dantuluris@google.com>
> Suggested-by: Gary Zibrat <gzibrat@google.com>
> 

Sorry for that.

Acked-by: Nadav Amit <namit@vmware.com>

Re: [PATCH] iommu/amd: Fix domain flush size when syncing iotlb

[Nadav Amit]

> On Apr 26, 2023, at 2:04 PM, Nadav Amit <namit@vmware.com> wrote:
> 
> 
> 
>> On Apr 26, 2023, at 1:32 PM, Jon Pan-Doh <pandoh@google.com> wrote:
>> 
>> When running on an AMD vIOMMU, we observed multiple invalidations (of
>> decreasing power of 2 aligned sizes) when unmapping a single page.
>> 
>> Domain flush takes gather bounds (end-start) as size param. However,
>> gather->end is defined as the last inclusive address (start + size - 1).
>> This leads to an off by 1 error.
>> 
>> With this patch, verified that 1 invalidation occurs when unmapping a
>> single page.
>> 
>> Fixes: a270be1b3fdf ("iommu/amd: Use only natural aligned flushes in a VM")
>> Signed-off-by: Jon Pan-Doh <pandoh@google.com>
>> Tested-by: Sudheer Dantuluri <dantuluris@google.com>
>> Suggested-by: Gary Zibrat <gzibrat@google.com>
>> 
> 
> Sorry for that.
> 
> Acked-by: Nadav Amit <namit@vmware.com>

P.S. I think it is stable-worthy.

Re: [PATCH] iommu/amd: Fix domain flush size when syncing iotlb

[Vasant Hegde]

On 4/27/2023 2:02 AM, Jon Pan-Doh wrote:
> When running on an AMD vIOMMU, we observed multiple invalidations (of
> decreasing power of 2 aligned sizes) when unmapping a single page.
> 
> Domain flush takes gather bounds (end-start) as size param. However,
> gather->end is defined as the last inclusive address (start + size - 1).
> This leads to an off by 1 error.
> 
> With this patch, verified that 1 invalidation occurs when unmapping a
> single page.
> 
> Fixes: a270be1b3fdf ("iommu/amd: Use only natural aligned flushes in a VM")
> Signed-off-by: Jon Pan-Doh <pandoh@google.com>
> Tested-by: Sudheer Dantuluri <dantuluris@google.com>
> Suggested-by: Gary Zibrat <gzibrat@google.com>

Thanks!

Reviewed-by: Vasant Hegde <vasant.hegde@amd.com>

-Vasant


> ---
>  drivers/iommu/amd/iommu.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
> index 5a505ba5467e..da45b1ab042d 100644
> --- a/drivers/iommu/amd/iommu.c
> +++ b/drivers/iommu/amd/iommu.c
> @@ -2378,7 +2378,7 @@ static void amd_iommu_iotlb_sync(struct iommu_domain *domain,
>  	unsigned long flags;
>  
>  	spin_lock_irqsave(&dom->lock, flags);
> -	domain_flush_pages(dom, gather->start, gather->end - gather->start, 1);
> +	domain_flush_pages(dom, gather->start, gather->end - gather->start + 1, 1);
>  	amd_iommu_domain_flush_complete(dom);
>  	spin_unlock_irqrestore(&dom->lock, flags);
>  }

Re: [PATCH] iommu/amd: Fix domain flush size when syncing iotlb

[Jon Pan-Doh]

Hi Joerg,

[Cc'ing stable@vger.kernel.org per Nadav's suggestion]

This bug fix seems to have gotten the necessary reviews (AMD and
previous commit author). Is it eligible to be applied?

Thanks,
Jon

On Wed, Apr 26, 2023 at 1:32 PM Jon Pan-Doh <pandoh@google.com> wrote:
>
> When running on an AMD vIOMMU, we observed multiple invalidations (of
> decreasing power of 2 aligned sizes) when unmapping a single page.
>
> Domain flush takes gather bounds (end-start) as size param. However,
> gather->end is defined as the last inclusive address (start + size - 1).
> This leads to an off by 1 error.
>
> With this patch, verified that 1 invalidation occurs when unmapping a
> single page.
>
> Fixes: a270be1b3fdf ("iommu/amd: Use only natural aligned flushes in a VM")
> Signed-off-by: Jon Pan-Doh <pandoh@google.com>
> Tested-by: Sudheer Dantuluri <dantuluris@google.com>
> Suggested-by: Gary Zibrat <gzibrat@google.com>
> ---
>  drivers/iommu/amd/iommu.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
> index 5a505ba5467e..da45b1ab042d 100644
> --- a/drivers/iommu/amd/iommu.c
> +++ b/drivers/iommu/amd/iommu.c
> @@ -2378,7 +2378,7 @@ static void amd_iommu_iotlb_sync(struct iommu_domain *domain,
>         unsigned long flags;
>
>         spin_lock_irqsave(&dom->lock, flags);
> -       domain_flush_pages(dom, gather->start, gather->end - gather->start, 1);
> +       domain_flush_pages(dom, gather->start, gather->end - gather->start + 1, 1);
>         amd_iommu_domain_flush_complete(dom);
>         spin_unlock_irqrestore(&dom->lock, flags);
>  }
> --
> 2.40.0.634.g4ca3ef3211-goog
>

Re: [PATCH] iommu/amd: Fix domain flush size when syncing iotlb

[Greg KH]

On Tue, May 16, 2023 at 01:59:16PM -0700, Jon Pan-Doh wrote:
> Hi Joerg,
> 
> [Cc'ing stable@vger.kernel.org per Nadav's suggestion]

<formletter>

This is not the correct way to submit patches for inclusion in the
stable kernel tree.  Please read:
    https://www.kernel.org/doc/html/latest/process/stable-kernel-rules.html
for how to do this properly.

</formletter>

Re: [PATCH] iommu/amd: Fix domain flush size when syncing iotlb

[Jon Pan-Doh]

On Tue, May 16, 2023 at 9:38 PM Greg KH <gregkh@linuxfoundation.org> wrote:
>
> On Tue, May 16, 2023 at 01:59:16PM -0700, Jon Pan-Doh wrote:
> > Hi Joerg,
> >
> > [Cc'ing stable@vger.kernel.org per Nadav's suggestion]
>
> <formletter>
>
> This is not the correct way to submit patches for inclusion in the
> stable kernel tree.  Please read:
>     https://www.kernel.org/doc/html/latest/process/stable-kernel-rules.html
> for how to do this properly.
>
> </formletter>

Thanks for the link. Added the Cc tag as a reply to the patch.
Apologies for top-posting previously.

Re: [PATCH] iommu/amd: Fix domain flush size when syncing iotlb

[Jon Pan-Doh]

On Wed, Apr 26, 2023 at 1:32 PM Jon Pan-Doh <pandoh@google.com> wrote:
>
> When running on an AMD vIOMMU, we observed multiple invalidations (of
> decreasing power of 2 aligned sizes) when unmapping a single page.
>
> Domain flush takes gather bounds (end-start) as size param. However,
> gather->end is defined as the last inclusive address (start + size - 1).
> This leads to an off by 1 error.
>
> With this patch, verified that 1 invalidation occurs when unmapping a
> single page.
>
> Fixes: a270be1b3fdf ("iommu/amd: Use only natural aligned flushes in a VM")
> Signed-off-by: Jon Pan-Doh <pandoh@google.com>
> Tested-by: Sudheer Dantuluri <dantuluris@google.com>
> Suggested-by: Gary Zibrat <gzibrat@google.com>

Cc: stable@vger.kernel.org

Re: [PATCH] iommu/amd: Fix domain flush size when syncing iotlb

[Nadav Amit]

> On May 16, 2023, at 10:49 PM, Jon Pan-Doh <pandoh@google.com> wrote:
> 
> !! External Email
> 
> On Wed, Apr 26, 2023 at 1:32 PM Jon Pan-Doh <pandoh@google.com> wrote:
>> 
>> When running on an AMD vIOMMU, we observed multiple invalidations (of
>> decreasing power of 2 aligned sizes) when unmapping a single page.
>> 
>> Domain flush takes gather bounds (end-start) as size param. However,
>> gather->end is defined as the last inclusive address (start + size - 1).
>> This leads to an off by 1 error.
>> 
>> With this patch, verified that 1 invalidation occurs when unmapping a
>> single page.
>> 
>> Fixes: a270be1b3fdf ("iommu/amd: Use only natural aligned flushes in a VM")
>> Signed-off-by: Jon Pan-Doh <pandoh@google.com>
>> Tested-by: Sudheer Dantuluri <dantuluris@google.com>
>> Suggested-by: Gary Zibrat <gzibrat@google.com>
> 
> Cc: stable@vger.kernel.org

Assuming the patch was still not applied, I think that Greg KH would
prefer that you send v2 that includes the “Cc: <stable@vger.kernel.org>”
(notice the “<“ and “>" signs) and ideally the version in which I added
the buggy code (sorry for that again) after a “#” sign.

I think it is version 5.15.

IOW: If the patch was still not applied by Joerg, please send v2 with:

Cc: <stable@vger.kernel.org <mailto:stable@vger.kernel.org>> # 5.15.x

In the commit log.

Re: [PATCH] iommu/amd: Fix domain flush size when syncing iotlb

[Jon Pan-Doh]

On Tue, May 16, 2023 at 10:58 PM Nadav Amit <namit@vmware.com> wrote:
> IOW: If the patch was still not applied by Joerg, please send v2 with:
>
> Cc: <stable@vger.kernel.org <mailto:stable@vger.kernel.org>> # 5.15.x
>
> In the commit log.
>

Thanks for the clarification/guidance. I've resent the patch for
review with added tags (Acked-by, Reviewed-by). Will re-read the patch
submission doc and try to avoid formatting issues in the future.

Re: [PATCH] iommu/amd: Fix domain flush size when syncing iotlb

[Joerg Roedel]

On Wed, Apr 26, 2023 at 01:32:56PM -0700, Jon Pan-Doh wrote:
>  drivers/iommu/amd/iommu.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)

Applied for 6.4, thanks.

Re: [PATCH] iommu/amd: Fix domain flush size when syncing iotlb

[Jon Pan-Doh]

Cc'ing stable@vger.kernel.org per Nadav's suggestion.