Re: Patch "iommu/amd: Use cmpxchg_double() when updating 128-bit IRTE" has been added to the 5.8-stable tree

Re: Patch "iommu/amd: Use cmpxchg_double() when updating 128-bit IRTE" has been added to the 5.8-stable tree

[Suravee Suthikulpanit]

Sasha,

This patch and the upstream commit 26e495f34107 ("iommu/amd: Restore IRTE.RemapEn bit after programming IRTE")
are related. Since the commit 26e495f34107 has been back-ported to linux-5.4.y branch of the stable tree,
this patch should also be add to linux-5.4.y as well.

Thanks,
Suravee

On 9/7/20 10:05 AM, Sasha Levin wrote:
> This is a note to let you know that I've just added the patch titled
> 
>      iommu/amd: Use cmpxchg_double() when updating 128-bit IRTE
> 
> to the 5.8-stable tree which can be found at:
>      https://nam11.safelinks.protection.outlook.com/?url=http%3A%2F%2Fwww.kernel.org%2Fgit%2F%3Fp%3Dlinux%2Fkernel%2Fgit%2Fstable%2Fstable-queue.git%3Ba%3Dsummary&data=02%7C01%7Csuravee.suthikulpanit%40amd.com%7C1e7eb9ea3f5c402d49fb08d852dadc51%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637350447241879928&sdata=O6CQHQFMlNkSLU1SHm5qHmWBN6hq2C8YUIWktEC0Rnc%3D&reserved=0
> 
> The filename of the patch is:
>       iommu-amd-use-cmpxchg_double-when-updating-128-bit-i.patch
> and it can be found in the queue-5.8 subdirectory.
> 
> If you, or anyone else, feels it should not be added to the stable tree,
> please let <stable@vger.kernel.org> know about it.
> 
> 
> 
> commit 31640157d429339f8fffc14a645cb2c9739e0f17
> Author: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
> Date:   Thu Sep 3 09:38:22 2020 +0000
> 
>      iommu/amd: Use cmpxchg_double() when updating 128-bit IRTE
>      
>      [ Upstream commit e52d58d54a321d4fe9d0ecdabe4f8774449f0d6e ]
>      
>      When using 128-bit interrupt-remapping table entry (IRTE) (a.k.a GA mode),
>      current driver disables interrupt remapping when it updates the IRTE
>      so that the upper and lower 64-bit values can be updated safely.
>      
>      However, this creates a small window, where the interrupt could
>      arrive and result in IO_PAGE_FAULT (for interrupt) as shown below.
>      
>        IOMMU Driver            Device IRQ
>        ============            ===========
>        irte.RemapEn=0
>             ...
>         change IRTE            IRQ from device ==> IO_PAGE_FAULT !!
>             ...
>        irte.RemapEn=1
>      
>      This scenario has been observed when changing irq affinity on a system
>      running I/O-intensive workload, in which the destination APIC ID
>      in the IRTE is updated.
>      
>      Instead, use cmpxchg_double() to update the 128-bit IRTE at once without
>      disabling the interrupt remapping. However, this means several features,
>      which require GA (128-bit IRTE) support will also be affected if cmpxchg16b
>      is not supported (which is unprecedented for AMD processors w/ IOMMU).
>      
>      Fixes: 880ac60e2538 ("iommu/amd: Introduce interrupt remapping ops structure")
>      Reported-by: Sean Osborne <sean.m.osborne@oracle.com>
>      Signed-off-by: Suravee Suthikulpanit <suravee.suthikulpanit@amd.com>
>      Tested-by: Erik Rockstrom <erik.rockstrom@oracle.com>
>      Reviewed-by: Joao Martins <joao.m.martins@oracle.com>
>      Link: https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flore.kernel.org%2Fr%2F20200903093822.52012-3-suravee.suthikulpanit%40amd.com&amp;data=02%7C01%7Csuravee.suthikulpanit%40amd.com%7C1e7eb9ea3f5c402d49fb08d852dadc51%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637350447241879928&amp;sdata=9pKWytGjGIiX0yaE9WC7ahNNnT5oGG8dyufBeqr7%2FkY%3D&amp;reserved=0
>      Signed-off-by: Joerg Roedel <jroedel@suse.de>
>      Signed-off-by: Sasha Levin <sashal@kernel.org>
> 
> diff --git a/drivers/iommu/Kconfig b/drivers/iommu/Kconfig
> index b0f308cb7f7c2..201b2718f0755 100644
> --- a/drivers/iommu/Kconfig
> +++ b/drivers/iommu/Kconfig
> @@ -143,7 +143,7 @@ config AMD_IOMMU
>   	select IOMMU_API
>   	select IOMMU_IOVA
>   	select IOMMU_DMA
> -	depends on X86_64 && PCI && ACPI
> +	depends on X86_64 && PCI && ACPI && HAVE_CMPXCHG_DOUBLE
>   	help
>   	  With this option you can enable support for AMD IOMMU hardware in
>   	  your system. An IOMMU is a hardware component which provides
> diff --git a/drivers/iommu/amd/init.c b/drivers/iommu/amd/init.c
> index 6ebd4825e3206..bf45f8e2c7edd 100644
> --- a/drivers/iommu/amd/init.c
> +++ b/drivers/iommu/amd/init.c
> @@ -1518,7 +1518,14 @@ static int __init init_iommu_one(struct amd_iommu *iommu, struct ivhd_header *h)
>   			iommu->mmio_phys_end = MMIO_REG_END_OFFSET;
>   		else
>   			iommu->mmio_phys_end = MMIO_CNTR_CONF_OFFSET;
> -		if (((h->efr_attr & (0x1 << IOMMU_FEAT_GASUP_SHIFT)) == 0))
> +
> +		/*
> +		 * Note: GA (128-bit IRTE) mode requires cmpxchg16b supports.
> +		 * GAM also requires GA mode. Therefore, we need to
> +		 * check cmpxchg16b support before enabling it.
> +		 */
> +		if (!boot_cpu_has(X86_FEATURE_CX16) ||
> +		    ((h->efr_attr & (0x1 << IOMMU_FEAT_GASUP_SHIFT)) == 0))
>   			amd_iommu_guest_ir = AMD_IOMMU_GUEST_IR_LEGACY;
>   		break;
>   	case 0x11:
> @@ -1527,8 +1534,18 @@ static int __init init_iommu_one(struct amd_iommu *iommu, struct ivhd_header *h)
>   			iommu->mmio_phys_end = MMIO_REG_END_OFFSET;
>   		else
>   			iommu->mmio_phys_end = MMIO_CNTR_CONF_OFFSET;
> -		if (((h->efr_reg & (0x1 << IOMMU_EFR_GASUP_SHIFT)) == 0))
> +
> +		/*
> +		 * Note: GA (128-bit IRTE) mode requires cmpxchg16b supports.
> +		 * XT, GAM also requires GA mode. Therefore, we need to
> +		 * check cmpxchg16b support before enabling them.
> +		 */
> +		if (!boot_cpu_has(X86_FEATURE_CX16) ||
> +		    ((h->efr_reg & (0x1 << IOMMU_EFR_GASUP_SHIFT)) == 0)) {
>   			amd_iommu_guest_ir = AMD_IOMMU_GUEST_IR_LEGACY;
> +			break;
> +		}
> +
>   		/*
>   		 * Note: Since iommu_update_intcapxt() leverages
>   		 * the IOMMU MMIO access to MSI capability block registers
> diff --git a/drivers/iommu/amd/iommu.c b/drivers/iommu/amd/iommu.c
> index d7b037891fb7e..200ee948f6ec1 100644
> --- a/drivers/iommu/amd/iommu.c
> +++ b/drivers/iommu/amd/iommu.c
> @@ -3283,6 +3283,7 @@ out:
>   static int modify_irte_ga(u16 devid, int index, struct irte_ga *irte,
>   			  struct amd_ir_data *data)
>   {
> +	bool ret;
>   	struct irq_remap_table *table;
>   	struct amd_iommu *iommu;
>   	unsigned long flags;
> @@ -3300,10 +3301,18 @@ static int modify_irte_ga(u16 devid, int index, struct irte_ga *irte,
>   
>   	entry = (struct irte_ga *)table->table;
>   	entry = &entry[index];
> -	entry->lo.fields_remap.valid = 0;
> -	entry->hi.val = irte->hi.val;
> -	entry->lo.val = irte->lo.val;
> -	entry->lo.fields_remap.valid = 1;
> +
> +	ret = cmpxchg_double(&entry->lo.val, &entry->hi.val,
> +			     entry->lo.val, entry->hi.val,
> +			     irte->lo.val, irte->hi.val);
> +	/*
> +	 * We use cmpxchg16 to atomically update the 128-bit IRTE,
> +	 * and it cannot be updated by the hardware or other processors
> +	 * behind us, so the return value of cmpxchg16 should be the
> +	 * same as the old value.
> +	 */
> +	WARN_ON(!ret);
> +
>   	if (data)
>   		data->ref = entry;
>   
> 

Re: Patch "iommu/amd: Use cmpxchg_double() when updating 128-bit IRTE" has been added to the 5.8-stable tree

[Greg KH]

On Wed, Sep 16, 2020 at 08:19:37PM +0700, Suravee Suthikulpanit wrote:
> Sasha,
> 
> This patch and the upstream commit 26e495f34107 ("iommu/amd: Restore IRTE.RemapEn bit after programming IRTE")
> are related. Since the commit 26e495f34107 has been back-ported to linux-5.4.y branch of the stable tree,
> this patch should also be add to linux-5.4.y as well.

It doesn't apply to 5.4.y at all, can someone please provide a working
backport?

thanks,

greg k-h

Re: Patch "iommu/amd: Use cmpxchg_double() when updating 128-bit IRTE" has been added to the 5.8-stable tree

[Suravee Suthikulpanit]

On 9/17/20 9:35 PM, Greg KH wrote:
> On Wed, Sep 16, 2020 at 08:19:37PM +0700, Suravee Suthikulpanit wrote:
>> Sasha,
>>
>> This patch and the upstream commit 26e495f34107 ("iommu/amd: Restore IRTE.RemapEn bit after programming IRTE")
>> are related. Since the commit 26e495f34107 has been back-ported to linux-5.4.y branch of the stable tree,
>> this patch should also be add to linux-5.4.y as well.
> 
> It doesn't apply to 5.4.y at all, can someone please provide a working
> backport?
> 
> thanks,
> 
> greg k-h
> 

Let me work on this and send upstream.

Thank you,
Suravee

Re: Patch "iommu/amd: Use cmpxchg_double() when updating 128-bit IRTE" has been added to the 5.8-stable tree

[Suravee Suthikulpanit]

Hi,

I have just sent out the patch for 5.4-stable branch.

Thank you,
Suravee

On 9/17/20 9:50 PM, Suravee Suthikulpanit wrote:
> 
> 
> On 9/17/20 9:35 PM, Greg KH wrote:
>> On Wed, Sep 16, 2020 at 08:19:37PM +0700, Suravee Suthikulpanit wrote:
>>> Sasha,
>>>
>>> This patch and the upstream commit 26e495f34107 ("iommu/amd: Restore IRTE.RemapEn bit after programming IRTE")
>>> are related. Since the commit 26e495f34107 has been back-ported to linux-5.4.y branch of the stable tree,
>>> this patch should also be add to linux-5.4.y as well.
>>
>> It doesn't apply to 5.4.y at all, can someone please provide a working
>> backport?
>>
>> thanks,
>>
>> greg k-h
>>
> 
> Let me work on this and send upstream.
> 
> Thank you,
> Suravee