RE: Make intel iommu disable swiotlb

RE: Make intel iommu disable swiotlb

[Adrian Mardare (amardare)]

(resending as plain-text)
Hi...
I'd like to submit the below patch.
While investigating a difference in memory consumption between the kernel in 5.4.x and in 6.6.x,
I noticed there is around ~80M less reported in MemTotal in 6.6.x versus 5.4.x.
This is because  in 6.6.x  the swiotlb is still enabled and consuming about 80M of kernel memory, even though
the intel iommu has been fully initialized.
This change will cause the swiotlb to be disabled /  memory to be released ( via swiotlb_exit ).
Thanks,
Adrian


From 27e63e727f23189e9c98d81fb445141f998c0808 Mon Sep 17 00:00:00 2001
From: Adrian Mardare <amardare@cisco.com>
Date: Sat, 25 Jan 2025 23:41:31 -0500
Subject: [PATCH] Make intel iommu disable swiotlb

On boards where intel iommu has been successfully
initialized, we need to disable the swiotlb as there
is no reason for swiotlb to be active past that.

Signed-off-by: Adrian Mardare <amardare@cisco.com>
---
drivers/iommu/intel/iommu.c | 3 +++
1 file changed, 3 insertions(+)

diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
index cc46098f875b..971023d33a70 100644
--- a/drivers/iommu/intel/iommu.c
+++ b/drivers/iommu/intel/iommu.c
@@ -3166,6 +3166,9 @@ int __init intel_iommu_init(void)
               pr_info("Intel(R) Virtualization Technology for Directed I/O\n");

                intel_iommu_enabled = 1;
+#ifdef CONFIG_SWIOTLB
+             x86_swiotlb_enable = false;
+#endif

                return 0;

-- 
2.34.1

Re: Make intel iommu disable swiotlb

[Robin Murphy]

On 2025-01-26 5:09 am, Adrian Mardare (amardare) wrote:
> (resending as plain-text)
> Hi...
> I'd like to submit the below patch.
> While investigating a difference in memory consumption between the kernel in 5.4.x and in 6.6.x,
> I noticed there is around ~80M less reported in MemTotal in 6.6.x versus 5.4.x.
> This is because  in 6.6.x  the swiotlb is still enabled and consuming about 80M of kernel memory, even though
> the intel iommu has been fully initialized.
> This change will cause the swiotlb to be disabled /  memory to be released ( via swiotlb_exit ).
> Thanks,
> Adrian
> 
> 
>  From 27e63e727f23189e9c98d81fb445141f998c0808 Mon Sep 17 00:00:00 2001
> From: Adrian Mardare <amardare@cisco.com>
> Date: Sat, 25 Jan 2025 23:41:31 -0500
> Subject: [PATCH] Make intel iommu disable swiotlb
> 
> On boards where intel iommu has been successfully
> initialized, we need to disable the swiotlb as there
> is no reason for swiotlb to be active past that.

Except there are *two* reasons: 32-bit devices in identity domains, and 
untrusted (e.g. Thunderbolt) devices...

See commits c5a5dc4cbbf4 ("iommu/vt-d: Don't switch off swiotlb if 
bounce page is used") and subsequently 327d5b2fee91 ("iommu/vt-d: Allow 
32bit devices to uses DMA domain")

Thanks,
Robin.

> Signed-off-by: Adrian Mardare <amardare@cisco.com>
> ---
> drivers/iommu/intel/iommu.c | 3 +++
> 1 file changed, 3 insertions(+)
> 
> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> index cc46098f875b..971023d33a70 100644
> --- a/drivers/iommu/intel/iommu.c
> +++ b/drivers/iommu/intel/iommu.c
> @@ -3166,6 +3166,9 @@ int __init intel_iommu_init(void)
>                 pr_info("Intel(R) Virtualization Technology for Directed I/O\n");
> 
>                  intel_iommu_enabled = 1;
> +#ifdef CONFIG_SWIOTLB
> +             x86_swiotlb_enable = false;
> +#endif
> 
>                  return 0;
> 

Re: Make intel iommu disable swiotlb

[Baolu Lu]

On 2025/1/26 13:09, Adrian Mardare (amardare) wrote:
> (resending as plain-text)
> Hi...
> I'd like to submit the below patch.
> While investigating a difference in memory consumption between the kernel in 5.4.x and in 6.6.x,
> I noticed there is around ~80M less reported in MemTotal in 6.6.x versus 5.4.x.
> This is because  in 6.6.x  the swiotlb is still enabled and consuming about 80M of kernel memory, even though
> the intel iommu has been fully initialized.
> This change will cause the swiotlb to be disabled /  memory to be released ( via swiotlb_exit ).
> Thanks,
> Adrian
> 
> 
>  From 27e63e727f23189e9c98d81fb445141f998c0808 Mon Sep 17 00:00:00 2001
> From: Adrian Mardare<amardare@cisco.com>
> Date: Sat, 25 Jan 2025 23:41:31 -0500
> Subject: [PATCH] Make intel iommu disable swiotlb
> 
> On boards where intel iommu has been successfully
> initialized, we need to disable the swiotlb as there
> is no reason for swiotlb to be active past that.
> 
> Signed-off-by: Adrian Mardare<amardare@cisco.com>
> ---
> drivers/iommu/intel/iommu.c | 3 +++
> 1 file changed, 3 insertions(+)
> 
> diff --git a/drivers/iommu/intel/iommu.c b/drivers/iommu/intel/iommu.c
> index cc46098f875b..971023d33a70 100644
> --- a/drivers/iommu/intel/iommu.c
> +++ b/drivers/iommu/intel/iommu.c
> @@ -3166,6 +3166,9 @@ int __init intel_iommu_init(void)
>                 pr_info("Intel(R) Virtualization Technology for Directed I/O\n");
> 
>                  intel_iommu_enabled = 1;
> +#ifdef CONFIG_SWIOTLB
> +             x86_swiotlb_enable = false;
> +#endif

Why does an iommu driver need to care about swiotlb? The extra memory
consumption might be a problem, but I don't think it should be fixed in
the iommu driver.

Thanks,
baolu

Re: Make intel iommu disable swiotlb

[Christoph Hellwig]

On Wed, Feb 05, 2025 at 06:35:20PM +0800, Baolu Lu wrote:
> Why does an iommu driver need to care about swiotlb? The extra memory
> consumption might be a problem, but I don't think it should be fixed in
> the iommu driver.

Not only should the iommu driver not care - the patch is broken.
We need swiotlb to bounce buffer partial pages for untrusted devices.

RE: Make intel iommu disable swiotlb

[Adrian Mardare (amardare)]

Yes I agree  that swiotlb is still needed and I'm withdrawing the patch.
Thanks,
Adrian

-----Original Message-----
From: Christoph Hellwig <hch@infradead.org> 
Sent: Wednesday, February 5, 2025 10:50 AM
To: Baolu Lu <baolu.lu@linux.intel.com>
Cc: Adrian Mardare (amardare) <amardare@cisco.com>; dwmw2@infradead.org; iommu@lists.linux.dev
Subject: Re: Make intel iommu disable swiotlb

On Wed, Feb 05, 2025 at 06:35:20PM +0800, Baolu Lu wrote:
> Why does an iommu driver need to care about swiotlb? The extra memory 
> consumption might be a problem, but I don't think it should be fixed 
> in the iommu driver.

Not only should the iommu driver not care - the patch is broken.
We need swiotlb to bounce buffer partial pages for untrusted devices.