[PATCH] iommu/vt-d: Make sure IOMMUs are off when intel_iommu=off

[PATCH] iommu/vt-d: Make sure IOMMUs are off when intel_iommu=off

[Joerg Roedel]

From: Joerg Roedel <jroedel-l3A5Bk7waGM@public.gmane.org>

When booting into a kexec kernel with intel_iommu=off, and
the previous kernel had intel_iommu=on, the IOMMU hardware
is still enabled and gets not disabled by the new kernel.

This causes the boot to fail because DMA is blocked by the
hardware. Disable the IOMMUs when we find it enabled in the
kexec kernel and boot with intel_iommu=off.

Signed-off-by: Joerg Roedel <jroedel-l3A5Bk7waGM@public.gmane.org>
---
 drivers/iommu/intel-iommu.c | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index d412a31..1662288 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -4730,6 +4730,15 @@ static int intel_iommu_cpu_dead(unsigned int cpu)
 	return 0;
 }
 
+static void intel_disable_iommus(void)
+{
+	struct intel_iommu *iommu = NULL;
+	struct dmar_drhd_unit *drhd;
+
+	for_each_iommu(iommu, drhd)
+		iommu_disable_translation(iommu);
+}
+
 static inline struct intel_iommu *dev_to_intel_iommu(struct device *dev)
 {
 	return container_of(dev, struct intel_iommu, iommu.dev);
@@ -4840,8 +4849,15 @@ int __init intel_iommu_init(void)
 		goto out_free_dmar;
 	}
 
-	if (no_iommu || dmar_disabled)
+	if (no_iommu || dmar_disabled) {
+		/*
+		 * Make sure the IOMMUs are switched off, even when we
+		 * boot into a kexec kernel and the previous kernel left
+		 * them enabled
+		 */
+		intel_disable_iommus();
 		goto out_free_dmar;
+	}
 
 	if (list_empty(&dmar_rmrr_units))
 		pr_info("No RMRR found\n");
-- 
1.9.1

Re: [PATCH] iommu/vt-d: Make sure IOMMUs are off when intel_iommu=off

[Jacob Pan]

On Wed, 29 Mar 2017 17:00:39 +0200
Joerg Roedel <joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org> wrote:

> From: Joerg Roedel <jroedel-l3A5Bk7waGM@public.gmane.org>
> 
> When booting into a kexec kernel with intel_iommu=off, and
> the previous kernel had intel_iommu=on, the IOMMU hardware
> is still enabled and gets not disabled by the new kernel.
> 
> This causes the boot to fail because DMA is blocked by the
> hardware. Disable the IOMMUs when we find it enabled in the
> kexec kernel and boot with intel_iommu=off.
> 
> Signed-off-by: Joerg Roedel <jroedel-l3A5Bk7waGM@public.gmane.org>
> ---
>  drivers/iommu/intel-iommu.c | 18 +++++++++++++++++-
>  1 file changed, 17 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
> index d412a31..1662288 100644
> --- a/drivers/iommu/intel-iommu.c
> +++ b/drivers/iommu/intel-iommu.c
> @@ -4730,6 +4730,15 @@ static int intel_iommu_cpu_dead(unsigned int
> cpu) return 0;
>  }
>  
> +static void intel_disable_iommus(void)
> +{
> +	struct intel_iommu *iommu = NULL;
> +	struct dmar_drhd_unit *drhd;
> +
> +	for_each_iommu(iommu, drhd)
> +		iommu_disable_translation(iommu);
should we also disable IR here? i.e.
iommu_disable_irq_remapping(iommu);

> +}
> +
>  static inline struct intel_iommu *dev_to_intel_iommu(struct device
> *dev) {
>  	return container_of(dev, struct intel_iommu, iommu.dev);
> @@ -4840,8 +4849,15 @@ int __init intel_iommu_init(void)
>  		goto out_free_dmar;
>  	}
>  
> -	if (no_iommu || dmar_disabled)
> +	if (no_iommu || dmar_disabled) {
> +		/*
> +		 * Make sure the IOMMUs are switched off, even when
> we
> +		 * boot into a kexec kernel and the previous kernel
> left
> +		 * them enabled
> +		 */
> +		intel_disable_iommus();
>  		goto out_free_dmar;
> +	}
>  
>  	if (list_empty(&dmar_rmrr_units))
>  		pr_info("No RMRR found\n");

[Jacob Pan]

Re: [PATCH] iommu/vt-d: Make sure IOMMUs are off when intel_iommu=off

[Joerg Roedel]

On Wed, Mar 29, 2017 at 08:51:53AM -0700, Jacob Pan wrote:
> On Wed, 29 Mar 2017 17:00:39 +0200
> Joerg Roedel <joro-zLv9SwRftAIdnm+yROfE0A@public.gmane.org> wrote:

> > +static void intel_disable_iommus(void)
> > +{
> > +	struct intel_iommu *iommu = NULL;
> > +	struct dmar_drhd_unit *drhd;
> > +
> > +	for_each_iommu(iommu, drhd)
> > +		iommu_disable_translation(iommu);
> should we also disable IR here? i.e.
> iommu_disable_irq_remapping(iommu);

No, irq remapping depends on another parameter (intremap=off) and needs
to be handled seperatly. I already looked at it and this needs to be
part of a larger effort as it required changes in code-paths that are
shared between Intel and AMD IOMMU drivers.


	Joerg

Re: [PATCH] iommu/vt-d: Make sure IOMMUs are off when intel_iommu=off

[Baoquan He]

Hi Joerg,

Do you plan to merge this one as urgent?

There's bug created about this issue on rhel, it would be great if it
can be put in next or merged so that we can back port it.

Thanks
Baoquan

On 03/29/17 at 05:00pm, Joerg Roedel wrote:
> From: Joerg Roedel <jroedel-l3A5Bk7waGM@public.gmane.org>
> 
> When booting into a kexec kernel with intel_iommu=off, and
> the previous kernel had intel_iommu=on, the IOMMU hardware
> is still enabled and gets not disabled by the new kernel.
> 
> This causes the boot to fail because DMA is blocked by the
> hardware. Disable the IOMMUs when we find it enabled in the
> kexec kernel and boot with intel_iommu=off.
> 
> Signed-off-by: Joerg Roedel <jroedel-l3A5Bk7waGM@public.gmane.org>
> ---
>  drivers/iommu/intel-iommu.c | 18 +++++++++++++++++-
>  1 file changed, 17 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
> index d412a31..1662288 100644
> --- a/drivers/iommu/intel-iommu.c
> +++ b/drivers/iommu/intel-iommu.c
> @@ -4730,6 +4730,15 @@ static int intel_iommu_cpu_dead(unsigned int cpu)
>  	return 0;
>  }
>  
> +static void intel_disable_iommus(void)
> +{
> +	struct intel_iommu *iommu = NULL;
> +	struct dmar_drhd_unit *drhd;
> +
> +	for_each_iommu(iommu, drhd)
> +		iommu_disable_translation(iommu);
> +}
> +
>  static inline struct intel_iommu *dev_to_intel_iommu(struct device *dev)
>  {
>  	return container_of(dev, struct intel_iommu, iommu.dev);
> @@ -4840,8 +4849,15 @@ int __init intel_iommu_init(void)
>  		goto out_free_dmar;
>  	}
>  
> -	if (no_iommu || dmar_disabled)
> +	if (no_iommu || dmar_disabled) {
> +		/*
> +		 * Make sure the IOMMUs are switched off, even when we
> +		 * boot into a kexec kernel and the previous kernel left
> +		 * them enabled
> +		 */
> +		intel_disable_iommus();
>  		goto out_free_dmar;
> +	}
>  
>  	if (list_empty(&dmar_rmrr_units))
>  		pr_info("No RMRR found\n");
> -- 
> 1.9.1
> 
> _______________________________________________
> iommu mailing list
> iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org
> https://lists.linuxfoundation.org/mailman/listinfo/iommu

Re: [PATCH] iommu/vt-d: Make sure IOMMUs are off when intel_iommu=off

[Joerg Roedel]

Hi Baoquan,

On Wed, Apr 12, 2017 at 09:40:56AM +0800, Baoquan He wrote:
> Do you plan to merge this one as urgent?
> 
> There's bug created about this issue on rhel, it would be great if it
> can be put in next or merged so that we can back port it.

No, I am not sending this for v4.11, because this issue existed forever
and is no regression. I queued it for v4.12 and the commit-id in the
iommu-tree will be the same as upstream, if you need that for your
backport.


	Joerg

Re: [PATCH] iommu/vt-d: Make sure IOMMUs are off when intel_iommu=off

[Baoquan He]

On 04/12/17 at 11:54pm, Joerg Roedel wrote:
> Hi Baoquan,
> 
> On Wed, Apr 12, 2017 at 09:40:56AM +0800, Baoquan He wrote:
> > Do you plan to merge this one as urgent?
> > 
> > There's bug created about this issue on rhel, it would be great if it
> > can be put in next or merged so that we can back port it.
> 
> No, I am not sending this for v4.11, because this issue existed forever
> and is no regression. I queued it for v4.12 and the commit-id in the
> iommu-tree will be the same as upstream, if you need that for your
> backport.

Yeah, agree. The commit-id should be enough. Thanks a lot!