[PATCH 00/17] Fixes and Cleanups for the Intel VT-d driver

[PATCH 00/17] Fixes and Cleanups for the Intel VT-d driver

[Joerg Roedel]

Hey,

here are a couple of patches to fix the fall-out from the
patch set fixing the kdump faults with VT-d enabled. A few
important issues have been fixed:

	* Moved sleeping functions out of atomic sections,
	  mainly iremap_cache/iounmap.

	* Mark domain ids used in the old kernel as reserved
	  in the kdump kernel, so that any new domains don't
	  interfere with domains from the old kernel

	* Mark all IRT entries from the old kernel as
	  reservered in the new kernel, so that we don't
	  overwrite an entry which might still be in use by
	  a device

	* Only try to load anything from the old kernel when
	  we are in a kdump kernel. If we find the iommu
	  hardware enabled and we are not in a kdump kernel,
	  just disable the iommu and proceed with
	  initialization.

	* Fix a compile error

Besides that I also cleaned up the code to simplify it:

	* Move necessary initialization steps before the
	  root entry table and the irq remapping table is
	  programmed into hardware. This makes sure QI is
	  initialized so that we can flush the iommu caches
	  when hardware got the new tables.

	* Make the new root entry table and irq remapping
	  table visible to hardware immediatly after they
	  are created and loaded with the data from the old
	  kernel. This allows to remove the code to update
	  both, the old and the new version of the table at
	  the same time.

	* Clean up log messages from the VT-d driver to have
	  a common prefix that can be grepped for.

	* Remove unused code

The changes have been tested by me an Baoquan He. I'll plan
to put them into the x86/vt-d branch for v4.2.

Regards,

	Joerg

Joerg Roedel (17):
  iommu/vt-d: Fix compile error when CONFIG_INTEL_IOMMU=n
  iommu/vt-d: Remove __iommu_save_to_oldmem() function
  iommu/vt-d: Make two functions static
  iommu/vt-d: Load old data structures only in kdump kernel
  iommu/vt-d: Mark root-entry present in set_root_entry
  iommu/vt-d: Rework loading of old root-entry table
  iommu/vt-d: Copy context-tables outside of spin_lock
  iommu/vt-d: Don't reuse domain-ids from old kernel
  iommu/vt-d: Clean up log messages in intel-iommu.c
  iommu/vt-d: Allocate irq remapping table bitmap with GFP_KERNEL
  iommu/vt-d: Move QI initialization to intel_setup_irq_remapping
  iommu/vt-d: Move EIM detection to intel_prepare_irq_remapping
  iommu/vt-d: Split up iommu_set_irq_remapping
  iommu/vt-d: Move kdump pointer intialization to __iommu_load_old_irte
  iommu/vt-d: Mark irt entries from old kernel as allocated
  iommu/vt-d: Copy old irte in intel_setup_irq_remapping
  iommu/vt-d: Remove update code for old ir-table

 drivers/iommu/dmar.c                |  26 +-
 drivers/iommu/intel-iommu.c         | 463 ++++++++++++++----------------------
 drivers/iommu/intel_irq_remapping.c | 263 +++++++++-----------
 include/linux/intel-iommu.h         |  17 +-
 4 files changed, 299 insertions(+), 470 deletions(-)

-- 
1.9.1

[PATCH 01/17] iommu/vt-d: Fix compile error when CONFIG_INTEL_IOMMU=n

[Joerg Roedel]

From: Joerg Roedel <jroedel@suse.de>

The patches for the Intel IOMMU driver to fix issues with
kdump introduced a compile error with:

CONFIG_DMAR_TABLE=y
CONFIG_INTEL_IOMMU=n
CONFIG_IRQ_REMAP=y

The error manifests as:

drivers/iommu/intel_irq_remapping.c: In function ‘modify_irte’:
drivers/iommu/intel_irq_remapping.c:149:11: error: ‘struct
intel_iommu’ has no member named ‘pre_enabled_ir’
  if (iommu->pre_enabled_ir)

Fix this by moving the declaration of the pre_enabled_ir
under #ifdef CONFIG_IRQ_REMAPPING instead of
CONFIG_INTEL_IOMMU.

Reported-by: Jim Davis <jim.epost@gmail.com>
Cc: Zhen-Hua Li <zhen-hual@hp.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
 include/linux/intel-iommu.h | 5 +++--
 1 file changed, 3 insertions(+), 2 deletions(-)

diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index d7a0b5d..a81ceee 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -344,8 +344,6 @@ struct intel_iommu {
 
 	/* whether translation is enabled prior to OS*/
 	u8		pre_enabled_trans;
-	/* whether interrupt remapping is enabled prior to OS*/
-	u8		pre_enabled_ir;
 
 	void __iomem	*root_entry_old_virt; /* mapped from old root entry */
 	unsigned long	root_entry_old_phys; /* root entry in old kernel */
@@ -357,6 +355,9 @@ struct intel_iommu {
 
 #ifdef CONFIG_IRQ_REMAP
 	struct ir_table *ir_table;	/* Interrupt remapping info */
+
+	/* whether interrupt remapping is enabled prior to OS*/
+	u8		pre_enabled_ir;
 #endif
 	struct device	*iommu_dev; /* IOMMU-sysfs device */
 	int		node;
-- 
1.9.1

[PATCH 02/17] iommu/vt-d: Remove __iommu_save_to_oldmem() function

[Joerg Roedel]

From: Joerg Roedel <jroedel@suse.de>

This function is unused and can be removed.

Tested-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
 drivers/iommu/intel-iommu.c | 36 ------------------------------------
 include/linux/intel-iommu.h |  2 --
 2 files changed, 38 deletions(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 60d5491..a061c3f 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -4978,42 +4978,6 @@ int __iommu_load_from_oldmem(void *to, unsigned long from, unsigned long size)
 }
 
 /*
- * Copy memory from a virtually-addressed area into a physically-addressed area
- */
-int __iommu_save_to_oldmem(unsigned long to, void *from, unsigned long size)
-{
-	unsigned long pfn;		/* Page Frame Number */
-	size_t csize = (size_t)size;	/* Num(bytes to copy) */
-	unsigned long offset;		/* Lower 12 bits of to */
-	void __iomem *virt_mem;
-	struct iommu_remapped_entry *mapped;
-
-	pfn = to >> VTD_PAGE_SHIFT;
-	offset = to & (~VTD_PAGE_MASK);
-
-	if (page_is_ram(pfn)) {
-		memcpy(pfn_to_kaddr(pfn) + offset, from, csize);
-	} else{
-		mapped = kzalloc(sizeof(struct iommu_remapped_entry),
-				GFP_KERNEL);
-		if (!mapped)
-			return -ENOMEM;
-
-		virt_mem = ioremap_cache((unsigned long)to, size);
-		if (!virt_mem) {
-			kfree(mapped);
-			return -ENOMEM;
-		}
-		memcpy(virt_mem, from, size);
-		mutex_lock(&__iommu_mem_list_lock);
-		mapped->mem = virt_mem;
-		list_add_tail(&mapped->list, &__iommu_remapped_mem);
-		mutex_unlock(&__iommu_mem_list_lock);
-	}
-	return size;
-}
-
-/*
  * Free the mapped memory for ioremap;
  */
 int __iommu_free_mapped_mem(void)
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index a81ceee..9526c28 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -393,8 +393,6 @@ extern const struct attribute_group *intel_iommu_groups[];
 
 extern int __iommu_load_from_oldmem(void *to, unsigned long from,
 					unsigned long size);
-extern int __iommu_save_to_oldmem(unsigned long to, void *from,
-					unsigned long size);
 extern int __iommu_free_mapped_mem(void);
 
 #endif
-- 
1.9.1

[PATCH 03/17] iommu/vt-d: Make two functions static

[Joerg Roedel]

From: Joerg Roedel <jroedel@suse.de>

These functions are only used in intel-iommu.c and can be
static:

	* __iommu_load_from_oldmem()
	* __iommu_free_mapped_mem()

Tested-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
 drivers/iommu/intel-iommu.c | 5 +++--
 include/linux/intel-iommu.h | 4 ----
 2 files changed, 3 insertions(+), 6 deletions(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index a061c3f..982b8c8 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -4942,7 +4942,8 @@ static struct context_entry *device_to_existing_context_entry(
 /*
  * Copy memory from a physically-addressed area into a virtually-addressed area
  */
-int __iommu_load_from_oldmem(void *to, unsigned long from, unsigned long size)
+static int __iommu_load_from_oldmem(void *to, unsigned long from,
+				    unsigned long size)
 {
 	unsigned long pfn;		/* Page Frame Number */
 	size_t csize = (size_t)size;	/* Num(bytes to copy) */
@@ -4980,7 +4981,7 @@ int __iommu_load_from_oldmem(void *to, unsigned long from, unsigned long size)
 /*
  * Free the mapped memory for ioremap;
  */
-int __iommu_free_mapped_mem(void)
+static int __iommu_free_mapped_mem(void)
 {
 	struct iommu_remapped_entry *mem_entry, *tmp;
 
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 9526c28..7e771c2 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -391,8 +391,4 @@ extern int dmar_ir_support(void);
 
 extern const struct attribute_group *intel_iommu_groups[];
 
-extern int __iommu_load_from_oldmem(void *to, unsigned long from,
-					unsigned long size);
-extern int __iommu_free_mapped_mem(void);
-
 #endif
-- 
1.9.1

[PATCH 04/17] iommu/vt-d: Load old data structures only in kdump kernel

[Joerg Roedel]

From: Joerg Roedel <jroedel@suse.de>

If we are not in a kdump kernel anybody could have left the
iommu enabled. We have no reason at all to trust the data
the data structures in this case, so don't even try to
preserve anything.

Tested-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
 drivers/iommu/intel-iommu.c         | 14 ++++++++++++++
 drivers/iommu/intel_irq_remapping.c | 12 +++++++-----
 2 files changed, 21 insertions(+), 5 deletions(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 982b8c8..d66bec6 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -40,6 +40,7 @@
 #include <linux/pci-ats.h>
 #include <linux/memblock.h>
 #include <linux/dma-contiguous.h>
+#include <linux/crash_dump.h>
 #include <asm/irq_remapping.h>
 #include <asm/cacheflush.h>
 #include <asm/iommu.h>
@@ -2943,6 +2944,19 @@ static int __init init_dmars(void)
 			goto free_iommu;
 
 		iommu_check_pre_te_status(iommu);
+
+		/*
+		 * We don't even try to preserve anything when we are not in a
+		 * kdump kernel.
+		 */
+		if (!is_kdump_kernel() && iommu->pre_enabled_trans) {
+			iommu_disable_translation(iommu);
+			iommu->pre_enabled_trans = 0;
+			g_translation_pre_enabled = 0;
+			pr_warn("Translation was enabled for %s but we are not in kdump mode\n",
+					iommu->name);
+		}
+
 		if (iommu->pre_enabled_trans) {
 			pr_info("IOMMU Copying translate tables from panicked kernel\n");
 			ret = intel_iommu_load_translation_tables(iommu);
diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index 46d80ad..c3f0686 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -8,6 +8,7 @@
 #include <linux/irq.h>
 #include <linux/intel-iommu.h>
 #include <linux/acpi.h>
+#include <linux/crash_dump.h>
 #include <asm/io_apic.h>
 #include <asm/smp.h>
 #include <asm/cpu.h>
@@ -666,11 +667,12 @@ static int __init intel_enable_irq_remapping(void)
 
 		iommu_check_pre_ir_status(iommu);
 
-		/*
-		 * Here we do not disable intr remapping,
-		 * if already enabled prior to OS handover.
-		 */
-		/* iommu_disable_irq_remapping(iommu); */
+		if (!is_kdump_kernel() && iommu->pre_enabled_ir) {
+			iommu_disable_irq_remapping(iommu);
+			iommu->pre_enabled_ir = 0;
+			pr_warn("IRQ remapping was enabled on %s but we are not in kdump mode\n",
+				iommu->name);
+		}
 
 		dmar_disable_qi(iommu);
 	}
-- 
1.9.1

[PATCH 05/17] iommu/vt-d: Mark root-entry present in set_root_entry

[Joerg Roedel]

From: Joerg Roedel <jroedel@suse.de>

When we do no memcpy from the old to the new kernel for the
root entry table, we have to set the present bit ourself.

Tested-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
 drivers/iommu/intel-iommu.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index d66bec6..cda0901 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -198,8 +198,7 @@ static inline bool root_present(struct root_entry *root)
 
 static inline void set_root_value(struct root_entry *root, unsigned long value)
 {
-	root->lo &= ~VTD_PAGE_MASK;
-	root->lo |= value & VTD_PAGE_MASK;
+	root->lo = (value & VTD_PAGE_MASK) | 1;
 }
 
 static inline struct context_entry *
-- 
1.9.1

[PATCH 06/17] iommu/vt-d: Rework loading of old root-entry table

[Joerg Roedel]

From: Joerg Roedel <jroedel@suse.de>

Instead of keeping pointers to the old data structures
around forever, just map the old root entry table to copy
the context entries and make the new table immidiatly
visible to the hardware.

Tested-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
 drivers/iommu/intel-iommu.c | 216 +++++++++++++++-----------------------------
 include/linux/intel-iommu.h |   3 -
 2 files changed, 74 insertions(+), 145 deletions(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index cda0901..c0b72e8 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -371,10 +371,6 @@ static struct context_entry *device_to_existing_context_entry(
 				struct intel_iommu *iommu,
 				u8 bus, u8 devfn);
 
-static void __iommu_load_old_root_entry(struct intel_iommu *iommu);
-
-static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index);
-
 /*
  * A structure used to store the address allocated by ioremap();
  * The we need to call iounmap() to free them out of spin_lock_irqsave/unlock;
@@ -392,7 +388,8 @@ static DEFINE_MUTEX(__iommu_mem_list_lock);
  * ------------------------------------------------------------------------
  */
 
-static int copy_root_entry_table(struct intel_iommu *iommu);
+static int copy_root_entry_table(struct intel_iommu *iommu,
+				 struct root_entry *old_re);
 
 static int intel_iommu_load_translation_tables(struct intel_iommu *iommu);
 
@@ -798,9 +795,6 @@ static inline struct context_entry *iommu_context_addr(struct intel_iommu *iommu
 		phy_addr = virt_to_phys((void *)context);
 		*entry = phy_addr | 1;
 		__iommu_flush_cache(iommu, entry, sizeof(*entry));
-
-		if (iommu->pre_enabled_trans)
-			__iommu_update_old_root_entry(iommu, bus);
 	}
 	return &context[devfn];
 }
@@ -918,12 +912,6 @@ static void free_context_table(struct intel_iommu *iommu)
 
 	}
 
-	if (iommu->pre_enabled_trans) {
-		iommu->root_entry_old_phys = 0;
-		root = iommu->root_entry_old_virt;
-		iommu->root_entry_old_virt = NULL;
-	}
-
 	free_pgtable_page(iommu->root_entry);
 	iommu->root_entry = NULL;
 
@@ -2942,6 +2930,20 @@ static int __init init_dmars(void)
 		if (ret)
 			goto free_iommu;
 
+		if (!ecap_pass_through(iommu->ecap))
+			hw_pass_through = 0;
+
+		intel_iommu_init_qi(iommu);
+
+		/*
+		 * TBD:
+		 * We could share the same root & context tables
+		 * among all IOMMU's. Need to Split it later.
+		 */
+		ret = iommu_alloc_root_entry(iommu);
+		if (ret)
+			goto free_iommu;
+
 		iommu_check_pre_te_status(iommu);
 
 		/*
@@ -2957,35 +2959,43 @@ static int __init init_dmars(void)
 		}
 
 		if (iommu->pre_enabled_trans) {
-			pr_info("IOMMU Copying translate tables from panicked kernel\n");
 			ret = intel_iommu_load_translation_tables(iommu);
 			if (ret) {
-				pr_err("IOMMU: Copy translate tables failed\n");
+				/*
+				 * We found the IOMMU with translation
+				 * enabled - but failed to copy over the
+				 * old root-entry table. Try to proceed
+				 * by disabling translation now and
+				 * allocating a clean root-entry table.
+				 * This might cause DMAR faults, but
+				 * probably the dump will still succeed.
+				 */
+				pr_err("Failed to copy translation tables from previous kernel for %s\n",
+						iommu->name);
 
-				/* Best to stop trying */
-				goto free_iommu;
+				iommu_disable_translation(iommu);
+				iommu->pre_enabled_trans = 0;
+			} else {
+				pr_info("Copied translation tables from previous kernel for %s\n",
+					iommu->name);
 			}
-			pr_info("IOMMU: root_cache:0x%12.12llx phys:0x%12.12llx\n",
-				(u64)iommu->root_entry,
-				(u64)iommu->root_entry_old_phys);
-		} else {
-			/*
-			 * TBD:
-			 * we could share the same root & context tables
-			 * among all IOMMU's. Need to Split it later.
-			 */
-			ret = iommu_alloc_root_entry(iommu);
-			if (ret)
-				goto free_iommu;
 		}
 
-		if (!ecap_pass_through(iommu->ecap))
-			hw_pass_through = 0;
+		/*
+		 * Make the new root entry visible. If IOMMU was
+		 * disabled before we are safe because translation is
+		 * not enabled yet, so any DMA access to RMRR regions
+		 * still works.
+		 * If we are in a kdump kernel and IOMMU was enabled
+		 * before, we must make sure that the hardware uses the
+		 * new root entry table from now on.
+		 */
+		iommu_flush_write_buffer(iommu);
+		iommu_set_root_entry(iommu);
+		iommu->flush.flush_context(iommu, 0, 0, 0, DMA_CCMD_GLOBAL_INVL);
+		iommu->flush.flush_iotlb(iommu, 0, 0, 0, DMA_TLB_GLOBAL_FLUSH);
 	}
 
-	for_each_active_iommu(iommu, drhd)
-		intel_iommu_init_qi(iommu);
-
 	if (iommu_pass_through)
 		iommu_identity_mapping |= IDENTMAP_ALL;
 
@@ -3063,21 +3073,14 @@ skip_new_domains_for_si_rmrr_isa:;
 			continue;
 		}
 
-		iommu_flush_write_buffer(iommu);
-
 		ret = dmar_set_interrupt(iommu);
 		if (ret)
 			goto free_iommu;
 
-		iommu_set_root_entry(iommu);
-
 		iommu->flush.flush_context(iommu, 0, 0, 0, DMA_CCMD_GLOBAL_INVL);
 		iommu->flush.flush_iotlb(iommu, 0, 0, 0, DMA_TLB_GLOBAL_FLUSH);
 
-		if (iommu->pre_enabled_trans) {
-			if (!(iommu->gcmd & DMA_GCMD_TE))
-				iommu_enable_translation(iommu);
-		} else
+		if (!iommu->pre_enabled_trans)
 			iommu_enable_translation(iommu);
 
 		iommu_disable_protect_mem_regions(iommu);
@@ -5009,94 +5012,37 @@ static int __iommu_free_mapped_mem(void)
 }
 
 /*
- * Load the old root entry table to new root entry table.
- */
-static void __iommu_load_old_root_entry(struct intel_iommu *iommu)
-{
-	if ((!iommu)
-		|| (!iommu->root_entry)
-		|| (!iommu->root_entry_old_virt)
-		|| (!iommu->root_entry_old_phys))
-		return;
-	memcpy(iommu->root_entry, iommu->root_entry_old_virt, PAGE_SIZE);
-
-	__iommu_flush_cache(iommu, iommu->root_entry, PAGE_SIZE);
-}
-
-/*
- * When the data in new root entry table is changed, this function
- * must be called to save the updated data to old root entry table.
- */
-static void __iommu_update_old_root_entry(struct intel_iommu *iommu, int index)
-{
-	u8 start;
-	unsigned long size;
-	void __iomem *to;
-	void *from;
-
-	if ((!iommu)
-		|| (!iommu->root_entry)
-		|| (!iommu->root_entry_old_virt)
-		|| (!iommu->root_entry_old_phys))
-		return;
-
-	if (index < -1 || index >= ROOT_ENTRY_NR)
-		return;
-
-	if (index == -1) {
-		start = 0;
-		size = ROOT_ENTRY_NR * sizeof(struct root_entry);
-	} else {
-		start = index * sizeof(struct root_entry);
-		size = sizeof(struct root_entry);
-	}
-	to = iommu->root_entry_old_virt;
-	from = iommu->root_entry;
-	memcpy(to + start, from + start, size);
-
-	__iommu_flush_cache(iommu, to + start, size);
-}
-
-/*
  * Load root entry tables from old kernel.
  */
-static int copy_root_entry_table(struct intel_iommu *iommu)
+static int copy_root_entry_table(struct intel_iommu *iommu,
+				 struct root_entry *old_re)
 {
-	u32 bus;				/* Index: root-entry-table */
-	struct root_entry  *re;			/* Virt(iterator: new table) */
-	unsigned long context_old_phys;		/* Phys(context table entry) */
-	struct context_entry *context_new_virt;	/* Virt(new context_entry) */
+	struct context_entry *context_new_virt;
+	unsigned long context_old_phys;
+	u32 bus;
 
-	/*
-	 * A new root entry table has been allocated ,
-	 * we need copy re from old kernel to the new allocated one.
-	 */
-
-	if (!iommu->root_entry_old_phys)
-		return -ENOMEM;
-
-	for (bus = 0, re = iommu->root_entry; bus < 256; bus += 1, re += 1) {
-		if (!root_present(re))
+	for (bus = 0; bus < 256; bus++, old_re++) {
+		if (!root_present(old_re))
 			continue;
 
-		context_old_phys = get_context_phys_from_root(re);
+		context_old_phys = get_context_phys_from_root(old_re);
 
 		if (!context_old_phys)
 			continue;
 
-		context_new_virt =
-			(struct context_entry *)alloc_pgtable_page(iommu->node);
+		context_new_virt = alloc_pgtable_page(iommu->node);
 
 		if (!context_new_virt)
 			return -ENOMEM;
 
 		__iommu_load_from_oldmem(context_new_virt,
-					context_old_phys,
-					VTD_PAGE_SIZE);
+					 context_old_phys,
+					 VTD_PAGE_SIZE);
 
 		__iommu_flush_cache(iommu, context_new_virt, VTD_PAGE_SIZE);
 
-		set_root_value(re, virt_to_phys(context_new_virt));
+		set_root_value(&iommu->root_entry[bus],
+			       virt_to_phys(context_new_virt));
 	}
 
 	return 0;
@@ -5108,49 +5054,35 @@ static int copy_root_entry_table(struct intel_iommu *iommu)
  */
 static int intel_iommu_load_translation_tables(struct intel_iommu *iommu)
 {
-	unsigned long long q;		/* quadword scratch */
-	int ret = 0;			/* Integer return code */
+	struct root_entry *old_re;
+	phys_addr_t old_re_phys;
 	unsigned long flags;
+	int ret = 0;
+	u64 q;
 
 	q = dmar_readq(iommu->reg + DMAR_RTADDR_REG);
 	if (!q)
 		return -1;
 
-	spin_lock_irqsave(&iommu->lock, flags);
-
-	/* Load the root-entry table from the old kernel
-	 * foreach context_entry_table in root_entry
-	 *   Copy each entry table from old kernel
-	 */
-	if (!iommu->root_entry) {
-		iommu->root_entry =
-			(struct root_entry *)alloc_pgtable_page(iommu->node);
-		if (!iommu->root_entry) {
-			spin_unlock_irqrestore(&iommu->lock, flags);
-			return -ENOMEM;
-		}
-	}
-
-	iommu->root_entry_old_phys = q & VTD_PAGE_MASK;
-	if (!iommu->root_entry_old_phys) {
-		pr_err("Could not read old root entry address.");
-		return -1;
-	}
+	old_re_phys = q & VTD_PAGE_MASK;
+	if (!old_re_phys)
+		return -ENODEV;
 
-	iommu->root_entry_old_virt = ioremap_cache(iommu->root_entry_old_phys,
-						VTD_PAGE_SIZE);
-	if (!iommu->root_entry_old_virt) {
-		pr_err("Could not map the old root entry.");
+	old_re = ioremap_cache(old_re_phys, PAGE_SIZE);
+	if (!old_re) {
+		pr_err("Could not map old root entry\n");
 		return -ENOMEM;
 	}
 
-	__iommu_load_old_root_entry(iommu);
-	ret = copy_root_entry_table(iommu);
+	spin_lock_irqsave(&iommu->lock, flags);
+
+	ret = copy_root_entry_table(iommu, old_re);
 	__iommu_flush_cache(iommu, iommu->root_entry, PAGE_SIZE);
-	__iommu_update_old_root_entry(iommu, -1);
 
 	spin_unlock_irqrestore(&iommu->lock, flags);
 
+	iounmap(old_re);
+
 	__iommu_free_mapped_mem();
 
 	return ret;
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 7e771c2..6c37de9 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -345,9 +345,6 @@ struct intel_iommu {
 	/* whether translation is enabled prior to OS*/
 	u8		pre_enabled_trans;
 
-	void __iomem	*root_entry_old_virt; /* mapped from old root entry */
-	unsigned long	root_entry_old_phys; /* root entry in old kernel */
-
 	struct iommu_flush flush;
 #endif
 	struct q_inval  *qi;            /* Queued invalidation info */
-- 
1.9.1

[PATCH 07/17] iommu/vt-d: Copy context-tables outside of spin_lock

[Joerg Roedel]

From: Joerg Roedel <jroedel@suse.de>

This allows to to ioremap_nocache and iounmap in the same
path, so we don't need to collect all pointers returned from
ioremap to free them after the spinlock is released.

Tested-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
 drivers/iommu/intel-iommu.c | 127 +++++++++++++++++++++++---------------------
 1 file changed, 66 insertions(+), 61 deletions(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index c0b72e8..2602b33 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -380,17 +380,11 @@ struct iommu_remapped_entry {
 	void __iomem *mem;
 };
 static LIST_HEAD(__iommu_remapped_mem);
-static DEFINE_MUTEX(__iommu_mem_list_lock);
 
-/* ========================================================================
+/*
  * Copy iommu translation tables from old kernel into new  kernel.
  * Entry to this set of functions is: intel_iommu_load_translation_tables()
- * ------------------------------------------------------------------------
  */
-
-static int copy_root_entry_table(struct intel_iommu *iommu,
-				 struct root_entry *old_re);
-
 static int intel_iommu_load_translation_tables(struct intel_iommu *iommu);
 
 static void unmap_device_dma(struct dmar_domain *domain,
@@ -4958,68 +4952,41 @@ static struct context_entry *device_to_existing_context_entry(
 /*
  * Copy memory from a physically-addressed area into a virtually-addressed area
  */
-static int __iommu_load_from_oldmem(void *to, unsigned long from,
-				    unsigned long size)
+static int copy_from_oldmem_phys(void *to, phys_addr_t from, size_t size)
 {
-	unsigned long pfn;		/* Page Frame Number */
-	size_t csize = (size_t)size;	/* Num(bytes to copy) */
-	unsigned long offset;		/* Lower 12 bits of to */
 	void __iomem *virt_mem;
-	struct iommu_remapped_entry *mapped;
+	unsigned long offset;
+	unsigned long pfn;
 
-	pfn = from >> VTD_PAGE_SHIFT;
+	pfn    = from >> VTD_PAGE_SHIFT;
 	offset = from & (~VTD_PAGE_MASK);
 
 	if (page_is_ram(pfn)) {
-		memcpy(to, pfn_to_kaddr(pfn) + offset, csize);
-	} else{
-
-		mapped = kzalloc(sizeof(struct iommu_remapped_entry),
-				GFP_KERNEL);
-		if (!mapped)
-			return -ENOMEM;
-
+		memcpy(to, pfn_to_kaddr(pfn) + offset, size);
+	} else {
 		virt_mem = ioremap_cache((unsigned long)from, size);
-		if (!virt_mem) {
-			kfree(mapped);
+		if (!virt_mem)
 			return -ENOMEM;
-		}
+
 		memcpy(to, virt_mem, size);
 
-		mutex_lock(&__iommu_mem_list_lock);
-		mapped->mem = virt_mem;
-		list_add_tail(&mapped->list, &__iommu_remapped_mem);
-		mutex_unlock(&__iommu_mem_list_lock);
+		iounmap(virt_mem);
 	}
-	return size;
-}
 
-/*
- * Free the mapped memory for ioremap;
- */
-static int __iommu_free_mapped_mem(void)
-{
-	struct iommu_remapped_entry *mem_entry, *tmp;
-
-	mutex_lock(&__iommu_mem_list_lock);
-	list_for_each_entry_safe(mem_entry, tmp, &__iommu_remapped_mem, list) {
-		iounmap(mem_entry->mem);
-		list_del(&mem_entry->list);
-		kfree(mem_entry);
-	}
-	mutex_unlock(&__iommu_mem_list_lock);
-	return 0;
+	return size;
 }
 
 /*
- * Load root entry tables from old kernel.
+ * Load context entry tables from old kernel.
  */
-static int copy_root_entry_table(struct intel_iommu *iommu,
-				 struct root_entry *old_re)
+static int copy_context_tables(struct intel_iommu *iommu,
+			       struct context_entry **tbl,
+			       struct root_entry *old_re)
 {
 	struct context_entry *context_new_virt;
-	unsigned long context_old_phys;
+	phys_addr_t context_old_phys;
 	u32 bus;
+	int ret;
 
 	for (bus = 0; bus < 256; bus++, old_re++) {
 		if (!root_present(old_re))
@@ -5030,22 +4997,48 @@ static int copy_root_entry_table(struct intel_iommu *iommu,
 		if (!context_old_phys)
 			continue;
 
+		ret = -ENOMEM;
 		context_new_virt = alloc_pgtable_page(iommu->node);
-
 		if (!context_new_virt)
-			return -ENOMEM;
-
-		__iommu_load_from_oldmem(context_new_virt,
-					 context_old_phys,
-					 VTD_PAGE_SIZE);
+			goto out_err;
+
+		ret = copy_from_oldmem_phys(context_new_virt, context_old_phys,
+					    VTD_PAGE_SIZE);
+		if (ret != VTD_PAGE_SIZE) {
+			pr_err("Failed to copy context table for bus %d from physical address 0x%llx\n",
+			       bus, context_old_phys);
+			free_pgtable_page(context_new_virt);
+			continue;
+		}
 
 		__iommu_flush_cache(iommu, context_new_virt, VTD_PAGE_SIZE);
 
-		set_root_value(&iommu->root_entry[bus],
-			       virt_to_phys(context_new_virt));
+		tbl[bus] = context_new_virt;
 	}
 
 	return 0;
+
+out_err:
+	for (bus = 0; bus < 256; bus++) {
+		free_pgtable_page(tbl[bus]);
+		tbl[bus] = NULL;
+	}
+
+	return ret;
+}
+
+static void update_root_entry_table(struct intel_iommu *iommu,
+				    struct context_entry **tbl)
+{
+	struct root_entry *re = iommu->root_entry;
+	u32 bus;
+
+	for (bus = 0; bus < 256; bus++, re++) {
+		if (!tbl[bus])
+			continue;
+
+		set_root_value(re, virt_to_phys(tbl[bus]));
+	}
 }
 
 /*
@@ -5054,6 +5047,7 @@ static int copy_root_entry_table(struct intel_iommu *iommu,
  */
 static int intel_iommu_load_translation_tables(struct intel_iommu *iommu)
 {
+	struct context_entry **ctxt_tbls;
 	struct root_entry *old_re;
 	phys_addr_t old_re_phys;
 	unsigned long flags;
@@ -5074,16 +5068,27 @@ static int intel_iommu_load_translation_tables(struct intel_iommu *iommu)
 		return -ENOMEM;
 	}
 
+	ret = -ENOMEM;
+	ctxt_tbls = kzalloc(256 * sizeof(void *), GFP_KERNEL);
+	if (!ctxt_tbls)
+		goto out_unmap;
+
+	ret = copy_context_tables(iommu, ctxt_tbls, old_re);
+	if (ret)
+		goto out_free;
+
 	spin_lock_irqsave(&iommu->lock, flags);
 
-	ret = copy_root_entry_table(iommu, old_re);
+	update_root_entry_table(iommu, ctxt_tbls);
 	__iommu_flush_cache(iommu, iommu->root_entry, PAGE_SIZE);
 
 	spin_unlock_irqrestore(&iommu->lock, flags);
 
-	iounmap(old_re);
+out_free:
+	kfree(ctxt_tbls);
 
-	__iommu_free_mapped_mem();
+out_unmap:
+	iounmap(old_re);
 
 	return ret;
 }
-- 
1.9.1

[PATCH 08/17] iommu/vt-d: Don't reuse domain-ids from old kernel

[Joerg Roedel]

From: Joerg Roedel <jroedel@suse.de>

Change the context table copy code to copy context
entrys one by one and check whether they are present and
mark the used domain-id as reserved in the allocation
bitmap. This way the domain-id will not be reused by new
domains allocated in the kdump kernel.

Tested-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
 drivers/iommu/intel-iommu.c | 80 ++++++++++++++-------------------------------
 1 file changed, 25 insertions(+), 55 deletions(-)

diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 2602b33..82239e3 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -367,10 +367,6 @@ static inline int first_pte_in_page(struct dma_pte *pte)
  * do the same thing as crashdump kernel.
  */
 
-static struct context_entry *device_to_existing_context_entry(
-				struct intel_iommu *iommu,
-				u8 bus, u8 devfn);
-
 /*
  * A structure used to store the address allocated by ioremap();
  * The we need to call iounmap() to free them out of spin_lock_irqsave/unlock;
@@ -2337,7 +2333,6 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 	unsigned long flags;
 	u8 bus, devfn;
 	int did = -1;   /* Default to "no domain_id supplied" */
-	struct context_entry *ce = NULL;
 
 	domain = find_domain(dev);
 	if (domain)
@@ -2372,19 +2367,6 @@ static struct dmar_domain *get_domain_for_dev(struct device *dev, int gaw)
 	if (!domain)
 		return NULL;
 
-	if (iommu->pre_enabled_trans) {
-		/*
-		 * if this device had a did in the old kernel
-		 * use its values instead of generating new ones
-		 */
-		ce = device_to_existing_context_entry(iommu, bus, devfn);
-
-		if (ce) {
-			did = context_domain_id(ce);
-			gaw = agaw_to_width(context_address_width(ce));
-		}
-	}
-
 	domain->id = iommu_attach_domain_with_id(domain, iommu, did);
 	if (domain->id < 0) {
 		free_domain_mem(domain);
@@ -4931,49 +4913,37 @@ static void __init check_tylersburg_isoch(void)
 	       vtisochctrl);
 }
 
-static struct context_entry *device_to_existing_context_entry(
-				struct intel_iommu *iommu,
-				u8 bus, u8 devfn)
-{
-	struct root_entry *root;
-	struct context_entry *context;
-	struct context_entry *ret = NULL;
-	unsigned long flags;
-
-	spin_lock_irqsave(&iommu->lock, flags);
-	root = &iommu->root_entry[bus];
-	context = get_context_addr_from_root(root);
-	if (context && context_present(context+devfn))
-		ret = &context[devfn];
-	spin_unlock_irqrestore(&iommu->lock, flags);
-	return ret;
-}
-
 /*
- * Copy memory from a physically-addressed area into a virtually-addressed area
+ * Copy one context table
  */
-static int copy_from_oldmem_phys(void *to, phys_addr_t from, size_t size)
+static int copy_one_context_table(struct intel_iommu *iommu,
+				  struct context_entry *ctxt_tbl,
+				  phys_addr_t old_table_phys)
 {
-	void __iomem *virt_mem;
-	unsigned long offset;
-	unsigned long pfn;
+	struct context_entry __iomem *ctxt_tbl_old, ce;
+	int did, devfn;
 
-	pfn    = from >> VTD_PAGE_SHIFT;
-	offset = from & (~VTD_PAGE_MASK);
+	ctxt_tbl_old = ioremap_cache(old_table_phys, VTD_PAGE_SIZE);
+	if (!ctxt_tbl_old)
+		return -ENOMEM;
 
-	if (page_is_ram(pfn)) {
-		memcpy(to, pfn_to_kaddr(pfn) + offset, size);
-	} else {
-		virt_mem = ioremap_cache((unsigned long)from, size);
-		if (!virt_mem)
-			return -ENOMEM;
+	for (devfn = 0; devfn < 256; devfn++) {
+		memcpy_fromio(&ce, &ctxt_tbl_old[devfn],
+			      sizeof(struct context_entry));
 
-		memcpy(to, virt_mem, size);
+		if (!context_present(&ce))
+			continue;
+
+		did = context_domain_id(&ce);
+		if (did >=0 && did < cap_ndoms(iommu->cap))
+			set_bit(did, iommu->domain_ids);
 
-		iounmap(virt_mem);
+		ctxt_tbl[devfn] = ce;
 	}
 
-	return size;
+	iounmap(ctxt_tbl_old);
+
+	return 0;
 }
 
 /*
@@ -5002,9 +4972,9 @@ static int copy_context_tables(struct intel_iommu *iommu,
 		if (!context_new_virt)
 			goto out_err;
 
-		ret = copy_from_oldmem_phys(context_new_virt, context_old_phys,
-					    VTD_PAGE_SIZE);
-		if (ret != VTD_PAGE_SIZE) {
+		ret = copy_one_context_table(iommu, context_new_virt,
+					     context_old_phys);
+		if (ret) {
 			pr_err("Failed to copy context table for bus %d from physical address 0x%llx\n",
 			       bus, context_old_phys);
 			free_pgtable_page(context_new_virt);
-- 
1.9.1

[PATCH 09/17] iommu/vt-d: Clean up log messages in intel-iommu.c

[Joerg Roedel]

From: Joerg Roedel <jroedel@suse.de>

Give them a common prefix and fix some message spellings.
Also make the meaning of some messages more clear.

Tested-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
 drivers/iommu/dmar.c                | 26 ++++++++++-----------
 drivers/iommu/intel-iommu.c         | 46 ++++++++++++++++++-------------------
 drivers/iommu/intel_irq_remapping.c | 22 ++++++++++--------
 3 files changed, 47 insertions(+), 47 deletions(-)

diff --git a/drivers/iommu/dmar.c b/drivers/iommu/dmar.c
index 9847613..a2f50c5 100644
--- a/drivers/iommu/dmar.c
+++ b/drivers/iommu/dmar.c
@@ -26,7 +26,7 @@
  * These routines are used by both DMA-remapping and Interrupt-remapping
  */
 
-#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt /* has to precede printk.h */
+#define pr_fmt(fmt) "DMAR: " fmt /* has to precede printk.h */
 
 #include <linux/pci.h>
 #include <linux/dmar.h>
@@ -555,7 +555,7 @@ static int dmar_walk_remapping_entries(struct acpi_dmar_header *start,
 			break;
 		} else if (next > end) {
 			/* Avoid passing table end */
-			pr_warn(FW_BUG "record passes table end\n");
+			pr_warn(FW_BUG "Record passes table end\n");
 			ret = -EINVAL;
 			break;
 		}
@@ -623,7 +623,7 @@ parse_dmar_table(void)
 		return -ENODEV;
 
 	if (dmar->width < PAGE_SHIFT - 1) {
-		pr_warn("Invalid DMAR haw\n");
+		pr_warn("Invalid DMAR host address width\n");
 		return -EINVAL;
 	}
 
@@ -802,7 +802,7 @@ int __init dmar_table_init(void)
 		ret = parse_dmar_table();
 		if (ret < 0) {
 			if (ret != -ENODEV)
-				pr_info("parse DMAR table failure.\n");
+				pr_info("DMAR table parse failure.\n");
 		} else  if (list_empty(&dmar_drhd_units)) {
 			pr_info("No DMAR devices found\n");
 			ret = -ENODEV;
@@ -847,7 +847,7 @@ dmar_validate_one_drhd(struct acpi_dmar_header *entry, void *arg)
 	else
 		addr = early_ioremap(drhd->address, VTD_PAGE_SIZE);
 	if (!addr) {
-		pr_warn("IOMMU: can't validate: %llx\n", drhd->address);
+		pr_warn("Can't validate DRHD address: %llx\n", drhd->address);
 		return -EINVAL;
 	}
 
@@ -921,14 +921,14 @@ static int map_iommu(struct intel_iommu *iommu, u64 phys_addr)
 	iommu->reg_size = VTD_PAGE_SIZE;
 
 	if (!request_mem_region(iommu->reg_phys, iommu->reg_size, iommu->name)) {
-		pr_err("IOMMU: can't reserve memory\n");
+		pr_err("Can't reserve MMIO register region\n");
 		err = -EBUSY;
 		goto out;
 	}
 
 	iommu->reg = ioremap(iommu->reg_phys, iommu->reg_size);
 	if (!iommu->reg) {
-		pr_err("IOMMU: can't map the region\n");
+		pr_err("Can't map MMIO registers\n");
 		err = -ENOMEM;
 		goto release;
 	}
@@ -952,13 +952,13 @@ static int map_iommu(struct intel_iommu *iommu, u64 phys_addr)
 		iommu->reg_size = map_size;
 		if (!request_mem_region(iommu->reg_phys, iommu->reg_size,
 					iommu->name)) {
-			pr_err("IOMMU: can't reserve memory\n");
+			pr_err("Can't reserve MMIO register region\n");
 			err = -EBUSY;
 			goto out;
 		}
 		iommu->reg = ioremap(iommu->reg_phys, iommu->reg_size);
 		if (!iommu->reg) {
-			pr_err("IOMMU: can't map the region\n");
+			pr_err("Can't map MMIO registers\n");
 			err = -ENOMEM;
 			goto release;
 		}
@@ -1014,14 +1014,14 @@ static int alloc_iommu(struct dmar_drhd_unit *drhd)
 		return -ENOMEM;
 
 	if (dmar_alloc_seq_id(iommu) < 0) {
-		pr_err("IOMMU: failed to allocate seq_id\n");
+		pr_err("Failed to allocate seq_id\n");
 		err = -ENOSPC;
 		goto error;
 	}
 
 	err = map_iommu(iommu, drhd->reg_base_addr);
 	if (err) {
-		pr_err("IOMMU: failed to map %s\n", iommu->name);
+		pr_err("Failed to map %s\n", iommu->name);
 		goto error_free_seq_id;
 	}
 
@@ -1644,7 +1644,7 @@ int dmar_set_interrupt(struct intel_iommu *iommu)
 
 	irq = dmar_alloc_hwirq();
 	if (irq <= 0) {
-		pr_err("IOMMU: no free vectors\n");
+		pr_err("No free irq vectors\n");
 		return -EINVAL;
 	}
 
@@ -1661,7 +1661,7 @@ int dmar_set_interrupt(struct intel_iommu *iommu)
 
 	ret = request_irq(irq, dmar_fault, IRQF_NO_THREAD, iommu->name, iommu);
 	if (ret)
-		pr_err("IOMMU: can't request irq\n");
+		pr_err("Can't request irq\n");
 	return ret;
 }
 
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 82239e3..49065a9 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -17,6 +17,8 @@
  *          Fenghua Yu <fenghua.yu@intel.com>
  */
 
+#define pr_fmt(fmt)	"DMAR: " fmt
+
 #include <linux/init.h>
 #include <linux/bitmap.h>
 #include <linux/debugfs.h>
@@ -1214,8 +1216,7 @@ static int iommu_alloc_root_entry(struct intel_iommu *iommu)
 
 	root = (struct root_entry *)alloc_pgtable_page(iommu->node);
 	if (!root) {
-		pr_err("IOMMU: allocating root entry for %s failed\n",
-			iommu->name);
+		pr_err("allocating root entry for %s failed\n", iommu->name);
 		return -ENOMEM;
 	}
 
@@ -1352,9 +1353,9 @@ static void __iommu_flush_iotlb(struct intel_iommu *iommu, u16 did,
 
 	/* check IOTLB invalidation granularity */
 	if (DMA_TLB_IAIG(val) == 0)
-		printk(KERN_ERR"IOMMU: flush IOTLB failed\n");
+		pr_err("flush IOTLB failed\n");
 	if (DMA_TLB_IAIG(val) != DMA_TLB_IIRG(type))
-		pr_debug("IOMMU: tlb flush request %Lx, actual %Lx\n",
+		pr_debug("tlb flush request %Lx, actual %Lx\n",
 			(unsigned long long)DMA_TLB_IIRG(type),
 			(unsigned long long)DMA_TLB_IAIG(val));
 }
@@ -1525,7 +1526,7 @@ static int iommu_init_domains(struct intel_iommu *iommu)
 	unsigned long nlongs;
 
 	ndomains = cap_ndoms(iommu->cap);
-	pr_debug("IOMMU%d: Number of Domains supported <%ld>\n",
+	pr_debug("Number of Domains supported iommu[%d] <%ld>\n",
 		 iommu->seq_id, ndomains);
 	nlongs = BITS_TO_LONGS(ndomains);
 
@@ -1536,14 +1537,14 @@ static int iommu_init_domains(struct intel_iommu *iommu)
 	 */
 	iommu->domain_ids = kcalloc(nlongs, sizeof(unsigned long), GFP_KERNEL);
 	if (!iommu->domain_ids) {
-		pr_err("IOMMU%d: allocating domain id array failed\n",
+		pr_err("Allocating domain id array for iommu[%d] failed\n",
 		       iommu->seq_id);
 		return -ENOMEM;
 	}
 	iommu->domains = kcalloc(ndomains, sizeof(struct dmar_domain *),
 			GFP_KERNEL);
 	if (!iommu->domains) {
-		pr_err("IOMMU%d: allocating domain array failed\n",
+		pr_err("Allocating domain array for iommu[%d] failed\n",
 		       iommu->seq_id);
 		kfree(iommu->domain_ids);
 		iommu->domain_ids = NULL;
@@ -1649,7 +1650,7 @@ static int iommu_attach_domain(struct dmar_domain *domain,
 	num = __iommu_attach_domain(domain, iommu);
 	spin_unlock_irqrestore(&iommu->lock, flags);
 	if (num < 0)
-		pr_err("IOMMU: no free domain ids\n");
+		pr_err("No free domain ids\n");
 
 	return num;
 }
@@ -1814,7 +1815,7 @@ static int domain_init(struct dmar_domain *domain, int guest_width)
 	sagaw = cap_sagaw(iommu->cap);
 	if (!test_bit(agaw, &sagaw)) {
 		/* hardware doesn't support it, choose a bigger one */
-		pr_debug("IOMMU: hardware doesn't support agaw %d\n", agaw);
+		pr_debug("Hardware doesn't support agaw %d\n", agaw);
 		agaw = find_next_bit(&sagaw, 5, agaw);
 		if (agaw >= 5)
 			return -ENODEV;
@@ -1890,7 +1891,7 @@ static int domain_context_mapping_one(struct dmar_domain *domain,
 	struct device_domain_info *info = NULL;
 
 	pr_debug("Set context mapping for %02x:%02x.%d\n",
-		bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
+		 bus, PCI_SLOT(devfn), PCI_FUNC(devfn));
 
 	BUG_ON(!domain->pgd);
 	BUG_ON(translation != CONTEXT_TT_PASS_THROUGH &&
@@ -1915,7 +1916,7 @@ static int domain_context_mapping_one(struct dmar_domain *domain,
 			id = iommu_attach_vm_domain(domain, iommu);
 			if (id < 0) {
 				spin_unlock_irqrestore(&iommu->lock, flags);
-				pr_err("IOMMU: no free domain ids\n");
+				pr_err("No free domain ids\n");
 				return -EFAULT;
 			}
 		}
@@ -2564,7 +2565,7 @@ static int __init si_domain_init(int hw)
 		return -EFAULT;
 	}
 
-	pr_debug("IOMMU: identity mapping domain is domain %d\n",
+	pr_debug("Identity mapping domain is domain %d\n",
 		 si_domain->id);
 
 	if (hw)
@@ -2764,7 +2765,7 @@ static int __init dev_prepare_static_identity_mapping(struct device *dev, int hw
 				  hw ? CONTEXT_TT_PASS_THROUGH :
 				       CONTEXT_TT_MULTI_LEVEL);
 	if (!ret)
-		pr_info("IOMMU: %s identity mapping for device %s\n",
+		pr_info("%s identity mapping for device %s\n",
 			hw ? "hardware" : "software", dev_name(dev));
 	else if (ret == -ENODEV)
 		/* device not associated with an iommu */
@@ -2842,12 +2843,11 @@ static void intel_iommu_init_qi(struct intel_iommu *iommu)
 		 */
 		iommu->flush.flush_context = __iommu_flush_context;
 		iommu->flush.flush_iotlb = __iommu_flush_iotlb;
-		pr_info("IOMMU: %s using Register based invalidation\n",
-			iommu->name);
+		pr_info("%s using Register based invalidation\n", iommu->name);
 	} else {
 		iommu->flush.flush_context = qi_flush_context;
 		iommu->flush.flush_iotlb = qi_flush_iotlb;
-		pr_info("IOMMU: %s using Queued invalidation\n", iommu->name);
+		pr_info("%s using Queued invalidation\n", iommu->name);
 	}
 }
 
@@ -3982,20 +3982,18 @@ static int intel_iommu_add(struct dmar_drhd_unit *dmaru)
 		return 0;
 
 	if (hw_pass_through && !ecap_pass_through(iommu->ecap)) {
-		pr_warn("IOMMU: %s doesn't support hardware pass through.\n",
+		pr_warn("%s doesn't support hardware pass through.\n",
 			iommu->name);
 		return -ENXIO;
 	}
 	if (!ecap_sc_support(iommu->ecap) &&
 	    domain_update_iommu_snooping(iommu)) {
-		pr_warn("IOMMU: %s doesn't support snooping.\n",
-			iommu->name);
+		pr_warn("%s doesn't support snooping.\n", iommu->name);
 		return -ENXIO;
 	}
 	sp = domain_update_iommu_superpage(iommu) - 1;
 	if (sp >= 0 && !(cap_super_page_val(iommu->cap) & (1 << sp))) {
-		pr_warn("IOMMU: %s doesn't support large page.\n",
-			iommu->name);
+		pr_warn("%s doesn't support large page.\n", iommu->name);
 		return -ENXIO;
 	}
 
@@ -4225,7 +4223,7 @@ static int intel_iommu_memory_notifier(struct notifier_block *nb,
 		start = mhp->start_pfn << PAGE_SHIFT;
 		end = ((mhp->start_pfn + mhp->nr_pages) << PAGE_SHIFT) - 1;
 		if (iommu_domain_identity_map(si_domain, start, end)) {
-			pr_warn("dmar: failed to build identity map for [%llx-%llx]\n",
+			pr_warn("Failed to build identity map for [%llx-%llx]\n",
 				start, end);
 			return NOTIFY_BAD;
 		}
@@ -4243,7 +4241,7 @@ static int intel_iommu_memory_notifier(struct notifier_block *nb,
 
 			iova = find_iova(&si_domain->iovad, start_vpfn);
 			if (iova == NULL) {
-				pr_debug("dmar: failed get IOVA for PFN %lx\n",
+				pr_debug("Failed get IOVA for PFN %lx\n",
 					 start_vpfn);
 				break;
 			}
@@ -4251,7 +4249,7 @@ static int intel_iommu_memory_notifier(struct notifier_block *nb,
 			iova = split_and_remove_iova(&si_domain->iovad, iova,
 						     start_vpfn, last_vpfn);
 			if (iova == NULL) {
-				pr_warn("dmar: failed to split IOVA PFN [%lx-%lx]\n",
+				pr_warn("Failed to split IOVA PFN [%lx-%lx]\n",
 					start_vpfn, last_vpfn);
 				return NOTIFY_BAD;
 			}
diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index c3f0686..34667d5 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -1,3 +1,6 @@
+
+#define pr_fmt(fmt)	"DMAR-IR: " fmt
+
 #include <linux/interrupt.h>
 #include <linux/dmar.h>
 #include <linux/spinlock.h>
@@ -105,8 +108,7 @@ static int alloc_irte(struct intel_iommu *iommu, int irq, u16 count)
 	}
 
 	if (mask > ecap_max_handle_mask(iommu->ecap)) {
-		printk(KERN_ERR
-		       "Requested mask %x exceeds the max invalidation handle"
+		pr_err("Requested mask %x exceeds the max invalidation handle"
 		       " mask value %Lx\n", mask,
 		       ecap_max_handle_mask(iommu->ecap));
 		return -1;
@@ -116,7 +118,7 @@ static int alloc_irte(struct intel_iommu *iommu, int irq, u16 count)
 	index = bitmap_find_free_region(table->bitmap,
 					INTR_REMAP_TABLE_ENTRIES, mask);
 	if (index < 0) {
-		pr_warn("IR%d: can't allocate an IRTE\n", iommu->seq_id);
+		pr_warn("Can't allocate an IRTE for IR[%d]\n", iommu->seq_id);
 	} else {
 		cfg->remapped = 1;
 		irq_iommu->iommu = iommu;
@@ -345,7 +347,7 @@ static int set_ioapic_sid(struct irte *irte, int apic)
 	up_read(&dmar_global_lock);
 
 	if (sid == 0) {
-		pr_warning("Failed to set source-id of IOAPIC (%d)\n", apic);
+		pr_warn("Failed to set source-id of IOAPIC (%d)\n", apic);
 		return -1;
 	}
 
@@ -372,7 +374,7 @@ static int set_hpet_sid(struct irte *irte, u8 id)
 	up_read(&dmar_global_lock);
 
 	if (sid == 0) {
-		pr_warning("Failed to set source-id of HPET block (%d)\n", id);
+		pr_warn("Failed to set source-id of HPET block (%d)\n", id);
 		return -1;
 	}
 
@@ -502,15 +504,15 @@ static int intel_setup_irq_remapping(struct intel_iommu *iommu)
 				 INTR_REMAP_PAGE_ORDER);
 
 	if (!pages) {
-		pr_err("IR%d: failed to allocate pages of order %d\n",
-		       iommu->seq_id, INTR_REMAP_PAGE_ORDER);
+		pr_err("Failed to allocate pages of order %d for IR[%d]\n",
+		       INTR_REMAP_PAGE_ORDER, iommu->seq_id);
 		goto out_free_table;
 	}
 
 	bitmap = kcalloc(BITS_TO_LONGS(INTR_REMAP_TABLE_ENTRIES),
 			 sizeof(long), GFP_ATOMIC);
 	if (bitmap == NULL) {
-		pr_err("IR%d: failed to allocate bitmap\n", iommu->seq_id);
+		pr_err("Failed to allocate bitmap for IR[%d]\n", iommu->seq_id);
 		goto out_free_pages;
 	}
 
@@ -592,7 +594,7 @@ static void __init intel_cleanup_irq_remapping(void)
 	}
 
 	if (x2apic_supported())
-		pr_warn("Failed to enable irq remapping.  You are vulnerable to irq-injection attacks.\n");
+		pr_warn("Failed to enable irq remapping. You are vulnerable to irq-injection attacks.\n");
 }
 
 static int __init intel_prepare_irq_remapping(void)
@@ -1385,7 +1387,7 @@ static void iommu_check_pre_ir_status(struct intel_iommu *iommu)
 
 	sts = readl(iommu->reg + DMAR_GSTS_REG);
 	if (sts & DMA_GSTS_IRES) {
-		pr_info("IR is enabled prior to OS.\n");
+		pr_info("IRQ remapping is enabled prior to OS.\n");
 		iommu->pre_enabled_ir = 1;
 	}
 }
-- 
1.9.1

[PATCH 10/17] iommu/vt-d: Allocate irq remapping table bitmap with GFP_KERNEL

[Joerg Roedel]

From: Joerg Roedel <jroedel@suse.de>

This function does not run in atomic context, so no reason
to do a GFP_ATOMIC allocation.

Tested-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
 drivers/iommu/intel_irq_remapping.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index 34667d5..d22518d 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -510,7 +510,7 @@ static int intel_setup_irq_remapping(struct intel_iommu *iommu)
 	}
 
 	bitmap = kcalloc(BITS_TO_LONGS(INTR_REMAP_TABLE_ENTRIES),
-			 sizeof(long), GFP_ATOMIC);
+			 sizeof(long), GFP_KERNEL | __GFP_ZERO);
 	if (bitmap == NULL) {
 		pr_err("Failed to allocate bitmap for IR[%d]\n", iommu->seq_id);
 		goto out_free_pages;
-- 
1.9.1

[PATCH 11/17] iommu/vt-d: Move QI initialization to intel_setup_irq_remapping

[Joerg Roedel]

From: Joerg Roedel <jroedel@suse.de>

This function needs a working QI later when kdump recovery
is moved to this function.

Tested-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
 drivers/iommu/intel_irq_remapping.c | 63 ++++++++++---------------------------
 1 file changed, 16 insertions(+), 47 deletions(-)

diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index d22518d..70e4955 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -519,6 +519,19 @@ static int intel_setup_irq_remapping(struct intel_iommu *iommu)
 	ir_table->base = page_address(pages);
 	ir_table->bitmap = bitmap;
 	iommu->ir_table = ir_table;
+
+	if (!iommu->qi) {
+		/*
+		 * Clear previous faults.
+		 */
+		dmar_fault(-1, iommu);
+		dmar_disable_qi(iommu);
+		if (dmar_enable_qi(iommu)) {
+			pr_err("Failed to enable queued invalidation\n");
+			goto out_free_pages;
+		}
+	}
+
 	return 0;
 
 out_free_pages:
@@ -655,18 +668,6 @@ static int __init intel_enable_irq_remapping(void)
 	}
 
 	for_each_iommu(iommu, drhd) {
-		/*
-		 * If the queued invalidation is already initialized,
-		 * shouldn't disable it.
-		 */
-		if (iommu->qi)
-			continue;
-
-		/*
-		 * Clear previous faults.
-		 */
-		dmar_fault(-1, iommu);
-
 		iommu_check_pre_ir_status(iommu);
 
 		if (!is_kdump_kernel() && iommu->pre_enabled_ir) {
@@ -675,8 +676,6 @@ static int __init intel_enable_irq_remapping(void)
 			pr_warn("IRQ remapping was enabled on %s but we are not in kdump mode\n",
 				iommu->name);
 		}
-
-		dmar_disable_qi(iommu);
 	}
 
 	/*
@@ -693,20 +692,6 @@ static int __init intel_enable_irq_remapping(void)
 		pr_info("Queued invalidation will be enabled to support x2apic and Intr-remapping.\n");
 
 	/*
-	 * Enable queued invalidation for all the DRHD's.
-	 */
-	for_each_iommu(iommu, drhd) {
-		int ret = dmar_enable_qi(iommu);
-
-		if (ret) {
-			printk(KERN_ERR "DRHD %Lx: failed to enable queued, "
-			       " invalidation, ecap %Lx, ret %d\n",
-			       drhd->reg_base_addr, iommu->ecap, ret);
-			goto error;
-		}
-	}
-
-	/*
 	 * Setup Interrupt-remapping for all the DRHD's now.
 	 */
 	for_each_iommu(iommu, drhd) {
@@ -1270,28 +1255,12 @@ static int dmar_ir_add(struct dmar_drhd_unit *dmaru, struct intel_iommu *iommu)
 	/* Setup Interrupt-remapping now. */
 	ret = intel_setup_irq_remapping(iommu);
 	if (ret) {
-		pr_err("DRHD %Lx: failed to allocate resource\n",
+		pr_err("DRHD %Lx: failed to setup IRQ remapping\n",
 		       iommu->reg_phys);
-		ir_remove_ioapic_hpet_scope(iommu);
-		return ret;
-	}
-
-	if (!iommu->qi) {
-		/* Clear previous faults. */
-		dmar_fault(-1, iommu);
-		iommu_disable_irq_remapping(iommu);
-		dmar_disable_qi(iommu);
-	}
-
-	/* Enable queued invalidation */
-	ret = dmar_enable_qi(iommu);
-	if (!ret) {
-		iommu_set_irq_remapping(iommu, eim);
-	} else {
-		pr_err("DRHD %Lx: failed to enable queued invalidation, ecap %Lx, ret %d\n",
-		       iommu->reg_phys, iommu->ecap, ret);
 		intel_teardown_irq_remapping(iommu);
 		ir_remove_ioapic_hpet_scope(iommu);
+	} else {
+		iommu_set_irq_remapping(iommu, eim);
 	}
 
 	return ret;
-- 
1.9.1

[PATCH 12/17] iommu/vt-d: Move EIM detection to intel_prepare_irq_remapping

[Joerg Roedel]

From: Joerg Roedel <jroedel@suse.de>

This needs to be set up before the ir-table can be
programmed to the iommu hardware register. This programming
will be moved to intel_setup_irq_remapping, so EIM needs to
be set up earlier too.

Tested-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
 drivers/iommu/intel_irq_remapping.c | 44 +++++++++++++++++--------------------
 1 file changed, 20 insertions(+), 24 deletions(-)

diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index 70e4955..35d73f6 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -637,10 +637,26 @@ static int __init intel_prepare_irq_remapping(void)
 		goto error;
 	}
 
+	if (x2apic_supported()) {
+		eim_mode = !dmar_x2apic_optout();
+		if (!eim_mode)
+			pr_info("BIOS set x2apic opt out bit. Use "
+				"'intremap=no_x2apic_optout' to enable x2apic\n");
+	}
+
 	/* First make sure all IOMMUs support IRQ remapping */
-	for_each_iommu(iommu, drhd)
+	for_each_iommu(iommu, drhd) {
 		if (!ecap_ir_support(iommu->ecap))
 			goto error;
+		if (eim_mode && !ecap_eim_support(iommu->ecap)) {
+			printk(KERN_INFO "DRHD %Lx: EIM not supported by DRHD, "
+			       " ecap %Lx\n", drhd->reg_base_addr, iommu->ecap);
+			eim_mode = 0;
+		}
+	}
+
+	if (eim_mode)
+		pr_info("Queued invalidation will be enabled to support x2apic and Intr-remapping.\n");
 
 	/* Do the allocations early */
 	for_each_iommu(iommu, drhd)
@@ -659,13 +675,6 @@ static int __init intel_enable_irq_remapping(void)
 	struct dmar_drhd_unit *drhd;
 	struct intel_iommu *iommu;
 	bool setup = false;
-	int eim = 0;
-
-	if (x2apic_supported()) {
-		eim = !dmar_x2apic_optout();
-		if (!eim)
-			pr_info("x2apic is disabled because BIOS sets x2apic opt out bit. You can use 'intremap=no_x2apic_optout' to override the BIOS setting.\n");
-	}
 
 	for_each_iommu(iommu, drhd) {
 		iommu_check_pre_ir_status(iommu);
@@ -679,19 +688,6 @@ static int __init intel_enable_irq_remapping(void)
 	}
 
 	/*
-	 * check for the Interrupt-remapping support
-	 */
-	for_each_iommu(iommu, drhd)
-		if (eim && !ecap_eim_support(iommu->ecap)) {
-			printk(KERN_INFO "DRHD %Lx: EIM not supported by DRHD, "
-			       " ecap %Lx\n", drhd->reg_base_addr, iommu->ecap);
-			eim = 0;
-		}
-	eim_mode = eim;
-	if (eim)
-		pr_info("Queued invalidation will be enabled to support x2apic and Intr-remapping.\n");
-
-	/*
 	 * Setup Interrupt-remapping for all the DRHD's now.
 	 */
 	for_each_iommu(iommu, drhd) {
@@ -705,7 +701,7 @@ static int __init intel_enable_irq_remapping(void)
 				INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
 			__iommu_load_old_irte(iommu);
 		} else
-			iommu_set_irq_remapping(iommu, eim);
+			iommu_set_irq_remapping(iommu, eim_mode);
 
 		setup = true;
 	}
@@ -722,9 +718,9 @@ static int __init intel_enable_irq_remapping(void)
 	 */
 	x86_io_apic_ops.print_entries = intel_ir_io_apic_print_entries;
 
-	pr_info("Enabled IRQ remapping in %s mode\n", eim ? "x2apic" : "xapic");
+	pr_info("Enabled IRQ remapping in %s mode\n", eim_mode ? "x2apic" : "xapic");
 
-	return eim ? IRQ_REMAP_X2APIC_MODE : IRQ_REMAP_XAPIC_MODE;
+	return eim_mode ? IRQ_REMAP_X2APIC_MODE : IRQ_REMAP_XAPIC_MODE;
 
 error:
 	intel_cleanup_irq_remapping();
-- 
1.9.1

[PATCH 13/17] iommu/vt-d: Split up iommu_set_irq_remapping

[Joerg Roedel]

From: Joerg Roedel <jroedel@suse.de>

Split out the part that enables irq remapping into the new
function iommu_enable_irq_remapping. This allows to program
the irq remapping table to the hardware without enabling irq
remapping.

Tested-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
 drivers/iommu/intel_irq_remapping.c | 14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)

diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index 35d73f6..c3d1e63 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -440,9 +440,9 @@ static int set_msi_sid(struct irte *irte, struct pci_dev *dev)
 
 static void iommu_set_irq_remapping(struct intel_iommu *iommu, int mode)
 {
+	unsigned long flags;
 	u64 addr;
 	u32 sts;
-	unsigned long flags;
 
 	addr = virt_to_phys((void *)iommu->ir_table->base);
 
@@ -463,6 +463,12 @@ static void iommu_set_irq_remapping(struct intel_iommu *iommu, int mode)
 	 * interrupt-remapping.
 	 */
 	qi_global_iec(iommu);
+}
+
+static void iommu_enable_irq_remapping(struct intel_iommu *iommu)
+{
+	unsigned long flags;
+	u32 sts;
 
 	raw_spin_lock_irqsave(&iommu->register_lock, flags);
 
@@ -700,8 +706,10 @@ static int __init intel_enable_irq_remapping(void)
 				iommu->ir_table->base_old_phys,
 				INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
 			__iommu_load_old_irte(iommu);
-		} else
+		} else {
 			iommu_set_irq_remapping(iommu, eim_mode);
+			iommu_enable_irq_remapping(iommu);
+		}
 
 		setup = true;
 	}
@@ -939,6 +947,7 @@ static int reenable_irq_remapping(int eim)
 
 		/* Set up interrupt remapping for iommu.*/
 		iommu_set_irq_remapping(iommu, eim);
+		iommu_enable_irq_remapping(iommu);
 		setup = true;
 	}
 
@@ -1257,6 +1266,7 @@ static int dmar_ir_add(struct dmar_drhd_unit *dmaru, struct intel_iommu *iommu)
 		ir_remove_ioapic_hpet_scope(iommu);
 	} else {
 		iommu_set_irq_remapping(iommu, eim);
+		iommu_enable_irq_remapping(iommu);
 	}
 
 	return ret;
-- 
1.9.1

[PATCH 14/17] iommu/vt-d: Move kdump pointer intialization to __iommu_load_old_irte

[Joerg Roedel]

From: Joerg Roedel <jroedel@suse.de>

We initialize the base_old_phys and base_old_virt in this
function. This cleans up the caller and makes the code more
readable. Also add a check for the size of the irq
remapping table of the old kernel, break out if it does not
match our size. Rename the function to iommu_load_old_irte
while at it.

Tested-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
 drivers/iommu/intel_irq_remapping.c | 50 +++++++++++++++++++++----------------
 include/linux/intel-iommu.h         |  1 +
 2 files changed, 29 insertions(+), 22 deletions(-)

diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index c3d1e63..fd6c25b 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -21,7 +21,7 @@
 
 #include "irq_remapping.h"
 
-static int __iommu_load_old_irte(struct intel_iommu *iommu);
+static int iommu_load_old_irte(struct intel_iommu *iommu);
 static int __iommu_update_old_irte(struct intel_iommu *iommu, int index);
 static void iommu_check_pre_ir_status(struct intel_iommu *iommu);
 
@@ -698,14 +698,7 @@ static int __init intel_enable_irq_remapping(void)
 	 */
 	for_each_iommu(iommu, drhd) {
 		if (iommu->pre_enabled_ir) {
-			unsigned long long q;
-
-			q = dmar_readq(iommu->reg + DMAR_IRTA_REG);
-			iommu->ir_table->base_old_phys = q & VTD_PAGE_MASK;
-			iommu->ir_table->base_old_virt = ioremap_cache(
-				iommu->ir_table->base_old_phys,
-				INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
-			__iommu_load_old_irte(iommu);
+			iommu_load_old_irte(iommu);
 		} else {
 			iommu_set_irq_remapping(iommu, eim_mode);
 			iommu_enable_irq_remapping(iommu);
@@ -1303,21 +1296,34 @@ int dmar_ir_hotplug(struct dmar_drhd_unit *dmaru, bool insert)
 	return ret;
 }
 
-static int __iommu_load_old_irte(struct intel_iommu *iommu)
+static int iommu_load_old_irte(struct intel_iommu *iommu)
 {
-	if ((!iommu)
-		|| (!iommu->ir_table)
-		|| (!iommu->ir_table->base)
-		|| (!iommu->ir_table->base_old_phys)
-		|| (!iommu->ir_table->base_old_virt))
-		return -1;
+	struct irte *old_ir_table;
+	phys_addr_t irt_phys;
+	size_t size;
+	u64 irta;
+
+	/* Check whether the old ir-table has the same size as ours */
+	irta = dmar_readq(iommu->reg + DMAR_IRTA_REG);
+	if ((irta & INTR_REMAP_TABLE_REG_SIZE_MASK)
+		 != INTR_REMAP_TABLE_REG_SIZE)
+		return -EINVAL;
+
+	irt_phys = irta & VTD_PAGE_MASK;
+	size     = INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte);
+
+	/* Map the old IR table */
+	old_ir_table = ioremap_cache(irt_phys, size);
+	if (!old_ir_table)
+		return -ENOMEM;
+
+	/* Copy data over */
+	memcpy(iommu->ir_table->base, old_ir_table, size);
 
-	memcpy(iommu->ir_table->base,
-		iommu->ir_table->base_old_virt,
-		INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+	__iommu_flush_cache(iommu, iommu->ir_table->base, size);
 
-	__iommu_flush_cache(iommu, iommu->ir_table->base,
-		INTR_REMAP_TABLE_ENTRIES*sizeof(struct irte));
+	iommu->ir_table->base_old_phys = irt_phys;
+	iommu->ir_table->base_old_virt = old_ir_table;
 
 	return 0;
 }
@@ -1327,7 +1333,7 @@ static int __iommu_update_old_irte(struct intel_iommu *iommu, int index)
 	int start;
 	unsigned long size;
 	void __iomem *to;
-	void *from;
+void *from;
 
 	if ((!iommu)
 		|| (!iommu->ir_table)
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 6c37de9..5aa8834 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -295,6 +295,7 @@ struct q_inval {
 /* 1MB - maximum possible interrupt remapping table size */
 #define INTR_REMAP_PAGE_ORDER	8
 #define INTR_REMAP_TABLE_REG_SIZE	0xf
+#define INTR_REMAP_TABLE_REG_SIZE_MASK  0xf
 
 #define INTR_REMAP_TABLE_ENTRIES	65536
 
-- 
1.9.1

[PATCH 15/17] iommu/vt-d: Mark irt entries from old kernel as allocated

[Joerg Roedel]

From: Joerg Roedel <jroedel@suse.de>

After we copied over all irq remapping entries from the old
kernel we need to mark the present entries as allocated in
our table too. Otherwise we might overwrite entries from the
previous kernel and cause DMAR faults.

Tested-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
 drivers/iommu/intel_irq_remapping.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index fd6c25b..f7b02ca 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -1300,6 +1300,7 @@ static int iommu_load_old_irte(struct intel_iommu *iommu)
 {
 	struct irte *old_ir_table;
 	phys_addr_t irt_phys;
+	unsigned int i;
 	size_t size;
 	u64 irta;
 
@@ -1325,6 +1326,15 @@ static int iommu_load_old_irte(struct intel_iommu *iommu)
 	iommu->ir_table->base_old_phys = irt_phys;
 	iommu->ir_table->base_old_virt = old_ir_table;
 
+	/*
+	 * Now check the table for used entries and mark those as
+	 * allocated in the bitmap
+	 */
+	for (i = 0; i < INTR_REMAP_TABLE_ENTRIES; i++) {
+		if (iommu->ir_table->base[i].present)
+			bitmap_set(iommu->ir_table->bitmap, i, 1);
+	}
+
 	return 0;
 }
 
-- 
1.9.1

[PATCH 16/17] iommu/vt-d: Copy old irte in intel_setup_irq_remapping

[Joerg Roedel]

From: Joerg Roedel <jroedel@suse.de>

Copy over the old irq remapping table in the
intel_setup_irq_remapping function. The reason is that
intel_enable_irq_remapping runs with irqs disabled and thus
we can not safely call ioremap. Therefore we need to copy in
the prepare stage when irqs are still enabled.

Tested-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
 drivers/iommu/intel_irq_remapping.c | 40 ++++++++++++++++++++-----------------
 1 file changed, 22 insertions(+), 18 deletions(-)

diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index f7b02ca..07a8e82 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -24,6 +24,7 @@
 static int iommu_load_old_irte(struct intel_iommu *iommu);
 static int __iommu_update_old_irte(struct intel_iommu *iommu, int index);
 static void iommu_check_pre_ir_status(struct intel_iommu *iommu);
+static void iommu_disable_irq_remapping(struct intel_iommu *iommu);
 
 struct ioapic_scope {
 	struct intel_iommu *iommu;
@@ -538,6 +539,26 @@ static int intel_setup_irq_remapping(struct intel_iommu *iommu)
 		}
 	}
 
+	iommu_check_pre_ir_status(iommu);
+
+	if (!is_kdump_kernel() && iommu->pre_enabled_ir) {
+		iommu_disable_irq_remapping(iommu);
+		iommu->pre_enabled_ir = 0;
+		pr_warn("IRQ remapping was enabled on %s but we are not in kdump mode\n",
+				iommu->name);
+	}
+
+	if (iommu->pre_enabled_ir) {
+		if (iommu_load_old_irte(iommu))
+			pr_err("Failed to copy IR table for %s from previous kernel\n",
+			       iommu->name);
+		else
+			pr_info("Copied IR table for %s from previous kernel\n",
+				iommu->name);
+	}
+
+	iommu_set_irq_remapping(iommu, eim_mode);
+
 	return 0;
 
 out_free_pages:
@@ -682,28 +703,11 @@ static int __init intel_enable_irq_remapping(void)
 	struct intel_iommu *iommu;
 	bool setup = false;
 
-	for_each_iommu(iommu, drhd) {
-		iommu_check_pre_ir_status(iommu);
-
-		if (!is_kdump_kernel() && iommu->pre_enabled_ir) {
-			iommu_disable_irq_remapping(iommu);
-			iommu->pre_enabled_ir = 0;
-			pr_warn("IRQ remapping was enabled on %s but we are not in kdump mode\n",
-				iommu->name);
-		}
-	}
-
 	/*
 	 * Setup Interrupt-remapping for all the DRHD's now.
 	 */
 	for_each_iommu(iommu, drhd) {
-		if (iommu->pre_enabled_ir) {
-			iommu_load_old_irte(iommu);
-		} else {
-			iommu_set_irq_remapping(iommu, eim_mode);
-			iommu_enable_irq_remapping(iommu);
-		}
-
+		iommu_enable_irq_remapping(iommu);
 		setup = true;
 	}
 
-- 
1.9.1

[PATCH 17/17] iommu/vt-d: Remove update code for old ir-table

[Joerg Roedel]

From: Joerg Roedel <jroedel@suse.de>

The code to update the irq remapping table of the old kernel
is not necessary anymore because we switch immidiatly to the
new table after we copied the data over.

Tested-by: Baoquan He <bhe@redhat.com>
Signed-off-by: Joerg Roedel <jroedel@suse.de>
---
 drivers/iommu/intel_irq_remapping.c | 44 -------------------------------------
 include/linux/intel-iommu.h         |  2 --
 2 files changed, 46 deletions(-)

diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index 07a8e82..b49aa8c 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -22,7 +22,6 @@
 #include "irq_remapping.h"
 
 static int iommu_load_old_irte(struct intel_iommu *iommu);
-static int __iommu_update_old_irte(struct intel_iommu *iommu, int index);
 static void iommu_check_pre_ir_status(struct intel_iommu *iommu);
 static void iommu_disable_irq_remapping(struct intel_iommu *iommu);
 
@@ -202,9 +201,6 @@ static int modify_irte(int irq, struct irte *irte_modified)
 	set_64bit(&irte->low, irte_modified->low);
 	set_64bit(&irte->high, irte_modified->high);
 
-	if (iommu->pre_enabled_ir)
-		__iommu_update_old_irte(iommu, index);
-
 	__iommu_flush_cache(iommu, irte, sizeof(*irte));
 
 	rc = qi_flush_iec(iommu, index, 0);
@@ -266,9 +262,6 @@ static int clear_entries(struct irq_2_iommu *irq_iommu)
 	bitmap_release_region(iommu->ir_table->bitmap, index,
 			      irq_iommu->irte_mask);
 
-	if (iommu->pre_enabled_ir)
-		__iommu_update_old_irte(iommu, -1);
-
 	return qi_flush_iec(iommu, index, irq_iommu->irte_mask);
 }
 
@@ -1327,9 +1320,6 @@ static int iommu_load_old_irte(struct intel_iommu *iommu)
 
 	__iommu_flush_cache(iommu, iommu->ir_table->base, size);
 
-	iommu->ir_table->base_old_phys = irt_phys;
-	iommu->ir_table->base_old_virt = old_ir_table;
-
 	/*
 	 * Now check the table for used entries and mark those as
 	 * allocated in the bitmap
@@ -1342,40 +1332,6 @@ static int iommu_load_old_irte(struct intel_iommu *iommu)
 	return 0;
 }
 
-static int __iommu_update_old_irte(struct intel_iommu *iommu, int index)
-{
-	int start;
-	unsigned long size;
-	void __iomem *to;
-void *from;
-
-	if ((!iommu)
-		|| (!iommu->ir_table)
-		|| (!iommu->ir_table->base)
-		|| (!iommu->ir_table->base_old_phys)
-		|| (!iommu->ir_table->base_old_virt))
-		return -1;
-
-	if (index < -1 || index >= INTR_REMAP_TABLE_ENTRIES)
-		return -1;
-
-	if (index == -1) {
-		start = 0;
-		size = INTR_REMAP_TABLE_ENTRIES * sizeof(struct irte);
-	} else {
-		start = index * sizeof(struct irte);
-		size = sizeof(struct irte);
-	}
-
-	to = iommu->ir_table->base_old_virt;
-	from = iommu->ir_table->base;
-	memcpy(to + start, from + start, size);
-
-	__iommu_flush_cache(iommu, to + start, size);
-
-	return 0;
-}
-
 static void iommu_check_pre_ir_status(struct intel_iommu *iommu)
 {
 	u32 sts;
diff --git a/include/linux/intel-iommu.h b/include/linux/intel-iommu.h
index 5aa8834..9fcdd9a 100644
--- a/include/linux/intel-iommu.h
+++ b/include/linux/intel-iommu.h
@@ -302,8 +302,6 @@ struct q_inval {
 struct ir_table {
 	struct irte *base;
 	unsigned long *bitmap;
-	void __iomem *base_old_virt;
-	unsigned long base_old_phys;
 };
 #endif
 
-- 
1.9.1

Re: [PATCH 00/17] Fixes and Cleanups for the Intel VT-d driver

[Li, ZhenHua]

Finished testing on my HP huge system, SuperDome X.
It works well.

Thanks
Zhenhua

On 06/05/2015 10:10 PM, Joerg Roedel wrote:
> Hey,
>
> here are a couple of patches to fix the fall-out from the
> patch set fixing the kdump faults with VT-d enabled. A few
> important issues have been fixed:
>
> 	* Moved sleeping functions out of atomic sections,
> 	  mainly iremap_cache/iounmap.
>
> 	* Mark domain ids used in the old kernel as reserved
> 	  in the kdump kernel, so that any new domains don't
> 	  interfere with domains from the old kernel
>
> 	* Mark all IRT entries from the old kernel as
> 	  reservered in the new kernel, so that we don't
> 	  overwrite an entry which might still be in use by
> 	  a device
>
> 	* Only try to load anything from the old kernel when
> 	  we are in a kdump kernel. If we find the iommu
> 	  hardware enabled and we are not in a kdump kernel,
> 	  just disable the iommu and proceed with
> 	  initialization.
>
> 	* Fix a compile error
>
> Besides that I also cleaned up the code to simplify it:
>
> 	* Move necessary initialization steps before the
> 	  root entry table and the irq remapping table is
> 	  programmed into hardware. This makes sure QI is
> 	  initialized so that we can flush the iommu caches
> 	  when hardware got the new tables.
>
> 	* Make the new root entry table and irq remapping
> 	  table visible to hardware immediatly after they
> 	  are created and loaded with the data from the old
> 	  kernel. This allows to remove the code to update
> 	  both, the old and the new version of the table at
> 	  the same time.
>
> 	* Clean up log messages from the VT-d driver to have
> 	  a common prefix that can be grepped for.
>
> 	* Remove unused code
>
> The changes have been tested by me an Baoquan He. I'll plan
> to put them into the x86/vt-d branch for v4.2.
>
> Regards,
>
> 	Joerg
>
> Joerg Roedel (17):
>    iommu/vt-d: Fix compile error when CONFIG_INTEL_IOMMU=n
>    iommu/vt-d: Remove __iommu_save_to_oldmem() function
>    iommu/vt-d: Make two functions static
>    iommu/vt-d: Load old data structures only in kdump kernel
>    iommu/vt-d: Mark root-entry present in set_root_entry
>    iommu/vt-d: Rework loading of old root-entry table
>    iommu/vt-d: Copy context-tables outside of spin_lock
>    iommu/vt-d: Don't reuse domain-ids from old kernel
>    iommu/vt-d: Clean up log messages in intel-iommu.c
>    iommu/vt-d: Allocate irq remapping table bitmap with GFP_KERNEL
>    iommu/vt-d: Move QI initialization to intel_setup_irq_remapping
>    iommu/vt-d: Move EIM detection to intel_prepare_irq_remapping
>    iommu/vt-d: Split up iommu_set_irq_remapping
>    iommu/vt-d: Move kdump pointer intialization to __iommu_load_old_irte
>    iommu/vt-d: Mark irt entries from old kernel as allocated
>    iommu/vt-d: Copy old irte in intel_setup_irq_remapping
>    iommu/vt-d: Remove update code for old ir-table
>
>   drivers/iommu/dmar.c                |  26 +-
>   drivers/iommu/intel-iommu.c         | 463 ++++++++++++++----------------------
>   drivers/iommu/intel_irq_remapping.c | 263 +++++++++-----------
>   include/linux/intel-iommu.h         |  17 +-
>   4 files changed, 299 insertions(+), 470 deletions(-)
>

Re: [PATCH 00/17] Fixes and Cleanups for the Intel VT-d driver

[Joerg Roedel]

On Mon, Jun 08, 2015 at 04:06:45PM +0800, Li, ZhenHua wrote:
> Finished testing on my HP huge system, SuperDome X.
> It works well.

Thanks for testing this, Zhen-Hua. Might I add your Tested-by to the
patches?


	Joerg

Re: [PATCH 00/17] Fixes and Cleanups for the Intel VT-d driver

[Li, ZhenHua]

My pleasure, thanks.

Zhenhua

On 06/08/2015 04:23 PM, Joerg Roedel wrote:
> On Mon, Jun 08, 2015 at 04:06:45PM +0800, Li, ZhenHua wrote:
>> Finished testing on my HP huge system, SuperDome X.
>> It works well.
>
> Thanks for testing this, Zhen-Hua. Might I add your Tested-by to the
> patches?
>
>
> 	Joerg
>