* [Patch Part2 V2 01/17] iommu/vt-d: avoid double free of g_iommus on error recovery path
2014-02-19 6:07 [Patch Part2 V2 00/17] Enhance DMAR drivers to handle PCI/memory hotplug events Jiang Liu
@ 2014-02-19 6:07 ` Jiang Liu
2014-02-19 6:07 ` [Patch Part2 V2 02/17] iommu/vt-d: avoid caching stale domain_device_info and fix memory leak Jiang Liu
` (16 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Jiang Liu @ 2014-02-19 6:07 UTC (permalink / raw)
To: Joerg Roedel, David Woodhouse, Yinghai Lu, Bjorn Helgaas,
Dan Williams, Vinod Koul, Rafael J . Wysocki
Cc: Jiang Liu, Ashok Raj, Yijing Wang, Tony Luck, iommu, linux-pci,
linux-kernel, dmaengine
Array 'g_iommus' may be freed twice on error recovery path in function
init_dmars() and free_dmar_iommu(), thus cause random system crash as
below.
[ 6.774301] IOMMU: dmar init failed
[ 6.778310] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[ 6.785615] software IO TLB [mem 0x76bcf000-0x7abcf000] (64MB) mapped at [ffff880076bcf000-ffff88007abcefff]
[ 6.796887] general protection fault: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 6.804173] Modules linked in:
[ 6.807731] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.14.0-rc1+ #108
[ 6.815122] Hardware name: Intel Corporation BRICKLAND/BRICKLAND, BIOS BRIVTIN1.86B.0047.R00.1402050741 02/05/2014
[ 6.836000] task: ffff880455a80000 ti: ffff880455a88000 task.ti: ffff880455a88000
[ 6.844487] RIP: 0010:[<ffffffff8143eea6>] [<ffffffff8143eea6>] memcpy+0x6/0x110
[ 6.853039] RSP: 0000:ffff880455a89cc8 EFLAGS: 00010293
[ 6.859064] RAX: ffff006568636163 RBX: ffff00656863616a RCX: 0000000000000005
[ 6.867134] RDX: 0000000000000005 RSI: ffffffff81cdc439 RDI: ffff006568636163
[ 6.875205] RBP: ffff880455a89d30 R08: 000000000001bc3b R09: 0000000000000000
[ 6.883275] R10: 0000000000000000 R11: ffffffff81cdc43e R12: ffff880455a89da8
[ 6.891338] R13: ffff006568636163 R14: 0000000000000005 R15: ffffffff81cdc439
[ 6.899408] FS: 0000000000000000(0000) GS:ffff88045b800000(0000) knlGS:0000000000000000
[ 6.908575] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 6.915088] CR2: ffff88047e1ff000 CR3: 0000000001e0e000 CR4: 00000000001407f0
[ 6.923160] Stack:
[ 6.925487] ffffffff8143c904 ffff88045b407e00 ffff006568636163 ffff006568636163
[ 6.934113] ffffffff8120a1a9 ffffffff81cdc43e 0000000000000007 0000000000000000
[ 6.942747] ffff880455a89da8 ffff006568636163 0000000000000007 ffffffff81cdc439
[ 6.951382] Call Trace:
[ 6.954197] [<ffffffff8143c904>] ? vsnprintf+0x124/0x6f0
[ 6.960323] [<ffffffff8120a1a9>] ? __kmalloc_track_caller+0x169/0x360
[ 6.967716] [<ffffffff81440e1b>] kvasprintf+0x6b/0x80
[ 6.973552] [<ffffffff81432bf1>] kobject_set_name_vargs+0x21/0x70
[ 6.980552] [<ffffffff8143393d>] kobject_init_and_add+0x4d/0x90
[ 6.987364] [<ffffffff812067c9>] ? __kmalloc+0x169/0x370
[ 6.993492] [<ffffffff8102dbbc>] ? cache_add_dev+0x17c/0x4f0
[ 7.000005] [<ffffffff8102ddfa>] cache_add_dev+0x3ba/0x4f0
[ 7.006327] [<ffffffff821a87ca>] ? i8237A_init_ops+0x14/0x14
[ 7.012842] [<ffffffff821a87f8>] cache_sysfs_init+0x2e/0x61
[ 7.019260] [<ffffffff81002162>] do_one_initcall+0xf2/0x220
[ 7.025679] [<ffffffff810a4a29>] ? parse_args+0x2c9/0x450
[ 7.031903] [<ffffffff8219d1b1>] kernel_init_freeable+0x1c9/0x25b
[ 7.038904] [<ffffffff8219c8d2>] ? do_early_param+0x8a/0x8a
[ 7.045322] [<ffffffff8184d5e0>] ? rest_init+0x150/0x150
[ 7.051447] [<ffffffff8184d5ee>] kernel_init+0xe/0x100
[ 7.057380] [<ffffffff8187b87c>] ret_from_fork+0x7c/0xb0
[ 7.063503] [<ffffffff8184d5e0>] ? rest_init+0x150/0x150
[ 7.069628] Code: 89 e5 53 48 89 fb 75 16 80 7f 3c 00 75 05 e8 d2 f9 ff ff 48 8b 43 58 48 2b 43 50 88 43 4e 5b 5d c3 90 90 90 90 48 89 f8 48 89 d1 <f3> a4 c3 03 83 e2 07 f3 48 a5 89 d1 f3 a4 c3 20 4c 8b 06 4c 8b
[ 7.094960] RIP [<ffffffff8143eea6>] memcpy+0x6/0x110
[ 7.100856] RSP <ffff880455a89cc8>
[ 7.104864] ---[ end trace b5d3fdc6c6c28083 ]---
[ 7.110142] Kernel panic - not syncing: Attempted to kill init! exitcode=0x0000000b
[ 7.110142]
[ 7.120540] Kernel Offset: 0x0 from 0xffffffff81000000 (relocation range: 0xffffffff80000000-0xffffffff9fffffff)
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
drivers/iommu/intel-iommu.c | 26 ++++++++++----------------
1 file changed, 10 insertions(+), 16 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index a22c86c..52be755 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1298,15 +1298,6 @@ static void free_dmar_iommu(struct intel_iommu *iommu)
g_iommus[iommu->seq_id] = NULL;
- /* if all iommus are freed, free g_iommus */
- for (i = 0; i < g_num_of_iommus; i++) {
- if (g_iommus[i])
- break;
- }
-
- if (i == g_num_of_iommus)
- kfree(g_iommus);
-
/* free context mapping */
free_context_table(iommu);
}
@@ -2461,7 +2452,7 @@ static int __init init_dmars(void)
sizeof(struct deferred_flush_tables), GFP_KERNEL);
if (!deferred_flush) {
ret = -ENOMEM;
- goto error;
+ goto free_g_iommus;
}
for_each_active_iommu(iommu, drhd) {
@@ -2469,7 +2460,7 @@ static int __init init_dmars(void)
ret = iommu_init_domains(iommu);
if (ret)
- goto error;
+ goto free_iommu;
/*
* TBD:
@@ -2479,7 +2470,7 @@ static int __init init_dmars(void)
ret = iommu_alloc_root_entry(iommu);
if (ret) {
printk(KERN_ERR "IOMMU: allocate root entry failed\n");
- goto error;
+ goto free_iommu;
}
if (!ecap_pass_through(iommu->ecap))
hw_pass_through = 0;
@@ -2548,7 +2539,7 @@ static int __init init_dmars(void)
ret = iommu_prepare_static_identity_mapping(hw_pass_through);
if (ret) {
printk(KERN_CRIT "Failed to setup IOMMU pass-through\n");
- goto error;
+ goto free_iommu;
}
}
/*
@@ -2606,7 +2597,7 @@ static int __init init_dmars(void)
ret = dmar_set_interrupt(iommu);
if (ret)
- goto error;
+ goto free_iommu;
iommu_set_root_entry(iommu);
@@ -2615,17 +2606,20 @@ static int __init init_dmars(void)
ret = iommu_enable_translation(iommu);
if (ret)
- goto error;
+ goto free_iommu;
iommu_disable_protect_mem_regions(iommu);
}
return 0;
-error:
+
+free_iommu:
for_each_active_iommu(iommu, drhd)
free_dmar_iommu(iommu);
kfree(deferred_flush);
+free_g_iommus:
kfree(g_iommus);
+error:
return ret;
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [Patch Part2 V2 02/17] iommu/vt-d: avoid caching stale domain_device_info and fix memory leak
2014-02-19 6:07 [Patch Part2 V2 00/17] Enhance DMAR drivers to handle PCI/memory hotplug events Jiang Liu
2014-02-19 6:07 ` [Patch Part2 V2 01/17] iommu/vt-d: avoid double free of g_iommus on error recovery path Jiang Liu
@ 2014-02-19 6:07 ` Jiang Liu
2014-02-19 6:07 ` [Patch Part2 V2 03/17] iommu/vt-d: avoid caching stale domain_device_info when hot-removing PCI device Jiang Liu
` (15 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Jiang Liu @ 2014-02-19 6:07 UTC (permalink / raw)
To: Joerg Roedel, David Woodhouse, Yinghai Lu, Bjorn Helgaas,
Dan Williams, Vinod Koul, Rafael J . Wysocki
Cc: Jiang Liu, Ashok Raj, Yijing Wang, Tony Luck, iommu, linux-pci,
linux-kernel, dmaengine
Function device_notifier() in intel-iommu.c fails to remove
device_domain_info data structures for PCI devices if they are
associated with si_domain because iommu_no_mapping() returns true
for those PCI devices. This will cause memory leak and caching of
stale information in domain->devices list.
So fix the issue by not calling iommu_no_mapping() and skipping check
of iommu_pass_through.
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
drivers/iommu/intel-iommu.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 52be755..f75d3ae 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -3600,14 +3600,14 @@ static int device_notifier(struct notifier_block *nb,
struct pci_dev *pdev = to_pci_dev(dev);
struct dmar_domain *domain;
- if (iommu_no_mapping(dev))
+ if (iommu_dummy(pdev))
return 0;
domain = find_domain(pdev);
if (!domain)
return 0;
- if (action == BUS_NOTIFY_UNBOUND_DRIVER && !iommu_pass_through) {
+ if (action == BUS_NOTIFY_UNBOUND_DRIVER) {
domain_remove_one_dev_info(domain, pdev);
if (!(domain->flags & DOMAIN_FLAG_VIRTUAL_MACHINE) &&
--
1.7.10.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [Patch Part2 V2 03/17] iommu/vt-d: avoid caching stale domain_device_info when hot-removing PCI device
2014-02-19 6:07 [Patch Part2 V2 00/17] Enhance DMAR drivers to handle PCI/memory hotplug events Jiang Liu
2014-02-19 6:07 ` [Patch Part2 V2 01/17] iommu/vt-d: avoid double free of g_iommus on error recovery path Jiang Liu
2014-02-19 6:07 ` [Patch Part2 V2 02/17] iommu/vt-d: avoid caching stale domain_device_info and fix memory leak Jiang Liu
@ 2014-02-19 6:07 ` Jiang Liu
2014-02-19 6:07 ` [Patch Part2 V2 04/17] iommu/vt-d: factor out dmar_alloc_dev_scope() for later reuse Jiang Liu
` (14 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Jiang Liu @ 2014-02-19 6:07 UTC (permalink / raw)
To: Joerg Roedel, David Woodhouse, Yinghai Lu, Bjorn Helgaas,
Dan Williams, Vinod Koul, Rafael J . Wysocki
Cc: Jiang Liu, Ashok Raj, Yijing Wang, Tony Luck, iommu, linux-pci,
linux-kernel, dmaengine
Function device_notifier() in intel-iommu.c only remove domain_device_info
data structure associated with a PCI device when handling PCI device
driver unbinding events. If a PCI device has never been bound to a PCI
device driver, there won't be BUS_NOTIFY_UNBOUND_DRIVER event when
hot-removing the PCI device. So associated domain_device_info data
structure may get lost.
On the other hand, if iommu_pass_through is enabled, function
iommu_prepare_static_indentify_mapping() will create domain_device_info
data structure for each PCIe to PCIe bridge and PCIe endpoint,
no matter whether there are drivers associated with those PCIe devices
or not. So those domain_device_info data structures will get lost when
hot-removing the assocated PCIe devices if they have never bound to
any PCI device driver.
To be even worse, it's not only an memory leak issue, but also an
caching of stale information bug because the memory are kept in
device_domain_list and domain->devices lists.
Fix the bug by trying to remove domain_device_info data structure when
handling BUS_NOTIFY_DEL_DEVICE event.
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
drivers/iommu/intel-iommu.c | 17 +++++++++--------
1 file changed, 9 insertions(+), 8 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index f75d3ae..5a411e8 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -3603,18 +3603,19 @@ static int device_notifier(struct notifier_block *nb,
if (iommu_dummy(pdev))
return 0;
+ if (action != BUS_NOTIFY_UNBOUND_DRIVER &&
+ action != BUS_NOTIFY_DEL_DEVICE)
+ return 0;
+
domain = find_domain(pdev);
if (!domain)
return 0;
- if (action == BUS_NOTIFY_UNBOUND_DRIVER) {
- domain_remove_one_dev_info(domain, pdev);
-
- if (!(domain->flags & DOMAIN_FLAG_VIRTUAL_MACHINE) &&
- !(domain->flags & DOMAIN_FLAG_STATIC_IDENTITY) &&
- list_empty(&domain->devices))
- domain_exit(domain);
- }
+ domain_remove_one_dev_info(domain, pdev);
+ if (!(domain->flags & DOMAIN_FLAG_VIRTUAL_MACHINE) &&
+ !(domain->flags & DOMAIN_FLAG_STATIC_IDENTITY) &&
+ list_empty(&domain->devices))
+ domain_exit(domain);
return 0;
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [Patch Part2 V2 04/17] iommu/vt-d: factor out dmar_alloc_dev_scope() for later reuse
2014-02-19 6:07 [Patch Part2 V2 00/17] Enhance DMAR drivers to handle PCI/memory hotplug events Jiang Liu
` (2 preceding siblings ...)
2014-02-19 6:07 ` [Patch Part2 V2 03/17] iommu/vt-d: avoid caching stale domain_device_info when hot-removing PCI device Jiang Liu
@ 2014-02-19 6:07 ` Jiang Liu
2014-02-19 6:07 ` [Patch Part2 V2 05/17] iommu/vt-d: move private structures and variables into intel-iommu.c Jiang Liu
` (13 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Jiang Liu @ 2014-02-19 6:07 UTC (permalink / raw)
To: Joerg Roedel, David Woodhouse, Yinghai Lu, Bjorn Helgaas,
Dan Williams, Vinod Koul, Rafael J . Wysocki
Cc: Jiang Liu, Ashok Raj, Yijing Wang, Tony Luck, iommu, linux-pci,
linux-kernel, dmaengine
Factor out function dmar_alloc_dev_scope() from dmar_parse_dev_scope()
for later reuse.
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
drivers/iommu/dmar.c | 28 ++++++++++++++++------------
include/linux/dmar.h | 1 +
2 files changed, 17 insertions(+), 12 deletions(-)
diff --git a/drivers/iommu/dmar.c b/drivers/iommu/dmar.c
index 1581565..58dde75 100644
--- a/drivers/iommu/dmar.c
+++ b/drivers/iommu/dmar.c
@@ -117,13 +117,9 @@ static int __init dmar_parse_one_dev_scope(struct acpi_dmar_device_scope *scope,
return 0;
}
-int __init dmar_parse_dev_scope(void *start, void *end, int *cnt,
- struct pci_dev ***devices, u16 segment)
+void *dmar_alloc_dev_scope(void *start, void *end, int *cnt)
{
struct acpi_dmar_device_scope *scope;
- void * tmp = start;
- int index;
- int ret;
*cnt = 0;
while (start < end) {
@@ -138,15 +134,24 @@ int __init dmar_parse_dev_scope(void *start, void *end, int *cnt,
start += scope->length;
}
if (*cnt == 0)
- return 0;
+ return NULL;
+
+ return kcalloc(*cnt, sizeof(struct pci_dev *), GFP_KERNEL);
+}
+
+int __init dmar_parse_dev_scope(void *start, void *end, int *cnt,
+ struct pci_dev ***devices, u16 segment)
+{
+ struct acpi_dmar_device_scope *scope;
+ int index, ret;
- *devices = kcalloc(*cnt, sizeof(struct pci_dev *), GFP_KERNEL);
- if (!*devices)
+ *devices = dmar_alloc_dev_scope(start, end, cnt);
+ if (*cnt == 0)
+ return 0;
+ else if (!*devices)
return -ENOMEM;
- start = tmp;
- index = 0;
- while (start < end) {
+ for (index = 0; start < end; start += scope->length) {
scope = start;
if (scope->entry_type == ACPI_DMAR_SCOPE_TYPE_ENDPOINT ||
scope->entry_type == ACPI_DMAR_SCOPE_TYPE_BRIDGE) {
@@ -158,7 +163,6 @@ int __init dmar_parse_dev_scope(void *start, void *end, int *cnt,
}
index ++;
}
- start += scope->length;
}
return 0;
diff --git a/include/linux/dmar.h b/include/linux/dmar.h
index eccb0c0..1b08ce8 100644
--- a/include/linux/dmar.h
+++ b/include/linux/dmar.h
@@ -69,6 +69,7 @@ extern int dmar_table_init(void);
extern int dmar_dev_scope_init(void);
extern int dmar_parse_dev_scope(void *start, void *end, int *cnt,
struct pci_dev ***devices, u16 segment);
+extern void *dmar_alloc_dev_scope(void *start, void *end, int *cnt);
extern void dmar_free_dev_scope(struct pci_dev ***devices, int *cnt);
/* Intel IOMMU detection */
--
1.7.10.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [Patch Part2 V2 05/17] iommu/vt-d: move private structures and variables into intel-iommu.c
2014-02-19 6:07 [Patch Part2 V2 00/17] Enhance DMAR drivers to handle PCI/memory hotplug events Jiang Liu
` (3 preceding siblings ...)
2014-02-19 6:07 ` [Patch Part2 V2 04/17] iommu/vt-d: factor out dmar_alloc_dev_scope() for later reuse Jiang Liu
@ 2014-02-19 6:07 ` Jiang Liu
2014-02-19 6:07 ` [Patch Part2 V2 06/17] iommu/vt-d: simplify function get_domain_for_dev() Jiang Liu
` (12 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Jiang Liu @ 2014-02-19 6:07 UTC (permalink / raw)
To: Joerg Roedel, David Woodhouse, Yinghai Lu, Bjorn Helgaas,
Dan Williams, Vinod Koul, Rafael J . Wysocki
Cc: Jiang Liu, Ashok Raj, Yijing Wang, Tony Luck, iommu, linux-pci,
linux-kernel, dmaengine
Move private structures and variables into intel-iommu.c, which will
help to simplify locking policy for hotplug. Also delete redundant
declarations.
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
drivers/iommu/intel-iommu.c | 31 +++++++++++++++++++++++++------
include/linux/dmar.h | 23 +----------------------
2 files changed, 26 insertions(+), 28 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 5a411e8..fffe3d1 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -380,6 +380,29 @@ struct device_domain_info {
struct dmar_domain *domain; /* pointer to domain */
};
+struct dmar_rmrr_unit {
+ struct list_head list; /* list of rmrr units */
+ struct acpi_dmar_header *hdr; /* ACPI header */
+ u64 base_address; /* reserved base address*/
+ u64 end_address; /* reserved end address */
+ struct pci_dev **devices; /* target devices */
+ int devices_cnt; /* target device count */
+};
+
+struct dmar_atsr_unit {
+ struct list_head list; /* list of ATSR units */
+ struct acpi_dmar_header *hdr; /* ACPI header */
+ struct pci_dev **devices; /* target devices */
+ int devices_cnt; /* target device count */
+ u8 include_all:1; /* include all ports */
+};
+
+static LIST_HEAD(dmar_atsr_units);
+static LIST_HEAD(dmar_rmrr_units);
+
+#define for_each_rmrr_units(rmrr) \
+ list_for_each_entry(rmrr, &dmar_rmrr_units, list)
+
static void flush_unmaps_timeout(unsigned long data);
static DEFINE_TIMER(unmap_timer, flush_unmaps_timeout, 0, 0);
@@ -403,6 +426,8 @@ static int timer_on;
static long list_size;
static void domain_remove_dev_info(struct dmar_domain *domain);
+static void domain_remove_one_dev_info(struct dmar_domain *domain,
+ struct pci_dev *pdev);
#ifdef CONFIG_INTEL_IOMMU_DEFAULT_ON
int dmar_disabled = 0;
@@ -2243,8 +2268,6 @@ static int __init si_domain_init(int hw)
return 0;
}
-static void domain_remove_one_dev_info(struct dmar_domain *domain,
- struct pci_dev *pdev);
static int identity_mapping(struct pci_dev *pdev)
{
struct device_domain_info *info;
@@ -3432,8 +3455,6 @@ static void __init init_iommu_pm_ops(void)
static inline void init_iommu_pm_ops(void) {}
#endif /* CONFIG_PM */
-LIST_HEAD(dmar_rmrr_units);
-
static void __init dmar_register_rmrr_unit(struct dmar_rmrr_unit *rmrr)
{
list_add(&rmrr->list, &dmar_rmrr_units);
@@ -3470,8 +3491,6 @@ rmrr_parse_dev(struct dmar_rmrr_unit *rmrru)
rmrr->segment);
}
-static LIST_HEAD(dmar_atsr_units);
-
int __init dmar_parse_one_atsr(struct acpi_dmar_header *hdr)
{
struct acpi_dmar_atsr *atsr;
diff --git a/include/linux/dmar.h b/include/linux/dmar.h
index 1b08ce8..ea599d4 100644
--- a/include/linux/dmar.h
+++ b/include/linux/dmar.h
@@ -139,28 +139,7 @@ extern int arch_setup_dmar_msi(unsigned int irq);
#ifdef CONFIG_INTEL_IOMMU
extern int iommu_detected, no_iommu;
-extern struct list_head dmar_rmrr_units;
-struct dmar_rmrr_unit {
- struct list_head list; /* list of rmrr units */
- struct acpi_dmar_header *hdr; /* ACPI header */
- u64 base_address; /* reserved base address*/
- u64 end_address; /* reserved end address */
- struct pci_dev **devices; /* target devices */
- int devices_cnt; /* target device count */
-};
-
-#define for_each_rmrr_units(rmrr) \
- list_for_each_entry(rmrr, &dmar_rmrr_units, list)
-
-struct dmar_atsr_unit {
- struct list_head list; /* list of ATSR units */
- struct acpi_dmar_header *hdr; /* ACPI header */
- struct pci_dev **devices; /* target devices */
- int devices_cnt; /* target device count */
- u8 include_all:1; /* include all ports */
-};
-
-int dmar_parse_rmrr_atsr_dev(void);
+extern int dmar_parse_rmrr_atsr_dev(void);
extern int dmar_parse_one_rmrr(struct acpi_dmar_header *header);
extern int dmar_parse_one_atsr(struct acpi_dmar_header *header);
extern int intel_iommu_init(void);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [Patch Part2 V2 06/17] iommu/vt-d: simplify function get_domain_for_dev()
2014-02-19 6:07 [Patch Part2 V2 00/17] Enhance DMAR drivers to handle PCI/memory hotplug events Jiang Liu
` (4 preceding siblings ...)
2014-02-19 6:07 ` [Patch Part2 V2 05/17] iommu/vt-d: move private structures and variables into intel-iommu.c Jiang Liu
@ 2014-02-19 6:07 ` Jiang Liu
2014-02-19 6:07 ` [Patch Part2 V2 07/17] iommu/vt-d: free resources if failed to create domain for PCIe endpoint Jiang Liu
` (11 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Jiang Liu @ 2014-02-19 6:07 UTC (permalink / raw)
To: Joerg Roedel, David Woodhouse, Yinghai Lu, Bjorn Helgaas,
Dan Williams, Vinod Koul, Rafael J . Wysocki
Cc: Jiang Liu, Ashok Raj, Yijing Wang, Tony Luck, iommu, linux-pci,
linux-kernel, dmaengine
Function get_domain_for_dev() is a little complex, simplify it
by factoring out dmar_search_domain_by_dev_info() and
dmar_insert_dev_info().
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
drivers/iommu/intel-iommu.c | 142 ++++++++++++++++++++-----------------------
1 file changed, 66 insertions(+), 76 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index fffe3d1..67b114e 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1957,18 +1957,71 @@ find_domain(struct pci_dev *pdev)
return NULL;
}
+static inline struct dmar_domain *
+dmar_search_domain_by_dev_info(int segment, int bus, int devfn)
+{
+ struct device_domain_info *info;
+
+ list_for_each_entry(info, &device_domain_list, global)
+ if (info->segment == segment && info->bus == bus &&
+ info->devfn == devfn)
+ return info->domain;
+
+ return NULL;
+}
+
+static int dmar_insert_dev_info(int segment, int bus, int devfn,
+ struct pci_dev *dev, struct dmar_domain **domp)
+{
+ struct dmar_domain *found, *domain = *domp;
+ struct device_domain_info *info;
+ unsigned long flags;
+
+ info = alloc_devinfo_mem();
+ if (!info)
+ return -ENOMEM;
+
+ info->segment = segment;
+ info->bus = bus;
+ info->devfn = devfn;
+ info->dev = dev;
+ info->domain = domain;
+ if (!dev)
+ domain->flags |= DOMAIN_FLAG_P2P_MULTIPLE_DEVICES;
+
+ spin_lock_irqsave(&device_domain_lock, flags);
+ if (dev)
+ found = find_domain(dev);
+ else
+ found = dmar_search_domain_by_dev_info(segment, bus, devfn);
+ if (found) {
+ spin_unlock_irqrestore(&device_domain_lock, flags);
+ free_devinfo_mem(info);
+ if (found != domain) {
+ domain_exit(domain);
+ *domp = found;
+ }
+ } else {
+ list_add(&info->link, &domain->devices);
+ list_add(&info->global, &device_domain_list);
+ if (dev)
+ dev->dev.archdata.iommu = info;
+ spin_unlock_irqrestore(&device_domain_lock, flags);
+ }
+
+ return 0;
+}
+
/* domain is initialized */
static struct dmar_domain *get_domain_for_dev(struct pci_dev *pdev, int gaw)
{
- struct dmar_domain *domain, *found = NULL;
+ struct dmar_domain *domain;
struct intel_iommu *iommu;
struct dmar_drhd_unit *drhd;
- struct device_domain_info *info, *tmp;
struct pci_dev *dev_tmp;
unsigned long flags;
int bus = 0, devfn = 0;
int segment;
- int ret;
domain = find_domain(pdev);
if (domain)
@@ -1986,41 +2039,29 @@ static struct dmar_domain *get_domain_for_dev(struct pci_dev *pdev, int gaw)
devfn = dev_tmp->devfn;
}
spin_lock_irqsave(&device_domain_lock, flags);
- list_for_each_entry(info, &device_domain_list, global) {
- if (info->segment == segment &&
- info->bus == bus && info->devfn == devfn) {
- found = info->domain;
- break;
- }
- }
+ domain = dmar_search_domain_by_dev_info(segment, bus, devfn);
spin_unlock_irqrestore(&device_domain_lock, flags);
/* pcie-pci bridge already has a domain, uses it */
- if (found) {
- domain = found;
+ if (domain)
goto found_domain;
- }
}
- domain = alloc_domain();
- if (!domain)
- goto error;
-
- /* Allocate new domain for the device */
drhd = dmar_find_matched_drhd_unit(pdev);
if (!drhd) {
printk(KERN_ERR "IOMMU: can't find DMAR for device %s\n",
pci_name(pdev));
- free_domain_mem(domain);
return NULL;
}
iommu = drhd->iommu;
- ret = iommu_attach_domain(domain, iommu);
- if (ret) {
+ /* Allocate and intialize new domain for the device */
+ domain = alloc_domain();
+ if (!domain)
+ goto error;
+ if (iommu_attach_domain(domain, iommu)) {
free_domain_mem(domain);
goto error;
}
-
if (domain_init(domain, gaw)) {
domain_exit(domain);
goto error;
@@ -2028,67 +2069,16 @@ static struct dmar_domain *get_domain_for_dev(struct pci_dev *pdev, int gaw)
/* register pcie-to-pci device */
if (dev_tmp) {
- info = alloc_devinfo_mem();
- if (!info) {
+ if (dmar_insert_dev_info(segment, bus, devfn, NULL, &domain)) {
domain_exit(domain);
goto error;
}
- info->segment = segment;
- info->bus = bus;
- info->devfn = devfn;
- info->dev = NULL;
- info->domain = domain;
- /* This domain is shared by devices under p2p bridge */
- domain->flags |= DOMAIN_FLAG_P2P_MULTIPLE_DEVICES;
-
- /* pcie-to-pci bridge already has a domain, uses it */
- found = NULL;
- spin_lock_irqsave(&device_domain_lock, flags);
- list_for_each_entry(tmp, &device_domain_list, global) {
- if (tmp->segment == segment &&
- tmp->bus == bus && tmp->devfn == devfn) {
- found = tmp->domain;
- break;
- }
- }
- if (found) {
- spin_unlock_irqrestore(&device_domain_lock, flags);
- free_devinfo_mem(info);
- domain_exit(domain);
- domain = found;
- } else {
- list_add(&info->link, &domain->devices);
- list_add(&info->global, &device_domain_list);
- spin_unlock_irqrestore(&device_domain_lock, flags);
- }
}
found_domain:
- info = alloc_devinfo_mem();
- if (!info)
- goto error;
- info->segment = segment;
- info->bus = pdev->bus->number;
- info->devfn = pdev->devfn;
- info->dev = pdev;
- info->domain = domain;
- spin_lock_irqsave(&device_domain_lock, flags);
- /* somebody is fast */
- found = find_domain(pdev);
- if (found != NULL) {
- spin_unlock_irqrestore(&device_domain_lock, flags);
- if (found != domain) {
- domain_exit(domain);
- domain = found;
- }
- free_devinfo_mem(info);
+ if (dmar_insert_dev_info(segment, pdev->bus->number, pdev->devfn,
+ pdev, &domain) == 0)
return domain;
- }
- list_add(&info->link, &domain->devices);
- list_add(&info->global, &device_domain_list);
- pdev->dev.archdata.iommu = info;
- spin_unlock_irqrestore(&device_domain_lock, flags);
- return domain;
error:
/* recheck it here, maybe others set it */
return find_domain(pdev);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [Patch Part2 V2 07/17] iommu/vt-d: free resources if failed to create domain for PCIe endpoint
2014-02-19 6:07 [Patch Part2 V2 00/17] Enhance DMAR drivers to handle PCI/memory hotplug events Jiang Liu
` (5 preceding siblings ...)
2014-02-19 6:07 ` [Patch Part2 V2 06/17] iommu/vt-d: simplify function get_domain_for_dev() Jiang Liu
@ 2014-02-19 6:07 ` Jiang Liu
2014-02-19 6:07 ` [Patch Part2 V2 08/17] iommu/vt-d: reduce duplicated code to handle virtual machine domains Jiang Liu
` (10 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Jiang Liu @ 2014-02-19 6:07 UTC (permalink / raw)
To: Joerg Roedel, David Woodhouse, Yinghai Lu, Bjorn Helgaas,
Dan Williams, Vinod Koul, Rafael J . Wysocki
Cc: Jiang Liu, Ashok Raj, Yijing Wang, Tony Luck, iommu, linux-pci,
linux-kernel, dmaengine
Enhance function get_domain_for_dev() to release allocated resources
if failed to create domain for PCIe endpoint, otherwise the allocated
resources will get lost.
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
drivers/iommu/intel-iommu.c | 15 ++++++++-------
1 file changed, 8 insertions(+), 7 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 67b114e..503cc73 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -2015,7 +2015,7 @@ static int dmar_insert_dev_info(int segment, int bus, int devfn,
/* domain is initialized */
static struct dmar_domain *get_domain_for_dev(struct pci_dev *pdev, int gaw)
{
- struct dmar_domain *domain;
+ struct dmar_domain *domain, *free = NULL;
struct intel_iommu *iommu;
struct dmar_drhd_unit *drhd;
struct pci_dev *dev_tmp;
@@ -2062,17 +2062,16 @@ static struct dmar_domain *get_domain_for_dev(struct pci_dev *pdev, int gaw)
free_domain_mem(domain);
goto error;
}
- if (domain_init(domain, gaw)) {
- domain_exit(domain);
+ free = domain;
+ if (domain_init(domain, gaw))
goto error;
- }
/* register pcie-to-pci device */
if (dev_tmp) {
- if (dmar_insert_dev_info(segment, bus, devfn, NULL, &domain)) {
- domain_exit(domain);
+ if (dmar_insert_dev_info(segment, bus, devfn, NULL, &domain))
goto error;
- }
+ else
+ free = NULL;
}
found_domain:
@@ -2080,6 +2079,8 @@ found_domain:
pdev, &domain) == 0)
return domain;
error:
+ if (free)
+ domain_exit(free);
/* recheck it here, maybe others set it */
return find_domain(pdev);
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [Patch Part2 V2 08/17] iommu/vt-d: reduce duplicated code to handle virtual machine domains
2014-02-19 6:07 [Patch Part2 V2 00/17] Enhance DMAR drivers to handle PCI/memory hotplug events Jiang Liu
` (6 preceding siblings ...)
2014-02-19 6:07 ` [Patch Part2 V2 07/17] iommu/vt-d: free resources if failed to create domain for PCIe endpoint Jiang Liu
@ 2014-02-19 6:07 ` Jiang Liu
2014-02-19 6:07 ` [Patch Part2 V2 09/17] iommu/vt-d: fix incorrect iommu_count for si_domain Jiang Liu
` (9 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Jiang Liu @ 2014-02-19 6:07 UTC (permalink / raw)
To: Joerg Roedel, David Woodhouse, Yinghai Lu, Bjorn Helgaas,
Dan Williams, Vinod Koul, Rafael J . Wysocki
Cc: Jiang Liu, Ashok Raj, Yijing Wang, Tony Luck, iommu, linux-pci,
linux-kernel, dmaengine
Reduce duplicated code to handle virtual machine domains, there's no
functionality changes. It also improves code readability.
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
drivers/iommu/intel-iommu.c | 177 +++++++++++--------------------------------
1 file changed, 45 insertions(+), 132 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 503cc73..6724772 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -425,9 +425,12 @@ static LIST_HEAD(unmaps_to_do);
static int timer_on;
static long list_size;
+static void domain_exit(struct dmar_domain *domain);
static void domain_remove_dev_info(struct dmar_domain *domain);
static void domain_remove_one_dev_info(struct dmar_domain *domain,
struct pci_dev *pdev);
+static void iommu_detach_dependent_devices(struct intel_iommu *iommu,
+ struct pci_dev *pdev);
#ifdef CONFIG_INTEL_IOMMU_DEFAULT_ON
int dmar_disabled = 0;
@@ -1286,10 +1289,6 @@ static int iommu_init_domains(struct intel_iommu *iommu)
return 0;
}
-
-static void domain_exit(struct dmar_domain *domain);
-static void vm_domain_exit(struct dmar_domain *domain);
-
static void free_dmar_iommu(struct intel_iommu *iommu)
{
struct dmar_domain *domain;
@@ -1304,12 +1303,8 @@ static void free_dmar_iommu(struct intel_iommu *iommu)
spin_lock_irqsave(&domain->iommu_lock, flags);
count = --domain->iommu_count;
spin_unlock_irqrestore(&domain->iommu_lock, flags);
- if (count == 0) {
- if (domain->flags & DOMAIN_FLAG_VIRTUAL_MACHINE)
- vm_domain_exit(domain);
- else
- domain_exit(domain);
- }
+ if (count == 0)
+ domain_exit(domain);
}
}
@@ -1327,8 +1322,10 @@ static void free_dmar_iommu(struct intel_iommu *iommu)
free_context_table(iommu);
}
-static struct dmar_domain *alloc_domain(void)
+static struct dmar_domain *alloc_domain(bool vm)
{
+ /* domain id for virtual machine, it won't be set in context */
+ static atomic_t vm_domid = ATOMIC_INIT(0);
struct dmar_domain *domain;
domain = alloc_domain_mem();
@@ -1336,8 +1333,15 @@ static struct dmar_domain *alloc_domain(void)
return NULL;
domain->nid = -1;
+ domain->iommu_count = 0;
memset(domain->iommu_bmp, 0, sizeof(domain->iommu_bmp));
domain->flags = 0;
+ spin_lock_init(&domain->iommu_lock);
+ INIT_LIST_HEAD(&domain->devices);
+ if (vm) {
+ domain->id = atomic_inc_return(&vm_domid);
+ domain->flags = DOMAIN_FLAG_VIRTUAL_MACHINE;
+ }
return domain;
}
@@ -1374,22 +1378,16 @@ static void iommu_detach_domain(struct dmar_domain *domain,
{
unsigned long flags;
int num, ndomains;
- int found = 0;
spin_lock_irqsave(&iommu->lock, flags);
ndomains = cap_ndoms(iommu->cap);
for_each_set_bit(num, iommu->domain_ids, ndomains) {
if (iommu->domains[num] == domain) {
- found = 1;
+ clear_bit(num, iommu->domain_ids);
+ iommu->domains[num] = NULL;
break;
}
}
-
- if (found) {
- clear_bit(num, iommu->domain_ids);
- clear_bit(iommu->seq_id, domain->iommu_bmp);
- iommu->domains[num] = NULL;
- }
spin_unlock_irqrestore(&iommu->lock, flags);
}
@@ -1461,8 +1459,6 @@ static int domain_init(struct dmar_domain *domain, int guest_width)
unsigned long sagaw;
init_iova_domain(&domain->iovad, DMA_32BIT_PFN);
- spin_lock_init(&domain->iommu_lock);
-
domain_reserve_special_ranges(domain);
/* calculate AGAW */
@@ -1481,7 +1477,6 @@ static int domain_init(struct dmar_domain *domain, int guest_width)
return -ENODEV;
}
domain->agaw = agaw;
- INIT_LIST_HEAD(&domain->devices);
if (ecap_coherent(iommu->ecap))
domain->iommu_coherency = 1;
@@ -1518,7 +1513,9 @@ static void domain_exit(struct dmar_domain *domain)
if (!intel_iommu_strict)
flush_unmaps_timeout(0);
+ /* remove associated devices */
domain_remove_dev_info(domain);
+
/* destroy iovas */
put_iova_domain(&domain->iovad);
@@ -1528,8 +1525,10 @@ static void domain_exit(struct dmar_domain *domain)
/* free page tables */
dma_pte_free_pagetable(domain, 0, DOMAIN_MAX_PFN(domain->gaw));
+ /* clear attached or cached domains */
for_each_active_iommu(iommu, drhd)
- if (test_bit(iommu->seq_id, domain->iommu_bmp))
+ if (domain->flags & DOMAIN_FLAG_VIRTUAL_MACHINE ||
+ test_bit(iommu->seq_id, domain->iommu_bmp))
iommu_detach_domain(domain, iommu);
free_domain_mem(domain);
@@ -1921,7 +1920,7 @@ static inline void unlink_domain_info(struct device_domain_info *info)
static void domain_remove_dev_info(struct dmar_domain *domain)
{
struct device_domain_info *info;
- unsigned long flags;
+ unsigned long flags, flags2;
struct intel_iommu *iommu;
spin_lock_irqsave(&device_domain_lock, flags);
@@ -1934,8 +1933,22 @@ static void domain_remove_dev_info(struct dmar_domain *domain)
iommu_disable_dev_iotlb(info);
iommu = device_to_iommu(info->segment, info->bus, info->devfn);
iommu_detach_dev(iommu, info->bus, info->devfn);
- free_devinfo_mem(info);
+ if (domain->flags & DOMAIN_FLAG_VIRTUAL_MACHINE) {
+ iommu_detach_dependent_devices(iommu, info->dev);
+ /* clear this iommu in iommu_bmp, update iommu count
+ * and capabilities
+ */
+ spin_lock_irqsave(&domain->iommu_lock, flags2);
+ if (test_and_clear_bit(iommu->seq_id,
+ domain->iommu_bmp)) {
+ domain->iommu_count--;
+ domain_update_iommu_cap(domain);
+ }
+ spin_unlock_irqrestore(&domain->iommu_lock, flags2);
+ }
+
+ free_devinfo_mem(info);
spin_lock_irqsave(&device_domain_lock, flags);
}
spin_unlock_irqrestore(&device_domain_lock, flags);
@@ -2055,7 +2068,7 @@ static struct dmar_domain *get_domain_for_dev(struct pci_dev *pdev, int gaw)
iommu = drhd->iommu;
/* Allocate and intialize new domain for the device */
- domain = alloc_domain();
+ domain = alloc_domain(false);
if (!domain)
goto error;
if (iommu_attach_domain(domain, iommu)) {
@@ -2220,10 +2233,12 @@ static int __init si_domain_init(int hw)
struct intel_iommu *iommu;
int nid, ret = 0;
- si_domain = alloc_domain();
+ si_domain = alloc_domain(false);
if (!si_domain)
return -EFAULT;
+ si_domain->flags = DOMAIN_FLAG_STATIC_IDENTITY;
+
for_each_active_iommu(iommu, drhd) {
ret = iommu_attach_domain(si_domain, iommu);
if (ret) {
@@ -2237,7 +2252,6 @@ static int __init si_domain_init(int hw)
return -EFAULT;
}
- si_domain->flags = DOMAIN_FLAG_STATIC_IDENTITY;
pr_debug("IOMMU: identity mapping domain is domain %d\n",
si_domain->id);
@@ -3810,67 +3824,11 @@ static void domain_remove_one_dev_info(struct dmar_domain *domain,
}
}
-static void vm_domain_remove_all_dev_info(struct dmar_domain *domain)
-{
- struct device_domain_info *info;
- struct intel_iommu *iommu;
- unsigned long flags1, flags2;
-
- spin_lock_irqsave(&device_domain_lock, flags1);
- while (!list_empty(&domain->devices)) {
- info = list_entry(domain->devices.next,
- struct device_domain_info, link);
- unlink_domain_info(info);
- spin_unlock_irqrestore(&device_domain_lock, flags1);
-
- iommu_disable_dev_iotlb(info);
- iommu = device_to_iommu(info->segment, info->bus, info->devfn);
- iommu_detach_dev(iommu, info->bus, info->devfn);
- iommu_detach_dependent_devices(iommu, info->dev);
-
- /* clear this iommu in iommu_bmp, update iommu count
- * and capabilities
- */
- spin_lock_irqsave(&domain->iommu_lock, flags2);
- if (test_and_clear_bit(iommu->seq_id,
- domain->iommu_bmp)) {
- domain->iommu_count--;
- domain_update_iommu_cap(domain);
- }
- spin_unlock_irqrestore(&domain->iommu_lock, flags2);
-
- free_devinfo_mem(info);
- spin_lock_irqsave(&device_domain_lock, flags1);
- }
- spin_unlock_irqrestore(&device_domain_lock, flags1);
-}
-
-/* domain id for virtual machine, it won't be set in context */
-static atomic_t vm_domid = ATOMIC_INIT(0);
-
-static struct dmar_domain *iommu_alloc_vm_domain(void)
-{
- struct dmar_domain *domain;
-
- domain = alloc_domain_mem();
- if (!domain)
- return NULL;
-
- domain->id = atomic_inc_return(&vm_domid);
- domain->nid = -1;
- memset(domain->iommu_bmp, 0, sizeof(domain->iommu_bmp));
- domain->flags = DOMAIN_FLAG_VIRTUAL_MACHINE;
-
- return domain;
-}
-
static int md_domain_init(struct dmar_domain *domain, int guest_width)
{
int adjust_width;
init_iova_domain(&domain->iovad, DMA_32BIT_PFN);
- spin_lock_init(&domain->iommu_lock);
-
domain_reserve_special_ranges(domain);
/* calculate AGAW */
@@ -3878,9 +3836,6 @@ static int md_domain_init(struct dmar_domain *domain, int guest_width)
adjust_width = guestwidth_to_adjustwidth(guest_width);
domain->agaw = width_to_agaw(adjust_width);
- INIT_LIST_HEAD(&domain->devices);
-
- domain->iommu_count = 0;
domain->iommu_coherency = 0;
domain->iommu_snooping = 0;
domain->iommu_superpage = 0;
@@ -3895,53 +3850,11 @@ static int md_domain_init(struct dmar_domain *domain, int guest_width)
return 0;
}
-static void iommu_free_vm_domain(struct dmar_domain *domain)
-{
- unsigned long flags;
- struct dmar_drhd_unit *drhd;
- struct intel_iommu *iommu;
- unsigned long i;
- unsigned long ndomains;
-
- for_each_active_iommu(iommu, drhd) {
- ndomains = cap_ndoms(iommu->cap);
- for_each_set_bit(i, iommu->domain_ids, ndomains) {
- if (iommu->domains[i] == domain) {
- spin_lock_irqsave(&iommu->lock, flags);
- clear_bit(i, iommu->domain_ids);
- iommu->domains[i] = NULL;
- spin_unlock_irqrestore(&iommu->lock, flags);
- break;
- }
- }
- }
-}
-
-static void vm_domain_exit(struct dmar_domain *domain)
-{
- /* Domain 0 is reserved, so dont process it */
- if (!domain)
- return;
-
- vm_domain_remove_all_dev_info(domain);
- /* destroy iovas */
- put_iova_domain(&domain->iovad);
-
- /* clear ptes */
- dma_pte_clear_range(domain, 0, DOMAIN_MAX_PFN(domain->gaw));
-
- /* free page tables */
- dma_pte_free_pagetable(domain, 0, DOMAIN_MAX_PFN(domain->gaw));
-
- iommu_free_vm_domain(domain);
- free_domain_mem(domain);
-}
-
static int intel_iommu_domain_init(struct iommu_domain *domain)
{
struct dmar_domain *dmar_domain;
- dmar_domain = iommu_alloc_vm_domain();
+ dmar_domain = alloc_domain(true);
if (!dmar_domain) {
printk(KERN_ERR
"intel_iommu_domain_init: dmar_domain == NULL\n");
@@ -3950,7 +3863,7 @@ static int intel_iommu_domain_init(struct iommu_domain *domain)
if (md_domain_init(dmar_domain, DEFAULT_DOMAIN_ADDRESS_WIDTH)) {
printk(KERN_ERR
"intel_iommu_domain_init() failed\n");
- vm_domain_exit(dmar_domain);
+ domain_exit(dmar_domain);
return -ENOMEM;
}
domain_update_iommu_cap(dmar_domain);
@@ -3968,7 +3881,7 @@ static void intel_iommu_domain_destroy(struct iommu_domain *domain)
struct dmar_domain *dmar_domain = domain->priv;
domain->priv = NULL;
- vm_domain_exit(dmar_domain);
+ domain_exit(dmar_domain);
}
static int intel_iommu_attach_device(struct iommu_domain *domain,
--
1.7.10.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [Patch Part2 V2 09/17] iommu/vt-d: fix incorrect iommu_count for si_domain
2014-02-19 6:07 [Patch Part2 V2 00/17] Enhance DMAR drivers to handle PCI/memory hotplug events Jiang Liu
` (7 preceding siblings ...)
2014-02-19 6:07 ` [Patch Part2 V2 08/17] iommu/vt-d: reduce duplicated code to handle virtual machine domains Jiang Liu
@ 2014-02-19 6:07 ` Jiang Liu
2014-02-19 6:07 ` [Patch Part2 V2 10/17] iommu/vt-d: check for NULL pointer when freeing IOMMU data structure Jiang Liu
` (8 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Jiang Liu @ 2014-02-19 6:07 UTC (permalink / raw)
To: Joerg Roedel, David Woodhouse, Yinghai Lu, Bjorn Helgaas,
Dan Williams, Vinod Koul, Rafael J . Wysocki
Cc: Jiang Liu, Ashok Raj, Yijing Wang, Tony Luck, iommu, linux-pci,
linux-kernel, dmaengine, Alex Williamson
The iommu_count field in si_domain(static identity domain) is
initialized to zero and never increases. It will underflow
when tearing down iommu unit in function free_dmar_iommu()
and leak memory. So refine code to correctly manage
si_domain->iommu_count.
Warning message caused by si_domain memory leak:
[ 14.609681] IOMMU: Setting RMRR:
[ 14.613496] Ignoring identity map for HW passthrough device 0000:00:1a.0 [0xbdcfd000 - 0xbdd1dfff]
[ 14.623809] Ignoring identity map for HW passthrough device 0000:00:1d.0 [0xbdcfd000 - 0xbdd1dfff]
[ 14.634162] IOMMU: Prepare 0-16MiB unity mapping for LPC
[ 14.640329] Ignoring identity map for HW passthrough device 0000:00:1f.0 [0x0 - 0xffffff]
[ 14.673360] IOMMU: dmar init failed
[ 14.678157] kmem_cache_destroy iommu_devinfo: Slab cache still has objects
[ 14.686076] CPU: 12 PID: 1 Comm: swapper/0 Not tainted 3.13.0-rc1-gerry+ #59
[ 14.694176] Hardware name: Intel Corporation LH Pass ........../SVRBD-ROW_T, BIOS SE5C600.86B.99.99.x059.091020121352 09/10/2012
[ 14.707412] 0000000000000000 ffff88042dd33db0 ffffffff8156223d ffff880c2cc37c00
[ 14.716407] ffff88042dd33dc8 ffffffff811790b1 ffff880c2d3533b8 ffff88042dd33e00
[ 14.725468] ffffffff81dc7a6a ffffffff81b1e8e0 ffffffff81f84058 ffffffff81d8a711
[ 14.734464] Call Trace:
[ 14.737453] [<ffffffff8156223d>] dump_stack+0x4d/0x66
[ 14.743430] [<ffffffff811790b1>] kmem_cache_destroy+0xf1/0x100
[ 14.750279] [<ffffffff81dc7a6a>] intel_iommu_init+0x122/0x56a
[ 14.757035] [<ffffffff81d8a711>] ? iommu_setup+0x27d/0x27d
[ 14.763491] [<ffffffff81d8a739>] pci_iommu_init+0x28/0x52
[ 14.769846] [<ffffffff81000342>] do_one_initcall+0x122/0x180
[ 14.776506] [<ffffffff81077738>] ? parse_args+0x1e8/0x320
[ 14.782866] [<ffffffff81d850e8>] kernel_init_freeable+0x1e1/0x26c
[ 14.789994] [<ffffffff81d84833>] ? do_early_param+0x88/0x88
[ 14.796556] [<ffffffff8154ffc0>] ? rest_init+0xd0/0xd0
[ 14.802626] [<ffffffff8154ffce>] kernel_init+0xe/0x130
[ 14.808698] [<ffffffff815756ac>] ret_from_fork+0x7c/0xb0
[ 14.814963] [<ffffffff8154ffc0>] ? rest_init+0xd0/0xd0
[ 14.821640] kmem_cache_destroy iommu_domain: Slab cache still has objects
[ 14.829456] CPU: 12 PID: 1 Comm: swapper/0 Not tainted 3.13.0-rc1-gerry+ #59
[ 14.837562] Hardware name: Intel Corporation LH Pass ........../SVRBD-ROW_T, BIOS SE5C600.86B.99.99.x059.091020121352 09/10/2012
[ 14.850803] 0000000000000000 ffff88042dd33db0 ffffffff8156223d ffff88102c1ee3c0
[ 14.861222] ffff88042dd33dc8 ffffffff811790b1 ffff880c2d3533b8 ffff88042dd33e00
[ 14.870284] ffffffff81dc7a76 ffffffff81b1e8e0 ffffffff81f84058 ffffffff81d8a711
[ 14.879271] Call Trace:
[ 14.882227] [<ffffffff8156223d>] dump_stack+0x4d/0x66
[ 14.888197] [<ffffffff811790b1>] kmem_cache_destroy+0xf1/0x100
[ 14.895034] [<ffffffff81dc7a76>] intel_iommu_init+0x12e/0x56a
[ 14.901781] [<ffffffff81d8a711>] ? iommu_setup+0x27d/0x27d
[ 14.908238] [<ffffffff81d8a739>] pci_iommu_init+0x28/0x52
[ 14.914594] [<ffffffff81000342>] do_one_initcall+0x122/0x180
[ 14.921244] [<ffffffff81077738>] ? parse_args+0x1e8/0x320
[ 14.927598] [<ffffffff81d850e8>] kernel_init_freeable+0x1e1/0x26c
[ 14.934738] [<ffffffff81d84833>] ? do_early_param+0x88/0x88
[ 14.941309] [<ffffffff8154ffc0>] ? rest_init+0xd0/0xd0
[ 14.947380] [<ffffffff8154ffce>] kernel_init+0xe/0x130
[ 14.953430] [<ffffffff815756ac>] ret_from_fork+0x7c/0xb0
[ 14.959689] [<ffffffff8154ffc0>] ? rest_init+0xd0/0xd0
[ 14.966299] kmem_cache_destroy iommu_iova: Slab cache still has objects
[ 14.973923] CPU: 12 PID: 1 Comm: swapper/0 Not tainted 3.13.0-rc1-gerry+ #59
[ 14.982020] Hardware name: Intel Corporation LH Pass ........../SVRBD-ROW_T, BIOS SE5C600.86B.99.99.x059.091020121352 09/10/2012
[ 14.995263] 0000000000000000 ffff88042dd33db0 ffffffff8156223d ffff88042cb5c980
[ 15.004265] ffff88042dd33dc8 ffffffff811790b1 ffff880c2d3533b8 ffff88042dd33e00
[ 15.013322] ffffffff81dc7a82 ffffffff81b1e8e0 ffffffff81f84058 ffffffff81d8a711
[ 15.022318] Call Trace:
[ 15.025238] [<ffffffff8156223d>] dump_stack+0x4d/0x66
[ 15.031202] [<ffffffff811790b1>] kmem_cache_destroy+0xf1/0x100
[ 15.038038] [<ffffffff81dc7a82>] intel_iommu_init+0x13a/0x56a
[ 15.044786] [<ffffffff81d8a711>] ? iommu_setup+0x27d/0x27d
[ 15.051242] [<ffffffff81d8a739>] pci_iommu_init+0x28/0x52
[ 15.057601] [<ffffffff81000342>] do_one_initcall+0x122/0x180
[ 15.064254] [<ffffffff81077738>] ? parse_args+0x1e8/0x320
[ 15.070608] [<ffffffff81d850e8>] kernel_init_freeable+0x1e1/0x26c
[ 15.077747] [<ffffffff81d84833>] ? do_early_param+0x88/0x88
[ 15.084300] [<ffffffff8154ffc0>] ? rest_init+0xd0/0xd0
[ 15.090362] [<ffffffff8154ffce>] kernel_init+0xe/0x130
[ 15.096431] [<ffffffff815756ac>] ret_from_fork+0x7c/0xb0
[ 15.102693] [<ffffffff8154ffc0>] ? rest_init+0xd0/0xd0
[ 15.189273] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
Cc: Alex Williamson <alex.williamson@redhat.com>
---
drivers/iommu/intel-iommu.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 6724772..a27d3eb 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1365,6 +1365,7 @@ static int iommu_attach_domain(struct dmar_domain *domain,
}
domain->id = num;
+ domain->iommu_count++;
set_bit(num, iommu->domain_ids);
set_bit(iommu->seq_id, domain->iommu_bmp);
iommu->domains[num] = domain;
@@ -1489,7 +1490,6 @@ static int domain_init(struct dmar_domain *domain, int guest_width)
domain->iommu_snooping = 0;
domain->iommu_superpage = fls(cap_super_page_val(iommu->cap));
- domain->iommu_count = 1;
domain->nid = iommu->node;
/* always allocate the top pgd */
--
1.7.10.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [Patch Part2 V2 10/17] iommu/vt-d: check for NULL pointer when freeing IOMMU data structure
2014-02-19 6:07 [Patch Part2 V2 00/17] Enhance DMAR drivers to handle PCI/memory hotplug events Jiang Liu
` (8 preceding siblings ...)
2014-02-19 6:07 ` [Patch Part2 V2 09/17] iommu/vt-d: fix incorrect iommu_count for si_domain Jiang Liu
@ 2014-02-19 6:07 ` Jiang Liu
2014-02-19 6:07 ` [Patch Part2 V2 11/17] iommu/vt-d: fix error in detect ATS capability Jiang Liu
` (7 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Jiang Liu @ 2014-02-19 6:07 UTC (permalink / raw)
To: Joerg Roedel, David Woodhouse, Yinghai Lu, Bjorn Helgaas,
Dan Williams, Vinod Koul, Rafael J . Wysocki
Cc: Jiang Liu, Ashok Raj, Yijing Wang, Tony Luck, iommu, linux-pci,
linux-kernel, dmaengine
Domain id 0 will be assigned to invalid translation without allocating
domain data structure if DMAR unit supports caching mode. So in function
free_dmar_iommu(), we should check whether the domain pointer is NULL,
otherwise it will cause system crash as below:
[ 6.790519] BUG: unable to handle kernel NULL pointer dereference at 00000000000000c8
[ 6.799520] IP: [<ffffffff810e2dc8>] __lock_acquire+0x11f8/0x1430
[ 6.806493] PGD 0
[ 6.817972] Oops: 0000 [#1] SMP DEBUG_PAGEALLOC
[ 6.823303] Modules linked in:
[ 6.826862] CPU: 0 PID: 1 Comm: swapper/0 Not tainted 3.14.0-rc1+ #126
[ 6.834252] Hardware name: Intel Corporation BRICKLAND/BRICKLAND, BIOS BRIVTIN1.86B.0047.R00.1402050741 02/05/2014
[ 6.845951] task: ffff880455a80000 ti: ffff880455a88000 task.ti: ffff880455a88000
[ 6.854437] RIP: 0010:[<ffffffff810e2dc8>] [<ffffffff810e2dc8>] __lock_acquire+0x11f8/0x1430
[ 6.864154] RSP: 0000:ffff880455a89ce0 EFLAGS: 00010046
[ 6.870179] RAX: 0000000000000046 RBX: 0000000000000002 RCX: 0000000000000000
[ 6.878249] RDX: 0000000000000000 RSI: 0000000000000000 RDI: 00000000000000c8
[ 6.886318] RBP: ffff880455a89d40 R08: 0000000000000002 R09: 0000000000000001
[ 6.894387] R10: 0000000000000000 R11: 0000000000000001 R12: ffff880455a80000
[ 6.902458] R13: 0000000000000000 R14: 00000000000000c8 R15: 0000000000000000
[ 6.910520] FS: 0000000000000000(0000) GS:ffff88045b800000(0000) knlGS:0000000000000000
[ 6.919687] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 6.926198] CR2: 00000000000000c8 CR3: 0000000001e0e000 CR4: 00000000001407f0
[ 6.934269] Stack:
[ 6.936588] ffffffffffffff10 ffffffff810f59db 0000000000000010 0000000000000246
[ 6.945219] ffff880455a89d10 0000000000000000 ffffffff82bcb980 0000000000000046
[ 6.953850] 0000000000000000 0000000000000000 0000000000000002 0000000000000000
[ 6.962482] Call Trace:
[ 6.965300] [<ffffffff810f59db>] ? vprintk_emit+0x4fb/0x5a0
[ 6.971716] [<ffffffff810e3185>] lock_acquire+0x185/0x200
[ 6.977941] [<ffffffff821fbbee>] ? init_dmars+0x839/0xa1d
[ 6.984167] [<ffffffff81870b06>] _raw_spin_lock_irqsave+0x56/0x90
[ 6.991158] [<ffffffff821fbbee>] ? init_dmars+0x839/0xa1d
[ 6.997380] [<ffffffff821fbbee>] init_dmars+0x839/0xa1d
[ 7.003410] [<ffffffff8147d575>] ? pci_get_dev_by_id+0x75/0xd0
[ 7.010119] [<ffffffff821fc146>] intel_iommu_init+0x2f0/0x502
[ 7.016735] [<ffffffff821a7947>] ? iommu_setup+0x27d/0x27d
[ 7.023056] [<ffffffff821a796f>] pci_iommu_init+0x28/0x52
[ 7.029282] [<ffffffff81002162>] do_one_initcall+0xf2/0x220
[ 7.035702] [<ffffffff810a4a29>] ? parse_args+0x2c9/0x450
[ 7.041919] [<ffffffff8219d1b1>] kernel_init_freeable+0x1c9/0x25b
[ 7.048919] [<ffffffff8219c8d2>] ? do_early_param+0x8a/0x8a
[ 7.055336] [<ffffffff8184d3f0>] ? rest_init+0x150/0x150
[ 7.061461] [<ffffffff8184d3fe>] kernel_init+0xe/0x100
[ 7.067393] [<ffffffff8187b5fc>] ret_from_fork+0x7c/0xb0
[ 7.073518] [<ffffffff8184d3f0>] ? rest_init+0x150/0x150
[ 7.079642] Code: 01 76 18 89 05 46 04 36 01 41 be 01 00 00 00 e9 2f 02 00 00 0f 1f 80 00 00 00 00 41 be 01 00 00 00 e9 1d 02 00 00 0f 1f 44 00 00 <49> 81 3e c0 31 34 82 b8 01 00 00 00 0f 44 d8 41 83 ff 01 0f 87
[ 7.104944] RIP [<ffffffff810e2dc8>] __lock_acquire+0x11f8/0x1430
[ 7.112008] RSP <ffff880455a89ce0>
[ 7.115988] CR2: 00000000000000c8
[ 7.119784] ---[ end trace 13d756f0f462c538 ]---
[ 7.125034] note: swapper/0[1] exited with preempt_count 1
[ 7.131285] Kernel panic - not syncing: Attempted to kill init! exitcode=0x00000009
[ 7.131285]
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
drivers/iommu/intel-iommu.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index a27d3eb..afbc8d6 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -1297,6 +1297,13 @@ static void free_dmar_iommu(struct intel_iommu *iommu)
if ((iommu->domains) && (iommu->domain_ids)) {
for_each_set_bit(i, iommu->domain_ids, cap_ndoms(iommu->cap)) {
+ /*
+ * Domain id 0 is reserved for invalid translation
+ * if hardware supports caching mode.
+ */
+ if (cap_caching_mode(iommu->cap) && i == 0)
+ continue;
+
domain = iommu->domains[i];
clear_bit(i, iommu->domain_ids);
--
1.7.10.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [Patch Part2 V2 11/17] iommu/vt-d: fix error in detect ATS capability
2014-02-19 6:07 [Patch Part2 V2 00/17] Enhance DMAR drivers to handle PCI/memory hotplug events Jiang Liu
` (9 preceding siblings ...)
2014-02-19 6:07 ` [Patch Part2 V2 10/17] iommu/vt-d: check for NULL pointer when freeing IOMMU data structure Jiang Liu
@ 2014-02-19 6:07 ` Jiang Liu
2014-02-19 6:07 ` [Patch Part2 V2 12/17] iommu/vt-d: introduce macro for_each_dev_scope() to walk device scope entries Jiang Liu
` (6 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Jiang Liu @ 2014-02-19 6:07 UTC (permalink / raw)
To: Joerg Roedel, David Woodhouse, Yinghai Lu, Bjorn Helgaas,
Dan Williams, Vinod Koul, Rafael J . Wysocki
Cc: Jiang Liu, Ashok Raj, Yijing Wang, Tony Luck, iommu, linux-pci,
linux-kernel, dmaengine
Current Intel IOMMU driver only matches a PCIe root port with the first
DRHD unit with the samge segment number. It will report false result
if there are multiple DRHD units with the same segment number, thus fail
to detect ATS capability for some PCIe devices.
This patch refines function dmar_find_matched_atsr_unit() to search all
DRHD units with the same segment number.
An example DMAR table entries as below:
[1D0h 0464 2] Subtable Type : 0002 <Root Port ATS Capability>
[1D2h 0466 2] Length : 0028
[1D4h 0468 1] Flags : 00
[1D5h 0469 1] Reserved : 00
[1D6h 0470 2] PCI Segment Number : 0000
[1D8h 0472 1] Device Scope Entry Type : 02
[1D9h 0473 1] Entry Length : 08
[1DAh 0474 2] Reserved : 0000
[1DCh 0476 1] Enumeration ID : 00
[1DDh 0477 1] PCI Bus Number : 00
[1DEh 0478 2] PCI Path : [02, 00]
[1E0h 0480 1] Device Scope Entry Type : 02
[1E1h 0481 1] Entry Length : 08
[1E2h 0482 2] Reserved : 0000
[1E4h 0484 1] Enumeration ID : 00
[1E5h 0485 1] PCI Bus Number : 00
[1E6h 0486 2] PCI Path : [03, 00]
[1E8h 0488 1] Device Scope Entry Type : 02
[1E9h 0489 1] Entry Length : 08
[1EAh 0490 2] Reserved : 0000
[1ECh 0492 1] Enumeration ID : 00
[1EDh 0493 1] PCI Bus Number : 00
[1EEh 0494 2] PCI Path : [03, 02]
[1F0h 0496 1] Device Scope Entry Type : 02
[1F1h 0497 1] Entry Length : 08
[1F2h 0498 2] Reserved : 0000
[1F4h 0500 1] Enumeration ID : 00
[1F5h 0501 1] PCI Bus Number : 00
[1F6h 0502 2] PCI Path : [03, 03]
[1F8h 0504 2] Subtable Type : 0002 <Root Port ATS Capability>
[1FAh 0506 2] Length : 0020
[1FCh 0508 1] Flags : 00
[1FDh 0509 1] Reserved : 00
[1FEh 0510 2] PCI Segment Number : 0000
[200h 0512 1] Device Scope Entry Type : 02
[201h 0513 1] Entry Length : 08
[202h 0514 2] Reserved : 0000
[204h 0516 1] Enumeration ID : 00
[205h 0517 1] PCI Bus Number : 40
[206h 0518 2] PCI Path : [02, 00]
[208h 0520 1] Device Scope Entry Type : 02
[209h 0521 1] Entry Length : 08
[20Ah 0522 2] Reserved : 0000
[20Ch 0524 1] Enumeration ID : 00
[20Dh 0525 1] PCI Bus Number : 40
[20Eh 0526 2] PCI Path : [02, 02]
[210h 0528 1] Device Scope Entry Type : 02
[211h 0529 1] Entry Length : 08
[212h 0530 2] Reserved : 0000
[214h 0532 1] Enumeration ID : 00
[215h 0533 1] PCI Bus Number : 40
[216h 0534 2] PCI Path : [03, 00]
[218h 0536 2] Subtable Type : 0002 <Root Port ATS Capability>
[21Ah 0538 2] Length : 0020
[21Ch 0540 1] Flags : 00
[21Dh 0541 1] Reserved : 00
[21Eh 0542 2] PCI Segment Number : 0000
[220h 0544 1] Device Scope Entry Type : 02
[221h 0545 1] Entry Length : 08
[222h 0546 2] Reserved : 0000
[224h 0548 1] Enumeration ID : 00
[225h 0549 1] PCI Bus Number : 80
[226h 0550 2] PCI Path : [02, 00]
[228h 0552 1] Device Scope Entry Type : 02
[229h 0553 1] Entry Length : 08
[22Ah 0554 2] Reserved : 0000
[22Ch 0556 1] Enumeration ID : 00
[22Dh 0557 1] PCI Bus Number : 80
[22Eh 0558 2] PCI Path : [02, 02]
[230h 0560 1] Device Scope Entry Type : 02
[231h 0561 1] Entry Length : 08
[232h 0562 2] Reserved : 0000
[234h 0564 1] Enumeration ID : 00
[235h 0565 1] PCI Bus Number : 80
[236h 0566 2] PCI Path : [03, 00]
[238h 0568 2] Subtable Type : 0002 <Root Port ATS Capability>
[23Ah 0570 2] Length : 0020
[23Ch 0572 1] Flags : 00
[23Dh 0573 1] Reserved : 00
[23Eh 0574 2] PCI Segment Number : 0000
[240h 0576 1] Device Scope Entry Type : 02
[241h 0577 1] Entry Length : 08
[242h 0578 2] Reserved : 0000
[244h 0580 1] Enumeration ID : 00
[245h 0581 1] PCI Bus Number : C0
[246h 0582 2] PCI Path : [02, 00]
[248h 0584 1] Device Scope Entry Type : 02
[249h 0585 1] Entry Length : 08
[24Ah 0586 2] Reserved : 0000
[24Ch 0588 1] Enumeration ID : 00
[24Dh 0589 1] PCI Bus Number : C0
[24Eh 0590 2] PCI Path : [02, 02]
[250h 0592 1] Device Scope Entry Type : 02
[251h 0593 1] Entry Length : 08
[252h 0594 2] Reserved : 0000
[254h 0596 1] Enumeration ID : 00
[255h 0597 1] PCI Bus Number : C0
[256h 0598 2] PCI Path : [03, 00]
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
drivers/iommu/intel-iommu.c | 37 +++++++++++++++++--------------------
1 file changed, 17 insertions(+), 20 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index afbc8d6..7732c43 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -3562,37 +3562,34 @@ int dmar_find_matched_atsr_unit(struct pci_dev *dev)
{
int i;
struct pci_bus *bus;
+ struct pci_dev *bridge = NULL;
struct acpi_dmar_atsr *atsr;
struct dmar_atsr_unit *atsru;
dev = pci_physfn(dev);
-
- list_for_each_entry(atsru, &dmar_atsr_units, list) {
- atsr = container_of(atsru->hdr, struct acpi_dmar_atsr, header);
- if (atsr->segment == pci_domain_nr(dev->bus))
- goto found;
- }
-
- return 0;
-
-found:
for (bus = dev->bus; bus; bus = bus->parent) {
- struct pci_dev *bridge = bus->self;
-
+ bridge = bus->self;
if (!bridge || !pci_is_pcie(bridge) ||
pci_pcie_type(bridge) == PCI_EXP_TYPE_PCI_BRIDGE)
return 0;
-
- if (pci_pcie_type(bridge) == PCI_EXP_TYPE_ROOT_PORT) {
- for (i = 0; i < atsru->devices_cnt; i++)
- if (atsru->devices[i] == bridge)
- return 1;
+ if (pci_pcie_type(bridge) == PCI_EXP_TYPE_ROOT_PORT)
break;
- }
}
+ if (!bridge)
+ return 0;
- if (atsru->include_all)
- return 1;
+ list_for_each_entry_rcu(atsru, &dmar_atsr_units, list) {
+ atsr = container_of(atsru->hdr, struct acpi_dmar_atsr, header);
+ if (atsr->segment != pci_domain_nr(dev->bus))
+ continue;
+
+ for (i = 0; i < atsru->devices_cnt; i++)
+ if (atsru->devices[i] == bridge)
+ return 1;
+
+ if (atsru->include_all)
+ return 1;
+ }
return 0;
}
--
1.7.10.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [Patch Part2 V2 12/17] iommu/vt-d: introduce macro for_each_dev_scope() to walk device scope entries
2014-02-19 6:07 [Patch Part2 V2 00/17] Enhance DMAR drivers to handle PCI/memory hotplug events Jiang Liu
` (10 preceding siblings ...)
2014-02-19 6:07 ` [Patch Part2 V2 11/17] iommu/vt-d: fix error in detect ATS capability Jiang Liu
@ 2014-02-19 6:07 ` Jiang Liu
2014-02-19 6:07 ` [Patch Part2 V2 13/17] iommu/vt-d: introduce a rwsem to protect global data structures Jiang Liu
` (5 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Jiang Liu @ 2014-02-19 6:07 UTC (permalink / raw)
To: Joerg Roedel, David Woodhouse, Yinghai Lu, Bjorn Helgaas,
Dan Williams, Vinod Koul, Rafael J . Wysocki
Cc: Jiang Liu, Ashok Raj, Yijing Wang, Tony Luck, iommu, linux-pci,
linux-kernel, dmaengine
Introduce for_each_dev_scope()/for_each_active_dev_scope() to walk
{active} device scope entries. This will help following RCU lock
related patches.
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
drivers/iommu/dmar.c | 14 +++---
drivers/iommu/intel-iommu.c | 100 +++++++++++++++++++++----------------------
include/linux/dmar.h | 6 +++
3 files changed, 64 insertions(+), 56 deletions(-)
diff --git a/drivers/iommu/dmar.c b/drivers/iommu/dmar.c
index 58dde75..4ae6df2 100644
--- a/drivers/iommu/dmar.c
+++ b/drivers/iommu/dmar.c
@@ -170,9 +170,12 @@ int __init dmar_parse_dev_scope(void *start, void *end, int *cnt,
void dmar_free_dev_scope(struct pci_dev ***devices, int *cnt)
{
+ int i;
+ struct pci_dev *tmp_dev;
+
if (*devices && *cnt) {
- while (--*cnt >= 0)
- pci_dev_put((*devices)[*cnt]);
+ for_each_active_dev_scope(*devices, *cnt, i, tmp_dev)
+ pci_dev_put(tmp_dev);
kfree(*devices);
*devices = NULL;
*cnt = 0;
@@ -402,10 +405,11 @@ static int dmar_pci_device_match(struct pci_dev *devices[], int cnt,
struct pci_dev *dev)
{
int index;
+ struct pci_dev *tmp;
while (dev) {
- for (index = 0; index < cnt; index++)
- if (dev == devices[index])
+ for_each_active_dev_scope(devices, cnt, index, tmp)
+ if (dev == tmp)
return 1;
/* Check our parent */
@@ -452,7 +456,7 @@ int __init dmar_dev_scope_init(void)
if (list_empty(&dmar_drhd_units))
goto fail;
- list_for_each_entry(drhd, &dmar_drhd_units, list) {
+ for_each_drhd_unit(drhd) {
ret = dmar_parse_dev(drhd);
if (ret)
goto fail;
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 7732c43..bb98e37 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -654,29 +654,31 @@ static void domain_update_iommu_cap(struct dmar_domain *domain)
static struct intel_iommu *device_to_iommu(int segment, u8 bus, u8 devfn)
{
struct dmar_drhd_unit *drhd = NULL;
+ struct intel_iommu *iommu;
+ struct pci_dev *dev;
int i;
- for_each_active_drhd_unit(drhd) {
+ for_each_active_iommu(iommu, drhd) {
if (segment != drhd->segment)
continue;
- for (i = 0; i < drhd->devices_cnt; i++) {
- if (drhd->devices[i] &&
- drhd->devices[i]->bus->number == bus &&
- drhd->devices[i]->devfn == devfn)
- return drhd->iommu;
- if (drhd->devices[i] &&
- drhd->devices[i]->subordinate &&
- drhd->devices[i]->subordinate->number <= bus &&
- drhd->devices[i]->subordinate->busn_res.end >= bus)
- return drhd->iommu;
+ for_each_active_dev_scope(drhd->devices,
+ drhd->devices_cnt, i, dev) {
+ if (dev->bus->number == bus && dev->devfn == devfn)
+ goto out;
+ if (dev->subordinate &&
+ dev->subordinate->number <= bus &&
+ dev->subordinate->busn_res.end >= bus)
+ goto out;
}
if (drhd->include_all)
- return drhd->iommu;
+ goto out;
}
+ iommu = NULL;
+out:
- return NULL;
+ return iommu;
}
static void domain_flush_cache(struct dmar_domain *domain,
@@ -2333,17 +2335,19 @@ static int domain_add_dev_info(struct dmar_domain *domain,
static bool device_has_rmrr(struct pci_dev *dev)
{
struct dmar_rmrr_unit *rmrr;
+ struct pci_dev *tmp;
int i;
for_each_rmrr_units(rmrr) {
- for (i = 0; i < rmrr->devices_cnt; i++) {
- /*
- * Return TRUE if this RMRR contains the device that
- * is passed in.
- */
- if (rmrr->devices[i] == dev)
+ /*
+ * Return TRUE if this RMRR contains the device that
+ * is passed in.
+ */
+ for_each_active_dev_scope(rmrr->devices,
+ rmrr->devices_cnt, i, tmp)
+ if (tmp == dev) {
return true;
- }
+ }
}
return false;
}
@@ -2593,14 +2597,9 @@ static int __init init_dmars(void)
*/
printk(KERN_INFO "IOMMU: Setting RMRR:\n");
for_each_rmrr_units(rmrr) {
- for (i = 0; i < rmrr->devices_cnt; i++) {
- pdev = rmrr->devices[i];
- /*
- * some BIOS lists non-exist devices in DMAR
- * table.
- */
- if (!pdev)
- continue;
+ /* some BIOS lists non-exist devices in DMAR table. */
+ for_each_active_dev_scope(rmrr->devices, rmrr->devices_cnt,
+ i, pdev) {
ret = iommu_prepare_rmrr_dev(rmrr, pdev);
if (ret)
printk(KERN_ERR
@@ -3288,13 +3287,14 @@ DECLARE_PCI_FIXUP_ENABLE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_IOAT_SNB, quir
static void __init init_no_remapping_devices(void)
{
struct dmar_drhd_unit *drhd;
+ struct pci_dev *dev;
+ int i;
for_each_drhd_unit(drhd) {
if (!drhd->include_all) {
- int i;
- for (i = 0; i < drhd->devices_cnt; i++)
- if (drhd->devices[i] != NULL)
- break;
+ for_each_active_dev_scope(drhd->devices,
+ drhd->devices_cnt, i, dev)
+ break;
/* ignore DMAR unit if no pci devices exist */
if (i == drhd->devices_cnt)
drhd->ignored = 1;
@@ -3302,15 +3302,13 @@ static void __init init_no_remapping_devices(void)
}
for_each_active_drhd_unit(drhd) {
- int i;
if (drhd->include_all)
continue;
- for (i = 0; i < drhd->devices_cnt; i++)
- if (drhd->devices[i] &&
- !IS_GFX_DEVICE(drhd->devices[i]))
+ for_each_active_dev_scope(drhd->devices,
+ drhd->devices_cnt, i, dev)
+ if (!IS_GFX_DEVICE(dev))
break;
-
if (i < drhd->devices_cnt)
continue;
@@ -3320,11 +3318,9 @@ static void __init init_no_remapping_devices(void)
intel_iommu_gfx_mapped = 1;
} else {
drhd->ignored = 1;
- for (i = 0; i < drhd->devices_cnt; i++) {
- if (!drhd->devices[i])
- continue;
- drhd->devices[i]->dev.archdata.iommu = DUMMY_DEVICE_DOMAIN_INFO;
- }
+ for_each_active_dev_scope(drhd->devices,
+ drhd->devices_cnt, i, dev)
+ dev->dev.archdata.iommu = DUMMY_DEVICE_DOMAIN_INFO;
}
}
}
@@ -3560,9 +3556,9 @@ static void intel_iommu_free_dmars(void)
int dmar_find_matched_atsr_unit(struct pci_dev *dev)
{
- int i;
+ int i, ret = 1;
struct pci_bus *bus;
- struct pci_dev *bridge = NULL;
+ struct pci_dev *bridge = NULL, *tmp;
struct acpi_dmar_atsr *atsr;
struct dmar_atsr_unit *atsru;
@@ -3583,22 +3579,24 @@ int dmar_find_matched_atsr_unit(struct pci_dev *dev)
if (atsr->segment != pci_domain_nr(dev->bus))
continue;
- for (i = 0; i < atsru->devices_cnt; i++)
- if (atsru->devices[i] == bridge)
- return 1;
+ for_each_dev_scope(atsru->devices, atsru->devices_cnt, i, tmp)
+ if (tmp == bridge)
+ goto out;
if (atsru->include_all)
- return 1;
+ goto out;
}
+ ret = 0;
+out:
- return 0;
+ return ret;
}
int __init dmar_parse_rmrr_atsr_dev(void)
{
struct dmar_rmrr_unit *rmrr;
struct dmar_atsr_unit *atsr;
- int ret = 0;
+ int ret;
list_for_each_entry(rmrr, &dmar_rmrr_units, list) {
ret = rmrr_parse_dev(rmrr);
@@ -3612,7 +3610,7 @@ int __init dmar_parse_rmrr_atsr_dev(void)
return ret;
}
- return ret;
+ return 0;
}
/*
diff --git a/include/linux/dmar.h b/include/linux/dmar.h
index ea599d4..4b77fd8 100644
--- a/include/linux/dmar.h
+++ b/include/linux/dmar.h
@@ -65,6 +65,12 @@ extern struct list_head dmar_drhd_units;
list_for_each_entry(drhd, &dmar_drhd_units, list) \
if (i=drhd->iommu, 0) {} else
+#define for_each_dev_scope(a, c, p, d) \
+ for ((p) = 0; ((d) = (p) < (c) ? (a)[(p)] : NULL, (p) < (c)); (p)++)
+
+#define for_each_active_dev_scope(a, c, p, d) \
+ for_each_dev_scope((a), (c), (p), (d)) if (!(d)) { continue; } else
+
extern int dmar_table_init(void);
extern int dmar_dev_scope_init(void);
extern int dmar_parse_dev_scope(void *start, void *end, int *cnt,
--
1.7.10.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [Patch Part2 V2 13/17] iommu/vt-d: introduce a rwsem to protect global data structures
2014-02-19 6:07 [Patch Part2 V2 00/17] Enhance DMAR drivers to handle PCI/memory hotplug events Jiang Liu
` (11 preceding siblings ...)
2014-02-19 6:07 ` [Patch Part2 V2 12/17] iommu/vt-d: introduce macro for_each_dev_scope() to walk device scope entries Jiang Liu
@ 2014-02-19 6:07 ` Jiang Liu
2014-02-19 6:07 ` [Patch Part2 V2 14/17] iommu/vt-d: use RCU to protect global resources in interrupt context Jiang Liu
` (4 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Jiang Liu @ 2014-02-19 6:07 UTC (permalink / raw)
To: Joerg Roedel, David Woodhouse, Yinghai Lu, Bjorn Helgaas,
Dan Williams, Vinod Koul, Rafael J . Wysocki
Cc: Jiang Liu, Ashok Raj, Yijing Wang, Tony Luck, iommu, linux-pci,
linux-kernel, dmaengine
Introduce a global rwsem dmar_global_lock, which will be used to
protect DMAR related global data structures from DMAR/PCI/memory
device hotplug operations in process context.
DMA and interrupt remapping related data structures are read most,
and only change when memory/PCI/DMAR hotplug event happens.
So a global rwsem solution is adopted for balance between simplicity
and performance.
For interrupt remapping driver, function intel_irq_remapping_supported(),
dmar_table_init(), intel_enable_irq_remapping(), disable_irq_remapping(),
reenable_irq_remapping() and enable_drhd_fault_handling() etc
are called during booting, suspending and resuming with interrupt
disabled, so no need to take the global lock.
For interrupt remapping entry allocation, the locking model is:
down_read(&dmar_global_lock);
/* Find corresponding iommu */
iommu = map_hpet_to_ir(id);
if (iommu)
/*
* Allocate remapping entry and mark entry busy,
* the IOMMU won't be hot-removed until the
* allocated entry has been released.
*/
index = alloc_irte(iommu, irq, 1);
up_read(&dmar_global_lock);
For DMA remmaping driver, we only uses the dmar_global_lock rwsem to
protect functions which are only called in process context. For any
function which may be called in interrupt context, we will use RCU
to protect them in following patches.
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
drivers/iommu/dmar.c | 19 +++++-
drivers/iommu/intel-iommu.c | 22 ++++---
drivers/iommu/intel_irq_remapping.c | 108 +++++++++++++++++++++++------------
include/linux/dmar.h | 2 +
4 files changed, 103 insertions(+), 48 deletions(-)
diff --git a/drivers/iommu/dmar.c b/drivers/iommu/dmar.c
index 4ae6df2..c9aca88 100644
--- a/drivers/iommu/dmar.c
+++ b/drivers/iommu/dmar.c
@@ -43,10 +43,19 @@
#include "irq_remapping.h"
-/* No locks are needed as DMA remapping hardware unit
- * list is constructed at boot time and hotplug of
- * these units are not supported by the architecture.
+/*
+ * Assumptions:
+ * 1) The hotplug framework guarentees that DMAR unit will be hot-added
+ * before IO devices managed by that unit.
+ * 2) The hotplug framework guarantees that DMAR unit will be hot-removed
+ * after IO devices managed by that unit.
+ * 3) Hotplug events are rare.
+ *
+ * Locking rules for DMA and interrupt remapping related global data structures:
+ * 1) Use dmar_global_lock in process context
+ * 2) Use RCU in interrupt context
*/
+DECLARE_RWSEM(dmar_global_lock);
LIST_HEAD(dmar_drhd_units);
struct acpi_table_header * __initdata dmar_tbl;
@@ -565,6 +574,7 @@ int __init detect_intel_iommu(void)
{
int ret;
+ down_write(&dmar_global_lock);
ret = dmar_table_detect();
if (ret)
ret = check_zero_address();
@@ -582,6 +592,7 @@ int __init detect_intel_iommu(void)
}
early_acpi_os_unmap_memory((void __iomem *)dmar_tbl, dmar_tbl_size);
dmar_tbl = NULL;
+ up_write(&dmar_global_lock);
return ret ? 1 : -ENODEV;
}
@@ -1394,10 +1405,12 @@ static int __init dmar_free_unused_resources(void)
if (irq_remapping_enabled || intel_iommu_enabled)
return 0;
+ down_write(&dmar_global_lock);
list_for_each_entry_safe(dmaru, dmaru_n, &dmar_drhd_units, list) {
list_del(&dmaru->list);
dmar_free_drhd(dmaru);
}
+ up_write(&dmar_global_lock);
return 0;
}
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index bb98e37..50d639a 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -3637,11 +3637,13 @@ static int device_notifier(struct notifier_block *nb,
if (!domain)
return 0;
+ down_read(&dmar_global_lock);
domain_remove_one_dev_info(domain, pdev);
if (!(domain->flags & DOMAIN_FLAG_VIRTUAL_MACHINE) &&
!(domain->flags & DOMAIN_FLAG_STATIC_IDENTITY) &&
list_empty(&domain->devices))
domain_exit(domain);
+ up_read(&dmar_global_lock);
return 0;
}
@@ -3659,6 +3661,13 @@ int __init intel_iommu_init(void)
/* VT-d is required for a TXT/tboot launch, so enforce that */
force_on = tboot_force_iommu();
+ if (iommu_init_mempool()) {
+ if (force_on)
+ panic("tboot: Failed to initialize iommu memory\n");
+ return -ENOMEM;
+ }
+
+ down_write(&dmar_global_lock);
if (dmar_table_init()) {
if (force_on)
panic("tboot: Failed to initialize DMAR table\n");
@@ -3681,12 +3690,6 @@ int __init intel_iommu_init(void)
if (no_iommu || dmar_disabled)
goto out_free_dmar;
- if (iommu_init_mempool()) {
- if (force_on)
- panic("tboot: Failed to initialize iommu memory\n");
- goto out_free_dmar;
- }
-
if (list_empty(&dmar_rmrr_units))
printk(KERN_INFO "DMAR: No RMRR found\n");
@@ -3696,7 +3699,7 @@ int __init intel_iommu_init(void)
if (dmar_init_reserved_ranges()) {
if (force_on)
panic("tboot: Failed to reserve iommu ranges\n");
- goto out_free_mempool;
+ goto out_free_reserved_range;
}
init_no_remapping_devices();
@@ -3708,6 +3711,7 @@ int __init intel_iommu_init(void)
printk(KERN_ERR "IOMMU: dmar init failed\n");
goto out_free_reserved_range;
}
+ up_write(&dmar_global_lock);
printk(KERN_INFO
"PCI-DMA: Intel(R) Virtualization Technology for Directed I/O\n");
@@ -3729,10 +3733,10 @@ int __init intel_iommu_init(void)
out_free_reserved_range:
put_iova_domain(&reserved_iova_list);
-out_free_mempool:
- iommu_exit_mempool();
out_free_dmar:
intel_iommu_free_dmars();
+ up_write(&dmar_global_lock);
+ iommu_exit_mempool();
return ret;
}
diff --git a/drivers/iommu/intel_irq_remapping.c b/drivers/iommu/intel_irq_remapping.c
index ef5f65d..9b17489 100644
--- a/drivers/iommu/intel_irq_remapping.c
+++ b/drivers/iommu/intel_irq_remapping.c
@@ -38,6 +38,17 @@ static struct ioapic_scope ir_ioapic[MAX_IO_APICS];
static struct hpet_scope ir_hpet[MAX_HPET_TBS];
static int ir_ioapic_num, ir_hpet_num;
+/*
+ * Lock ordering:
+ * ->dmar_global_lock
+ * ->irq_2_ir_lock
+ * ->qi->q_lock
+ * ->iommu->register_lock
+ * Note:
+ * intel_irq_remap_ops.{supported,prepare,enable,disable,reenable} are called
+ * in single-threaded environment with interrupt disabled, so no need to tabke
+ * the dmar_global_lock.
+ */
static DEFINE_RAW_SPINLOCK(irq_2_ir_lock);
static int __init parse_ioapics_under_ir(void);
@@ -307,12 +318,14 @@ static int set_ioapic_sid(struct irte *irte, int apic)
if (!irte)
return -1;
+ down_read(&dmar_global_lock);
for (i = 0; i < MAX_IO_APICS; i++) {
if (ir_ioapic[i].id == apic) {
sid = (ir_ioapic[i].bus << 8) | ir_ioapic[i].devfn;
break;
}
}
+ up_read(&dmar_global_lock);
if (sid == 0) {
pr_warning("Failed to set source-id of IOAPIC (%d)\n", apic);
@@ -332,12 +345,14 @@ static int set_hpet_sid(struct irte *irte, u8 id)
if (!irte)
return -1;
+ down_read(&dmar_global_lock);
for (i = 0; i < MAX_HPET_TBS; i++) {
if (ir_hpet[i].id == id) {
sid = (ir_hpet[i].bus << 8) | ir_hpet[i].devfn;
break;
}
}
+ up_read(&dmar_global_lock);
if (sid == 0) {
pr_warning("Failed to set source-id of HPET block (%d)\n", id);
@@ -794,10 +809,16 @@ static int __init parse_ioapics_under_ir(void)
static int __init ir_dev_scope_init(void)
{
+ int ret;
+
if (!irq_remapping_enabled)
return 0;
- return dmar_dev_scope_init();
+ down_write(&dmar_global_lock);
+ ret = dmar_dev_scope_init();
+ up_write(&dmar_global_lock);
+
+ return ret;
}
rootfs_initcall(ir_dev_scope_init);
@@ -878,23 +899,27 @@ static int intel_setup_ioapic_entry(int irq,
struct io_apic_irq_attr *attr)
{
int ioapic_id = mpc_ioapic_id(attr->ioapic);
- struct intel_iommu *iommu = map_ioapic_to_ir(ioapic_id);
+ struct intel_iommu *iommu;
struct IR_IO_APIC_route_entry *entry;
struct irte irte;
int index;
+ down_read(&dmar_global_lock);
+ iommu = map_ioapic_to_ir(ioapic_id);
if (!iommu) {
pr_warn("No mapping iommu for ioapic %d\n", ioapic_id);
- return -ENODEV;
- }
-
- entry = (struct IR_IO_APIC_route_entry *)route_entry;
-
- index = alloc_irte(iommu, irq, 1);
- if (index < 0) {
- pr_warn("Failed to allocate IRTE for ioapic %d\n", ioapic_id);
- return -ENOMEM;
+ index = -ENODEV;
+ } else {
+ index = alloc_irte(iommu, irq, 1);
+ if (index < 0) {
+ pr_warn("Failed to allocate IRTE for ioapic %d\n",
+ ioapic_id);
+ index = -ENOMEM;
+ }
}
+ up_read(&dmar_global_lock);
+ if (index < 0)
+ return index;
prepare_irte(&irte, vector, destination);
@@ -913,6 +938,7 @@ static int intel_setup_ioapic_entry(int irq,
irte.avail, irte.vector, irte.dest_id,
irte.sid, irte.sq, irte.svt);
+ entry = (struct IR_IO_APIC_route_entry *)route_entry;
memset(entry, 0, sizeof(*entry));
entry->index2 = (index >> 15) & 0x1;
@@ -1043,20 +1069,23 @@ static int intel_msi_alloc_irq(struct pci_dev *dev, int irq, int nvec)
struct intel_iommu *iommu;
int index;
+ down_read(&dmar_global_lock);
iommu = map_dev_to_ir(dev);
if (!iommu) {
printk(KERN_ERR
"Unable to map PCI %s to iommu\n", pci_name(dev));
- return -ENOENT;
+ index = -ENOENT;
+ } else {
+ index = alloc_irte(iommu, irq, nvec);
+ if (index < 0) {
+ printk(KERN_ERR
+ "Unable to allocate %d IRTE for PCI %s\n",
+ nvec, pci_name(dev));
+ index = -ENOSPC;
+ }
}
+ up_read(&dmar_global_lock);
- index = alloc_irte(iommu, irq, nvec);
- if (index < 0) {
- printk(KERN_ERR
- "Unable to allocate %d IRTE for PCI %s\n", nvec,
- pci_name(dev));
- return -ENOSPC;
- }
return index;
}
@@ -1064,33 +1093,40 @@ static int intel_msi_setup_irq(struct pci_dev *pdev, unsigned int irq,
int index, int sub_handle)
{
struct intel_iommu *iommu;
+ int ret = -ENOENT;
+ down_read(&dmar_global_lock);
iommu = map_dev_to_ir(pdev);
- if (!iommu)
- return -ENOENT;
- /*
- * setup the mapping between the irq and the IRTE
- * base index, the sub_handle pointing to the
- * appropriate interrupt remap table entry.
- */
- set_irte_irq(irq, iommu, index, sub_handle);
+ if (iommu) {
+ /*
+ * setup the mapping between the irq and the IRTE
+ * base index, the sub_handle pointing to the
+ * appropriate interrupt remap table entry.
+ */
+ set_irte_irq(irq, iommu, index, sub_handle);
+ ret = 0;
+ }
+ up_read(&dmar_global_lock);
- return 0;
+ return ret;
}
static int intel_setup_hpet_msi(unsigned int irq, unsigned int id)
{
- struct intel_iommu *iommu = map_hpet_to_ir(id);
+ int ret = -1;
+ struct intel_iommu *iommu;
int index;
- if (!iommu)
- return -1;
-
- index = alloc_irte(iommu, irq, 1);
- if (index < 0)
- return -1;
+ down_read(&dmar_global_lock);
+ iommu = map_hpet_to_ir(id);
+ if (iommu) {
+ index = alloc_irte(iommu, irq, 1);
+ if (index >= 0)
+ ret = 0;
+ }
+ up_read(&dmar_global_lock);
- return 0;
+ return ret;
}
struct irq_remap_ops intel_irq_remap_ops = {
diff --git a/include/linux/dmar.h b/include/linux/dmar.h
index 4b77fd8..8f06a01 100644
--- a/include/linux/dmar.h
+++ b/include/linux/dmar.h
@@ -25,6 +25,7 @@
#include <linux/types.h>
#include <linux/msi.h>
#include <linux/irqreturn.h>
+#include <linux/rwsem.h>
struct acpi_dmar_header;
@@ -48,6 +49,7 @@ struct dmar_drhd_unit {
struct intel_iommu *iommu;
};
+extern struct rw_semaphore dmar_global_lock;
extern struct list_head dmar_drhd_units;
#define for_each_drhd_unit(drhd) \
--
1.7.10.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [Patch Part2 V2 14/17] iommu/vt-d: use RCU to protect global resources in interrupt context
2014-02-19 6:07 [Patch Part2 V2 00/17] Enhance DMAR drivers to handle PCI/memory hotplug events Jiang Liu
` (12 preceding siblings ...)
2014-02-19 6:07 ` [Patch Part2 V2 13/17] iommu/vt-d: introduce a rwsem to protect global data structures Jiang Liu
@ 2014-02-19 6:07 ` Jiang Liu
2014-02-19 6:07 ` [Patch Part2 V2 15/17] iommu/vt-d, PCI: update DRHD/RMRR/ATSR device scope caches when PCI hotplug happens Jiang Liu
` (3 subsequent siblings)
17 siblings, 0 replies; 22+ messages in thread
From: Jiang Liu @ 2014-02-19 6:07 UTC (permalink / raw)
To: Joerg Roedel, David Woodhouse, Yinghai Lu, Bjorn Helgaas,
Dan Williams, Vinod Koul, Rafael J . Wysocki
Cc: Jiang Liu, Ashok Raj, Yijing Wang, Tony Luck, iommu, linux-pci,
linux-kernel, dmaengine
Global DMA and interrupt remapping resources may be accessed in
interrupt context, so use RCU instead of rwsem to protect them
in such cases.
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
drivers/iommu/dmar.c | 33 ++++++++++++++++++++-------------
drivers/iommu/intel-iommu.c | 20 ++++++++++++++++----
include/linux/dmar.h | 23 +++++++++++++++++------
3 files changed, 53 insertions(+), 23 deletions(-)
diff --git a/drivers/iommu/dmar.c b/drivers/iommu/dmar.c
index c9aca88..6e4d851 100644
--- a/drivers/iommu/dmar.c
+++ b/drivers/iommu/dmar.c
@@ -71,13 +71,13 @@ static void __init dmar_register_drhd_unit(struct dmar_drhd_unit *drhd)
* the very end.
*/
if (drhd->include_all)
- list_add_tail(&drhd->list, &dmar_drhd_units);
+ list_add_tail_rcu(&drhd->list, &dmar_drhd_units);
else
- list_add(&drhd->list, &dmar_drhd_units);
+ list_add_rcu(&drhd->list, &dmar_drhd_units);
}
static int __init dmar_parse_one_dev_scope(struct acpi_dmar_device_scope *scope,
- struct pci_dev **dev, u16 segment)
+ struct pci_dev __rcu **dev, u16 segment)
{
struct pci_bus *bus;
struct pci_dev *pdev = NULL;
@@ -122,7 +122,9 @@ static int __init dmar_parse_one_dev_scope(struct acpi_dmar_device_scope *scope,
pci_name(pdev));
return -EINVAL;
}
- *dev = pdev;
+
+ rcu_assign_pointer(*dev, pdev);
+
return 0;
}
@@ -149,7 +151,7 @@ void *dmar_alloc_dev_scope(void *start, void *end, int *cnt)
}
int __init dmar_parse_dev_scope(void *start, void *end, int *cnt,
- struct pci_dev ***devices, u16 segment)
+ struct pci_dev __rcu ***devices, u16 segment)
{
struct acpi_dmar_device_scope *scope;
int index, ret;
@@ -177,7 +179,7 @@ int __init dmar_parse_dev_scope(void *start, void *end, int *cnt,
return 0;
}
-void dmar_free_dev_scope(struct pci_dev ***devices, int *cnt)
+void dmar_free_dev_scope(struct pci_dev __rcu ***devices, int *cnt)
{
int i;
struct pci_dev *tmp_dev;
@@ -186,9 +188,10 @@ void dmar_free_dev_scope(struct pci_dev ***devices, int *cnt)
for_each_active_dev_scope(*devices, *cnt, i, tmp_dev)
pci_dev_put(tmp_dev);
kfree(*devices);
- *devices = NULL;
- *cnt = 0;
}
+
+ *devices = NULL;
+ *cnt = 0;
}
/**
@@ -410,7 +413,7 @@ parse_dmar_table(void)
return ret;
}
-static int dmar_pci_device_match(struct pci_dev *devices[], int cnt,
+static int dmar_pci_device_match(struct pci_dev __rcu *devices[], int cnt,
struct pci_dev *dev)
{
int index;
@@ -431,11 +434,12 @@ static int dmar_pci_device_match(struct pci_dev *devices[], int cnt,
struct dmar_drhd_unit *
dmar_find_matched_drhd_unit(struct pci_dev *dev)
{
- struct dmar_drhd_unit *dmaru = NULL;
+ struct dmar_drhd_unit *dmaru;
struct acpi_dmar_hardware_unit *drhd;
dev = pci_physfn(dev);
+ rcu_read_lock();
for_each_drhd_unit(dmaru) {
drhd = container_of(dmaru->hdr,
struct acpi_dmar_hardware_unit,
@@ -443,14 +447,17 @@ dmar_find_matched_drhd_unit(struct pci_dev *dev)
if (dmaru->include_all &&
drhd->segment == pci_domain_nr(dev->bus))
- return dmaru;
+ goto out;
if (dmar_pci_device_match(dmaru->devices,
dmaru->devices_cnt, dev))
- return dmaru;
+ goto out;
}
+ dmaru = NULL;
+out:
+ rcu_read_unlock();
- return NULL;
+ return dmaru;
}
int __init dmar_dev_scope_init(void)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index 50d639a..e1679a6 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -385,14 +385,14 @@ struct dmar_rmrr_unit {
struct acpi_dmar_header *hdr; /* ACPI header */
u64 base_address; /* reserved base address*/
u64 end_address; /* reserved end address */
- struct pci_dev **devices; /* target devices */
+ struct pci_dev __rcu **devices; /* target devices */
int devices_cnt; /* target device count */
};
struct dmar_atsr_unit {
struct list_head list; /* list of ATSR units */
struct acpi_dmar_header *hdr; /* ACPI header */
- struct pci_dev **devices; /* target devices */
+ struct pci_dev __rcu **devices; /* target devices */
int devices_cnt; /* target device count */
u8 include_all:1; /* include all ports */
};
@@ -634,12 +634,15 @@ static void domain_update_iommu_superpage(struct dmar_domain *domain)
}
/* set iommu_superpage to the smallest common denominator */
+ rcu_read_lock();
for_each_active_iommu(iommu, drhd) {
mask &= cap_super_page_val(iommu->cap);
if (!mask) {
break;
}
}
+ rcu_read_unlock();
+
domain->iommu_superpage = fls(mask);
}
@@ -658,6 +661,7 @@ static struct intel_iommu *device_to_iommu(int segment, u8 bus, u8 devfn)
struct pci_dev *dev;
int i;
+ rcu_read_lock();
for_each_active_iommu(iommu, drhd) {
if (segment != drhd->segment)
continue;
@@ -677,6 +681,7 @@ static struct intel_iommu *device_to_iommu(int segment, u8 bus, u8 devfn)
}
iommu = NULL;
out:
+ rcu_read_unlock();
return iommu;
}
@@ -1535,10 +1540,12 @@ static void domain_exit(struct dmar_domain *domain)
dma_pte_free_pagetable(domain, 0, DOMAIN_MAX_PFN(domain->gaw));
/* clear attached or cached domains */
+ rcu_read_lock();
for_each_active_iommu(iommu, drhd)
if (domain->flags & DOMAIN_FLAG_VIRTUAL_MACHINE ||
test_bit(iommu->seq_id, domain->iommu_bmp))
iommu_detach_domain(domain, iommu);
+ rcu_read_unlock();
free_domain_mem(domain);
}
@@ -2338,6 +2345,7 @@ static bool device_has_rmrr(struct pci_dev *dev)
struct pci_dev *tmp;
int i;
+ rcu_read_lock();
for_each_rmrr_units(rmrr) {
/*
* Return TRUE if this RMRR contains the device that
@@ -2346,9 +2354,11 @@ static bool device_has_rmrr(struct pci_dev *dev)
for_each_active_dev_scope(rmrr->devices,
rmrr->devices_cnt, i, tmp)
if (tmp == dev) {
+ rcu_read_unlock();
return true;
}
}
+ rcu_read_unlock();
return false;
}
@@ -3512,7 +3522,7 @@ int __init dmar_parse_one_atsr(struct acpi_dmar_header *hdr)
atsru->hdr = hdr;
atsru->include_all = atsr->flags & 0x1;
- list_add(&atsru->list, &dmar_atsr_units);
+ list_add_rcu(&atsru->list, &dmar_atsr_units);
return 0;
}
@@ -3574,6 +3584,7 @@ int dmar_find_matched_atsr_unit(struct pci_dev *dev)
if (!bridge)
return 0;
+ rcu_read_lock();
list_for_each_entry_rcu(atsru, &dmar_atsr_units, list) {
atsr = container_of(atsru->hdr, struct acpi_dmar_atsr, header);
if (atsr->segment != pci_domain_nr(dev->bus))
@@ -3588,6 +3599,7 @@ int dmar_find_matched_atsr_unit(struct pci_dev *dev)
}
ret = 0;
out:
+ rcu_read_unlock();
return ret;
}
@@ -3604,7 +3616,7 @@ int __init dmar_parse_rmrr_atsr_dev(void)
return ret;
}
- list_for_each_entry(atsr, &dmar_atsr_units, list) {
+ list_for_each_entry_rcu(atsr, &dmar_atsr_units, list) {
ret = atsr_parse_dev(atsr);
if (ret)
return ret;
diff --git a/include/linux/dmar.h b/include/linux/dmar.h
index 8f06a01..bedebab 100644
--- a/include/linux/dmar.h
+++ b/include/linux/dmar.h
@@ -26,6 +26,7 @@
#include <linux/msi.h>
#include <linux/irqreturn.h>
#include <linux/rwsem.h>
+#include <linux/rcupdate.h>
struct acpi_dmar_header;
@@ -41,7 +42,7 @@ struct dmar_drhd_unit {
struct list_head list; /* list of drhd units */
struct acpi_dmar_header *hdr; /* ACPI header */
u64 reg_base_addr; /* register base address*/
- struct pci_dev **devices; /* target device array */
+ struct pci_dev __rcu **devices;/* target device array */
int devices_cnt; /* target device count */
u16 segment; /* PCI domain */
u8 ignored:1; /* ignore drhd */
@@ -53,22 +54,31 @@ extern struct rw_semaphore dmar_global_lock;
extern struct list_head dmar_drhd_units;
#define for_each_drhd_unit(drhd) \
- list_for_each_entry(drhd, &dmar_drhd_units, list)
+ list_for_each_entry_rcu(drhd, &dmar_drhd_units, list)
#define for_each_active_drhd_unit(drhd) \
- list_for_each_entry(drhd, &dmar_drhd_units, list) \
+ list_for_each_entry_rcu(drhd, &dmar_drhd_units, list) \
if (drhd->ignored) {} else
#define for_each_active_iommu(i, drhd) \
- list_for_each_entry(drhd, &dmar_drhd_units, list) \
+ list_for_each_entry_rcu(drhd, &dmar_drhd_units, list) \
if (i=drhd->iommu, drhd->ignored) {} else
#define for_each_iommu(i, drhd) \
- list_for_each_entry(drhd, &dmar_drhd_units, list) \
+ list_for_each_entry_rcu(drhd, &dmar_drhd_units, list) \
if (i=drhd->iommu, 0) {} else
+static inline bool dmar_rcu_check(void)
+{
+ return rwsem_is_locked(&dmar_global_lock) ||
+ system_state == SYSTEM_BOOTING;
+}
+
+#define dmar_rcu_dereference(p) rcu_dereference_check((p), dmar_rcu_check())
+
#define for_each_dev_scope(a, c, p, d) \
- for ((p) = 0; ((d) = (p) < (c) ? (a)[(p)] : NULL, (p) < (c)); (p)++)
+ for ((p) = 0; ((d) = (p) < (c) ? dmar_rcu_dereference((a)[(p)]) : \
+ NULL, (p) < (c)); (p)++)
#define for_each_active_dev_scope(a, c, p, d) \
for_each_dev_scope((a), (c), (p), (d)) if (!(d)) { continue; } else
@@ -78,6 +88,7 @@ extern int dmar_dev_scope_init(void);
extern int dmar_parse_dev_scope(void *start, void *end, int *cnt,
struct pci_dev ***devices, u16 segment);
extern void *dmar_alloc_dev_scope(void *start, void *end, int *cnt);
+extern void dmar_free_dev_scope(struct pci_dev __rcu ***devices, int *cnt);
extern void dmar_free_dev_scope(struct pci_dev ***devices, int *cnt);
/* Intel IOMMU detection */
--
1.7.10.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [Patch Part2 V2 15/17] iommu/vt-d, PCI: update DRHD/RMRR/ATSR device scope caches when PCI hotplug happens
2014-02-19 6:07 [Patch Part2 V2 00/17] Enhance DMAR drivers to handle PCI/memory hotplug events Jiang Liu
` (13 preceding siblings ...)
2014-02-19 6:07 ` [Patch Part2 V2 14/17] iommu/vt-d: use RCU to protect global resources in interrupt context Jiang Liu
@ 2014-02-19 6:07 ` Jiang Liu
2014-03-06 18:25 ` David Woodhouse
2014-02-19 6:07 ` [Patch Part2 V2 16/17] iommu/vt-d, PCI: unify the way to process DMAR device scope array Jiang Liu
` (2 subsequent siblings)
17 siblings, 1 reply; 22+ messages in thread
From: Jiang Liu @ 2014-02-19 6:07 UTC (permalink / raw)
To: Joerg Roedel, David Woodhouse, Yinghai Lu, Bjorn Helgaas,
Dan Williams, Vinod Koul, Rafael J . Wysocki
Cc: Jiang Liu, Ashok Raj, Yijing Wang, Tony Luck, iommu, linux-pci,
linux-kernel, dmaengine
Current Intel DMAR/IOMMU driver assumes that all PCI devices associated
with DMAR/RMRR/ATSR device scope arrays are created at boot time and
won't change at runtime, so it caches pointers of associated PCI device
object. That assumption may be wrong now due to:
1) introduction of PCI host bridge hotplug
2) PCI device hotplug through sysfs interfaces.
Wang Yijing has tried to solve this issue by caching <bus, dev, func>
tupple instead of the PCI device object pointer, but that's still
unreliable because PCI bus number may change in case of hotplug.
Please refer to http://lkml.org/lkml/2013/11/5/64
Message from Yingjing's mail:
after remove and rescan a pci device
[ 611.857095] dmar: DRHD: handling fault status reg 2
[ 611.857109] dmar: DMAR:[DMA Read] Request device [86:00.3] fault addr ffff7000
[ 611.857109] DMAR:[fault reason 02] Present bit in context entry is clear
[ 611.857524] dmar: DRHD: handling fault status reg 102
[ 611.857534] dmar: DMAR:[DMA Read] Request device [86:00.3] fault addr ffff6000
[ 611.857534] DMAR:[fault reason 02] Present bit in context entry is clear
[ 611.857936] dmar: DRHD: handling fault status reg 202
[ 611.857947] dmar: DMAR:[DMA Read] Request device [86:00.3] fault addr ffff5000
[ 611.857947] DMAR:[fault reason 02] Present bit in context entry is clear
[ 611.858351] dmar: DRHD: handling fault status reg 302
[ 611.858362] dmar: DMAR:[DMA Read] Request device [86:00.3] fault addr ffff4000
[ 611.858362] DMAR:[fault reason 02] Present bit in context entry is clear
[ 611.860819] IPv6: ADDRCONF(NETDEV_UP): eth3: link is not ready
[ 611.860983] dmar: DRHD: handling fault status reg 402
[ 611.860995] dmar: INTR-REMAP: Request device [[86:00.3] fault index a4
[ 611.860995] INTR-REMAP:[fault reason 34] Present field in the IRTE entry is clear
This patch introduces a new mechanism to update the DRHD/RMRR/ATSR device scope
caches by hooking PCI bus notification.
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
drivers/iommu/dmar.c | 207 +++++++++++++++++++++++++++++++++++++++++++
drivers/iommu/intel-iommu.c | 54 +++++++++++
include/linux/dmar.h | 24 ++++-
3 files changed, 283 insertions(+), 2 deletions(-)
diff --git a/drivers/iommu/dmar.c b/drivers/iommu/dmar.c
index 6e4d851..bf6bfd1 100644
--- a/drivers/iommu/dmar.c
+++ b/drivers/iommu/dmar.c
@@ -194,6 +194,209 @@ void dmar_free_dev_scope(struct pci_dev __rcu ***devices, int *cnt)
*cnt = 0;
}
+/* Optimize out kzalloc()/kfree() for normal cases */
+static char dmar_pci_notify_info_buf[64];
+
+static struct dmar_pci_notify_info *
+dmar_alloc_pci_notify_info(struct pci_dev *dev, unsigned long event)
+{
+ int level = 0;
+ size_t size;
+ struct pci_dev *tmp;
+ struct dmar_pci_notify_info *info;
+
+ BUG_ON(dev->is_virtfn);
+
+ /* Only generate path[] for device addition event */
+ if (event == BUS_NOTIFY_ADD_DEVICE)
+ for (tmp = dev; tmp; tmp = tmp->bus->self)
+ level++;
+
+ size = sizeof(*info) + level * sizeof(struct acpi_dmar_pci_path);
+ if (size <= sizeof(dmar_pci_notify_info_buf)) {
+ info = (struct dmar_pci_notify_info *)dmar_pci_notify_info_buf;
+ } else {
+ info = kzalloc(size, GFP_KERNEL);
+ if (!info) {
+ pr_warn("Out of memory when allocating notify_info "
+ "for %s.\n", pci_name(dev));
+ return NULL;
+ }
+ }
+
+ info->event = event;
+ info->dev = dev;
+ info->seg = pci_domain_nr(dev->bus);
+ info->level = level;
+ if (event == BUS_NOTIFY_ADD_DEVICE) {
+ for (tmp = dev, level--; tmp; tmp = tmp->bus->self) {
+ info->path[level].device = PCI_SLOT(tmp->devfn);
+ info->path[level].function = PCI_FUNC(tmp->devfn);
+ if (pci_is_root_bus(tmp->bus))
+ info->bus = tmp->bus->number;
+ }
+ }
+
+ return info;
+}
+
+static inline void dmar_free_pci_notify_info(struct dmar_pci_notify_info *info)
+{
+ if ((void *)info != dmar_pci_notify_info_buf)
+ kfree(info);
+}
+
+static bool dmar_match_pci_path(struct dmar_pci_notify_info *info, int bus,
+ struct acpi_dmar_pci_path *path, int count)
+{
+ int i;
+
+ if (info->bus != bus)
+ return false;
+ if (info->level != count)
+ return false;
+
+ for (i = 0; i < count; i++) {
+ if (path[i].device != info->path[i].device ||
+ path[i].function != info->path[i].function)
+ return false;
+ }
+
+ return true;
+}
+
+/* Return: > 0 if match found, 0 if no match found, < 0 if error happens */
+int dmar_insert_dev_scope(struct dmar_pci_notify_info *info,
+ void *start, void*end, u16 segment,
+ struct pci_dev __rcu **devices, int devices_cnt)
+{
+ int i, level;
+ struct pci_dev *tmp, *dev = info->dev;
+ struct acpi_dmar_device_scope *scope;
+ struct acpi_dmar_pci_path *path;
+
+ if (segment != info->seg)
+ return 0;
+
+ for (; start < end; start += scope->length) {
+ scope = start;
+ if (scope->entry_type != ACPI_DMAR_SCOPE_TYPE_ENDPOINT &&
+ scope->entry_type != ACPI_DMAR_SCOPE_TYPE_BRIDGE)
+ continue;
+
+ path = (struct acpi_dmar_pci_path *)(scope + 1);
+ level = (scope->length - sizeof(*scope)) / sizeof(*path);
+ if (!dmar_match_pci_path(info, scope->bus, path, level))
+ continue;
+
+ if ((scope->entry_type == ACPI_DMAR_SCOPE_TYPE_ENDPOINT) ^
+ (dev->hdr_type == PCI_HEADER_TYPE_NORMAL)) {
+ pr_warn("Device scope type does not match for %s\n",
+ pci_name(dev));
+ return -EINVAL;
+ }
+
+ for_each_dev_scope(devices, devices_cnt, i, tmp)
+ if (tmp == NULL) {
+ rcu_assign_pointer(devices[i],
+ pci_dev_get(dev));
+ return 1;
+ }
+ BUG_ON(i >= devices_cnt);
+ }
+
+ return 0;
+}
+
+int dmar_remove_dev_scope(struct dmar_pci_notify_info *info, u16 segment,
+ struct pci_dev __rcu **devices, int count)
+{
+ int index;
+ struct pci_dev *tmp;
+
+ if (info->seg != segment)
+ return 0;
+
+ for_each_active_dev_scope(devices, count, index, tmp)
+ if (tmp == info->dev) {
+ rcu_assign_pointer(devices[index], NULL);
+ synchronize_rcu();
+ pci_dev_put(tmp);
+ return 1;
+ }
+
+ return 0;
+}
+
+static int dmar_pci_bus_add_dev(struct dmar_pci_notify_info *info)
+{
+ int ret = 0;
+ struct dmar_drhd_unit *dmaru;
+ struct acpi_dmar_hardware_unit *drhd;
+
+ for_each_drhd_unit(dmaru) {
+ if (dmaru->include_all)
+ continue;
+
+ drhd = container_of(dmaru->hdr,
+ struct acpi_dmar_hardware_unit, header);
+ ret = dmar_insert_dev_scope(info, (void *)(drhd + 1),
+ ((void *)drhd) + drhd->header.length,
+ dmaru->segment,
+ dmaru->devices, dmaru->devices_cnt);
+ if (ret != 0)
+ break;
+ }
+ if (ret >= 0)
+ ret = dmar_iommu_notify_scope_dev(info);
+
+ return ret;
+}
+
+static void dmar_pci_bus_del_dev(struct dmar_pci_notify_info *info)
+{
+ struct dmar_drhd_unit *dmaru;
+
+ for_each_drhd_unit(dmaru)
+ if (dmar_remove_dev_scope(info, dmaru->segment,
+ dmaru->devices, dmaru->devices_cnt))
+ break;
+ dmar_iommu_notify_scope_dev(info);
+}
+
+static int dmar_pci_bus_notifier(struct notifier_block *nb,
+ unsigned long action, void *data)
+{
+ struct pci_dev *pdev = to_pci_dev(data);
+ struct dmar_pci_notify_info *info;
+
+ /* Only care about add/remove events for physical functions */
+ if (pdev->is_virtfn)
+ return NOTIFY_DONE;
+ if (action != BUS_NOTIFY_ADD_DEVICE && action != BUS_NOTIFY_DEL_DEVICE)
+ return NOTIFY_DONE;
+
+ info = dmar_alloc_pci_notify_info(pdev, action);
+ if (!info)
+ return NOTIFY_DONE;
+
+ down_write(&dmar_global_lock);
+ if (action == BUS_NOTIFY_ADD_DEVICE)
+ dmar_pci_bus_add_dev(info);
+ else if (action == BUS_NOTIFY_DEL_DEVICE)
+ dmar_pci_bus_del_dev(info);
+ up_write(&dmar_global_lock);
+
+ dmar_free_pci_notify_info(info);
+
+ return NOTIFY_OK;
+}
+
+static struct notifier_block dmar_pci_bus_nb = {
+ .notifier_call = dmar_pci_bus_notifier,
+ .priority = INT_MIN,
+};
+
/**
* dmar_parse_one_drhd - parses exactly one DMA remapping hardware definition
* structure which uniquely represent one DMA remapping hardware unit
@@ -482,6 +685,8 @@ int __init dmar_dev_scope_init(void)
if (ret)
goto fail;
+ bus_register_notifier(&pci_bus_type, &dmar_pci_bus_nb);
+
dmar_dev_scope_initialized = 1;
return 0;
@@ -1412,6 +1617,8 @@ static int __init dmar_free_unused_resources(void)
if (irq_remapping_enabled || intel_iommu_enabled)
return 0;
+ bus_unregister_notifier(&pci_bus_type, &dmar_pci_bus_nb);
+
down_write(&dmar_global_lock);
list_for_each_entry_safe(dmaru, dmaru_n, &dmar_drhd_units, list) {
list_del(&dmaru->list);
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index e1679a6..d9c0dc5 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -3625,6 +3625,60 @@ int __init dmar_parse_rmrr_atsr_dev(void)
return 0;
}
+int dmar_iommu_notify_scope_dev(struct dmar_pci_notify_info *info)
+{
+ int ret = 0;
+ struct dmar_rmrr_unit *rmrru;
+ struct dmar_atsr_unit *atsru;
+ struct acpi_dmar_atsr *atsr;
+ struct acpi_dmar_reserved_memory *rmrr;
+
+ if (!intel_iommu_enabled && system_state != SYSTEM_BOOTING)
+ return 0;
+
+ list_for_each_entry(rmrru, &dmar_rmrr_units, list) {
+ rmrr = container_of(rmrru->hdr,
+ struct acpi_dmar_reserved_memory, header);
+ if (info->event == BUS_NOTIFY_ADD_DEVICE) {
+ ret = dmar_insert_dev_scope(info, (void *)(rmrr + 1),
+ ((void *)rmrr) + rmrr->header.length,
+ rmrr->segment, rmrru->devices,
+ rmrru->devices_cnt);
+ if (ret > 0)
+ break;
+ else if(ret < 0)
+ return ret;
+ } else if (info->event == BUS_NOTIFY_DEL_DEVICE) {
+ if (dmar_remove_dev_scope(info, rmrr->segment,
+ rmrru->devices, rmrru->devices_cnt))
+ break;
+ }
+ }
+
+ list_for_each_entry(atsru, &dmar_atsr_units, list) {
+ if (atsru->include_all)
+ continue;
+
+ atsr = container_of(atsru->hdr, struct acpi_dmar_atsr, header);
+ if (info->event == BUS_NOTIFY_ADD_DEVICE) {
+ ret = dmar_insert_dev_scope(info, (void *)(atsr + 1),
+ (void *)atsr + atsr->header.length,
+ atsr->segment, atsru->devices,
+ atsru->devices_cnt);
+ if (ret > 0)
+ break;
+ else if(ret < 0)
+ return ret;
+ } else if (info->event == BUS_NOTIFY_DEL_DEVICE) {
+ if (dmar_remove_dev_scope(info, atsr->segment,
+ atsru->devices, atsru->devices_cnt))
+ break;
+ }
+ }
+
+ return 0;
+}
+
/*
* Here we only respond to action of unbound device from driver.
*
diff --git a/include/linux/dmar.h b/include/linux/dmar.h
index bedebab..4e19643 100644
--- a/include/linux/dmar.h
+++ b/include/linux/dmar.h
@@ -50,6 +50,15 @@ struct dmar_drhd_unit {
struct intel_iommu *iommu;
};
+struct dmar_pci_notify_info {
+ struct pci_dev *dev;
+ unsigned long event;
+ int bus;
+ u16 seg;
+ u16 level;
+ struct acpi_dmar_pci_path path[];
+} __attribute__((packed));
+
extern struct rw_semaphore dmar_global_lock;
extern struct list_head dmar_drhd_units;
@@ -89,12 +98,18 @@ extern int dmar_parse_dev_scope(void *start, void *end, int *cnt,
struct pci_dev ***devices, u16 segment);
extern void *dmar_alloc_dev_scope(void *start, void *end, int *cnt);
extern void dmar_free_dev_scope(struct pci_dev __rcu ***devices, int *cnt);
-extern void dmar_free_dev_scope(struct pci_dev ***devices, int *cnt);
-
+extern int dmar_insert_dev_scope(struct dmar_pci_notify_info *info,
+ void *start, void*end, u16 segment,
+ struct pci_dev __rcu **devices,
+ int devices_cnt);
+extern int dmar_remove_dev_scope(struct dmar_pci_notify_info *info,
+ u16 segment, struct pci_dev __rcu **devices,
+ int count);
/* Intel IOMMU detection */
extern int detect_intel_iommu(void);
extern int enable_drhd_fault_handling(void);
#else
+struct dmar_pci_notify_info;
static inline int detect_intel_iommu(void)
{
return -ENODEV;
@@ -161,6 +176,7 @@ extern int iommu_detected, no_iommu;
extern int dmar_parse_rmrr_atsr_dev(void);
extern int dmar_parse_one_rmrr(struct acpi_dmar_header *header);
extern int dmar_parse_one_atsr(struct acpi_dmar_header *header);
+extern int dmar_iommu_notify_scope_dev(struct dmar_pci_notify_info *info);
extern int intel_iommu_init(void);
#else /* !CONFIG_INTEL_IOMMU: */
static inline int intel_iommu_init(void) { return -ENODEV; }
@@ -176,6 +192,10 @@ static inline int dmar_parse_rmrr_atsr_dev(void)
{
return 0;
}
+static inline int dmar_iommu_notify_scope_dev(struct dmar_pci_notify_info *info)
+{
+ return 0;
+}
#endif /* CONFIG_INTEL_IOMMU */
#endif /* __DMAR_H__ */
--
1.7.10.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* Re: [Patch Part2 V2 15/17] iommu/vt-d, PCI: update DRHD/RMRR/ATSR device scope caches when PCI hotplug happens
2014-02-19 6:07 ` [Patch Part2 V2 15/17] iommu/vt-d, PCI: update DRHD/RMRR/ATSR device scope caches when PCI hotplug happens Jiang Liu
@ 2014-03-06 18:25 ` David Woodhouse
2014-03-10 8:46 ` Jiang Liu
0 siblings, 1 reply; 22+ messages in thread
From: David Woodhouse @ 2014-03-06 18:25 UTC (permalink / raw)
To: Jiang Liu
Cc: Joerg Roedel, Yinghai Lu, Bjorn Helgaas, Dan Williams, Vinod Koul,
Rafael J . Wysocki, Tony Luck, linux-pci, linux-kernel, dmaengine,
iommu
[-- Attachment #1: Type: text/plain, Size: 1132 bytes --]
On Wed, 2014-02-19 at 14:07 +0800, Jiang Liu wrote:
> Current Intel DMAR/IOMMU driver assumes that all PCI devices
> associated
> with DMAR/RMRR/ATSR device scope arrays are created at boot time and
> won't change at runtime, so it caches pointers of associated PCI
> device
> object. That assumption may be wrong now due to:
> 1) introduction of PCI host bridge hotplug
> 2) PCI device hotplug through sysfs interfaces.
...
> This patch introduces a new mechanism to update the DRHD/RMRR/ATSR
> device scope caches by hooking PCI bus notification.
Hm, this seems overly complex. Can't we just abandon the
dmaru->devices[] array completely? In dmar_find_matched_drhd_unit()
can't we just compare the path in the scope entries directly to the PCI
device we are trying to find a DRHD for?
We then cache the result in dev->archdata.iommu for ever (well, until
hot-unplug) anyway, so it shouldn't even be any less efficient to do it
on demand, right?
That's true at least for dmar_find_matched_drhd_unit(). Perhaps we'd
need to find a way to cache the result for
dmar_find_matched_atsr_unit()?
--
dwmw2
[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5745 bytes --]
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Patch Part2 V2 15/17] iommu/vt-d, PCI: update DRHD/RMRR/ATSR device scope caches when PCI hotplug happens
2014-03-06 18:25 ` David Woodhouse
@ 2014-03-10 8:46 ` Jiang Liu
2014-03-10 13:04 ` David Woodhouse
0 siblings, 1 reply; 22+ messages in thread
From: Jiang Liu @ 2014-03-10 8:46 UTC (permalink / raw)
To: David Woodhouse
Cc: Joerg Roedel, Yinghai Lu, Bjorn Helgaas, Dan Williams, Vinod Koul,
Rafael J . Wysocki, Tony Luck, linux-pci, linux-kernel, dmaengine,
iommu
Hi David,
Good suggestion! It should make hotplug logic simpler too.
Will try to work out some patches for it.
Thanks!
Gerry
On 2014/3/7 2:25, David Woodhouse wrote:
> On Wed, 2014-02-19 at 14:07 +0800, Jiang Liu wrote:
>> Current Intel DMAR/IOMMU driver assumes that all PCI devices
>> associated
>> with DMAR/RMRR/ATSR device scope arrays are created at boot time and
>> won't change at runtime, so it caches pointers of associated PCI
>> device
>> object. That assumption may be wrong now due to:
>> 1) introduction of PCI host bridge hotplug
>> 2) PCI device hotplug through sysfs interfaces.
> ...
>> This patch introduces a new mechanism to update the DRHD/RMRR/ATSR
>> device scope caches by hooking PCI bus notification.
>
> Hm, this seems overly complex. Can't we just abandon the
> dmaru->devices[] array completely? In dmar_find_matched_drhd_unit()
> can't we just compare the path in the scope entries directly to the PCI
> device we are trying to find a DRHD for?
>
> We then cache the result in dev->archdata.iommu for ever (well, until
> hot-unplug) anyway, so it shouldn't even be any less efficient to do it
> on demand, right?
>
> That's true at least for dmar_find_matched_drhd_unit(). Perhaps we'd
> need to find a way to cache the result for
> dmar_find_matched_atsr_unit()?
>
^ permalink raw reply [flat|nested] 22+ messages in thread
* Re: [Patch Part2 V2 15/17] iommu/vt-d, PCI: update DRHD/RMRR/ATSR device scope caches when PCI hotplug happens
2014-03-10 8:46 ` Jiang Liu
@ 2014-03-10 13:04 ` David Woodhouse
0 siblings, 0 replies; 22+ messages in thread
From: David Woodhouse @ 2014-03-10 13:04 UTC (permalink / raw)
To: Jiang Liu
Cc: Joerg Roedel, Yinghai Lu, Bjorn Helgaas, Dan Williams, Vinod Koul,
Rafael J . Wysocki, Tony Luck, linux-pci, linux-kernel, dmaengine,
iommu
[-- Attachment #1: Type: text/plain, Size: 716 bytes --]
On Mon, 2014-03-10 at 16:46 +0800, Jiang Liu wrote:
> Hi David,
> Good suggestion! It should make hotplug logic simpler too.
> Will try to work out some patches for it.
Before doing so, please be aware of the WIP at
http://git.infradead.org/users/dwmw2/linux-andd.git
This is the support for DMA mapping for ACPI devices. Largely thrown
together on a plane above the Atlantic over the last day, so mostly
untested, but a lot of it is fairly simply switching to using 'struct
device' instead of assuming 'struct pci_dev' everywhere.
For now I've taken the simple approach of using the ->devices array, and
*expanding* it to contain the bus+devfn to use for ACPI devices too.
--
dwmw2
[-- Attachment #2: smime.p7s --]
[-- Type: application/x-pkcs7-signature, Size: 5745 bytes --]
^ permalink raw reply [flat|nested] 22+ messages in thread
* [Patch Part2 V2 16/17] iommu/vt-d, PCI: unify the way to process DMAR device scope array
2014-02-19 6:07 [Patch Part2 V2 00/17] Enhance DMAR drivers to handle PCI/memory hotplug events Jiang Liu
` (14 preceding siblings ...)
2014-02-19 6:07 ` [Patch Part2 V2 15/17] iommu/vt-d, PCI: update DRHD/RMRR/ATSR device scope caches when PCI hotplug happens Jiang Liu
@ 2014-02-19 6:07 ` Jiang Liu
2014-02-19 6:07 ` [Patch Part2 V2 17/17] iommu/vt-d: update IOMMU state when memory hotplug happens Jiang Liu
2014-03-05 7:48 ` [Patch Part2 V2 00/17] Enhance DMAR drivers to handle PCI/memory hotplug events Joerg Roedel
17 siblings, 0 replies; 22+ messages in thread
From: Jiang Liu @ 2014-02-19 6:07 UTC (permalink / raw)
To: Joerg Roedel, David Woodhouse, Yinghai Lu, Bjorn Helgaas,
Dan Williams, Vinod Koul, Rafael J . Wysocki
Cc: Jiang Liu, Ashok Raj, Yijing Wang, Tony Luck, iommu, linux-pci,
linux-kernel, dmaengine
Now we have a PCI bus notification based mechanism to update DMAR
device scope array, we could extend the mechanism to support boot
time initialization too, which will help to unify and simplify
the implementation.
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
drivers/iommu/dmar.c | 163 ++++++++++++-------------------------------
drivers/iommu/intel-iommu.c | 71 +++++--------------
include/linux/dmar.h | 5 --
3 files changed, 61 insertions(+), 178 deletions(-)
diff --git a/drivers/iommu/dmar.c b/drivers/iommu/dmar.c
index bf6bfd1..b19f9f4 100644
--- a/drivers/iommu/dmar.c
+++ b/drivers/iommu/dmar.c
@@ -60,6 +60,7 @@ LIST_HEAD(dmar_drhd_units);
struct acpi_table_header * __initdata dmar_tbl;
static acpi_size dmar_tbl_size;
+static int dmar_dev_scope_status = 1;
static int alloc_iommu(struct dmar_drhd_unit *drhd);
static void free_iommu(struct intel_iommu *iommu);
@@ -76,58 +77,6 @@ static void __init dmar_register_drhd_unit(struct dmar_drhd_unit *drhd)
list_add_rcu(&drhd->list, &dmar_drhd_units);
}
-static int __init dmar_parse_one_dev_scope(struct acpi_dmar_device_scope *scope,
- struct pci_dev __rcu **dev, u16 segment)
-{
- struct pci_bus *bus;
- struct pci_dev *pdev = NULL;
- struct acpi_dmar_pci_path *path;
- int count;
-
- bus = pci_find_bus(segment, scope->bus);
- path = (struct acpi_dmar_pci_path *)(scope + 1);
- count = (scope->length - sizeof(struct acpi_dmar_device_scope))
- / sizeof(struct acpi_dmar_pci_path);
-
- while (count) {
- if (pdev)
- pci_dev_put(pdev);
- /*
- * Some BIOSes list non-exist devices in DMAR table, just
- * ignore it
- */
- if (!bus) {
- pr_warn("Device scope bus [%d] not found\n", scope->bus);
- break;
- }
- pdev = pci_get_slot(bus, PCI_DEVFN(path->device, path->function));
- if (!pdev) {
- /* warning will be printed below */
- break;
- }
- path ++;
- count --;
- bus = pdev->subordinate;
- }
- if (!pdev) {
- pr_warn("Device scope device [%04x:%02x:%02x.%02x] not found\n",
- segment, scope->bus, path->device, path->function);
- return 0;
- }
- if ((scope->entry_type == ACPI_DMAR_SCOPE_TYPE_ENDPOINT && \
- pdev->subordinate) || (scope->entry_type == \
- ACPI_DMAR_SCOPE_TYPE_BRIDGE && !pdev->subordinate)) {
- pci_dev_put(pdev);
- pr_warn("Device scope type does not match for %s\n",
- pci_name(pdev));
- return -EINVAL;
- }
-
- rcu_assign_pointer(*dev, pdev);
-
- return 0;
-}
-
void *dmar_alloc_dev_scope(void *start, void *end, int *cnt)
{
struct acpi_dmar_device_scope *scope;
@@ -150,35 +99,6 @@ void *dmar_alloc_dev_scope(void *start, void *end, int *cnt)
return kcalloc(*cnt, sizeof(struct pci_dev *), GFP_KERNEL);
}
-int __init dmar_parse_dev_scope(void *start, void *end, int *cnt,
- struct pci_dev __rcu ***devices, u16 segment)
-{
- struct acpi_dmar_device_scope *scope;
- int index, ret;
-
- *devices = dmar_alloc_dev_scope(start, end, cnt);
- if (*cnt == 0)
- return 0;
- else if (!*devices)
- return -ENOMEM;
-
- for (index = 0; start < end; start += scope->length) {
- scope = start;
- if (scope->entry_type == ACPI_DMAR_SCOPE_TYPE_ENDPOINT ||
- scope->entry_type == ACPI_DMAR_SCOPE_TYPE_BRIDGE) {
- ret = dmar_parse_one_dev_scope(scope,
- &(*devices)[index], segment);
- if (ret) {
- dmar_free_dev_scope(devices, cnt);
- return ret;
- }
- index ++;
- }
- }
-
- return 0;
-}
-
void dmar_free_dev_scope(struct pci_dev __rcu ***devices, int *cnt)
{
int i;
@@ -220,6 +140,8 @@ dmar_alloc_pci_notify_info(struct pci_dev *dev, unsigned long event)
if (!info) {
pr_warn("Out of memory when allocating notify_info "
"for %s.\n", pci_name(dev));
+ if (dmar_dev_scope_status == 0)
+ dmar_dev_scope_status = -ENOMEM;
return NULL;
}
}
@@ -349,6 +271,8 @@ static int dmar_pci_bus_add_dev(struct dmar_pci_notify_info *info)
}
if (ret >= 0)
ret = dmar_iommu_notify_scope_dev(info);
+ if (ret < 0 && dmar_dev_scope_status == 0)
+ dmar_dev_scope_status = ret;
return ret;
}
@@ -418,9 +342,21 @@ dmar_parse_one_drhd(struct acpi_dmar_header *header)
dmaru->reg_base_addr = drhd->address;
dmaru->segment = drhd->segment;
dmaru->include_all = drhd->flags & 0x1; /* BIT0: INCLUDE_ALL */
+ if (!dmaru->include_all) {
+ dmaru->devices = dmar_alloc_dev_scope((void *)(drhd + 1),
+ ((void *)drhd) + drhd->header.length,
+ &dmaru->devices_cnt);
+ if (dmaru->devices_cnt && dmaru->devices == NULL) {
+ kfree(dmaru);
+ return -ENOMEM;
+ }
+ }
ret = alloc_iommu(dmaru);
if (ret) {
+ if (!dmaru->include_all)
+ dmar_free_dev_scope(&dmaru->devices,
+ &dmaru->devices_cnt);
kfree(dmaru);
return ret;
}
@@ -437,21 +373,6 @@ static void dmar_free_drhd(struct dmar_drhd_unit *dmaru)
kfree(dmaru);
}
-static int __init dmar_parse_dev(struct dmar_drhd_unit *dmaru)
-{
- struct acpi_dmar_hardware_unit *drhd;
-
- drhd = (struct acpi_dmar_hardware_unit *) dmaru->hdr;
-
- if (dmaru->include_all)
- return 0;
-
- return dmar_parse_dev_scope((void *)(drhd + 1),
- ((void *)drhd) + drhd->header.length,
- &dmaru->devices_cnt, &dmaru->devices,
- drhd->segment);
-}
-
#ifdef CONFIG_ACPI_NUMA
static int __init
dmar_parse_one_rhsa(struct acpi_dmar_header *header)
@@ -665,34 +586,35 @@ out:
int __init dmar_dev_scope_init(void)
{
- static int dmar_dev_scope_initialized;
- struct dmar_drhd_unit *drhd;
- int ret = -ENODEV;
+ struct pci_dev *dev = NULL;
+ struct dmar_pci_notify_info *info;
- if (dmar_dev_scope_initialized)
- return dmar_dev_scope_initialized;
+ if (dmar_dev_scope_status != 1)
+ return dmar_dev_scope_status;
- if (list_empty(&dmar_drhd_units))
- goto fail;
+ if (list_empty(&dmar_drhd_units)) {
+ dmar_dev_scope_status = -ENODEV;
+ } else {
+ dmar_dev_scope_status = 0;
+
+ for_each_pci_dev(dev) {
+ if (dev->is_virtfn)
+ continue;
+
+ info = dmar_alloc_pci_notify_info(dev,
+ BUS_NOTIFY_ADD_DEVICE);
+ if (!info) {
+ return dmar_dev_scope_status;
+ } else {
+ dmar_pci_bus_add_dev(info);
+ dmar_free_pci_notify_info(info);
+ }
+ }
- for_each_drhd_unit(drhd) {
- ret = dmar_parse_dev(drhd);
- if (ret)
- goto fail;
+ bus_register_notifier(&pci_bus_type, &dmar_pci_bus_nb);
}
- ret = dmar_parse_rmrr_atsr_dev();
- if (ret)
- goto fail;
-
- bus_register_notifier(&pci_bus_type, &dmar_pci_bus_nb);
-
- dmar_dev_scope_initialized = 1;
- return 0;
-
-fail:
- dmar_dev_scope_initialized = ret;
- return ret;
+ return dmar_dev_scope_status;
}
@@ -1617,7 +1539,8 @@ static int __init dmar_free_unused_resources(void)
if (irq_remapping_enabled || intel_iommu_enabled)
return 0;
- bus_unregister_notifier(&pci_bus_type, &dmar_pci_bus_nb);
+ if (dmar_dev_scope_status != 1 && !list_empty(&dmar_drhd_units))
+ bus_unregister_notifier(&pci_bus_type, &dmar_pci_bus_nb);
down_write(&dmar_global_lock);
list_for_each_entry_safe(dmaru, dmaru_n, &dmar_drhd_units, list) {
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index d9c0dc5..dd576c0 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -3473,11 +3473,6 @@ static void __init init_iommu_pm_ops(void)
static inline void init_iommu_pm_ops(void) {}
#endif /* CONFIG_PM */
-static void __init dmar_register_rmrr_unit(struct dmar_rmrr_unit *rmrr)
-{
- list_add(&rmrr->list, &dmar_rmrr_units);
-}
-
int __init dmar_parse_one_rmrr(struct acpi_dmar_header *header)
{
@@ -3492,21 +3487,17 @@ int __init dmar_parse_one_rmrr(struct acpi_dmar_header *header)
rmrr = (struct acpi_dmar_reserved_memory *)header;
rmrru->base_address = rmrr->base_address;
rmrru->end_address = rmrr->end_address;
+ rmrru->devices = dmar_alloc_dev_scope((void *)(rmrr + 1),
+ ((void *)rmrr) + rmrr->header.length,
+ &rmrru->devices_cnt);
+ if (rmrru->devices_cnt && rmrru->devices == NULL) {
+ kfree(rmrru);
+ return -ENOMEM;
+ }
- dmar_register_rmrr_unit(rmrru);
- return 0;
-}
+ list_add(&rmrru->list, &dmar_rmrr_units);
-static int __init
-rmrr_parse_dev(struct dmar_rmrr_unit *rmrru)
-{
- struct acpi_dmar_reserved_memory *rmrr;
-
- rmrr = (struct acpi_dmar_reserved_memory *) rmrru->hdr;
- return dmar_parse_dev_scope((void *)(rmrr + 1),
- ((void *)rmrr) + rmrr->header.length,
- &rmrru->devices_cnt, &rmrru->devices,
- rmrr->segment);
+ return 0;
}
int __init dmar_parse_one_atsr(struct acpi_dmar_header *hdr)
@@ -3521,26 +3512,21 @@ int __init dmar_parse_one_atsr(struct acpi_dmar_header *hdr)
atsru->hdr = hdr;
atsru->include_all = atsr->flags & 0x1;
+ if (!atsru->include_all) {
+ atsru->devices = dmar_alloc_dev_scope((void *)(atsr + 1),
+ (void *)atsr + atsr->header.length,
+ &atsru->devices_cnt);
+ if (atsru->devices_cnt && atsru->devices == NULL) {
+ kfree(atsru);
+ return -ENOMEM;
+ }
+ }
list_add_rcu(&atsru->list, &dmar_atsr_units);
return 0;
}
-static int __init atsr_parse_dev(struct dmar_atsr_unit *atsru)
-{
- struct acpi_dmar_atsr *atsr;
-
- if (atsru->include_all)
- return 0;
-
- atsr = container_of(atsru->hdr, struct acpi_dmar_atsr, header);
- return dmar_parse_dev_scope((void *)(atsr + 1),
- (void *)atsr + atsr->header.length,
- &atsru->devices_cnt, &atsru->devices,
- atsr->segment);
-}
-
static void intel_iommu_free_atsr(struct dmar_atsr_unit *atsru)
{
dmar_free_dev_scope(&atsru->devices, &atsru->devices_cnt);
@@ -3604,27 +3590,6 @@ out:
return ret;
}
-int __init dmar_parse_rmrr_atsr_dev(void)
-{
- struct dmar_rmrr_unit *rmrr;
- struct dmar_atsr_unit *atsr;
- int ret;
-
- list_for_each_entry(rmrr, &dmar_rmrr_units, list) {
- ret = rmrr_parse_dev(rmrr);
- if (ret)
- return ret;
- }
-
- list_for_each_entry_rcu(atsr, &dmar_atsr_units, list) {
- ret = atsr_parse_dev(atsr);
- if (ret)
- return ret;
- }
-
- return 0;
-}
-
int dmar_iommu_notify_scope_dev(struct dmar_pci_notify_info *info)
{
int ret = 0;
diff --git a/include/linux/dmar.h b/include/linux/dmar.h
index 4e19643..0a92e4d 100644
--- a/include/linux/dmar.h
+++ b/include/linux/dmar.h
@@ -173,7 +173,6 @@ extern int arch_setup_dmar_msi(unsigned int irq);
#ifdef CONFIG_INTEL_IOMMU
extern int iommu_detected, no_iommu;
-extern int dmar_parse_rmrr_atsr_dev(void);
extern int dmar_parse_one_rmrr(struct acpi_dmar_header *header);
extern int dmar_parse_one_atsr(struct acpi_dmar_header *header);
extern int dmar_iommu_notify_scope_dev(struct dmar_pci_notify_info *info);
@@ -188,10 +187,6 @@ static inline int dmar_parse_one_atsr(struct acpi_dmar_header *header)
{
return 0;
}
-static inline int dmar_parse_rmrr_atsr_dev(void)
-{
- return 0;
-}
static inline int dmar_iommu_notify_scope_dev(struct dmar_pci_notify_info *info)
{
return 0;
--
1.7.10.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* [Patch Part2 V2 17/17] iommu/vt-d: update IOMMU state when memory hotplug happens
2014-02-19 6:07 [Patch Part2 V2 00/17] Enhance DMAR drivers to handle PCI/memory hotplug events Jiang Liu
` (15 preceding siblings ...)
2014-02-19 6:07 ` [Patch Part2 V2 16/17] iommu/vt-d, PCI: unify the way to process DMAR device scope array Jiang Liu
@ 2014-02-19 6:07 ` Jiang Liu
2014-03-05 7:48 ` [Patch Part2 V2 00/17] Enhance DMAR drivers to handle PCI/memory hotplug events Joerg Roedel
17 siblings, 0 replies; 22+ messages in thread
From: Jiang Liu @ 2014-02-19 6:07 UTC (permalink / raw)
To: Joerg Roedel, David Woodhouse, Yinghai Lu, Bjorn Helgaas,
Dan Williams, Vinod Koul, Rafael J . Wysocki
Cc: Jiang Liu, Ashok Raj, Yijing Wang, Tony Luck, iommu, linux-pci,
linux-kernel, dmaengine
If static identity domain is created, IOMMU driver needs to update
si_domain page table when memory hotplug event happens. Otherwise
PCI device DMA operations can't access the hot-added memory regions.
Signed-off-by: Jiang Liu <jiang.liu@linux.intel.com>
---
drivers/iommu/intel-iommu.c | 71 ++++++++++++++++++++++++++++++++++++++++++-
drivers/iommu/iova.c | 64 ++++++++++++++++++++++++++++++++++----
include/linux/iova.h | 2 ++
3 files changed, 130 insertions(+), 7 deletions(-)
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index dd576c0..484d669 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -33,6 +33,7 @@
#include <linux/dmar.h>
#include <linux/dma-mapping.h>
#include <linux/mempool.h>
+#include <linux/memory.h>
#include <linux/timer.h>
#include <linux/iova.h>
#include <linux/iommu.h>
@@ -3683,6 +3684,73 @@ static struct notifier_block device_nb = {
.notifier_call = device_notifier,
};
+static int intel_iommu_memory_notifier(struct notifier_block *nb,
+ unsigned long val, void *v)
+{
+ struct memory_notify *mhp = v;
+ unsigned long long start, end;
+ unsigned long start_vpfn, last_vpfn;
+
+ switch (val) {
+ case MEM_GOING_ONLINE:
+ start = mhp->start_pfn << PAGE_SHIFT;
+ end = ((mhp->start_pfn + mhp->nr_pages) << PAGE_SHIFT) - 1;
+ if (iommu_domain_identity_map(si_domain, start, end)) {
+ pr_warn("dmar: failed to build identity map for [%llx-%llx]\n",
+ start, end);
+ return NOTIFY_BAD;
+ }
+ break;
+
+ case MEM_OFFLINE:
+ case MEM_CANCEL_ONLINE:
+ start_vpfn = mm_to_dma_pfn(mhp->start_pfn);
+ last_vpfn = mm_to_dma_pfn(mhp->start_pfn + mhp->nr_pages - 1);
+ while (start_vpfn <= last_vpfn) {
+ struct iova *iova;
+ struct dmar_drhd_unit *drhd;
+ struct intel_iommu *iommu;
+
+ iova = find_iova(&si_domain->iovad, start_vpfn);
+ if (iova == NULL) {
+ pr_debug("dmar: failed get IOVA for PFN %lx\n",
+ start_vpfn);
+ break;
+ }
+
+ iova = split_and_remove_iova(&si_domain->iovad, iova,
+ start_vpfn, last_vpfn);
+ if (iova == NULL) {
+ pr_warn("dmar: failed to split IOVA PFN [%lx-%lx]\n",
+ start_vpfn, last_vpfn);
+ return NOTIFY_BAD;
+ }
+
+ rcu_read_lock();
+ for_each_active_iommu(iommu, drhd)
+ iommu_flush_iotlb_psi(iommu, si_domain->id,
+ iova->pfn_lo,
+ iova->pfn_hi - iova->pfn_lo + 1, 0);
+ rcu_read_unlock();
+ dma_pte_clear_range(si_domain, iova->pfn_lo,
+ iova->pfn_hi);
+ dma_pte_free_pagetable(si_domain, iova->pfn_lo,
+ iova->pfn_hi);
+
+ start_vpfn = iova->pfn_hi + 1;
+ free_iova_mem(iova);
+ }
+ break;
+ }
+
+ return NOTIFY_OK;
+}
+
+static struct notifier_block intel_iommu_memory_nb = {
+ .notifier_call = intel_iommu_memory_notifier,
+ .priority = 0
+};
+
int __init intel_iommu_init(void)
{
int ret = -ENODEV;
@@ -3755,8 +3823,9 @@ int __init intel_iommu_init(void)
init_iommu_pm_ops();
bus_set_iommu(&pci_bus_type, &intel_iommu_ops);
-
bus_register_notifier(&pci_bus_type, &device_nb);
+ if (si_domain && !hw_pass_through)
+ register_memory_notifier(&intel_iommu_memory_nb);
intel_iommu_enabled = 1;
diff --git a/drivers/iommu/iova.c b/drivers/iommu/iova.c
index 67da6cff..f6b17e6 100644
--- a/drivers/iommu/iova.c
+++ b/drivers/iommu/iova.c
@@ -342,19 +342,30 @@ __is_range_overlap(struct rb_node *node,
return 0;
}
+static inline struct iova *
+alloc_and_init_iova(unsigned long pfn_lo, unsigned long pfn_hi)
+{
+ struct iova *iova;
+
+ iova = alloc_iova_mem();
+ if (iova) {
+ iova->pfn_lo = pfn_lo;
+ iova->pfn_hi = pfn_hi;
+ }
+
+ return iova;
+}
+
static struct iova *
__insert_new_range(struct iova_domain *iovad,
unsigned long pfn_lo, unsigned long pfn_hi)
{
struct iova *iova;
- iova = alloc_iova_mem();
- if (!iova)
- return iova;
+ iova = alloc_and_init_iova(pfn_lo, pfn_hi);
+ if (iova)
+ iova_insert_rbtree(&iovad->rbroot, iova);
- iova->pfn_hi = pfn_hi;
- iova->pfn_lo = pfn_lo;
- iova_insert_rbtree(&iovad->rbroot, iova);
return iova;
}
@@ -433,3 +444,44 @@ copy_reserved_iova(struct iova_domain *from, struct iova_domain *to)
}
spin_unlock_irqrestore(&from->iova_rbtree_lock, flags);
}
+
+struct iova *
+split_and_remove_iova(struct iova_domain *iovad, struct iova *iova,
+ unsigned long pfn_lo, unsigned long pfn_hi)
+{
+ unsigned long flags;
+ struct iova *prev = NULL, *next = NULL;
+
+ spin_lock_irqsave(&iovad->iova_rbtree_lock, flags);
+ if (iova->pfn_lo < pfn_lo) {
+ prev = alloc_and_init_iova(iova->pfn_lo, pfn_lo - 1);
+ if (prev == NULL)
+ goto error;
+ }
+ if (iova->pfn_hi > pfn_hi) {
+ next = alloc_and_init_iova(pfn_hi + 1, iova->pfn_hi);
+ if (next == NULL)
+ goto error;
+ }
+
+ __cached_rbnode_delete_update(iovad, iova);
+ rb_erase(&iova->node, &iovad->rbroot);
+
+ if (prev) {
+ iova_insert_rbtree(&iovad->rbroot, prev);
+ iova->pfn_lo = pfn_lo;
+ }
+ if (next) {
+ iova_insert_rbtree(&iovad->rbroot, next);
+ iova->pfn_hi = pfn_hi;
+ }
+ spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags);
+
+ return iova;
+
+error:
+ spin_unlock_irqrestore(&iovad->iova_rbtree_lock, flags);
+ if (prev)
+ free_iova_mem(prev);
+ return NULL;
+}
diff --git a/include/linux/iova.h b/include/linux/iova.h
index 76a0759..3277f47 100644
--- a/include/linux/iova.h
+++ b/include/linux/iova.h
@@ -47,5 +47,7 @@ void copy_reserved_iova(struct iova_domain *from, struct iova_domain *to);
void init_iova_domain(struct iova_domain *iovad, unsigned long pfn_32bit);
struct iova *find_iova(struct iova_domain *iovad, unsigned long pfn);
void put_iova_domain(struct iova_domain *iovad);
+struct iova *split_and_remove_iova(struct iova_domain *iovad,
+ struct iova *iova, unsigned long pfn_lo, unsigned long pfn_hi);
#endif
--
1.7.10.4
^ permalink raw reply related [flat|nested] 22+ messages in thread* Re: [Patch Part2 V2 00/17] Enhance DMAR drivers to handle PCI/memory hotplug events
2014-02-19 6:07 [Patch Part2 V2 00/17] Enhance DMAR drivers to handle PCI/memory hotplug events Jiang Liu
` (16 preceding siblings ...)
2014-02-19 6:07 ` [Patch Part2 V2 17/17] iommu/vt-d: update IOMMU state when memory hotplug happens Jiang Liu
@ 2014-03-05 7:48 ` Joerg Roedel
17 siblings, 0 replies; 22+ messages in thread
From: Joerg Roedel @ 2014-03-05 7:48 UTC (permalink / raw)
To: Jiang Liu
Cc: David Woodhouse, Yinghai Lu, Bjorn Helgaas, Dan Williams,
Vinod Koul, Rafael J . Wysocki, Ashok Raj, Yijing Wang, Tony Luck,
iommu, linux-pci, linux-kernel, dmaengine
On Wed, Feb 19, 2014 at 02:07:20PM +0800, Jiang Liu wrote:
> Jiang Liu (17):
> iommu/vt-d: avoid double free of g_iommus on error recovery path
> iommu/vt-d: avoid caching stale domain_device_info and fix memory
> leak
> iommu/vt-d: avoid caching stale domain_device_info when hot-removing
> PCI device
> iommu/vt-d: factor out dmar_alloc_dev_scope() for later reuse
> iommu/vt-d: move private structures and variables into intel-iommu.c
> iommu/vt-d: simplify function get_domain_for_dev()
> iommu/vt-d: free resources if failed to create domain for PCIe
> endpoint
> iommu/vt-d: reduce duplicated code to handle virtual machine domains
> iommu/vt-d: fix incorrect iommu_count for si_domain
> iommu/vt-d: check for NULL pointer when freeing IOMMU data structure
> iommu/vt-d: fix error in detect ATS capability
> iommu/vt-d: introduce macro for_each_dev_scope() to walk device scope
> entries
> iommu/vt-d: introduce a rwsem to protect global data structures
> iommu/vt-d: use RCU to protect global resources in interrupt context
> iommu/vt-d, PCI: update DRHD/RMRR/ATSR device scope caches when PCI
> hotplug happens
> iommu/vt-d, PCI: unify the way to process DMAR device scope array
> iommu/vt-d: update IOMMU state when memory hotplug happens
>
> drivers/iommu/dmar.c | 412 +++++++++++++------
> drivers/iommu/intel-iommu.c | 750 ++++++++++++++++++-----------------
> drivers/iommu/intel_irq_remapping.c | 108 +++--
> drivers/iommu/iova.c | 64 ++-
> include/linux/dmar.h | 74 ++--
> include/linux/iova.h | 2 +
> 6 files changed, 848 insertions(+), 562 deletions(-)
Applied, thanks Jiang.
^ permalink raw reply [flat|nested] 22+ messages in thread