iommu.lists.linux-foundation.org archive mirror
 help / color / mirror / Atom feed
* RE: [PATCH] iommu/vt-d: Fix broken device issue when using iommu=pt
@ 2014-09-24 21:56 Rob Roschewsk
       [not found] ` <1597491265.153338.1411595791727.JavaMail.open-xchange-91tEjvOFe9KcT/DCa4qSTkaJx/dRlJfr5NbjCUgZEJk@public.gmane.org>
  0 siblings, 1 reply; 17+ messages in thread
From: Rob Roschewsk @ 2014-09-24 21:56 UTC (permalink / raw)
  To: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA


[-- Attachment #1.1: Type: text/plain, Size: 1653 bytes --]

Hello All .... wonder if there has been any movement on this issue .... I'm
having a similar issue 
 
I'm running an HP dl380 gen 8 with an Emulex OneConnect 10Gb iSCSI (14e4:164c)
(rev 11)
also known as " Hewlett-Packard Company NC373i Integrated Multifunction Gigabit
Server Adapter"
 
03:00.2 Mass storage controller: Emulex Corporation OneConnect 10Gb iSCSI
Initiator (be3) (rev 01)

03:00.3 Mass storage controller: Emulex Corporation OneConnect 10Gb iSCSI
Initiator (be3) (rev 01)

Seems as soon as the iscsi target is contacted the following messages appear in
the log every few seconds ....

Sep 24 15:25:30  kernel: [ 78.682675] dmar: DRHD: handling fault status reg 2
Sep 24 15:25:30  kernel: [ 78.699797] dmar: DMAR:[DMA Write] Request device
[03:00.2] fault addr ffffc000
Sep 24 15:25:30  kernel: [ 78.699797] DMAR:[fault reason 05] PTE Write access is
not set
Sep 24 15:25:30  kernel: [ 78.934546] dmar: DRHD: handling fault status reg 102
Sep 24 15:25:30  kernel: [ 78.934549] dmar: DMAR:[DMA Write] Request device
[03:00.2] fault addr ffffc000
Sep 24 15:25:30  kernel: [ 78.934549] DMAR:[fault reason 05] PTE Write access is
not set
Sep 24 15:25:30  kernel: [ 78.935359] dmar: DRHD: handling fault status reg 202

If I pull intel_iommu=on out of the kernel command line the problem goes away.

System is running Ubuntu 14.04.1 LTS 

3.13.0-32-generic #57-Ubuntu SMP Tue Jul 15 03:51:08 UTC 2014 x86_64 x86_64
x86_64 GNU/Linux

Looking for help .... I'm not sure where I should be looking next ..... I need
SR-IOV for other adapters in the box.

I have a system up that I can pull any data from that might be required.

Thanks,

--> Rob

 

[-- Attachment #1.2: Type: text/html, Size: 2771 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]



^ permalink raw reply	[flat|nested] 17+ messages in thread
* [PATCH] iommu/vt-d: Fix broken device issue when using iommu=pt
@ 2014-08-11  2:54 Yijing Wang
       [not found] ` <1407725674-27271-1-git-send-email-wangyijing-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
  0 siblings, 1 reply; 17+ messages in thread
From: Yijing Wang @ 2014-08-11  2:54 UTC (permalink / raw)
  To: Joerg Roedel, David Woodhouse
  Cc: iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA, Jiang Liu

We found some strange devices in HP C7000 and Huawei Server. These devices
can not be enumerated by OS, but they still did DMA read/write without OS 
management. Because iommu will not create the DMA mapping for these devices,
the DMA read/write will be blocked by iommu hardware.

Eg.
 \-[0000:00]-+-00.0  Intel Corporation Xeon E5/Core i7 DMI2
             +-01.0-[11]--
			 +-01.1-[02]--
			 +-02.0-[04]--+-00.0  Emulex Corporation OneConnect 10Gb NIC (be3)
	         |            +-00.1  Emulex Corporation OneConnect 10Gb NIC (be3)
	         |            +-00.2  Emulex Corporation OneConnect 10Gb iSCSI Initiator (be3)
	         |            \-00.3  Emulex Corporation OneConnect 10Gb iSCSI Initiator (be3)
	         +-02.1-[12]--
Kernel only found four devices in bus 0x04, but we found following DMA errors in dmesg.

[ 1438.477262] DRHD: handling fault status reg 402
[ 1438.498278] DMAR:[DMA Write] Request device [04:00.4] fault addr bdf70000 
[ 1438.498280] DMAR:[fault reason 02] Present bit in context entry is clear
[ 1438.566458] DMAR:[DMA Write] Request device [04:00.5] fault addr bdf70000 
[ 1438.566460] DMAR:[fault reason 02] Present bit in context entry is clear
[ 1438.635211] DMAR:[DMA Write] Request device [04:00.6] fault addr bdf70000 
[ 1438.635213] DMAR:[fault reason 02] Present bit in context entry is clear
[ 1438.703849] DMAR:[DMA Write] Request device [04:00.7] fault addr bdf70000 
[ 1438.703851] DMAR:[fault reason 02] Present bit in context entry is clear

Signed-off-by: Yijing Wang <wangyijing-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
---
 arch/x86/include/asm/iommu.h |    2 ++
 arch/x86/kernel/pci-dma.c    |    8 ++++++++
 drivers/iommu/intel-iommu.c  |   41 +++++++++++++++++++++++++++++++++++++++++
 3 files changed, 51 insertions(+), 0 deletions(-)

diff --git a/arch/x86/include/asm/iommu.h b/arch/x86/include/asm/iommu.h
index 345c99c..5e3a2d8 100644
--- a/arch/x86/include/asm/iommu.h
+++ b/arch/x86/include/asm/iommu.h
@@ -5,6 +5,8 @@ extern struct dma_map_ops nommu_dma_ops;
 extern int force_iommu, no_iommu;
 extern int iommu_detected;
 extern int iommu_pass_through;
+extern int iommu_pt_force_bus;
+extern int iommu_pt_force_domain;
 
 /* 10 seconds */
 #define DMAR_OPERATION_TIMEOUT ((cycles_t) tsc_khz*10*1000)
diff --git a/arch/x86/kernel/pci-dma.c b/arch/x86/kernel/pci-dma.c
index a25e202..bf21d97 100644
--- a/arch/x86/kernel/pci-dma.c
+++ b/arch/x86/kernel/pci-dma.c
@@ -44,6 +44,8 @@ int iommu_detected __read_mostly = 0;
  * guests and not for driver dma translation.
  */
 int iommu_pass_through __read_mostly;
+int iommu_pt_force_bus = -1;
+int iommu_pt_force_domain = -1;
 
 extern struct iommu_table_entry __iommu_table[], __iommu_table_end[];
 
@@ -146,6 +148,7 @@ void dma_generic_free_coherent(struct device *dev, size_t size, void *vaddr,
  */
 static __init int iommu_setup(char *p)
 {
+	char *end;
 	iommu_merge = 1;
 
 	if (!p)
@@ -192,6 +195,11 @@ static __init int iommu_setup(char *p)
 #endif
 		if (!strncmp(p, "pt", 2))
 			iommu_pass_through = 1;
+		if (!strncmp(p, "pt_force=", 9)) {
+			iommu_pass_through = 1;
+			iommu_pt_force_domain = simple_strtol(p+9, &end, 0);
+			iommu_pt_force_bus = simple_strtol(end+1, NULL, 0);
+		}
 
 		gart_parse_options(p);
 
diff --git a/drivers/iommu/intel-iommu.c b/drivers/iommu/intel-iommu.c
index d1f5caa..49757f1 100644
--- a/drivers/iommu/intel-iommu.c
+++ b/drivers/iommu/intel-iommu.c
@@ -2705,6 +2705,47 @@ static int __init iommu_prepare_static_identity_mapping(int hw)
 				return ret;
 		}
 
+	/* We found some strange devices in HP c7000 and other platforms that
+	 * can not be enumerated by OS, but they did DMA read/write without
+	 * driver management, so we should create the pt mapping for these
+	 * devices to avoid DMA errors. Add iommu=pt_force=segment:busnum to
+	 * force to do pt context mapping in the bus number.
+	 */
+	if (iommu_pt_force_bus >= 0 && iommu_pt_force_bus >= 0) {
+		int found = 0;
+
+		iommu = NULL;
+		for_each_active_iommu(iommu, drhd) {
+			if (iommu_pt_force_domain != drhd->segment)
+				continue;
+
+			for_each_active_dev_scope(drhd->devices, drhd->devices_cnt, i, dev) {
+				if (!dev_is_pci(dev))
+					continue;
+
+				pdev = to_pci_dev(dev);
+				if (pdev->bus->number == iommu_pt_force_bus ||
+						(pdev->subordinate
+						 && pdev->subordinate->number <= iommu_pt_force_bus
+						 && pdev->subordinate->busn_res.end >= iommu_pt_force_bus)) {
+					found = 1;
+					break;
+				}
+			}
+
+			if (drhd->include_all) {
+				found = 1;
+				break;
+			}
+		}
+
+		if (found && iommu)
+			for (i = 0; i < 256; i++)
+				domain_context_mapping_one(si_domain, iommu, iommu_pt_force_bus,
+						i,  hw ? CONTEXT_TT_PASS_THROUGH :
+						CONTEXT_TT_MULTI_LEVEL);
+	}
+
 	return 0;
 }
 
-- 
1.7.1

^ permalink raw reply related	[flat|nested] 17+ messages in thread
end of thread, other threads:[~2014-09-25 14:29 UTC | newest]

Thread overview: 17+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2014-09-24 21:56 [PATCH] iommu/vt-d: Fix broken device issue when using iommu=pt Rob Roschewsk
     [not found] ` <1597491265.153338.1411595791727.JavaMail.open-xchange-91tEjvOFe9KcT/DCa4qSTkaJx/dRlJfr5NbjCUgZEJk@public.gmane.org>
2014-09-25  2:36   ` Yijing Wang
     [not found]     ` <54237F92.1090601-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
2014-09-25 13:56       ` Rob Roschewsk
     [not found]         ` <CAE1O4xrQYpc4o6MbN8b7kkgwTbjj+Pszu5WUwM_vErUfyQYfNw-JsoAwUIsXosN+BqQ9rBEUg@public.gmane.org>
2014-09-25 14:29           ` Linda Knippers
  -- strict thread matches above, loose matches on Subject: below --
2014-08-11  2:54 Yijing Wang
     [not found] ` <1407725674-27271-1-git-send-email-wangyijing-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
2014-08-11  3:15   ` Jiang Liu
     [not found]     ` <53E8355C.4010906-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
2014-08-11  3:46       ` Yijing Wang
     [not found]         ` <53E83C9E.9060405-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
2014-08-11  4:56           ` Jiang Liu
     [not found]             ` <53E84CEA.7080402-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
2014-08-11  8:26               ` Yijing Wang
2014-08-11  4:43   ` Alex Williamson
     [not found]     ` <1407732187.9800.11.camel-85EaTFmN5p//9pzu0YdTqQ@public.gmane.org>
2014-08-11  8:36       ` Yijing Wang
2014-08-11 14:59       ` Linda Knippers
     [not found]         ` <53E8DA5E.8090406-VXdhtT5mjnY@public.gmane.org>
2014-08-12  1:37           ` Yijing Wang
     [not found]             ` <53E96FE0.7080600-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
2014-08-12  2:34               ` Jiang Liu
2014-08-12  3:18               ` Jiang Liu
     [not found]                 ` <53E9876F.9040300-VuQAYsv1563Yd54FQh9/CA@public.gmane.org>
2014-08-12  3:48                   ` Yijing Wang
2014-08-14 16:07               ` Linda Knippers

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).