[PATCH v3 00/25] VT-d unit test

[PATCH v3 00/25] VT-d unit test

[Peter Xu]

This is v3 of vt-d unit test series.

Patch "libcflat: add IS_ALIGNED() macro, and page sizes" is picked up
by Drew in the ARM GIC framework series, so please feel free to drop
it when needed.

Online repo:

  https://github.com/xzpeter/kvm-unit-tests.git iommu-ut-v3

Please review. Thanks,

v3:
- is_power_of_2(): remove outer "()" [Drew]
- fix indent for vtd_root_table() [Drew]
- pci_find_dev(): keep {} to avoid touching more codes [Drew]
- pci_scan_bars(): check whether bar is 64bits [Drew]
- pci_enable_defaults(): make it return nothing [Drew]
- fix strnang blanks in lib/x86/intel-iommu.h [Drew]
- pci_setup_msi(): fix correct assert() inside [Drew]
- squash patches: 03+17 08+09 10+11 12+14 [Drew]
- remove all "()" for raw defines (numbers) in macros [Drew]

v2:
- move cpu_relax patch to the beginning, and use them in all places
  [Drew]
- replace all corresponding 256 into PCI_DEVFN_MAX, as well for
  PCI_BAR_NUM [Drew]
- adding is_power_of_2() to replace ONE_BIT_ONLY() [Drew]
- add SZ_64K macro [Drew]
- declare pci_config_write[wb] in lib/asm-generic/pci-host-bridge.h [Alex]
- edu_reg_read/write() add "l" in func name [Drew]
- drop pci_set_master(), instead, provide pci_cmd_set_clr() [Drew]
- change return code into bool (always) for functions that apply
  [Drew]
- keep old pci_find_dev() interface [Drew/Alex]
- use __raw_{read|write}*() for both vt-d and edu register read/writes
  [Alex]
- remove pci_ prefix for all pci_dev fields [Drew]
- replace 0xff in cap_handlers[0xff] into (PCI_CAP_ID_MAX + 1) [Drew]
- make x86/unittest.cfg simpler by using q35 directly with eim=off
  [Drew]

RFC -> v1:
- when init edu device fail, report_skip() rather than return error
  [Radim]
- use asserts rather than "static bool inited" to avoid multiple init
  of components (affects patch 1/2) [Drew]
- moving the first two patches out of the series [Drew]
- int vtd_init(), do not setup_idt() since smp_init() did it [Drew]
- when edu do not have MSI enabled, skip interrupt test [Radim]
- rename vtd_reg_*() into vtd_{read|write}[lq](), and move them to
  header file [Drew]
- use PAGE_MASK when able [Drew]
- use "&" instead of "|" in intel-iommu init test (three places)
  [Drew]
- use "vtd_init()" in unit test [Drew]
- mention that where intel-iommu.h comes from [Drew]
- re-written vtd_gcmd_or(), make it also work on even hardware [Drew]
- remove most of the oneline wrapper for VT-d registers, instead, use
  vtd_{read|write}* with register names [Drew]
- remove useless BDF helpers [Drew]
- move edu device macros into header file [Drew]
- make edu_check_alive static inline [Drew]
- remove all useless wrappers in pci-edu.c [Drew]
- remove pci_dma_dir_t and all its users, instead, use "bool
  from_device" [Drew]
- not use typedef for structs, to follow Linux/kvm-unit-tests coding
  style [Drew]
- let pci_dev_init() clean and simple, then provide
  pci_enable_defaults() for more complicated things [Drew]
- add one more patch to add intel-iommu test into x86/unittest [Radim]
- use 0x60 intr request instead of factorial to trigger edu device
  interrupt [Drew]
- ...and some other changes I just forgot to note down...

Currently only a very small test scope is covered:

* VT-d init
* DMAR: 4 bytes copy
* IR: MSI

However this series could be a base point to add more test cases for
VT-d. The problem is, there are many IOMMU error conditions which are
very hard to be triggered in a real guest (IOMMU has merely no
interface for guest user, and it's totally running in the background).
This piece of work can be a start point if we want to do more
complicated things and play around with Intel IOMMU devices (also for
IOMMU regression tests).

Please review. Thanks,

=================

To run the test:

./x86/run ./x86/intel-iommu.flat \
    -M q35,kernel-irqchip=split -global ioapic.version=0x20 \
    -device intel-iommu,intremap=on -device edu

Sample output:

pxdev:kvm-unit-tests [new-iommu-ut]# ./iommu_run.sh
/root/git/qemu/bin/x86_64-softmmu/qemu-system-x86_64 -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio
+-device pci-testdev -kernel ./x86/intel-iommu.flat -M q35,kernel-irqchip=split -global ioapic.version=0x20 -device intel-iommu,intremap=on -device edu
enabling apic
paging enabled
cr0 = 80010011
cr3 = 7fff000
cr4 = 20
VT-d version:   0x10
     cap:       0x0012008c22260206
     ecap:      0x0000000000f00f1a
PASS: init status check
PASS: fault status check
PASS: QI enablement
DMAR table address: 0x0000000007ff9000
PASS: DMAR table setup
IR table address: 0x0000000007ff8000
PASS: IR table setup
PASS: DMAR enablement
PASS: IR enablement
PASS: DMAR support 39 bits address width
PASS: DMAR support huge pages
PCI: init dev 0x0020 BAR 0 [MEM] addr 0xfea00000
PCI detected cap 0x5
Detected MSI for device 0x20 offset 0x40
allocated vt-d root entry for PCI bus 0
allocated vt-d context entry for devfn 0x20
map 4K page IOVA 0x0 to 0x7ff7000 (sid=0x0020)
edu device DMA start TO addr 0x0 size 0x4 off 0x0
edu device DMA start FROM addr 0x4 size 0x4 off 0x0
PASS: DMAR 4B memcpy test
INTR: setup IRTE index 0
MSI: dev 0x20 init 64bit address: addr=0xfee00010, data=0x0
PASS: EDU factorial INTR test

Alexander Gordeev (12):
  pci: Fix coding style in generic PCI files
  pci: x86: Rename pci_config_read() to pci_config_readl()
  pci: Add 'extern' to public function declarations
  pci: x86: Add remaining PCI configuration space accessors
  pci: Factor out pci_bar_get()
  pci: Rework pci_bar_addr()
  pci: Add pci_bar_set_addr()
  pci: Add pci_dev_exists()
  pci: Add pci_print()
  pci: Add generic ECAM host support
  pci: Add pci-testdev PCI bus test device
  arm/arm64: pci: Add pci-testdev PCI device operation test

Peter Xu (13):
  x86/asm: add cpu_relax()
  libcflat: introduce is_power_of_2()
  x86: intel-iommu: add vt-d init test
  libcflat: add IS_ALIGNED() macro, and page sizes
  libcflat: moving MIN/MAX here
  vm/page: provide PGDIR_OFFSET() macro
  pci: introduce struct pci_dev
  pci: provide pci_scan_bars()
  pci: provide pci_enable_defaults()
  pci: edu: introduce pci-edu helpers
  x86: intel-iommu: add dmar test
  pci: add msi support for 32/64bit address
  x86: intel-iommu: add IR MSI test

 arm/Makefile.common               |   6 +-
 arm/pci-test.c                    |  27 ++++
 arm/run                           |   7 +-
 arm/unittests.cfg                 |   4 +
 lib/alloc.c                       |   3 -
 lib/arm/asm/pci.h                 |   1 +
 lib/arm64/asm/pci.h               |   1 +
 lib/asm-generic/pci-host-bridge.h |  28 ++++
 lib/libcflat.h                    |  14 ++
 lib/pci-edu.c                     |  73 +++++++++
 lib/pci-edu.h                     |  83 ++++++++++
 lib/pci-host-generic.c            | 321 +++++++++++++++++++++++++++++++++++++
 lib/pci-host-generic.h            |  46 ++++++
 lib/pci-testdev.c                 | 194 ++++++++++++++++++++++
 lib/pci.c                         | 327 +++++++++++++++++++++++++++++++++++---
 lib/pci.h                         |  66 +++++++-
 lib/x86/asm/barrier.h             |  11 ++
 lib/x86/asm/page.h                |   3 +
 lib/x86/asm/pci.h                 |  46 +++++-
 lib/x86/intel-iommu.c             | 313 ++++++++++++++++++++++++++++++++++++
 lib/x86/intel-iommu.h             | 142 +++++++++++++++++
 lib/x86/vm.c                      |   4 +-
 x86/Makefile.common               |   1 +
 x86/Makefile.x86_64               |   2 +
 x86/intel-iommu.c                 | 120 ++++++++++++++
 x86/unittests.cfg                 |   7 +
 x86/vmexit.c                      |  27 ++--
 27 files changed, 1826 insertions(+), 51 deletions(-)
 create mode 100644 arm/pci-test.c
 create mode 100644 lib/arm/asm/pci.h
 create mode 100644 lib/arm64/asm/pci.h
 create mode 100644 lib/asm-generic/pci-host-bridge.h
 create mode 100644 lib/pci-edu.c
 create mode 100644 lib/pci-edu.h
 create mode 100644 lib/pci-host-generic.c
 create mode 100644 lib/pci-host-generic.h
 create mode 100644 lib/pci-testdev.c
 create mode 100644 lib/x86/intel-iommu.c
 create mode 100644 lib/x86/intel-iommu.h
 create mode 100644 x86/intel-iommu.c

-- 
2.7.4

[PATCH v3 01/25] pci: Fix coding style in generic PCI files

[Peter Xu]

From: Alexander Gordeev <agordeev@redhat.com>

Cc: Thomas Huth <thuth@redhat.com>
Cc: Andrew Jones <drjones@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alexander Gordeev <agordeev@redhat.com>
---
 lib/pci.c | 38 ++++++++++++++++++++------------------
 lib/pci.h |  3 ++-
 2 files changed, 22 insertions(+), 19 deletions(-)

diff --git a/lib/pci.c b/lib/pci.c
index 0058d70..43cd0ea 100644
--- a/lib/pci.c
+++ b/lib/pci.c
@@ -10,34 +10,36 @@
 /* Scan bus look for a specific device. Only bus 0 scanned for now. */
 pcidevaddr_t pci_find_dev(uint16_t vendor_id, uint16_t device_id)
 {
-    unsigned dev;
-    for (dev = 0; dev < 256; ++dev) {
-    uint32_t id = pci_config_read(dev, 0);
-    if ((id & 0xFFFF) == vendor_id && (id >> 16) == device_id) {
-        return dev;
-    }
-    }
-    return PCIDEVADDR_INVALID;
+	pcidevaddr_t dev;
+
+	for (dev = 0; dev < 256; ++dev) {
+		uint32_t id = pci_config_read(dev, 0);
+
+		if ((id & 0xFFFF) == vendor_id && (id >> 16) == device_id)
+			return dev;
+	}
+
+	return PCIDEVADDR_INVALID;
 }
 
 unsigned long pci_bar_addr(pcidevaddr_t dev, int bar_num)
 {
-    uint32_t bar = pci_config_read(dev, PCI_BASE_ADDRESS_0 + bar_num * 4);
-    if (bar & PCI_BASE_ADDRESS_SPACE_IO) {
-        return bar & PCI_BASE_ADDRESS_IO_MASK;
-    } else {
-        return bar & PCI_BASE_ADDRESS_MEM_MASK;
-    }
+	uint32_t bar = pci_config_read(dev, PCI_BASE_ADDRESS_0 + bar_num * 4);
+
+	if (bar & PCI_BASE_ADDRESS_SPACE_IO)
+		return bar & PCI_BASE_ADDRESS_IO_MASK;
+	else
+		return bar & PCI_BASE_ADDRESS_MEM_MASK;
 }
 
 bool pci_bar_is_memory(pcidevaddr_t dev, int bar_num)
 {
-    uint32_t bar = pci_config_read(dev, PCI_BASE_ADDRESS_0 + bar_num * 4);
-    return !(bar & PCI_BASE_ADDRESS_SPACE_IO);
+	uint32_t bar = pci_config_read(dev, PCI_BASE_ADDRESS_0 + bar_num * 4);
+
+	return !(bar & PCI_BASE_ADDRESS_SPACE_IO);
 }
 
 bool pci_bar_is_valid(pcidevaddr_t dev, int bar_num)
 {
-    uint32_t bar = pci_config_read(dev, PCI_BASE_ADDRESS_0 + bar_num * 4);
-    return bar;
+	return pci_config_read(dev, PCI_BASE_ADDRESS_0 + bar_num * 4);
 }
diff --git a/lib/pci.h b/lib/pci.h
index 9160cfb..54fbf22 100644
--- a/lib/pci.h
+++ b/lib/pci.h
@@ -12,8 +12,9 @@
 
 typedef uint16_t pcidevaddr_t;
 enum {
-    PCIDEVADDR_INVALID = 0xffff,
+	PCIDEVADDR_INVALID = 0xffff,
 };
+
 pcidevaddr_t pci_find_dev(uint16_t vendor_id, uint16_t device_id);
 unsigned long pci_bar_addr(pcidevaddr_t dev, int bar_num);
 bool pci_bar_is_memory(pcidevaddr_t dev, int bar_num);
-- 
2.7.4

[PATCH v3 02/25] pci: x86: Rename pci_config_read() to pci_config_readl()

[Peter Xu]

From: Alexander Gordeev <agordeev@redhat.com>

Cc: Thomas Huth <thuth@redhat.com>
Cc: Andrew Jones <drjones@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Signed-off-by: Alexander Gordeev <agordeev@redhat.com>
---
 lib/pci.c         | 8 ++++----
 lib/x86/asm/pci.h | 2 +-
 2 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/lib/pci.c b/lib/pci.c
index 43cd0ea..e0b4514 100644
--- a/lib/pci.c
+++ b/lib/pci.c
@@ -13,7 +13,7 @@ pcidevaddr_t pci_find_dev(uint16_t vendor_id, uint16_t device_id)
 	pcidevaddr_t dev;
 
 	for (dev = 0; dev < 256; ++dev) {
-		uint32_t id = pci_config_read(dev, 0);
+		uint32_t id = pci_config_readl(dev, 0);
 
 		if ((id & 0xFFFF) == vendor_id && (id >> 16) == device_id)
 			return dev;
@@ -24,7 +24,7 @@ pcidevaddr_t pci_find_dev(uint16_t vendor_id, uint16_t device_id)
 
 unsigned long pci_bar_addr(pcidevaddr_t dev, int bar_num)
 {
-	uint32_t bar = pci_config_read(dev, PCI_BASE_ADDRESS_0 + bar_num * 4);
+	uint32_t bar = pci_config_readl(dev, PCI_BASE_ADDRESS_0 + bar_num * 4);
 
 	if (bar & PCI_BASE_ADDRESS_SPACE_IO)
 		return bar & PCI_BASE_ADDRESS_IO_MASK;
@@ -34,12 +34,12 @@ unsigned long pci_bar_addr(pcidevaddr_t dev, int bar_num)
 
 bool pci_bar_is_memory(pcidevaddr_t dev, int bar_num)
 {
-	uint32_t bar = pci_config_read(dev, PCI_BASE_ADDRESS_0 + bar_num * 4);
+	uint32_t bar = pci_config_readl(dev, PCI_BASE_ADDRESS_0 + bar_num * 4);
 
 	return !(bar & PCI_BASE_ADDRESS_SPACE_IO);
 }
 
 bool pci_bar_is_valid(pcidevaddr_t dev, int bar_num)
 {
-	return pci_config_read(dev, PCI_BASE_ADDRESS_0 + bar_num * 4);
+	return pci_config_readl(dev, PCI_BASE_ADDRESS_0 + bar_num * 4);
 }
diff --git a/lib/x86/asm/pci.h b/lib/x86/asm/pci.h
index cddde41..d00438f 100644
--- a/lib/x86/asm/pci.h
+++ b/lib/x86/asm/pci.h
@@ -9,7 +9,7 @@
 #include "pci.h"
 #include "x86/asm/io.h"
 
-static inline uint32_t pci_config_read(pcidevaddr_t dev, uint8_t reg)
+static inline uint32_t pci_config_readl(pcidevaddr_t dev, uint8_t reg)
 {
     uint32_t index = reg | (dev << 8) | (0x1 << 31);
     outl(index, 0xCF8);
-- 
2.7.4

[PATCH v3 03/25] pci: Add 'extern' to public function declarations

[Peter Xu]

From: Alexander Gordeev <agordeev@redhat.com>

Cc: Thomas Huth <thuth@redhat.com>
Cc: Andrew Jones <drjones@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Signed-off-by: Alexander Gordeev <agordeev@redhat.com>
---
 lib/pci.h | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/lib/pci.h b/lib/pci.h
index 54fbf22..066fac7 100644
--- a/lib/pci.h
+++ b/lib/pci.h
@@ -15,10 +15,10 @@ enum {
 	PCIDEVADDR_INVALID = 0xffff,
 };
 
-pcidevaddr_t pci_find_dev(uint16_t vendor_id, uint16_t device_id);
-unsigned long pci_bar_addr(pcidevaddr_t dev, int bar_num);
-bool pci_bar_is_memory(pcidevaddr_t dev, int bar_num);
-bool pci_bar_is_valid(pcidevaddr_t dev, int bar_num);
+extern pcidevaddr_t pci_find_dev(uint16_t vendor_id, uint16_t device_id);
+extern unsigned long pci_bar_addr(pcidevaddr_t dev, int bar_num);
+extern bool pci_bar_is_memory(pcidevaddr_t dev, int bar_num);
+extern bool pci_bar_is_valid(pcidevaddr_t dev, int bar_num);
 
 /*
  * pci-testdev is a driver for the pci-testdev qemu pci device. The
-- 
2.7.4

[PATCH v3 04/25] pci: x86: Add remaining PCI configuration space accessors

[Peter Xu]

From: Alexander Gordeev <agordeev@redhat.com>

Cc: Thomas Huth <thuth@redhat.com>
Cc: Andrew Jones <drjones@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alexander Gordeev <agordeev@redhat.com>
---
 lib/pci.c         |  5 ++---
 lib/x86/asm/pci.h | 38 ++++++++++++++++++++++++++++++++++++--
 2 files changed, 38 insertions(+), 5 deletions(-)

diff --git a/lib/pci.c b/lib/pci.c
index e0b4514..b05ecfa 100644
--- a/lib/pci.c
+++ b/lib/pci.c
@@ -13,9 +13,8 @@ pcidevaddr_t pci_find_dev(uint16_t vendor_id, uint16_t device_id)
 	pcidevaddr_t dev;
 
 	for (dev = 0; dev < 256; ++dev) {
-		uint32_t id = pci_config_readl(dev, 0);
-
-		if ((id & 0xFFFF) == vendor_id && (id >> 16) == device_id)
+		if (pci_config_readw(dev, PCI_VENDOR_ID) == vendor_id &&
+		    pci_config_readw(dev, PCI_DEVICE_ID) == device_id)
 			return dev;
 	}
 
diff --git a/lib/x86/asm/pci.h b/lib/x86/asm/pci.h
index d00438f..4862ab5 100644
--- a/lib/x86/asm/pci.h
+++ b/lib/x86/asm/pci.h
@@ -9,11 +9,45 @@
 #include "pci.h"
 #include "x86/asm/io.h"
 
+#define PCI_CONF1_ADDRESS(dev, reg)	((0x1 << 31) | (dev << 8) | reg)
+
+static inline uint8_t pci_config_readb(pcidevaddr_t dev, uint8_t reg)
+{
+    outl(PCI_CONF1_ADDRESS(dev, reg), 0xCF8);
+    return inb(0xCFC);
+}
+
+static inline uint16_t pci_config_readw(pcidevaddr_t dev, uint8_t reg)
+{
+    outl(PCI_CONF1_ADDRESS(dev, reg), 0xCF8);
+    return inw(0xCFC);
+}
+
 static inline uint32_t pci_config_readl(pcidevaddr_t dev, uint8_t reg)
 {
-    uint32_t index = reg | (dev << 8) | (0x1 << 31);
-    outl(index, 0xCF8);
+    outl(PCI_CONF1_ADDRESS(dev, reg), 0xCF8);
     return inl(0xCFC);
 }
 
+static inline void pci_config_writeb(pcidevaddr_t dev, uint8_t reg,
+                                     uint8_t val)
+{
+    outl(PCI_CONF1_ADDRESS(dev, reg), 0xCF8);
+    outb(val, 0xCFC);
+}
+
+static inline void pci_config_writew(pcidevaddr_t dev, uint8_t reg,
+                                     uint16_t val)
+{
+    outl(PCI_CONF1_ADDRESS(dev, reg), 0xCF8);
+    outw(val, 0xCFC);
+}
+
+static inline void pci_config_writel(pcidevaddr_t dev, uint8_t reg,
+                                     uint32_t val)
+{
+    outl(PCI_CONF1_ADDRESS(dev, reg), 0xCF8);
+    outl(val, 0xCFC);
+}
+
 #endif
-- 
2.7.4

[PATCH v3 05/25] pci: Factor out pci_bar_get()

[Peter Xu]

From: Alexander Gordeev <agordeev@redhat.com>

Cc: Thomas Huth <thuth@redhat.com>
Cc: Andrew Jones <drjones@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alexander Gordeev <agordeev@redhat.com>
---
 lib/pci.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/lib/pci.c b/lib/pci.c
index b05ecfa..ce481bb 100644
--- a/lib/pci.c
+++ b/lib/pci.c
@@ -21,9 +21,14 @@ pcidevaddr_t pci_find_dev(uint16_t vendor_id, uint16_t device_id)
 	return PCIDEVADDR_INVALID;
 }
 
+static uint32_t pci_bar_get(pcidevaddr_t dev, int bar_num)
+{
+	return pci_config_readl(dev, PCI_BASE_ADDRESS_0 + bar_num * 4);
+}
+
 unsigned long pci_bar_addr(pcidevaddr_t dev, int bar_num)
 {
-	uint32_t bar = pci_config_readl(dev, PCI_BASE_ADDRESS_0 + bar_num * 4);
+	uint32_t bar = pci_bar_get(dev, bar_num);
 
 	if (bar & PCI_BASE_ADDRESS_SPACE_IO)
 		return bar & PCI_BASE_ADDRESS_IO_MASK;
@@ -33,12 +38,12 @@ unsigned long pci_bar_addr(pcidevaddr_t dev, int bar_num)
 
 bool pci_bar_is_memory(pcidevaddr_t dev, int bar_num)
 {
-	uint32_t bar = pci_config_readl(dev, PCI_BASE_ADDRESS_0 + bar_num * 4);
+	uint32_t bar = pci_bar_get(dev, bar_num);
 
 	return !(bar & PCI_BASE_ADDRESS_SPACE_IO);
 }
 
 bool pci_bar_is_valid(pcidevaddr_t dev, int bar_num)
 {
-	return pci_config_readl(dev, PCI_BASE_ADDRESS_0 + bar_num * 4);
+	return pci_bar_get(dev, bar_num);
 }
-- 
2.7.4

[PATCH v3 06/25] pci: Rework pci_bar_addr()

[Peter Xu]

From: Alexander Gordeev <agordeev@redhat.com>

This update makes pci_bar_addr() interface 64 bit BARs aware and
introduces a concept of PCI address translation.

An architecutre should implement pci_translate_addr() interface
in order to provide mapping between PCI bus address and CPU
physical address.

Cc: Thomas Huth <thuth@redhat.com>
Cc: Andrew Jones <drjones@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Reviewed-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Alexander Gordeev <agordeev@redhat.com>
---
 lib/pci.c         | 73 +++++++++++++++++++++++++++++++++++++++++++++++++++----
 lib/pci.h         | 17 ++++++++++++-
 lib/x86/asm/pci.h |  6 +++++
 3 files changed, 90 insertions(+), 6 deletions(-)

diff --git a/lib/pci.c b/lib/pci.c
index ce481bb..3bf45fb 100644
--- a/lib/pci.c
+++ b/lib/pci.c
@@ -21,19 +21,71 @@ pcidevaddr_t pci_find_dev(uint16_t vendor_id, uint16_t device_id)
 	return PCIDEVADDR_INVALID;
 }
 
+static uint32_t pci_bar_mask(uint32_t bar)
+{
+	return (bar & PCI_BASE_ADDRESS_SPACE_IO) ?
+		PCI_BASE_ADDRESS_IO_MASK : PCI_BASE_ADDRESS_MEM_MASK;
+}
+
 static uint32_t pci_bar_get(pcidevaddr_t dev, int bar_num)
 {
 	return pci_config_readl(dev, PCI_BASE_ADDRESS_0 + bar_num * 4);
 }
 
-unsigned long pci_bar_addr(pcidevaddr_t dev, int bar_num)
+phys_addr_t pci_bar_addr(pcidevaddr_t dev, int bar_num)
 {
 	uint32_t bar = pci_bar_get(dev, bar_num);
+	uint32_t mask = pci_bar_mask(bar);
+	uint64_t addr = bar & mask;
 
-	if (bar & PCI_BASE_ADDRESS_SPACE_IO)
-		return bar & PCI_BASE_ADDRESS_IO_MASK;
-	else
-		return bar & PCI_BASE_ADDRESS_MEM_MASK;
+	if (pci_bar_is64(dev, bar_num))
+		addr |= (uint64_t)pci_bar_get(dev, bar_num + 1) << 32;
+
+	return pci_translate_addr(dev, addr);
+}
+
+/*
+ * To determine the amount of address space needed by a PCI device,
+ * one must save the original value of the BAR, write a value of
+ * all 1's to the register, and then read it back. The amount of
+ * memory can be then determined by masking the information bits,
+ * performing a bitwise NOT, and incrementing the value by 1.
+ *
+ * The following pci_bar_size_helper() and pci_bar_size() functions
+ * implement the algorithm.
+ */
+static uint32_t pci_bar_size_helper(pcidevaddr_t dev, int bar_num)
+{
+	int off = PCI_BASE_ADDRESS_0 + bar_num * 4;
+	uint32_t bar, val;
+
+	bar = pci_config_readl(dev, off);
+	pci_config_writel(dev, off, ~0u);
+	val = pci_config_readl(dev, off);
+	pci_config_writel(dev, off, bar);
+
+	return val;
+}
+
+phys_addr_t pci_bar_size(pcidevaddr_t dev, int bar_num)
+{
+	uint32_t bar, size;
+
+	size = pci_bar_size_helper(dev, bar_num);
+	if (!size)
+		return 0;
+
+	bar = pci_bar_get(dev, bar_num);
+	size &= pci_bar_mask(bar);
+
+	if (pci_bar_is64(dev, bar_num)) {
+		phys_addr_t size64 = pci_bar_size_helper(dev, bar_num + 1);
+		size64 = (size64 << 32) | size;
+
+		return ~size64 + 1;
+	} else {
+		return ~size + 1;
+	}
 }
 
 bool pci_bar_is_memory(pcidevaddr_t dev, int bar_num)
@@ -47,3 +99,14 @@ bool pci_bar_is_valid(pcidevaddr_t dev, int bar_num)
 {
 	return pci_bar_get(dev, bar_num);
 }
+
+bool pci_bar_is64(pcidevaddr_t dev, int bar_num)
+{
+	uint32_t bar = pci_bar_get(dev, bar_num);
+
+	if (bar & PCI_BASE_ADDRESS_SPACE_IO)
+		return false;
+
+	return (bar & PCI_BASE_ADDRESS_MEM_TYPE_MASK) ==
+		      PCI_BASE_ADDRESS_MEM_TYPE_64;
+}
diff --git a/lib/pci.h b/lib/pci.h
index 066fac7..8eec236 100644
--- a/lib/pci.h
+++ b/lib/pci.h
@@ -16,7 +16,22 @@ enum {
 };
 
 extern pcidevaddr_t pci_find_dev(uint16_t vendor_id, uint16_t device_id);
-extern unsigned long pci_bar_addr(pcidevaddr_t dev, int bar_num);
+
+/*
+ * @bar_num in all BAR access functions below is the index of the 32-bit
+ * register starting from the PCI_BASE_ADDRESS_0 offset.
+ *
+ * In cases where the BAR size is 64-bit, a caller should still provide
+ * @bar_num in terms of 32-bit words. For example, if a device has a 64-bit
+ * BAR#0 and a 32-bit BAR#1, then caller should provide 2 to address BAR#1,
+ * not 1.
+ *
+ * It is expected the caller is aware of the device BAR layout and never
+ * tries to address the middle of a 64-bit register.
+ */
+extern phys_addr_t pci_bar_addr(pcidevaddr_t dev, int bar_num);
+extern phys_addr_t pci_bar_size(pcidevaddr_t dev, int bar_num);
+extern bool pci_bar_is64(pcidevaddr_t dev, int bar_num);
 extern bool pci_bar_is_memory(pcidevaddr_t dev, int bar_num);
 extern bool pci_bar_is_valid(pcidevaddr_t dev, int bar_num);
 
diff --git a/lib/x86/asm/pci.h b/lib/x86/asm/pci.h
index 4862ab5..c937e5c 100644
--- a/lib/x86/asm/pci.h
+++ b/lib/x86/asm/pci.h
@@ -50,4 +50,10 @@ static inline void pci_config_writel(pcidevaddr_t dev, uint8_t reg,
     outl(val, 0xCFC);
 }
 
+static inline
+phys_addr_t pci_translate_addr(pcidevaddr_t dev __unused, uint64_t addr)
+{
+    return addr;
+}
+
 #endif
-- 
2.7.4

[PATCH v3 07/25] pci: Add pci_bar_set_addr()

[Peter Xu]

From: Alexander Gordeev <agordeev@redhat.com>

Because the counterpart to pci_bar_set_addr() setter is
pci_bar_addr() getter, these names become inconsistent.
Rename pci_bar_addr() to pci_bar_get_addr() also to make
the resulting names conform to each other.

Cc: Thomas Huth <thuth@redhat.com>
Cc: Andrew Jones <drjones@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Signed-off-by: Alexander Gordeev <agordeev@redhat.com>
---
 lib/pci.c    | 12 +++++++++++-
 lib/pci.h    |  3 ++-
 x86/vmexit.c |  4 ++--
 3 files changed, 15 insertions(+), 4 deletions(-)

diff --git a/lib/pci.c b/lib/pci.c
index 3bf45fb..74936f3 100644
--- a/lib/pci.c
+++ b/lib/pci.c
@@ -32,7 +32,7 @@ static uint32_t pci_bar_get(pcidevaddr_t dev, int bar_num)
 	return pci_config_readl(dev, PCI_BASE_ADDRESS_0 + bar_num * 4);
 }
 
-phys_addr_t pci_bar_addr(pcidevaddr_t dev, int bar_num)
+phys_addr_t pci_bar_get_addr(pcidevaddr_t dev, int bar_num)
 {
 	uint32_t bar = pci_bar_get(dev, bar_num);
 	uint32_t mask = pci_bar_mask(bar);
@@ -44,6 +44,16 @@ phys_addr_t pci_bar_addr(pcidevaddr_t dev, int bar_num)
 	return pci_translate_addr(dev, addr);
 }
 
+void pci_bar_set_addr(pcidevaddr_t dev, int bar_num, phys_addr_t addr)
+{
+	int off = PCI_BASE_ADDRESS_0 + bar_num * 4;
+
+	pci_config_writel(dev, off, (uint32_t)addr);
+
+	if (pci_bar_is64(dev, bar_num))
+		pci_config_writel(dev, off + 4, (uint32_t)(addr >> 32));
+}
+
 /*
  * To determine the amount of address space needed by a PCI device,
  * one must save the original value of the BAR, write a value of
diff --git a/lib/pci.h b/lib/pci.h
index 8eec236..1c20308 100644
--- a/lib/pci.h
+++ b/lib/pci.h
@@ -29,7 +29,8 @@ extern pcidevaddr_t pci_find_dev(uint16_t vendor_id, uint16_t device_id);
  * It is expected the caller is aware of the device BAR layout and never
  * tries to address the middle of a 64-bit register.
  */
-extern phys_addr_t pci_bar_addr(pcidevaddr_t dev, int bar_num);
+extern phys_addr_t pci_bar_get_addr(pcidevaddr_t dev, int bar_num);
+extern void pci_bar_set_addr(pcidevaddr_t dev, int bar_num, phys_addr_t addr);
 extern phys_addr_t pci_bar_size(pcidevaddr_t dev, int bar_num);
 extern bool pci_bar_is64(pcidevaddr_t dev, int bar_num);
 extern bool pci_bar_is_memory(pcidevaddr_t dev, int bar_num);
diff --git a/x86/vmexit.c b/x86/vmexit.c
index c2e1e49..2d99d5f 100644
--- a/x86/vmexit.c
+++ b/x86/vmexit.c
@@ -392,10 +392,10 @@ int main(int ac, char **av)
 				continue;
 			}
 			if (pci_bar_is_memory(pcidev, i)) {
-				membar = pci_bar_addr(pcidev, i);
+				membar = pci_bar_get_addr(pcidev, i);
 				pci_test.memaddr = ioremap(membar, PAGE_SIZE);
 			} else {
-				pci_test.iobar = pci_bar_addr(pcidev, i);
+				pci_test.iobar = pci_bar_get_addr(pcidev, i);
 			}
 		}
 		printf("pci-testdev at 0x%x membar %lx iobar %x\n",
-- 
2.7.4

[PATCH v3 08/25] pci: Add pci_dev_exists()

[Peter Xu]

From: Alexander Gordeev <agordeev@redhat.com>

Cc: Thomas Huth <thuth@redhat.com>
Cc: Andrew Jones <drjones@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Signed-off-by: Alexander Gordeev <agordeev@redhat.com>
---
 lib/pci.c | 6 ++++++
 lib/pci.h | 1 +
 2 files changed, 7 insertions(+)

diff --git a/lib/pci.c b/lib/pci.c
index 74936f3..42f47d9 100644
--- a/lib/pci.c
+++ b/lib/pci.c
@@ -7,6 +7,12 @@
 #include "pci.h"
 #include "asm/pci.h"
 
+bool pci_dev_exists(pcidevaddr_t dev)
+{
+	return (pci_config_readw(dev, PCI_VENDOR_ID) != 0xffff &&
+		pci_config_readw(dev, PCI_DEVICE_ID) != 0xffff);
+}
+
 /* Scan bus look for a specific device. Only bus 0 scanned for now. */
 pcidevaddr_t pci_find_dev(uint16_t vendor_id, uint16_t device_id)
 {
diff --git a/lib/pci.h b/lib/pci.h
index 1c20308..1462aa2 100644
--- a/lib/pci.h
+++ b/lib/pci.h
@@ -15,6 +15,7 @@ enum {
 	PCIDEVADDR_INVALID = 0xffff,
 };
 
+extern bool pci_dev_exists(pcidevaddr_t dev);
 extern pcidevaddr_t pci_find_dev(uint16_t vendor_id, uint16_t device_id);
 
 /*
-- 
2.7.4

[PATCH v3 09/25] pci: Add pci_print()

[Peter Xu]

From: Alexander Gordeev <agordeev@redhat.com>

Cc: Thomas Huth <thuth@redhat.com>
Cc: Andrew Jones <drjones@redhat.com>
Cc: Peter Xu <peterx@redhat.com>
Suggested-by: Andrew Jones <drjones@redhat.com>
Reviewed-by: Andrew Jones <drjones@redhat.com>
Signed-off-by: Alexander Gordeev <agordeev@redhat.com>
---
 lib/pci.c | 89 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 lib/pci.h |  3 +++
 2 files changed, 92 insertions(+)

diff --git a/lib/pci.c b/lib/pci.c
index 42f47d9..e03c67c 100644
--- a/lib/pci.c
+++ b/lib/pci.c
@@ -126,3 +126,92 @@ bool pci_bar_is64(pcidevaddr_t dev, int bar_num)
 	return (bar & PCI_BASE_ADDRESS_MEM_TYPE_MASK) ==
 		      PCI_BASE_ADDRESS_MEM_TYPE_64;
 }
+
+static void pci_bar_print(pcidevaddr_t dev, int bar_num)
+{
+	phys_addr_t size, start, end;
+	uint32_t bar;
+
+	size = pci_bar_size(dev, bar_num);
+	if (!size)
+		return;
+
+	bar = pci_bar_get(dev, bar_num);
+	start = pci_bar_get_addr(dev, bar_num);
+	end = start + size - 1;
+
+	if (pci_bar_is64(dev, bar_num)) {
+		printf("BAR#%d,%d [%" PRIx64 "-%" PRIx64 " ",
+		       bar_num, bar_num + 1, start, end);
+	} else {
+		printf("BAR#%d [%02x-%02x ",
+		       bar_num, (uint32_t)start, (uint32_t)end);
+	}
+
+	if (bar & PCI_BASE_ADDRESS_SPACE_IO) {
+		printf("PIO");
+	} else {
+		printf("MEM");
+		switch (bar & PCI_BASE_ADDRESS_MEM_TYPE_MASK) {
+		case PCI_BASE_ADDRESS_MEM_TYPE_32:
+			printf("32");
+			break;
+		case PCI_BASE_ADDRESS_MEM_TYPE_1M:
+			printf("1M");
+			break;
+		case PCI_BASE_ADDRESS_MEM_TYPE_64:
+			printf("64");
+			break;
+		default:
+			assert(0);
+		}
+	}
+
+	if (bar & PCI_BASE_ADDRESS_MEM_PREFETCH)
+		printf("/p");
+
+	printf("]");
+}
+
+static void pci_dev_print_id(pcidevaddr_t dev)
+{
+	printf("00.%02x.%1x %04x:%04x", dev / 8, dev % 8,
+		pci_config_readw(dev, PCI_VENDOR_ID),
+		pci_config_readw(dev, PCI_DEVICE_ID));
+}
+
+static void pci_dev_print(pcidevaddr_t dev)
+{
+	uint8_t header = pci_config_readb(dev, PCI_HEADER_TYPE);
+	uint8_t progif = pci_config_readb(dev, PCI_CLASS_PROG);
+	uint8_t subclass = pci_config_readb(dev, PCI_CLASS_DEVICE);
+	uint8_t class = pci_config_readb(dev, PCI_CLASS_DEVICE + 1);
+	int i;
+
+	pci_dev_print_id(dev);
+	printf(" type %02x progif %02x class %02x subclass %02x\n",
+	       header, progif, class, subclass);
+
+	if ((header & PCI_HEADER_TYPE_MASK) != PCI_HEADER_TYPE_NORMAL)
+		return;
+
+	for (i = 0; i < 6; i++) {
+		if (pci_bar_size(dev, i)) {
+			printf("\t");
+			pci_bar_print(dev, i);
+			printf("\n");
+		}
+		if (pci_bar_is64(dev, i))
+			i++;
+	}
+}
+
+void pci_print(void)
+{
+	pcidevaddr_t dev;
+
+	for (dev = 0; dev < 256; ++dev) {
+		if (pci_dev_exists(dev))
+			pci_dev_print(dev);
+	}
+}
diff --git a/lib/pci.h b/lib/pci.h
index 1462aa2..fc0940a 100644
--- a/lib/pci.h
+++ b/lib/pci.h
@@ -15,6 +15,7 @@ enum {
 	PCIDEVADDR_INVALID = 0xffff,
 };
 
+extern void pci_print(void);
 extern bool pci_dev_exists(pcidevaddr_t dev);
 extern pcidevaddr_t pci_find_dev(uint16_t vendor_id, uint16_t device_id);
 
@@ -57,4 +58,6 @@ struct pci_test_dev_hdr {
 	uint8_t  name[];
 };
 
+#define  PCI_HEADER_TYPE_MASK		0x7f
+
 #endif /* PCI_H */
-- 
2.7.4

Re: [PATCH v3 00/25] VT-d unit test

[Peter Xu]

Please ignore this v3 series.

This series is incorrectly configured with base branch, so wrong
patches are sent. I did stop it as long as I found the problem, but
still lots of patches were sent already.

Will repost later. Sorry for the noise!

On Mon, Nov 14, 2016 at 05:18:56PM -0500, Peter Xu wrote:
> This is v3 of vt-d unit test series.
> 
> Patch "libcflat: add IS_ALIGNED() macro, and page sizes" is picked up
> by Drew in the ARM GIC framework series, so please feel free to drop
> it when needed.
> 
> Online repo:
> 
>   https://github.com/xzpeter/kvm-unit-tests.git iommu-ut-v3
> 
> Please review. Thanks,
> 
> v3:
> - is_power_of_2(): remove outer "()" [Drew]
> - fix indent for vtd_root_table() [Drew]
> - pci_find_dev(): keep {} to avoid touching more codes [Drew]
> - pci_scan_bars(): check whether bar is 64bits [Drew]
> - pci_enable_defaults(): make it return nothing [Drew]
> - fix strnang blanks in lib/x86/intel-iommu.h [Drew]
> - pci_setup_msi(): fix correct assert() inside [Drew]
> - squash patches: 03+17 08+09 10+11 12+14 [Drew]
> - remove all "()" for raw defines (numbers) in macros [Drew]
> 
> v2:
> - move cpu_relax patch to the beginning, and use them in all places
>   [Drew]
> - replace all corresponding 256 into PCI_DEVFN_MAX, as well for
>   PCI_BAR_NUM [Drew]
> - adding is_power_of_2() to replace ONE_BIT_ONLY() [Drew]
> - add SZ_64K macro [Drew]
> - declare pci_config_write[wb] in lib/asm-generic/pci-host-bridge.h [Alex]
> - edu_reg_read/write() add "l" in func name [Drew]
> - drop pci_set_master(), instead, provide pci_cmd_set_clr() [Drew]
> - change return code into bool (always) for functions that apply
>   [Drew]
> - keep old pci_find_dev() interface [Drew/Alex]
> - use __raw_{read|write}*() for both vt-d and edu register read/writes
>   [Alex]
> - remove pci_ prefix for all pci_dev fields [Drew]
> - replace 0xff in cap_handlers[0xff] into (PCI_CAP_ID_MAX + 1) [Drew]
> - make x86/unittest.cfg simpler by using q35 directly with eim=off
>   [Drew]
> 
> RFC -> v1:
> - when init edu device fail, report_skip() rather than return error
>   [Radim]
> - use asserts rather than "static bool inited" to avoid multiple init
>   of components (affects patch 1/2) [Drew]
> - moving the first two patches out of the series [Drew]
> - int vtd_init(), do not setup_idt() since smp_init() did it [Drew]
> - when edu do not have MSI enabled, skip interrupt test [Radim]
> - rename vtd_reg_*() into vtd_{read|write}[lq](), and move them to
>   header file [Drew]
> - use PAGE_MASK when able [Drew]
> - use "&" instead of "|" in intel-iommu init test (three places)
>   [Drew]
> - use "vtd_init()" in unit test [Drew]
> - mention that where intel-iommu.h comes from [Drew]
> - re-written vtd_gcmd_or(), make it also work on even hardware [Drew]
> - remove most of the oneline wrapper for VT-d registers, instead, use
>   vtd_{read|write}* with register names [Drew]
> - remove useless BDF helpers [Drew]
> - move edu device macros into header file [Drew]
> - make edu_check_alive static inline [Drew]
> - remove all useless wrappers in pci-edu.c [Drew]
> - remove pci_dma_dir_t and all its users, instead, use "bool
>   from_device" [Drew]
> - not use typedef for structs, to follow Linux/kvm-unit-tests coding
>   style [Drew]
> - let pci_dev_init() clean and simple, then provide
>   pci_enable_defaults() for more complicated things [Drew]
> - add one more patch to add intel-iommu test into x86/unittest [Radim]
> - use 0x60 intr request instead of factorial to trigger edu device
>   interrupt [Drew]
> - ...and some other changes I just forgot to note down...
> 
> Currently only a very small test scope is covered:
> 
> * VT-d init
> * DMAR: 4 bytes copy
> * IR: MSI
> 
> However this series could be a base point to add more test cases for
> VT-d. The problem is, there are many IOMMU error conditions which are
> very hard to be triggered in a real guest (IOMMU has merely no
> interface for guest user, and it's totally running in the background).
> This piece of work can be a start point if we want to do more
> complicated things and play around with Intel IOMMU devices (also for
> IOMMU regression tests).
> 
> Please review. Thanks,
> 
> =================
> 
> To run the test:
> 
> ./x86/run ./x86/intel-iommu.flat \
>     -M q35,kernel-irqchip=split -global ioapic.version=0x20 \
>     -device intel-iommu,intremap=on -device edu
> 
> Sample output:
> 
> pxdev:kvm-unit-tests [new-iommu-ut]# ./iommu_run.sh
> /root/git/qemu/bin/x86_64-softmmu/qemu-system-x86_64 -enable-kvm -device pc-testdev -device isa-debug-exit,iobase=0xf4,iosize=0x4 -vnc none -serial stdio
> +-device pci-testdev -kernel ./x86/intel-iommu.flat -M q35,kernel-irqchip=split -global ioapic.version=0x20 -device intel-iommu,intremap=on -device edu
> enabling apic
> paging enabled
> cr0 = 80010011
> cr3 = 7fff000
> cr4 = 20
> VT-d version:   0x10
>      cap:       0x0012008c22260206
>      ecap:      0x0000000000f00f1a
> PASS: init status check
> PASS: fault status check
> PASS: QI enablement
> DMAR table address: 0x0000000007ff9000
> PASS: DMAR table setup
> IR table address: 0x0000000007ff8000
> PASS: IR table setup
> PASS: DMAR enablement
> PASS: IR enablement
> PASS: DMAR support 39 bits address width
> PASS: DMAR support huge pages
> PCI: init dev 0x0020 BAR 0 [MEM] addr 0xfea00000
> PCI detected cap 0x5
> Detected MSI for device 0x20 offset 0x40
> allocated vt-d root entry for PCI bus 0
> allocated vt-d context entry for devfn 0x20
> map 4K page IOVA 0x0 to 0x7ff7000 (sid=0x0020)
> edu device DMA start TO addr 0x0 size 0x4 off 0x0
> edu device DMA start FROM addr 0x4 size 0x4 off 0x0
> PASS: DMAR 4B memcpy test
> INTR: setup IRTE index 0
> MSI: dev 0x20 init 64bit address: addr=0xfee00010, data=0x0
> PASS: EDU factorial INTR test
> 
> Alexander Gordeev (12):
>   pci: Fix coding style in generic PCI files
>   pci: x86: Rename pci_config_read() to pci_config_readl()
>   pci: Add 'extern' to public function declarations
>   pci: x86: Add remaining PCI configuration space accessors
>   pci: Factor out pci_bar_get()
>   pci: Rework pci_bar_addr()
>   pci: Add pci_bar_set_addr()
>   pci: Add pci_dev_exists()
>   pci: Add pci_print()
>   pci: Add generic ECAM host support
>   pci: Add pci-testdev PCI bus test device
>   arm/arm64: pci: Add pci-testdev PCI device operation test
> 
> Peter Xu (13):
>   x86/asm: add cpu_relax()
>   libcflat: introduce is_power_of_2()
>   x86: intel-iommu: add vt-d init test
>   libcflat: add IS_ALIGNED() macro, and page sizes
>   libcflat: moving MIN/MAX here
>   vm/page: provide PGDIR_OFFSET() macro
>   pci: introduce struct pci_dev
>   pci: provide pci_scan_bars()
>   pci: provide pci_enable_defaults()
>   pci: edu: introduce pci-edu helpers
>   x86: intel-iommu: add dmar test
>   pci: add msi support for 32/64bit address
>   x86: intel-iommu: add IR MSI test
> 
>  arm/Makefile.common               |   6 +-
>  arm/pci-test.c                    |  27 ++++
>  arm/run                           |   7 +-
>  arm/unittests.cfg                 |   4 +
>  lib/alloc.c                       |   3 -
>  lib/arm/asm/pci.h                 |   1 +
>  lib/arm64/asm/pci.h               |   1 +
>  lib/asm-generic/pci-host-bridge.h |  28 ++++
>  lib/libcflat.h                    |  14 ++
>  lib/pci-edu.c                     |  73 +++++++++
>  lib/pci-edu.h                     |  83 ++++++++++
>  lib/pci-host-generic.c            | 321 +++++++++++++++++++++++++++++++++++++
>  lib/pci-host-generic.h            |  46 ++++++
>  lib/pci-testdev.c                 | 194 ++++++++++++++++++++++
>  lib/pci.c                         | 327 +++++++++++++++++++++++++++++++++++---
>  lib/pci.h                         |  66 +++++++-
>  lib/x86/asm/barrier.h             |  11 ++
>  lib/x86/asm/page.h                |   3 +
>  lib/x86/asm/pci.h                 |  46 +++++-
>  lib/x86/intel-iommu.c             | 313 ++++++++++++++++++++++++++++++++++++
>  lib/x86/intel-iommu.h             | 142 +++++++++++++++++
>  lib/x86/vm.c                      |   4 +-
>  x86/Makefile.common               |   1 +
>  x86/Makefile.x86_64               |   2 +
>  x86/intel-iommu.c                 | 120 ++++++++++++++
>  x86/unittests.cfg                 |   7 +
>  x86/vmexit.c                      |  27 ++--
>  27 files changed, 1826 insertions(+), 51 deletions(-)
>  create mode 100644 arm/pci-test.c
>  create mode 100644 lib/arm/asm/pci.h
>  create mode 100644 lib/arm64/asm/pci.h
>  create mode 100644 lib/asm-generic/pci-host-bridge.h
>  create mode 100644 lib/pci-edu.c
>  create mode 100644 lib/pci-edu.h
>  create mode 100644 lib/pci-host-generic.c
>  create mode 100644 lib/pci-host-generic.h
>  create mode 100644 lib/pci-testdev.c
>  create mode 100644 lib/x86/intel-iommu.c
>  create mode 100644 lib/x86/intel-iommu.h
>  create mode 100644 x86/intel-iommu.c
> 
> -- 
> 2.7.4
> 

-- peterx