[RFC PATCH 00/33] vfio: Introduce selftests for VFIO

[RFC PATCH 00/33] vfio: Introduce selftests for VFIO

[David Matlack]

This series introduces VFIO selftests, located in
tools/testing/selftests/vfio/.

VFIO selftests aim to enable kernel developers to write and run tests
that take the form of userspace programs that interact with VFIO and
IOMMUFD uAPIs. VFIO selftests can be used to write functional tests for
new features, regression tests for bugs, and performance tests for
optimizations.

These tests are designed to interact with real PCI devices, i.e. they do
not rely on mocking out or faking any behavior in the kernel. This
allows the tests to exercise not only VFIO but also IOMMUFD, the IOMMU
driver, interrupt remapping, IRQ handling, etc.

We chose selftests to host these tests primarily to enable integration
with the existing KVM selftests. As explained in the next section,
enabling KVM developers to test the interaction between VFIO and KVM is
one of the motivators of this series.

Motivation
-----------------------------------------------------------------------

The main motivation for this series is upcoming development in the
kernel to support Hypervisor Live Updates [1][2]. Live Update is a
specialized reboot process where selected devices are kept operational
and their kernel state is preserved and recreated across a kexec. For
devices, DMA and interrupts may continue during the reboot. VFIO-bound
devices are the main target, since the first usecase of Live Updates is
to enable host kernel upgrades in a Cloud Computing environment without
disrupting running customer VMs.

To prepare for upcoming support for Live Updates in VFIO, IOMMUFD, IOMMU
drivers, the PCI layer, etc., we'd like to first lay the ground work for
exercising and testing VFIO from kernel selftests. This way when we
eventually upstream support for Live Updates, we can also upstream tests
for those changes, rather than purely relying on Live Update integration
tests which would be hard to share and reproduce upstream.

But even without Live Updates, VFIO and IOMMUFD are becoming an
increasingly critical component of running KVM-based VMs in cloud
environments. Virtualized networking and storage are increasingly being
offloaded to smart NICs/cards, and demand for high performance
networking, storage, and AI are also leading to NICs, SSDs, and GPUs
being directly attached to VMs via VFIO.

VFIO selftests increases our ability to test in several ways.

 - It enables developers sending VFIO, IOMMUFD, etc. commits upstream to
   test their changes against all existing VFIO selftests, reducing the
   probability of regressions.

 - It enables developers sending VFIO, IOMMUFD, etc. commits upstream to
   include tests alongside their changes, increasing the quality of the
   code that is merged.

 - It enables testing the interaction between VFIO and KVM. There are
   some paths in KVM that are only exercised through VFIO, such as IRQ
   bypass. VFIO selftests provides a helper library to enable KVM
   developers to write KVM selftests to test those interactions [3].

Design
-----------------------------------------------------------------------

VFIO selftests are designed around interacting with with VFIO-managed PCI
devices. As such, the core data struture is struct vfio_pci_device, which
represents a single PCI device.

  struct vfio_pci_device *device;

  device = vfio_pci_device_init("0000:6a:01.0", iommu_mode);

  ...

  vfio_pci_device_cleanup(device);

vfio_pci_device_init() sets up a container or iommufd, depending on the
iommu_mode argument, to manage DMA mappings, fetches information about
the device and what interrupts it supports from VFIO and caches it, and
mmap()s all mappable BARs for the test to use.

There are helper methods that operate on struct vfio_pci_device to do
things like read and write to PCI config space, enable/disable IRQs, and
map memory for DMA,

struct vfio_pci_device and its methods do not care about what device
they are actually interacting with. It can be a GPU, a NIC, an SSD, etc.

To keep things simple initially, VFIO selftests only support a single
device per group and per container/iommufd. But it should be possible to
relax those restrictions in the future, e.g. to enable testing with
multiple devices in the same container/iommufd.

Driver Framework
-----------------------------------------------------------------------

In order to support VFIO selftests where a device is generating DMA and
interrupts on command, the VFIO selftests supports a driver framework.

This framework abstracts away device-specific details allowing VFIO
selftests to be written in a generic way, and then run against different
devices depending on what hardware developers have access to.

The framework also aims to support carrying drivers out-of-tree, e.g.
so that companies can run VFIO selftests with custom/test hardware.

Drivers must implement the following methods:

 - probe():        Check if the driver supports a given device.
 - init():         Initialize the driver.
 - remove():       Deinitialize the driver and reset the device.
 - memcpy_start(): Kick off a series of repeated memcpys (DMA reads and
                   DMA writes).
 - memcpy_wait():  Wait for a memcpy operation to complete.
 - send_msi():     Make the device send an MSI interrupt.

memcpy_start/wait() are for generating DMA. We separate the operation
into 2 steps so that tests can trigger a long-running DMA operation. We
expect to use this to stress test Live Updates by kicking off a
long-running mempcy operation and then performing a Live Update. These
methods are required to not generate any interrupts.

send_msi() is used for testing MSI and MSI-x interrupts. The driver
tells the test which MSI it will be using via device->driver.msi.

It's the responsibility of the test to set up a region of memory
and map it into the device for use by the driver, e.g. for in-memory
descriptors, before calling init().

A demo of the driver framework can be found in
tools/testing/selftests/vfio/vfio_pci_driver_test.c.

In addition, this series introduces a new KVM selftest to demonstrate
delivering a device MSI directly into a guest, which can be found in
tools/testing/selftests/kvm/vfio_pci_device_irq_test.c.

Tests
-----------------------------------------------------------------------

There are 5 tests in this series, mostly to demonstrate as a
proof-of-concept:

 - tools/testing/selftests/vfio/vfio_pci_device_test.c
 - tools/testing/selftests/vfio/vfio_pci_driver_test.c
 - tools/testing/selftests/vfio/vfio_iommufd_setup_test.c
 - tools/testing/selftests/vfio/vfio_dma_mapping_test.c
 - tools/testing/selftests/kvm/vfio_pci_device_irq_test.c

Integrating with KVM selftests
-----------------------------------------------------------------------

To support testing the interactions between VFIO and KVM, the VFIO
selftests support sharing its library with the KVM selftest. The patches
at the end of this series demonstrate how that works.

Essentially, we allow the KVM selftests to build their own copy of
tools/testing/selftests/vfio/lib/ and link it into KVM selftests
binaries. This requires minimal changes to the KVM selftests Makefile.

Future Areas of Development
-----------------------------------------------------------------------

Library:

 - Driver support for devices that can be used on AMD, ARM, and other
   platforms.
 - Driver support for a device available in QEMU VMs.
 - Support for tests that use multiple devices.
 - Support for IOMMU groups with multiple devices.
 - Support for multiple devices sharing the same container/iommufd.
 - Sharing TEST_ASSERT() macros and other common code between KVM
   and VFIO selftests.

Tests:

 - DMA mapping performance tests for BARs/HugeTLB/etc.
 - Live Update selftests.
 - Porting Sean's KVM selftest for posted interrupts to use the VFIO
   selftests library [3]

This series can also be found on GitHub:

  https://github.com/dmatlack/linux/tree/vfio/selftests/rfc

Cc: Alex Williamson <alex.williamson@redhat.com>
Cc: Jason Gunthorpe <jgg@nvidia.com>
Cc: Kevin Tian <kevin.tian@intel.com>
Cc: Paolo Bonzini <pbonzini@redhat.com>
Cc: Sean Christopherson <seanjc@google.com>
Cc: Vipin Sharma <vipinsh@google.com>
Cc: Josh Hilke <jrhilke@google.com>
Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
Cc: Saeed Mahameed <saeedm@nvidia.com>
Cc: Saeed Mahameed <saeedm@nvidia.com>
Cc: Adithya Jayachandran <ajayachandra@nvidia.com>
Cc: Parav Pandit <parav@nvidia.com>
Cc: Leon Romanovsky <leonro@nvidia.com>
Cc: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>

[1] https://lore.kernel.org/all/f35359d5-63e1-8390-619f-67961443bfe1@google.com/
[2] https://lore.kernel.org/all/20250515182322.117840-1-pasha.tatashin@soleen.com/
[3] https://lore.kernel.org/kvm/20250404193923.1413163-68-seanjc@google.com/

David Matlack (28):
  selftests: Create tools/testing/selftests/vfio
  vfio: selftests: Add a helper library for VFIO selftests
  vfio: selftests: Introduce vfio_pci_device_test
  tools headers: Add stub definition for __iomem
  tools headers: Import asm-generic MMIO helpers
  tools headers: Import x86 MMIO helper overrides
  tools headers: Import iosubmit_cmds512()
  tools headers: Import drivers/dma/ioat/{hw.h,registers.h}
  tools headers: Import drivers/dma/idxd/registers.h
  tools headers: Import linux/pci_ids.h
  vfio: selftests: Keep track of DMA regions mapped into the device
  vfio: selftests: Enable asserting MSI eventfds not firing
  vfio: selftests: Add a helper for matching vendor+device IDs
  vfio: selftests: Add driver framework
  vfio: sefltests: Add vfio_pci_driver_test
  vfio: selftests: Add driver for Intel CBDMA
  vfio: selftests: Add driver for Intel DSA
  vfio: selftests: Move helper to get cdev path to libvfio
  vfio: selftests: Encapsulate IOMMU mode
  vfio: selftests: Add [-i iommu_mode] option to all tests
  vfio: selftests: Add vfio_type1v2_mode
  vfio: selftests: Add iommufd_compat_type1{,v2} modes
  vfio: selftests: Add iommufd mode
  vfio: selftests: Make iommufd the default iommu_mode
  vfio: selftests: Add a script to help with running VFIO selftests
  KVM: selftests: Build and link sefltests/vfio/lib into KVM selftests
  KVM: selftests: Test sending a vfio-pci device IRQ to a VM
  KVM: selftests: Use real device MSIs in vfio_pci_device_irq_test

Josh Hilke (5):
  vfio: selftests: Test basic VFIO and IOMMUFD integration
  vfio: selftests: Move vfio dma mapping test to their own file
  vfio: selftests: Add test to reset vfio device.
  vfio: selftests: Use command line to set hugepage size for DMA mapping
    test
  vfio: selftests: Validate 2M/1G HugeTLB are mapped as 2M/1G in IOMMU

 MAINTAINERS                                   |    7 +
 tools/arch/x86/include/asm/io.h               |  101 +
 tools/arch/x86/include/asm/special_insns.h    |   27 +
 tools/include/asm-generic/io.h                |  482 +++
 tools/include/asm/io.h                        |   11 +
 tools/include/drivers/dma/idxd/registers.h    |  601 +++
 tools/include/drivers/dma/ioat/hw.h           |  270 ++
 tools/include/drivers/dma/ioat/registers.h    |  251 ++
 tools/include/linux/compiler.h                |    4 +
 tools/include/linux/io.h                      |    4 +-
 tools/include/linux/pci_ids.h                 | 3212 +++++++++++++++++
 tools/testing/selftests/Makefile              |    1 +
 tools/testing/selftests/kvm/Makefile.kvm      |    6 +-
 .../testing/selftests/kvm/include/kvm_util.h  |    4 +
 tools/testing/selftests/kvm/lib/kvm_util.c    |   21 +
 .../selftests/kvm/vfio_pci_device_irq_test.c  |  173 +
 tools/testing/selftests/vfio/.gitignore       |    7 +
 tools/testing/selftests/vfio/Makefile         |   20 +
 .../testing/selftests/vfio/lib/drivers/dsa.c  |  416 +++
 .../testing/selftests/vfio/lib/drivers/ioat.c |  235 ++
 .../selftests/vfio/lib/include/vfio_util.h    |  271 ++
 tools/testing/selftests/vfio/lib/libvfio.mk   |   26 +
 .../selftests/vfio/lib/vfio_pci_device.c      |  573 +++
 .../selftests/vfio/lib/vfio_pci_driver.c      |  126 +
 tools/testing/selftests/vfio/run.sh           |  110 +
 .../selftests/vfio/vfio_dma_mapping_test.c    |  239 ++
 .../selftests/vfio/vfio_iommufd_setup_test.c  |  133 +
 .../selftests/vfio/vfio_pci_device_test.c     |  195 +
 .../selftests/vfio/vfio_pci_driver_test.c     |  256 ++
 29 files changed, 7780 insertions(+), 2 deletions(-)
 create mode 100644 tools/arch/x86/include/asm/io.h
 create mode 100644 tools/arch/x86/include/asm/special_insns.h
 create mode 100644 tools/include/asm-generic/io.h
 create mode 100644 tools/include/asm/io.h
 create mode 100644 tools/include/drivers/dma/idxd/registers.h
 create mode 100644 tools/include/drivers/dma/ioat/hw.h
 create mode 100644 tools/include/drivers/dma/ioat/registers.h
 create mode 100644 tools/include/linux/pci_ids.h
 create mode 100644 tools/testing/selftests/kvm/vfio_pci_device_irq_test.c
 create mode 100644 tools/testing/selftests/vfio/.gitignore
 create mode 100644 tools/testing/selftests/vfio/Makefile
 create mode 100644 tools/testing/selftests/vfio/lib/drivers/dsa.c
 create mode 100644 tools/testing/selftests/vfio/lib/drivers/ioat.c
 create mode 100644 tools/testing/selftests/vfio/lib/include/vfio_util.h
 create mode 100644 tools/testing/selftests/vfio/lib/libvfio.mk
 create mode 100644 tools/testing/selftests/vfio/lib/vfio_pci_device.c
 create mode 100644 tools/testing/selftests/vfio/lib/vfio_pci_driver.c
 create mode 100755 tools/testing/selftests/vfio/run.sh
 create mode 100644 tools/testing/selftests/vfio/vfio_dma_mapping_test.c
 create mode 100644 tools/testing/selftests/vfio/vfio_iommufd_setup_test.c
 create mode 100644 tools/testing/selftests/vfio/vfio_pci_device_test.c
 create mode 100644 tools/testing/selftests/vfio/vfio_pci_driver_test.c


base-commit: a11a72229881d8ac1d52ea727101bc9c744189c1
prerequisite-patch-id: 3bae97c9e1093148763235f47a84fa040b512d04
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 01/33] selftests: Create tools/testing/selftests/vfio

[David Matlack]

Create the directory tools/testing/selftests/vfio with a stub Makefile
and hook it up to the top-level selftests Makefile.

This directory will be used in subsequent commits to host selftests for
the VFIO subsystem.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 MAINTAINERS                             | 7 +++++++
 tools/testing/selftests/Makefile        | 1 +
 tools/testing/selftests/vfio/.gitignore | 7 +++++++
 tools/testing/selftests/vfio/Makefile   | 2 ++
 4 files changed, 17 insertions(+)
 create mode 100644 tools/testing/selftests/vfio/.gitignore
 create mode 100644 tools/testing/selftests/vfio/Makefile

diff --git a/MAINTAINERS b/MAINTAINERS
index 3c6f9e70ace0..ff17343c7fa3 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -25493,6 +25493,7 @@ F:	drivers/vfio/
 F:	include/linux/vfio.h
 F:	include/linux/vfio_pci_core.h
 F:	include/uapi/linux/vfio.h
+F:	tools/testing/selftests/vfio/
 
 VFIO FSL-MC DRIVER
 L:	kvm@vger.kernel.org
@@ -25565,6 +25566,12 @@ L:	virtualization@lists.linux.dev
 S:	Maintained
 F:	drivers/vfio/pci/virtio
 
+VFIO SELFTESTS
+R:	David Matlack <dmatlack@google.com>
+L:	kvm@vger.kernel.org
+S:	Maintained
+F:	tools/testing/selftests/vfio/
+
 VGA_SWITCHEROO
 R:	Lukas Wunner <lukas@wunner.de>
 S:	Maintained
diff --git a/tools/testing/selftests/Makefile b/tools/testing/selftests/Makefile
index 80fb84fa3cfc..7681ebb6bd02 100644
--- a/tools/testing/selftests/Makefile
+++ b/tools/testing/selftests/Makefile
@@ -120,6 +120,7 @@ TARGETS += uevent
 TARGETS += user_events
 TARGETS += vDSO
 TARGETS += mm
+TARGETS += vfio
 TARGETS += x86
 TARGETS += x86/bugs
 TARGETS += zram
diff --git a/tools/testing/selftests/vfio/.gitignore b/tools/testing/selftests/vfio/.gitignore
new file mode 100644
index 000000000000..6d9381d60172
--- /dev/null
+++ b/tools/testing/selftests/vfio/.gitignore
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: GPL-2.0-only
+*
+!/**/
+!*.c
+!*.h
+!*.S
+!*.sh
diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftests/vfio/Makefile
new file mode 100644
index 000000000000..2bba39aff5d9
--- /dev/null
+++ b/tools/testing/selftests/vfio/Makefile
@@ -0,0 +1,2 @@
+CFLAGS = $(KHDR_INCLUDES)
+include ../lib.mk
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 02/33] vfio: selftests: Add a helper library for VFIO selftests

[David Matlack]

Add a basic helper library to be used by VFIO selftests.

The basic unit of the library is struct vfio_pci_device, which
represents a single PCI device that is bound to the vfio-pci driver. The
library currently only supports a single device per group and container,
and VFIO IOMMU types.

The code in this library was heavily based on prior work done by
Raghavendra Rao Ananta <rananta@google.com>, and the VFIO_ASSERT*()
macros were written by Vipin Sharma <vipinsh@google.com>.

Separate that Makefile rules for building the library into a separate
script so that the library can be built by and linked into KVM selftests
in a subsequent commit.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 tools/testing/selftests/vfio/Makefile         |  13 +
 .../selftests/vfio/lib/include/vfio_util.h    | 126 +++++++
 tools/testing/selftests/vfio/lib/libvfio.mk   |  17 +
 .../selftests/vfio/lib/vfio_pci_device.c      | 330 ++++++++++++++++++
 4 files changed, 486 insertions(+)
 create mode 100644 tools/testing/selftests/vfio/lib/include/vfio_util.h
 create mode 100644 tools/testing/selftests/vfio/lib/libvfio.mk
 create mode 100644 tools/testing/selftests/vfio/lib/vfio_pci_device.c

diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftests/vfio/Makefile
index 2bba39aff5d9..f8933dcdc8e1 100644
--- a/tools/testing/selftests/vfio/Makefile
+++ b/tools/testing/selftests/vfio/Makefile
@@ -1,2 +1,15 @@
 CFLAGS = $(KHDR_INCLUDES)
 include ../lib.mk
+include lib/libvfio.mk
+
+CFLAGS += -I$(top_srcdir)/tools/include
+CFLAGS += -MD
+
+$(TEST_GEN_PROGS_EXTENDED): %: %.o $(LIBVFIO_O)
+	$(CC) $(CFLAGS) $(CPPFLAGS) $(LDFLAGS) $< $(LIBVFIO_O) $(LDLIBS) -o $@
+
+TEST_GEN_PROGS_EXTENDED_O = $(patsubst %, %.o, $(TEST_GEN_PROGS_EXTENDED))
+TEST_DEP_FILES = $(patsubst %.o, %.d, $(TEST_GEN_PROGS_EXTENDED_O) $(LIBVFIO_O))
+-include $(TEST_DEP_FILES)
+
+EXTRA_CLEAN += $(TEST_GEN_PROGS_EXTENDED_O) $(TEST_DEP_FILES)
diff --git a/tools/testing/selftests/vfio/lib/include/vfio_util.h b/tools/testing/selftests/vfio/lib/include/vfio_util.h
new file mode 100644
index 000000000000..ea54754a8b16
--- /dev/null
+++ b/tools/testing/selftests/vfio/lib/include/vfio_util.h
@@ -0,0 +1,126 @@
+/* SPDX-License-Identifier: GPL-2.0-only */
+#ifndef SELFTESTS_VFIO_LIB_INCLUDE_VFIO_UTIL_H
+#define SELFTESTS_VFIO_LIB_INCLUDE_VFIO_UTIL_H
+
+#include <string.h>
+#include <linux/vfio.h>
+#include <linux/list.h>
+#include <linux/pci_regs.h>
+
+#include "../../../kselftest.h"
+
+#define VFIO_LOG_AND_EXIT(...) do {		\
+	fprintf(stderr, "  " __VA_ARGS__);	\
+	fprintf(stderr, "\n");			\
+	exit(KSFT_FAIL);			\
+} while (0)
+
+#define VFIO_ASSERT_OP(_lhs, _rhs, _op, ...) do {				\
+	typeof(_lhs) __lhs = (_lhs);						\
+	typeof(_rhs) __rhs = (_rhs);						\
+										\
+	if (__lhs _op __rhs)							\
+		break;								\
+										\
+	fprintf(stderr, "%s:%u: Assertion Failure\n\n", __FILE__, __LINE__);	\
+	fprintf(stderr, "  Expression: " #_lhs " " #_op " " #_rhs "\n");	\
+	fprintf(stderr, "  Observed: %#lx %s %#lx\n",				\
+			(u64)__lhs, #_op, (u64)__rhs);				\
+	fprintf(stderr, "  [errno: %d - %s\n]", errno, strerror(errno));	\
+	VFIO_LOG_AND_EXIT(__VA_ARGS__);						\
+} while (0)
+
+#define VFIO_ASSERT_EQ(_a, _b, ...) VFIO_ASSERT_OP(_a, _b, ==, ##__VA_ARGS__)
+#define VFIO_ASSERT_NE(_a, _b, ...) VFIO_ASSERT_OP(_a, _b, !=, ##__VA_ARGS__)
+#define VFIO_ASSERT_LT(_a, _b, ...) VFIO_ASSERT_OP(_a, _b, <, ##__VA_ARGS__)
+#define VFIO_ASSERT_LE(_a, _b, ...) VFIO_ASSERT_OP(_a, _b, <=, ##__VA_ARGS__)
+#define VFIO_ASSERT_GT(_a, _b, ...) VFIO_ASSERT_OP(_a, _b, >, ##__VA_ARGS__)
+#define VFIO_ASSERT_GE(_a, _b, ...) VFIO_ASSERT_OP(_a, _b, >=, ##__VA_ARGS__)
+#define VFIO_ASSERT_TRUE(_a, ...) VFIO_ASSERT_NE(false, (_a), ##__VA_ARGS__)
+#define VFIO_ASSERT_FALSE(_a, ...) VFIO_ASSERT_EQ(false, (_a), ##__VA_ARGS__)
+#define VFIO_ASSERT_NULL(_a, ...) VFIO_ASSERT_EQ(NULL, _a, ##__VA_ARGS__)
+#define VFIO_ASSERT_NOT_NULL(_a, ...) VFIO_ASSERT_NE(NULL, _a, ##__VA_ARGS__)
+
+#define VFIO_FAIL(_fmt, ...) do {				\
+	fprintf(stderr, "%s:%u: FAIL\n\n", __FILE__, __LINE__);	\
+	VFIO_LOG_AND_EXIT(_fmt, ##__VA_ARGS__);			\
+} while (0)
+
+struct vfio_pci_bar {
+	struct vfio_region_info info;
+	void *vaddr;
+};
+
+struct vfio_pci_device {
+	int fd;
+	int group_fd;
+	int container_fd;
+
+	struct vfio_device_info info;
+	struct vfio_region_info config_space;
+	struct vfio_pci_bar bars[PCI_STD_NUM_BARS];
+
+	struct vfio_irq_info msi_info;
+	struct vfio_irq_info msix_info;
+
+	/* eventfds for MSI and MSI-x interrupts */
+	int msi_eventfds[PCI_MSIX_FLAGS_QSIZE + 1];
+};
+
+struct vfio_pci_device *vfio_pci_device_init(const char *bdf, int iommu_type);
+void vfio_pci_device_cleanup(struct vfio_pci_device *device);
+
+void vfio_pci_dma_map(struct vfio_pci_device *device, u64 iova, u64 size,
+		      void *vaddr);
+void vfio_pci_dma_unmap(struct vfio_pci_device *device, u64 iova, u64 size);
+
+void vfio_pci_config_access(struct vfio_pci_device *device, bool write,
+			    size_t config, size_t size, void *data);
+
+#define vfio_pci_config_read(_device, _offset, _type) ({			    \
+	_type __data;								    \
+	vfio_pci_config_access((_device), false, _offset, sizeof(__data), &__data); \
+	__data;									    \
+})
+
+#define vfio_pci_config_readb(_d, _o) vfio_pci_config_read(_d, _o, u8)
+#define vfio_pci_config_readw(_d, _o) vfio_pci_config_read(_d, _o, u16)
+#define vfio_pci_config_readl(_d, _o) vfio_pci_config_read(_d, _o, u32)
+
+#define vfio_pci_config_write(_device, _offset, _value, _type) do {		  \
+	_type __data = (_value);						  \
+	vfio_pci_config_access((_device), true, _offset, sizeof(_type), &__data); \
+} while (0)
+
+#define vfio_pci_config_writeb(_d, _o, _v) vfio_pci_config_write(_d, _o, _v, u8)
+#define vfio_pci_config_writew(_d, _o, _v) vfio_pci_config_write(_d, _o, _v, u16)
+#define vfio_pci_config_writel(_d, _o, _v) vfio_pci_config_write(_d, _o, _v, u32)
+
+void vfio_pci_irq_enable(struct vfio_pci_device *device, u32 index,
+			 u32 vector, int count);
+void vfio_pci_irq_disable(struct vfio_pci_device *device, u32 index);
+void vfio_pci_irq_trigger(struct vfio_pci_device *device, u32 index, u32 vector);
+
+static inline void vfio_pci_msi_enable(struct vfio_pci_device *device,
+				       u32 vector, int count)
+{
+	vfio_pci_irq_enable(device, VFIO_PCI_MSI_IRQ_INDEX, vector, count);
+}
+
+static inline void vfio_pci_msi_disable(struct vfio_pci_device *device)
+{
+	vfio_pci_irq_disable(device, VFIO_PCI_MSI_IRQ_INDEX);
+}
+
+static inline void vfio_pci_msix_enable(struct vfio_pci_device *device,
+					u32 vector, int count)
+{
+	vfio_pci_irq_enable(device, VFIO_PCI_MSIX_IRQ_INDEX, vector, count);
+}
+
+static inline void vfio_pci_msix_disable(struct vfio_pci_device *device)
+{
+	vfio_pci_irq_disable(device, VFIO_PCI_MSIX_IRQ_INDEX);
+}
+
+#endif /* SELFTESTS_VFIO_LIB_INCLUDE_VFIO_UTIL_H */
diff --git a/tools/testing/selftests/vfio/lib/libvfio.mk b/tools/testing/selftests/vfio/lib/libvfio.mk
new file mode 100644
index 000000000000..388533f8d407
--- /dev/null
+++ b/tools/testing/selftests/vfio/lib/libvfio.mk
@@ -0,0 +1,17 @@
+VFIO_DIR := $(selfdir)/vfio
+
+LIBVFIO_C := lib/vfio_pci_device.c
+
+LIBVFIO_O := $(patsubst %.c, $(OUTPUT)/%.o, $(LIBVFIO_C))
+
+LIBVFIO_O_DIRS := $(shell dirname $(LIBVFIO_O) | uniq)
+
+CFLAGS += -I$(VFIO_DIR)/lib/include
+
+$(LIBVFIO_O): $(OUTPUT)/%.o : $(VFIO_DIR)/%.c $(LIBVFIO_O_DIRS)
+	$(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c $< -o $@
+
+$(LIBVFIO_O_DIRS):
+	mkdir -p $@
+
+EXTRA_CLEAN += $(LIBVFIO_O)
diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_device.c b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
new file mode 100644
index 000000000000..922f21138532
--- /dev/null
+++ b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
@@ -0,0 +1,330 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include <fcntl.h>
+#include <libgen.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
+
+#include <sys/eventfd.h>
+#include <sys/ioctl.h>
+#include <sys/mman.h>
+
+#include <linux/limits.h>
+#include <linux/mman.h>
+#include <linux/types.h>
+#include <linux/vfio.h>
+
+#include "../../../kselftest.h"
+#include <vfio_util.h>
+
+#define VFIO_DEV_PATH	"/dev/vfio/vfio"
+#define PCI_SYSFS_PATH	"/sys/bus/pci/devices"
+
+#define ioctl_assert(_fd, _op, _arg) do {					  \
+	void *__arg = (_arg);							  \
+	int __ret = ioctl((_fd), (_op), (__arg));				  \
+	VFIO_ASSERT_EQ(__ret, 0,						  \
+		    "ioctl(%s, %s, %s) returned %d\n", #_fd, #_op, #_arg, __ret); \
+} while (0)
+
+static void vfio_pci_irq_set(struct vfio_pci_device *device,
+			     u32 index, u32 vector, u32 count, int *fds)
+{
+	u8 buf[sizeof(struct vfio_irq_set) + sizeof(int) * count] = {};
+	struct vfio_irq_set *irq = (void *)&buf;
+	int *irq_fds = (void *)&irq->data;
+
+	irq->argsz = sizeof(buf);
+	irq->flags = VFIO_IRQ_SET_ACTION_TRIGGER;
+	irq->index = index;
+	irq->start = vector;
+	irq->count = count;
+
+	if (count) {
+		irq->flags |= VFIO_IRQ_SET_DATA_EVENTFD;
+		memcpy(irq_fds, fds, sizeof(int) * count);
+	} else {
+		irq->flags |= VFIO_IRQ_SET_DATA_NONE;
+	}
+
+	ioctl_assert(device->fd, VFIO_DEVICE_SET_IRQS, irq);
+}
+
+void vfio_pci_irq_trigger(struct vfio_pci_device *device, u32 index, u32 vector)
+{
+	struct vfio_irq_set irq = {
+		.argsz = sizeof(irq),
+		.flags = VFIO_IRQ_SET_ACTION_TRIGGER | VFIO_IRQ_SET_DATA_NONE,
+		.index = index,
+		.start = vector,
+		.count = 1,
+	};
+
+	ioctl_assert(device->fd, VFIO_DEVICE_SET_IRQS, &irq);
+}
+
+static void check_supported_irq_index(u32 index)
+{
+	/* VFIO selftests only supports MSI and MSI-x for now. */
+	VFIO_ASSERT_TRUE(index == VFIO_PCI_MSI_IRQ_INDEX ||
+			 index == VFIO_PCI_MSIX_IRQ_INDEX,
+			 "Unsupported IRQ index: %u\n", index);
+}
+
+void vfio_pci_irq_enable(struct vfio_pci_device *device, u32 index, u32 vector,
+			 int count)
+{
+	int i;
+
+	check_supported_irq_index(index);
+
+	for (i = vector; i < vector + count; i++) {
+		VFIO_ASSERT_LT(device->msi_eventfds[i], 0);
+		device->msi_eventfds[i] = eventfd(0, 0);
+		VFIO_ASSERT_GE(device->msi_eventfds[i], 0);
+	}
+
+	vfio_pci_irq_set(device, index, vector, count, device->msi_eventfds + vector);
+}
+
+void vfio_pci_irq_disable(struct vfio_pci_device *device, u32 index)
+{
+	int i;
+
+	check_supported_irq_index(index);
+
+	for (i = 0; i < ARRAY_SIZE(device->msi_eventfds); i++) {
+		if (device->msi_eventfds[i] < 0)
+			continue;
+
+		VFIO_ASSERT_EQ(close(device->msi_eventfds[i]), 0);
+		device->msi_eventfds[i] = -1;
+	}
+
+	vfio_pci_irq_set(device, index, 0, 0, NULL);
+}
+
+static void vfio_pci_irq_get(struct vfio_pci_device *device, u32 index,
+			     struct vfio_irq_info *irq_info)
+{
+	irq_info->argsz = sizeof(*irq_info);
+	irq_info->index = index;
+
+	ioctl_assert(device->fd, VFIO_DEVICE_GET_IRQ_INFO, irq_info);
+}
+
+void vfio_pci_dma_map(struct vfio_pci_device *device, u64 iova, u64 size, void *vaddr)
+{
+	struct vfio_iommu_type1_dma_map map = {
+		.argsz = sizeof(map),
+		.flags = VFIO_DMA_MAP_FLAG_READ | VFIO_DMA_MAP_FLAG_WRITE,
+		.vaddr = (u64)vaddr,
+		.iova = iova,
+		.size = size,
+	};
+
+	ioctl_assert(device->container_fd, VFIO_IOMMU_MAP_DMA, &map);
+}
+
+void vfio_pci_dma_unmap(struct vfio_pci_device *device, u64 iova, u64 size)
+{
+	struct vfio_iommu_type1_dma_unmap unmap = {
+		.argsz = sizeof(unmap),
+		.iova = iova,
+		.size = size,
+	};
+
+	ioctl_assert(device->container_fd, VFIO_IOMMU_UNMAP_DMA, &unmap);
+}
+
+static void vfio_pci_region_get(struct vfio_pci_device *device, int index,
+				struct vfio_region_info *info)
+{
+	memset(info, 0, sizeof(*info));
+
+	info->argsz = sizeof(*info);
+	info->index = index;
+
+	ioctl_assert(device->fd, VFIO_DEVICE_GET_REGION_INFO, info);
+}
+
+static void vfio_pci_bar_map(struct vfio_pci_device *device, int index)
+{
+	struct vfio_pci_bar *bar = &device->bars[index];
+	int prot = 0;
+
+	VFIO_ASSERT_LT(index, PCI_STD_NUM_BARS);
+	VFIO_ASSERT_NULL(bar->vaddr);
+	VFIO_ASSERT_TRUE(bar->info.flags & VFIO_REGION_INFO_FLAG_MMAP);
+
+	if (bar->info.flags & VFIO_REGION_INFO_FLAG_READ)
+		prot |= PROT_READ;
+	if (bar->info.flags & VFIO_REGION_INFO_FLAG_WRITE)
+		prot |= PROT_WRITE;
+
+	bar->vaddr = mmap(NULL, bar->info.size, prot, MAP_FILE | MAP_SHARED,
+			  device->fd, bar->info.offset);
+	VFIO_ASSERT_NE(bar->vaddr, MAP_FAILED);
+}
+
+static void vfio_pci_bar_unmap(struct vfio_pci_device *device, int index)
+{
+	struct vfio_pci_bar *bar = &device->bars[index];
+
+	VFIO_ASSERT_LT(index, PCI_STD_NUM_BARS);
+	VFIO_ASSERT_NOT_NULL(bar->vaddr);
+
+	VFIO_ASSERT_EQ(munmap(bar->vaddr, bar->info.size), 0);
+	bar->vaddr = NULL;
+}
+
+static void vfio_pci_bar_unmap_all(struct vfio_pci_device *device)
+{
+	int i;
+
+	for (i = 0; i < PCI_STD_NUM_BARS; i++) {
+		if (device->bars[i].vaddr)
+			vfio_pci_bar_unmap(device, i);
+	}
+}
+
+void vfio_pci_config_access(struct vfio_pci_device *device, bool write,
+			    size_t config, size_t size, void *data)
+{
+	struct vfio_region_info *config_space = &device->config_space;
+	int ret;
+
+	if (write)
+		ret = pwrite(device->fd, data, size, config_space->offset + config);
+	else
+		ret = pread(device->fd, data, size, config_space->offset + config);
+
+	VFIO_ASSERT_EQ(ret, size, "Failed to %s PCI config space: 0x%lx\n",
+		       write ? "write to" : "read from", config);
+}
+
+static unsigned int vfio_pci_get_group_from_dev(const char *bdf)
+{
+	char dev_iommu_group_path[PATH_MAX] = {0};
+	char sysfs_path[PATH_MAX] = {0};
+	unsigned int group;
+	int ret;
+
+	snprintf(sysfs_path, PATH_MAX, "%s/%s/iommu_group", PCI_SYSFS_PATH, bdf);
+
+	ret = readlink(sysfs_path, dev_iommu_group_path, sizeof(dev_iommu_group_path));
+	VFIO_ASSERT_NE(ret, -1, "Failed to get the IOMMU group for device: %s\n", bdf);
+
+	ret = sscanf(basename(dev_iommu_group_path), "%u", &group);
+	VFIO_ASSERT_EQ(ret, 1, "Failed to get the IOMMU group for device: %s\n", bdf);
+
+	return group;
+}
+
+static void vfio_pci_container_setup(struct vfio_pci_device *device)
+{
+	int version;
+
+	device->container_fd = open(VFIO_DEV_PATH, O_RDWR);
+	VFIO_ASSERT_GE(device->container_fd, 0, "open(%s) failed\n", VFIO_DEV_PATH);
+
+	version = ioctl(device->container_fd, VFIO_GET_API_VERSION);
+	VFIO_ASSERT_EQ(version, VFIO_API_VERSION);
+}
+
+static void vfio_pci_group_setup(struct vfio_pci_device *device, const char *bdf)
+{
+	struct vfio_group_status group_status = {
+		.argsz = sizeof(group_status),
+	};
+	char group_path[32];
+	int group;
+
+	group = vfio_pci_get_group_from_dev(bdf);
+	snprintf(group_path, sizeof(group_path), "/dev/vfio/%d", group);
+
+	device->group_fd = open(group_path, O_RDWR);
+	VFIO_ASSERT_GE(device->group_fd, 0, "open(%s) failed\n", group_path);
+
+	ioctl_assert(device->group_fd, VFIO_GROUP_GET_STATUS, &group_status);
+	VFIO_ASSERT_TRUE(group_status.flags & VFIO_GROUP_FLAGS_VIABLE);
+
+	ioctl_assert(device->group_fd, VFIO_GROUP_SET_CONTAINER, &device->container_fd);
+}
+
+static void vfio_pci_iommu_setup(struct vfio_pci_device *device, unsigned long iommu_type)
+{
+	int ret;
+
+	ret = ioctl(device->container_fd, VFIO_CHECK_EXTENSION, iommu_type);
+	VFIO_ASSERT_GT(ret, 0, "VFIO IOMMU type %lu not supported\n", iommu_type);
+
+	ioctl_assert(device->container_fd, VFIO_SET_IOMMU, (void *)iommu_type);
+}
+
+static void vfio_pci_device_setup(struct vfio_pci_device *device, const char *bdf)
+{
+	int i;
+
+	device->fd = ioctl(device->group_fd, VFIO_GROUP_GET_DEVICE_FD, bdf);
+	VFIO_ASSERT_GE(device->fd, 0);
+
+	device->info.argsz = sizeof(device->info);
+	ioctl_assert(device->fd, VFIO_DEVICE_GET_INFO, &device->info);
+
+	vfio_pci_region_get(device, VFIO_PCI_CONFIG_REGION_INDEX, &device->config_space);
+
+	/* Sanity check VFIO does not advertise mmap for config space */
+	VFIO_ASSERT_TRUE(!(device->config_space.flags & VFIO_REGION_INFO_FLAG_MMAP),
+			 "PCI config space should not support mmap()\n");
+
+	for (i = 0; i < PCI_STD_NUM_BARS; i++) {
+		struct vfio_pci_bar *bar = device->bars + i;
+
+		vfio_pci_region_get(device, i, &bar->info);
+		if (bar->info.flags & VFIO_REGION_INFO_FLAG_MMAP)
+			vfio_pci_bar_map(device, i);
+	}
+
+	vfio_pci_irq_get(device, VFIO_PCI_MSI_IRQ_INDEX, &device->msi_info);
+	vfio_pci_irq_get(device, VFIO_PCI_MSIX_IRQ_INDEX, &device->msix_info);
+
+	for (i = 0; i < ARRAY_SIZE(device->msi_eventfds); i++)
+		device->msi_eventfds[i] = -1;
+}
+
+struct vfio_pci_device *vfio_pci_device_init(const char *bdf, int iommu_type)
+{
+	struct vfio_pci_device *device;
+
+	device = calloc(1, sizeof(*device));
+	VFIO_ASSERT_NOT_NULL(device);
+
+	vfio_pci_container_setup(device);
+	vfio_pci_group_setup(device, bdf);
+	vfio_pci_iommu_setup(device, iommu_type);
+	vfio_pci_device_setup(device, bdf);
+
+	return device;
+}
+
+void vfio_pci_device_cleanup(struct vfio_pci_device *device)
+{
+	int i;
+
+	vfio_pci_bar_unmap_all(device);
+
+	VFIO_ASSERT_EQ(close(device->fd), 0);
+
+	for (i = 0; i < ARRAY_SIZE(device->msi_eventfds); i++) {
+		if (device->msi_eventfds[i] < 0)
+			continue;
+
+		VFIO_ASSERT_EQ(close(device->msi_eventfds[i]), 0);
+	}
+
+	VFIO_ASSERT_EQ(close(device->group_fd), 0);
+	VFIO_ASSERT_EQ(close(device->container_fd), 0);
+
+	free(device);
+}
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 03/33] vfio: selftests: Introduce vfio_pci_device_test

[David Matlack]

Introduce a basic VFIO selftest called vfio_pci_device_test to
demonstrate the functionality of the VFIO selftest library and provide
some test coverage of basic VFIO operations, including:

 - Mapping and unmapping DMA
 - Mapping and unmapping BARs
 - Enabling, triggering, and disabling MSI and MSI-x
 - Reading and writing to PCI config space

This test should work with most PCI devices, as long as they are bound
to vfio-pci.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 tools/testing/selftests/vfio/Makefile         |   1 +
 .../selftests/vfio/vfio_pci_device_test.c     | 180 ++++++++++++++++++
 2 files changed, 181 insertions(+)
 create mode 100644 tools/testing/selftests/vfio/vfio_pci_device_test.c

diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftests/vfio/Makefile
index f8933dcdc8e1..89313d63414f 100644
--- a/tools/testing/selftests/vfio/Makefile
+++ b/tools/testing/selftests/vfio/Makefile
@@ -1,4 +1,5 @@
 CFLAGS = $(KHDR_INCLUDES)
+TEST_GEN_PROGS_EXTENDED += vfio_pci_device_test
 include ../lib.mk
 include lib/libvfio.mk
 
diff --git a/tools/testing/selftests/vfio/vfio_pci_device_test.c b/tools/testing/selftests/vfio/vfio_pci_device_test.c
new file mode 100644
index 000000000000..7906b2684dd3
--- /dev/null
+++ b/tools/testing/selftests/vfio/vfio_pci_device_test.c
@@ -0,0 +1,180 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include <fcntl.h>
+#include <stdlib.h>
+
+#include <sys/ioctl.h>
+#include <sys/mman.h>
+
+#include <linux/limits.h>
+#include <linux/pci_regs.h>
+#include <linux/sizes.h>
+#include <linux/vfio.h>
+
+#include <vfio_util.h>
+
+#include "../kselftest_harness.h"
+
+const char *device_bdf;
+
+/*
+ * Limit the number of MSIs enabled/disabled by the test regardless of the
+ * number of MSIs the device itself supports, e.g. to avoid hitting IRTE limits.
+ */
+#define MAX_TEST_MSI 16U
+
+FIXTURE(vfio_pci_device_test) {
+	struct vfio_pci_device *device;
+};
+
+FIXTURE_SETUP(vfio_pci_device_test) {
+	self->device = vfio_pci_device_init(device_bdf, VFIO_TYPE1_IOMMU);
+}
+
+FIXTURE_TEARDOWN(vfio_pci_device_test) {
+	vfio_pci_device_cleanup(self->device);
+}
+
+TEST_F(vfio_pci_device_test, dma_map_unmap)
+{
+	const u64 size = SZ_2M;
+	const u64 iova = SZ_4G;
+	void *mem;
+
+	mem = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0);
+	ASSERT_NE(mem, MAP_FAILED);
+
+	vfio_pci_dma_map(self->device, iova, size, mem);
+	printf("Mapped HVA %p (size 0x%lx) at IOVA 0x%lx\n", mem, size, iova);
+	vfio_pci_dma_unmap(self->device, iova, size);
+
+	ASSERT_TRUE(!munmap(mem, SZ_2M));
+}
+
+#define read_pci_id_from_sysfs(_file) ({							\
+	char __sysfs_path[PATH_MAX];								\
+	char __buf[32];										\
+	int __fd;										\
+												\
+	snprintf(__sysfs_path, PATH_MAX, "/sys/bus/pci/devices/%s/%s", device_bdf, _file);	\
+	ASSERT_GT((__fd = open(__sysfs_path, O_RDONLY)), 0);					\
+	ASSERT_GT(read(__fd, __buf, ARRAY_SIZE(__buf)), 0);					\
+	ASSERT_EQ(0, close(__fd));								\
+	(u16)strtoul(__buf, NULL, 0);								\
+})
+
+TEST_F(vfio_pci_device_test, config_space_read_write)
+{
+	u16 vendor, device;
+	u16 command;
+
+	/* Check that Vendor and Device match what the kernel reports. */
+	vendor = read_pci_id_from_sysfs("vendor");
+	device = read_pci_id_from_sysfs("device");
+
+	ASSERT_EQ(vendor, vfio_pci_config_readw(self->device, PCI_VENDOR_ID));
+	ASSERT_EQ(device, vfio_pci_config_readw(self->device, PCI_DEVICE_ID));
+
+	printf("Vendor: %04x, Device: %04x\n", vendor, device);
+
+	command = vfio_pci_config_readw(self->device, PCI_COMMAND);
+	ASSERT_FALSE(command & PCI_COMMAND_MASTER);
+
+	vfio_pci_config_writew(self->device, PCI_COMMAND, command | PCI_COMMAND_MASTER);
+	command = vfio_pci_config_readw(self->device, PCI_COMMAND);
+	ASSERT_TRUE(command & PCI_COMMAND_MASTER);
+	printf("Enabled Bus Mastering (command: %04x)\n", command);
+
+	vfio_pci_config_writew(self->device, PCI_COMMAND, command & ~PCI_COMMAND_MASTER);
+	command = vfio_pci_config_readw(self->device, PCI_COMMAND);
+	ASSERT_FALSE(command & PCI_COMMAND_MASTER);
+	printf("Disabled Bus Mastering (command: %04x)\n", command);
+}
+
+TEST_F(vfio_pci_device_test, validate_bars)
+{
+	struct vfio_pci_bar *bar;
+	int i;
+
+	for (i = 0; i < PCI_STD_NUM_BARS; i++) {
+		bar = &self->device->bars[i];
+
+		if (!(bar->info.flags & VFIO_REGION_INFO_FLAG_MMAP)) {
+			printf("BAR %d does not support mmap()\n", i);
+			ASSERT_EQ(NULL, bar->vaddr);
+			continue;
+		}
+
+		/*
+		 * BARs that support mmap() should be automatically mapped by
+		 * vfio_pci_device_init().
+		 */
+		ASSERT_NE(NULL, bar->vaddr);
+		ASSERT_NE(0, bar->info.size);
+		printf("BAR %d mapped at %p (size 0x%llx)\n", i, bar->vaddr, bar->info.size);
+	}
+}
+
+FIXTURE(vfio_pci_irq_test) {
+	struct vfio_pci_device *device;
+};
+
+FIXTURE_VARIANT(vfio_pci_irq_test) {
+	int irq_index;
+};
+
+FIXTURE_VARIANT_ADD(vfio_pci_irq_test, msi) {
+	.irq_index = VFIO_PCI_MSI_IRQ_INDEX,
+};
+
+FIXTURE_VARIANT_ADD(vfio_pci_irq_test, msix) {
+	.irq_index = VFIO_PCI_MSIX_IRQ_INDEX,
+};
+
+FIXTURE_SETUP(vfio_pci_irq_test) {
+	self->device = vfio_pci_device_init(device_bdf, VFIO_TYPE1_IOMMU);
+}
+
+FIXTURE_TEARDOWN(vfio_pci_irq_test) {
+	vfio_pci_device_cleanup(self->device);
+}
+
+TEST_F(vfio_pci_irq_test, enable_trigger_disable)
+{
+	bool msix = variant->irq_index == VFIO_PCI_MSIX_IRQ_INDEX;
+	u32 count;
+	u64 value;
+	int i;
+
+	if (msix)
+		count = self->device->msix_info.count;
+	else
+		count = self->device->msi_info.count;
+
+	count = min(count, MAX_TEST_MSI);
+
+	if (!count)
+		SKIP(return, "MSI%s: not supported\n", msix ? "-x" : "");
+
+	vfio_pci_irq_enable(self->device, variant->irq_index, 0, count);
+	printf("MSI%s: enabled %d interrupts\n", msix ? "-x" : "", count);
+
+	for (i = 0; i < count; i++) {
+		vfio_pci_irq_trigger(self->device, variant->irq_index, i);
+		ASSERT_EQ(8, read(self->device->msi_eventfds[i], &value, 8));
+		ASSERT_EQ(1, value);
+	}
+
+	vfio_pci_irq_disable(self->device, variant->irq_index);
+}
+
+int main(int argc, char *argv[])
+{
+	if (argc != 2) {
+		fprintf(stderr, "usage: %s segment:bus:device.function\n", argv[0]);
+		return KSFT_FAIL;
+	}
+
+	device_bdf = argv[1];
+
+	return test_harness_run(1, argv);
+}
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 04/33] vfio: selftests: Test basic VFIO and IOMMUFD integration

[David Matlack]

From: Josh Hilke <jrhilke@google.com>

Add a vfio test suite which verifies that userspace can bind and unbind
devices, allocate I/O address space, and attach a device to an IOMMU
domain using the cdev + IOMMUfd VFIO interface.

Signed-off-by: Josh Hilke <jrhilke@google.com>
Signed-off-by: David Matlack <dmatlack@google.com>
---
 tools/testing/selftests/vfio/Makefile         |   1 +
 .../selftests/vfio/vfio_iommufd_setup_test.c  | 163 ++++++++++++++++++
 2 files changed, 164 insertions(+)
 create mode 100644 tools/testing/selftests/vfio/vfio_iommufd_setup_test.c

diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftests/vfio/Makefile
index 89313d63414f..175d6ed9ba6e 100644
--- a/tools/testing/selftests/vfio/Makefile
+++ b/tools/testing/selftests/vfio/Makefile
@@ -1,4 +1,5 @@
 CFLAGS = $(KHDR_INCLUDES)
+TEST_GEN_PROGS_EXTENDED += vfio_iommufd_setup_test
 TEST_GEN_PROGS_EXTENDED += vfio_pci_device_test
 include ../lib.mk
 include lib/libvfio.mk
diff --git a/tools/testing/selftests/vfio/vfio_iommufd_setup_test.c b/tools/testing/selftests/vfio/vfio_iommufd_setup_test.c
new file mode 100644
index 000000000000..851032f79a31
--- /dev/null
+++ b/tools/testing/selftests/vfio/vfio_iommufd_setup_test.c
@@ -0,0 +1,163 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <assert.h>
+#include <dirent.h>
+#include <fcntl.h>
+
+#include <uapi/linux/types.h>
+#include <linux/limits.h>
+#include <linux/sizes.h>
+#include <linux/vfio.h>
+#include <linux/iommufd.h>
+
+#include <stdint.h>
+#include <stdio.h>
+#include <string.h>
+#include <sys/ioctl.h>
+#include <unistd.h>
+
+#include "../kselftest_harness.h"
+
+static const char iommu_dev_path[] = "/dev/iommu";
+char cdev_path[PATH_MAX] = { '\0' };
+
+static void set_cdev_path(const char *bdf)
+{
+	char dir_path[PATH_MAX];
+	DIR *dir;
+	struct dirent *entry;
+
+	snprintf(dir_path, sizeof(dir_path), "/sys/bus/pci/devices/%s/vfio-dev/", bdf);
+
+	dir = opendir(dir_path);
+	assert(dir);
+
+	/* Find the file named "vfio<number>" */
+	while ((entry = readdir(dir)) != NULL) {
+		if (!strncmp("vfio", entry->d_name, 4)) {
+			snprintf(cdev_path, sizeof(cdev_path), "/dev/vfio/devices/%s",
+				 entry->d_name);
+			break;
+		}
+	}
+
+	assert(strlen(cdev_path) > 0);
+
+	closedir(dir);
+}
+
+static int vfio_device_bind_iommufd_ioctl(int cdev_fd, int iommufd)
+{
+	struct vfio_device_bind_iommufd bind_args = {
+		.argsz = sizeof(bind_args),
+		.iommufd = iommufd,
+	};
+
+	return ioctl(cdev_fd, VFIO_DEVICE_BIND_IOMMUFD, &bind_args);
+}
+
+static int vfio_device_get_info_ioctl(int cdev_fd)
+{
+	struct vfio_device_info info_args = { .argsz = sizeof(info_args) };
+
+	return ioctl(cdev_fd, VFIO_DEVICE_GET_INFO, &info_args);
+}
+
+static int vfio_device_ioas_alloc_ioctl(int iommufd, struct iommu_ioas_alloc *alloc_args)
+{
+	*alloc_args = (struct iommu_ioas_alloc){
+		.size = sizeof(struct iommu_ioas_alloc),
+	};
+
+	return ioctl(iommufd, IOMMU_IOAS_ALLOC, alloc_args);
+}
+
+static int vfio_device_attach_iommufd_pt_ioctl(int cdev_fd, u32 pt_id)
+{
+	struct vfio_device_attach_iommufd_pt attach_args = {
+		.argsz = sizeof(attach_args),
+		.pt_id = pt_id,
+	};
+
+	return ioctl(cdev_fd, VFIO_DEVICE_ATTACH_IOMMUFD_PT, &attach_args);
+}
+
+static int vfio_device_detach_iommufd_pt_ioctl(int cdev_fd)
+{
+	struct vfio_device_detach_iommufd_pt detach_args = {
+		.argsz = sizeof(detach_args),
+	};
+
+	return ioctl(cdev_fd, VFIO_DEVICE_DETACH_IOMMUFD_PT, &detach_args);
+}
+
+FIXTURE(vfio_cdev)
+{
+	int cdev_fd;
+	int iommufd;
+};
+
+FIXTURE_SETUP(vfio_cdev)
+{
+	ASSERT_LE(0, (self->cdev_fd = open(cdev_path, O_RDWR, 0)));
+	ASSERT_LE(0, (self->iommufd = open(iommu_dev_path, O_RDWR, 0)));
+}
+
+FIXTURE_TEARDOWN(vfio_cdev)
+{
+	ASSERT_EQ(0, close(self->cdev_fd));
+	ASSERT_EQ(0, close(self->iommufd));
+}
+
+TEST_F(vfio_cdev, bind)
+{
+	ASSERT_EQ(0, vfio_device_bind_iommufd_ioctl(self->cdev_fd, self->iommufd));
+	ASSERT_EQ(0, vfio_device_get_info_ioctl(self->cdev_fd));
+}
+
+TEST_F(vfio_cdev, get_info_without_bind_fails)
+{
+	ASSERT_NE(0, vfio_device_get_info_ioctl(self->cdev_fd));
+}
+
+TEST_F(vfio_cdev, bind_bad_iommufd_fails)
+{
+	ASSERT_NE(0, vfio_device_bind_iommufd_ioctl(self->cdev_fd, -2));
+}
+
+TEST_F(vfio_cdev, repeated_bind_fails)
+{
+	ASSERT_EQ(0, vfio_device_bind_iommufd_ioctl(self->cdev_fd, self->iommufd));
+	ASSERT_NE(0, vfio_device_bind_iommufd_ioctl(self->cdev_fd, self->iommufd));
+}
+
+TEST_F(vfio_cdev, attach_detatch_pt)
+{
+	struct iommu_ioas_alloc alloc_args;
+
+	ASSERT_EQ(0, vfio_device_bind_iommufd_ioctl(self->cdev_fd, self->iommufd));
+	ASSERT_EQ(0, vfio_device_ioas_alloc_ioctl(self->iommufd, &alloc_args));
+	ASSERT_EQ(0, vfio_device_attach_iommufd_pt_ioctl(self->cdev_fd, alloc_args.out_ioas_id));
+	ASSERT_EQ(0, vfio_device_detach_iommufd_pt_ioctl(self->cdev_fd));
+}
+
+TEST_F(vfio_cdev, attach_invalid_pt_fails)
+{
+	ASSERT_EQ(0, vfio_device_bind_iommufd_ioctl(self->cdev_fd, self->iommufd));
+	ASSERT_NE(0, vfio_device_attach_iommufd_pt_ioctl(self->cdev_fd, UINT32_MAX));
+}
+
+int main(int argc, char *argv[])
+{
+	char *bdf;
+
+	if (argc != 2) {
+		printf("Usage: %s bus:device:function\n", argv[0]);
+		return 1;
+	}
+
+	bdf = argv[1];
+	set_cdev_path(bdf);
+	printf("Using cdev device %s\n", cdev_path);
+
+	return test_harness_run(1, argv);
+}
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 05/33] vfio: selftests: Move vfio dma mapping test to their own file

[David Matlack]

From: Josh Hilke <jrhilke@google.com>

Move the dma_map_unmap test from vfio_pci_device_test to a new test:
vfio_dma_mapping_test. We are going to add more complex dma mapping
tests, so it makes sense to separate this from the vfio pci device
test which is more of a sanity check for vfio pci functionality.

Signed-off-by: Josh Hilke <jrhilke@google.com>
Signed-off-by: David Matlack <dmatlack@google.com>
---
 tools/testing/selftests/vfio/Makefile         |  1 +
 .../selftests/vfio/vfio_dma_mapping_test.c    | 57 +++++++++++++++++++
 .../selftests/vfio/vfio_pci_device_test.c     | 16 ------
 3 files changed, 58 insertions(+), 16 deletions(-)
 create mode 100644 tools/testing/selftests/vfio/vfio_dma_mapping_test.c

diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftests/vfio/Makefile
index 175d6ed9ba6e..2cc97176c6be 100644
--- a/tools/testing/selftests/vfio/Makefile
+++ b/tools/testing/selftests/vfio/Makefile
@@ -1,4 +1,5 @@
 CFLAGS = $(KHDR_INCLUDES)
+TEST_GEN_PROGS_EXTENDED += vfio_dma_mapping_test
 TEST_GEN_PROGS_EXTENDED += vfio_iommufd_setup_test
 TEST_GEN_PROGS_EXTENDED += vfio_pci_device_test
 include ../lib.mk
diff --git a/tools/testing/selftests/vfio/vfio_dma_mapping_test.c b/tools/testing/selftests/vfio/vfio_dma_mapping_test.c
new file mode 100644
index 000000000000..e020f3eb6910
--- /dev/null
+++ b/tools/testing/selftests/vfio/vfio_dma_mapping_test.c
@@ -0,0 +1,57 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include <fcntl.h>
+
+#include <sys/mman.h>
+
+#include <linux/sizes.h>
+#include <linux/vfio.h>
+
+#include <vfio_util.h>
+
+#include "../kselftest_harness.h"
+
+const char *device_bdf;
+
+FIXTURE(vfio_dma_mapping_test)
+{
+	struct vfio_pci_device *device;
+};
+
+FIXTURE_SETUP(vfio_dma_mapping_test)
+{
+	self->device = vfio_pci_device_init(device_bdf, VFIO_TYPE1_IOMMU);
+}
+
+FIXTURE_TEARDOWN(vfio_dma_mapping_test)
+{
+	vfio_pci_device_cleanup(self->device);
+}
+
+TEST_F(vfio_dma_mapping_test, dma_map_unmap)
+{
+	const u64 size = SZ_2M;
+	const u64 iova = SZ_4G;
+	void *mem;
+
+	mem = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0);
+	ASSERT_NE(mem, MAP_FAILED);
+
+	vfio_pci_dma_map(self->device, iova, size, mem);
+	printf("Mapped HVA %p (size 0x%lx) at IOVA 0x%lx\n", mem, size, iova);
+	vfio_pci_dma_unmap(self->device, iova, size);
+
+	ASSERT_TRUE(!munmap(mem, size));
+}
+
+int main(int argc, char *argv[])
+{
+	if (argc != 2) {
+		fprintf(stderr, "usage: %s segment:bus:device.function\n",
+			argv[0]);
+		return KSFT_FAIL;
+	}
+
+	device_bdf = argv[1];
+
+	return test_harness_run(1, argv);
+}
diff --git a/tools/testing/selftests/vfio/vfio_pci_device_test.c b/tools/testing/selftests/vfio/vfio_pci_device_test.c
index 7906b2684dd3..41166f4e9468 100644
--- a/tools/testing/selftests/vfio/vfio_pci_device_test.c
+++ b/tools/testing/selftests/vfio/vfio_pci_device_test.c
@@ -34,22 +34,6 @@ FIXTURE_TEARDOWN(vfio_pci_device_test) {
 	vfio_pci_device_cleanup(self->device);
 }
 
-TEST_F(vfio_pci_device_test, dma_map_unmap)
-{
-	const u64 size = SZ_2M;
-	const u64 iova = SZ_4G;
-	void *mem;
-
-	mem = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0);
-	ASSERT_NE(mem, MAP_FAILED);
-
-	vfio_pci_dma_map(self->device, iova, size, mem);
-	printf("Mapped HVA %p (size 0x%lx) at IOVA 0x%lx\n", mem, size, iova);
-	vfio_pci_dma_unmap(self->device, iova, size);
-
-	ASSERT_TRUE(!munmap(mem, SZ_2M));
-}
-
 #define read_pci_id_from_sysfs(_file) ({							\
 	char __sysfs_path[PATH_MAX];								\
 	char __buf[32];										\
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 06/33] vfio: selftests: Add test to reset vfio device.

[David Matlack]

From: Josh Hilke <jrhilke@google.com>

Add a test to vfio_pci_device_test which resets the device. If reset is
not supported by the device, the test is skipped.

Signed-off-by: Josh Hilke <jrhilke@google.com>
Signed-off-by: David Matlack <dmatlack@google.com>
---
 tools/testing/selftests/vfio/lib/include/vfio_util.h | 1 +
 tools/testing/selftests/vfio/lib/vfio_pci_device.c   | 5 +++++
 tools/testing/selftests/vfio/vfio_pci_device_test.c  | 8 ++++++++
 3 files changed, 14 insertions(+)

diff --git a/tools/testing/selftests/vfio/lib/include/vfio_util.h b/tools/testing/selftests/vfio/lib/include/vfio_util.h
index ea54754a8b16..4e5c6d395a12 100644
--- a/tools/testing/selftests/vfio/lib/include/vfio_util.h
+++ b/tools/testing/selftests/vfio/lib/include/vfio_util.h
@@ -69,6 +69,7 @@ struct vfio_pci_device {
 
 struct vfio_pci_device *vfio_pci_device_init(const char *bdf, int iommu_type);
 void vfio_pci_device_cleanup(struct vfio_pci_device *device);
+void vfio_pci_device_reset(struct vfio_pci_device *device);
 
 void vfio_pci_dma_map(struct vfio_pci_device *device, u64 iova, u64 size,
 		      void *vaddr);
diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_device.c b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
index 922f21138532..a2747de9ad69 100644
--- a/tools/testing/selftests/vfio/lib/vfio_pci_device.c
+++ b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
@@ -328,3 +328,8 @@ void vfio_pci_device_cleanup(struct vfio_pci_device *device)
 
 	free(device);
 }
+
+void vfio_pci_device_reset(struct vfio_pci_device *device)
+{
+	ioctl_assert(device->fd, VFIO_DEVICE_RESET, NULL);
+}
diff --git a/tools/testing/selftests/vfio/vfio_pci_device_test.c b/tools/testing/selftests/vfio/vfio_pci_device_test.c
index 41166f4e9468..b969ecdfb042 100644
--- a/tools/testing/selftests/vfio/vfio_pci_device_test.c
+++ b/tools/testing/selftests/vfio/vfio_pci_device_test.c
@@ -151,6 +151,14 @@ TEST_F(vfio_pci_irq_test, enable_trigger_disable)
 	vfio_pci_irq_disable(self->device, variant->irq_index);
 }
 
+TEST_F(vfio_pci_device_test, reset)
+{
+	if (!(self->device->info.flags & VFIO_DEVICE_FLAGS_RESET))
+		SKIP(return, "Device does not support reset\n");
+
+	vfio_pci_device_reset(self->device);
+}
+
 int main(int argc, char *argv[])
 {
 	if (argc != 2) {
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 07/33] vfio: selftests: Use command line to set hugepage size for DMA mapping test

[David Matlack]

From: Josh Hilke <jrhilke@google.com>

Add a command line arg to vfio_dma_mapping_test to choose the size of
page which is mapped in VFIO.

Signed-off-by: Josh Hilke <jrhilke@google.com>
Signed-off-by: David Matlack <dmatlack@google.com>
---
 .../selftests/vfio/vfio_dma_mapping_test.c    | 92 +++++++++++++++----
 1 file changed, 75 insertions(+), 17 deletions(-)

diff --git a/tools/testing/selftests/vfio/vfio_dma_mapping_test.c b/tools/testing/selftests/vfio/vfio_dma_mapping_test.c
index e020f3eb6910..4b838a06a5fe 100644
--- a/tools/testing/selftests/vfio/vfio_dma_mapping_test.c
+++ b/tools/testing/selftests/vfio/vfio_dma_mapping_test.c
@@ -1,8 +1,11 @@
 // SPDX-License-Identifier: GPL-2.0-only
-#include <fcntl.h>
-
+#include <stdio.h>
+#include <string.h>
 #include <sys/mman.h>
+#include <unistd.h>
 
+#include <linux/limits.h>
+#include <linux/mman.h>
 #include <linux/sizes.h>
 #include <linux/vfio.h>
 
@@ -10,7 +13,12 @@
 
 #include "../kselftest_harness.h"
 
-const char *device_bdf;
+static struct {
+	u64 size;
+	u64 iova;
+	int mmap_flags;
+	const char *bdf;
+} test_config;
 
 FIXTURE(vfio_dma_mapping_test)
 {
@@ -19,7 +27,7 @@ FIXTURE(vfio_dma_mapping_test)
 
 FIXTURE_SETUP(vfio_dma_mapping_test)
 {
-	self->device = vfio_pci_device_init(device_bdf, VFIO_TYPE1_IOMMU);
+	self->device = vfio_pci_device_init(test_config.bdf, VFIO_TYPE1_IOMMU);
 }
 
 FIXTURE_TEARDOWN(vfio_dma_mapping_test)
@@ -29,29 +37,79 @@ FIXTURE_TEARDOWN(vfio_dma_mapping_test)
 
 TEST_F(vfio_dma_mapping_test, dma_map_unmap)
 {
-	const u64 size = SZ_2M;
-	const u64 iova = SZ_4G;
 	void *mem;
 
-	mem = mmap(NULL, size, PROT_READ | PROT_WRITE, MAP_SHARED | MAP_ANONYMOUS, -1, 0);
+	mem = mmap(NULL, test_config.size, PROT_READ | PROT_WRITE,
+		   test_config.mmap_flags, -1, 0);
 	ASSERT_NE(mem, MAP_FAILED);
 
-	vfio_pci_dma_map(self->device, iova, size, mem);
-	printf("Mapped HVA %p (size 0x%lx) at IOVA 0x%lx\n", mem, size, iova);
-	vfio_pci_dma_unmap(self->device, iova, size);
+	vfio_pci_dma_map(self->device, test_config.iova, test_config.size, mem);
+	printf("Mapped HVA %p (size 0x%lx) at IOVA 0x%lx\n", mem,
+	       test_config.size, test_config.iova);
+
+	vfio_pci_dma_unmap(self->device, test_config.iova, test_config.size);
+
+	ASSERT_TRUE(!munmap(mem, test_config.size));
+}
 
-	ASSERT_TRUE(!munmap(mem, size));
+static void help(const char *name)
+{
+	printf("Usage: %s [-b backing_src] segment:bus:device.function\n"
+	       "  -b: Which backing memory to use (default: anonymous)\n"
+	       "\n"
+	       "      anonymous\n"
+	       "      anonymous_hugetlb_2mb\n"
+	       "      anonymous_hugetlb_1gb\n",
+	       name);
+	exit(1);
+}
+
+static int set_backing_src(const char *backing_src)
+{
+	if (!backing_src)
+		return 1;
+
+	test_config.mmap_flags = MAP_PRIVATE | MAP_ANONYMOUS;
+
+	if (!strcmp(backing_src, "anonymous")) {
+		test_config.size = SZ_4K;
+		return 0;
+	} else if (!strcmp(backing_src, "anonymous_hugetlb_2mb")) {
+		test_config.size = SZ_2M;
+		test_config.mmap_flags |= MAP_HUGETLB | MAP_HUGE_2MB;
+		return 0;
+	} else if (!strcmp(backing_src, "anonymous_hugetlb_1gb")) {
+		test_config.size = SZ_1G;
+		test_config.mmap_flags |= MAP_HUGETLB | MAP_HUGE_1GB;
+		return 0;
+	}
+
+	fprintf(stderr, "Unrecognized value for -b: %s\n", backing_src);
+	return 1;
 }
 
 int main(int argc, char *argv[])
 {
-	if (argc != 2) {
-		fprintf(stderr, "usage: %s segment:bus:device.function\n",
-			argv[0]);
-		return KSFT_FAIL;
+	const char *backing_src = "anonymous";
+	int c;
+
+	while ((c = getopt(argc, argv, "b:")) != -1) {
+		switch (c) {
+		case 'b':
+			backing_src = optarg;
+			break;
+		default:
+			help(argv[0]);
+		}
 	}
 
-	device_bdf = argv[1];
+	if (set_backing_src(backing_src))
+		help(argv[0]);
+
+	if (optind >= argc)
+		help(argv[0]);
+
+	test_config.bdf = argv[optind];
 
-	return test_harness_run(1, argv);
+	return test_harness_run(0, NULL);
 }
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 08/33] vfio: selftests: Validate 2M/1G HugeTLB are mapped as 2M/1G in IOMMU

[David Matlack]

From: Josh Hilke <jrhilke@google.com>

Update vfio dma mapping test to verify that the IOMMU uses 2M and 1G
mappings when 2M and 1G HugeTLB pages are mapped into a device
respectively.

This validation is done by inspecting the contents of the I/O page
tables via /sys/kernel/debug/iommu/intel/. This validation is skipped if
that directory is not available (i.e. non-Intel IOMMUs).

Signed-off-by: Josh Hilke <jrhilke@google.com>
[reword commit message, refactor code]
Signed-off-by: David Matlack <dmatlack@google.com>
---
 .../selftests/vfio/vfio_dma_mapping_test.c    | 126 +++++++++++++++++-
 1 file changed, 119 insertions(+), 7 deletions(-)

diff --git a/tools/testing/selftests/vfio/vfio_dma_mapping_test.c b/tools/testing/selftests/vfio/vfio_dma_mapping_test.c
index 4b838a06a5fe..c4920267ae42 100644
--- a/tools/testing/selftests/vfio/vfio_dma_mapping_test.c
+++ b/tools/testing/selftests/vfio/vfio_dma_mapping_test.c
@@ -20,6 +20,83 @@ static struct {
 	const char *bdf;
 } test_config;
 
+struct iommu_mapping {
+	u64 pgd;
+	u64 p4d;
+	u64 pud;
+	u64 pmd;
+	u64 pte;
+};
+
+static void parse_next_value(char **line, u64 *value)
+{
+	char *token;
+
+	token = strtok_r(*line, " \t|\n", line);
+	if (!token)
+		return;
+
+	/* Caller verifies `value`. No need to check return value. */
+	sscanf(token, "0x%lx", value);
+}
+
+static int intel_iommu_mapping_get(const char *bdf, u64 iova,
+				   struct iommu_mapping *mapping)
+{
+	char iommu_mapping_path[PATH_MAX], line[PATH_MAX];
+	u64 line_iova = -1;
+	int ret = -ENOENT;
+	FILE *file;
+	char *rest;
+
+	snprintf(iommu_mapping_path, sizeof(iommu_mapping_path),
+		 "/sys/kernel/debug/iommu/intel/%s/domain_translation_struct",
+		 bdf);
+
+	printf("Searching for IOVA 0x%lx in %s\n", iova, iommu_mapping_path);
+
+	file = fopen(iommu_mapping_path, "r");
+	VFIO_ASSERT_NOT_NULL(file, "fopen(%s) failed", iommu_mapping_path);
+
+	while (fgets(line, sizeof(line), file)) {
+		rest = line;
+
+		parse_next_value(&rest, &line_iova);
+		if (line_iova != (iova / getpagesize()))
+			continue;
+
+		/*
+		 * Ensure each struct field is initialized in case of empty
+		 * page table values.
+		 */
+		memset(mapping, 0, sizeof(*mapping));
+		parse_next_value(&rest, &mapping->pgd);
+		parse_next_value(&rest, &mapping->p4d);
+		parse_next_value(&rest, &mapping->pud);
+		parse_next_value(&rest, &mapping->pmd);
+		parse_next_value(&rest, &mapping->pte);
+
+		ret = 0;
+		break;
+	}
+
+	fclose(file);
+
+	if (ret)
+		printf("IOVA not found\n");
+
+	return ret;
+}
+
+static int iommu_mapping_get(const char *bdf, u64 iova,
+			     struct iommu_mapping *mapping)
+{
+	if (access("/sys/kernel/debug/iommu/intel", F_OK))
+		return intel_iommu_mapping_get(bdf, iova, mapping);
+
+	return -EOPNOTSUPP;
+}
+
 FIXTURE(vfio_dma_mapping_test)
 {
 	struct vfio_pci_device *device;
@@ -37,19 +114,54 @@ FIXTURE_TEARDOWN(vfio_dma_mapping_test)
 
 TEST_F(vfio_dma_mapping_test, dma_map_unmap)
 {
+	const int prot = PROT_READ | PROT_WRITE;
+	const u64 iova = test_config.iova;
+	const u64 size = test_config.size;
+	struct iommu_mapping mapping;
 	void *mem;
+	int rc;
 
-	mem = mmap(NULL, test_config.size, PROT_READ | PROT_WRITE,
-		   test_config.mmap_flags, -1, 0);
+	mem = mmap(NULL, size, prot, test_config.mmap_flags, -1, 0);
 	ASSERT_NE(mem, MAP_FAILED);
 
-	vfio_pci_dma_map(self->device, test_config.iova, test_config.size, mem);
-	printf("Mapped HVA %p (size 0x%lx) at IOVA 0x%lx\n", mem,
-	       test_config.size, test_config.iova);
+	vfio_pci_dma_map(self->device, iova, size, mem);
+	printf("Mapped HVA %p (size 0x%lx) at IOVA 0x%lx\n", mem, size, iova);
+
+	rc = iommu_mapping_get(test_config.bdf, iova, &mapping);
+	if (rc == -EOPNOTSUPP)
+		goto unmap;
+
+	ASSERT_EQ(0, rc);
+	printf("Found IOMMU mappings for IOVA 0x%lx:\n", iova);
+	printf("PGD: 0x%016lx\n", mapping.pgd);
+	printf("P4D: 0x%016lx\n", mapping.p4d);
+	printf("PUD: 0x%016lx\n", mapping.pud);
+	printf("PMD: 0x%016lx\n", mapping.pmd);
+	printf("PTE: 0x%016lx\n", mapping.pte);
+
+	switch (size) {
+	case SZ_4K:
+		ASSERT_NE(0, mapping.pte);
+		break;
+	case SZ_2M:
+		ASSERT_EQ(0, mapping.pte);
+		ASSERT_NE(0, mapping.pmd);
+		break;
+	case SZ_1G:
+		ASSERT_EQ(0, mapping.pte);
+		ASSERT_EQ(0, mapping.pmd);
+		ASSERT_NE(0, mapping.pud);
+		break;
+	default:
+		VFIO_FAIL("Unrecognized size: 0x%lx\n", size);
+	}
 
-	vfio_pci_dma_unmap(self->device, test_config.iova, test_config.size);
+unmap:
+	vfio_pci_dma_unmap(self->device, iova, size);
+	printf("Unmapped IOVA 0x%lx\n", iova);
+	ASSERT_NE(0, iommu_mapping_get(test_config.bdf, iova, &mapping));
 
-	ASSERT_TRUE(!munmap(mem, test_config.size));
+	ASSERT_TRUE(!munmap(mem, size));
 }
 
 static void help(const char *name)
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 09/33] tools headers: Add stub definition for __iomem

[David Matlack]

Add an empty definition for __iomem so that kernel headers that use
__iomem can be imported into tools/include/ with less modifications.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 tools/include/linux/compiler.h | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tools/include/linux/compiler.h b/tools/include/linux/compiler.h
index 9c05a59f0184..df503ed8e038 100644
--- a/tools/include/linux/compiler.h
+++ b/tools/include/linux/compiler.h
@@ -116,6 +116,10 @@
 # define __force
 #endif
 
+#ifndef __iomem
+# define __iomem
+#endif
+
 #ifndef __weak
 # define __weak			__attribute__((weak))
 #endif
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 10/33] tools headers: Import asm-generic MMIO helpers

[David Matlack]

Import the asm-generic MMIO helper functions from the kernel headers
into tools/include/. The top-level include is <linux/io.h> which then
includes the arch-specific <asm/io.h>, which then includes
<asm-generic/io.h>. This layout is chosen to match the kernel header
layout and to appease checkpatch.pl (which warns against including
<asm/io.h> or <asm-generic/io.h> directly).

Changes made when importing:

 - Add missing includes at the top.
 - Stub out mmiowb_set_pending().
 - Stub out _THIS_IP_.
 - Stub out log_*_mmio() calls.
 - Drop the CONFIG_64BIT checks, since tools/include/linux/types.h
   always defines u64.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 tools/include/asm-generic/io.h | 482 +++++++++++++++++++++++++++++++++
 tools/include/asm/io.h         |   7 +
 tools/include/linux/io.h       |   4 +-
 3 files changed, 492 insertions(+), 1 deletion(-)
 create mode 100644 tools/include/asm-generic/io.h
 create mode 100644 tools/include/asm/io.h

diff --git a/tools/include/asm-generic/io.h b/tools/include/asm-generic/io.h
new file mode 100644
index 000000000000..e5a0b07ad452
--- /dev/null
+++ b/tools/include/asm-generic/io.h
@@ -0,0 +1,482 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _TOOLS_ASM_GENERIC_IO_H
+#define _TOOLS_ASM_GENERIC_IO_H
+
+#include <asm/barrier.h>
+#include <asm/byteorder.h>
+
+#include <linux/compiler.h>
+#include <linux/kernel.h>
+#include <linux/types.h>
+
+#ifndef mmiowb_set_pending
+#define mmiowb_set_pending() do { } while (0)
+#endif
+
+#ifndef __io_br
+#define __io_br()      barrier()
+#endif
+
+/* prevent prefetching of coherent DMA data ahead of a dma-complete */
+#ifndef __io_ar
+#ifdef rmb
+#define __io_ar(v)      rmb()
+#else
+#define __io_ar(v)      barrier()
+#endif
+#endif
+
+/* flush writes to coherent DMA data before possibly triggering a DMA read */
+#ifndef __io_bw
+#ifdef wmb
+#define __io_bw()      wmb()
+#else
+#define __io_bw()      barrier()
+#endif
+#endif
+
+/* serialize device access against a spin_unlock, usually handled there. */
+#ifndef __io_aw
+#define __io_aw()      mmiowb_set_pending()
+#endif
+
+#ifndef __io_pbw
+#define __io_pbw()     __io_bw()
+#endif
+
+#ifndef __io_paw
+#define __io_paw()     __io_aw()
+#endif
+
+#ifndef __io_pbr
+#define __io_pbr()     __io_br()
+#endif
+
+#ifndef __io_par
+#define __io_par(v)     __io_ar(v)
+#endif
+
+#ifndef _THIS_IP_
+#define _THIS_IP_ 0
+#endif
+
+static inline void log_write_mmio(u64 val, u8 width, volatile void __iomem *addr,
+				  unsigned long caller_addr, unsigned long caller_addr0) {}
+static inline void log_post_write_mmio(u64 val, u8 width, volatile void __iomem *addr,
+				       unsigned long caller_addr, unsigned long caller_addr0) {}
+static inline void log_read_mmio(u8 width, const volatile void __iomem *addr,
+				 unsigned long caller_addr, unsigned long caller_addr0) {}
+static inline void log_post_read_mmio(u64 val, u8 width, const volatile void __iomem *addr,
+				      unsigned long caller_addr, unsigned long caller_addr0) {}
+
+/*
+ * __raw_{read,write}{b,w,l,q}() access memory in native endianness.
+ *
+ * On some architectures memory mapped IO needs to be accessed differently.
+ * On the simple architectures, we just read/write the memory location
+ * directly.
+ */
+
+#ifndef __raw_readb
+#define __raw_readb __raw_readb
+static inline u8 __raw_readb(const volatile void __iomem *addr)
+{
+	return *(const volatile u8 __force *)addr;
+}
+#endif
+
+#ifndef __raw_readw
+#define __raw_readw __raw_readw
+static inline u16 __raw_readw(const volatile void __iomem *addr)
+{
+	return *(const volatile u16 __force *)addr;
+}
+#endif
+
+#ifndef __raw_readl
+#define __raw_readl __raw_readl
+static inline u32 __raw_readl(const volatile void __iomem *addr)
+{
+	return *(const volatile u32 __force *)addr;
+}
+#endif
+
+#ifndef __raw_readq
+#define __raw_readq __raw_readq
+static inline u64 __raw_readq(const volatile void __iomem *addr)
+{
+	return *(const volatile u64 __force *)addr;
+}
+#endif
+
+#ifndef __raw_writeb
+#define __raw_writeb __raw_writeb
+static inline void __raw_writeb(u8 value, volatile void __iomem *addr)
+{
+	*(volatile u8 __force *)addr = value;
+}
+#endif
+
+#ifndef __raw_writew
+#define __raw_writew __raw_writew
+static inline void __raw_writew(u16 value, volatile void __iomem *addr)
+{
+	*(volatile u16 __force *)addr = value;
+}
+#endif
+
+#ifndef __raw_writel
+#define __raw_writel __raw_writel
+static inline void __raw_writel(u32 value, volatile void __iomem *addr)
+{
+	*(volatile u32 __force *)addr = value;
+}
+#endif
+
+#ifndef __raw_writeq
+#define __raw_writeq __raw_writeq
+static inline void __raw_writeq(u64 value, volatile void __iomem *addr)
+{
+	*(volatile u64 __force *)addr = value;
+}
+#endif
+
+/*
+ * {read,write}{b,w,l,q}() access little endian memory and return result in
+ * native endianness.
+ */
+
+#ifndef readb
+#define readb readb
+static inline u8 readb(const volatile void __iomem *addr)
+{
+	u8 val;
+
+	log_read_mmio(8, addr, _THIS_IP_, _RET_IP_);
+	__io_br();
+	val = __raw_readb(addr);
+	__io_ar(val);
+	log_post_read_mmio(val, 8, addr, _THIS_IP_, _RET_IP_);
+	return val;
+}
+#endif
+
+#ifndef readw
+#define readw readw
+static inline u16 readw(const volatile void __iomem *addr)
+{
+	u16 val;
+
+	log_read_mmio(16, addr, _THIS_IP_, _RET_IP_);
+	__io_br();
+	val = __le16_to_cpu((__le16 __force)__raw_readw(addr));
+	__io_ar(val);
+	log_post_read_mmio(val, 16, addr, _THIS_IP_, _RET_IP_);
+	return val;
+}
+#endif
+
+#ifndef readl
+#define readl readl
+static inline u32 readl(const volatile void __iomem *addr)
+{
+	u32 val;
+
+	log_read_mmio(32, addr, _THIS_IP_, _RET_IP_);
+	__io_br();
+	val = __le32_to_cpu((__le32 __force)__raw_readl(addr));
+	__io_ar(val);
+	log_post_read_mmio(val, 32, addr, _THIS_IP_, _RET_IP_);
+	return val;
+}
+#endif
+
+#ifndef readq
+#define readq readq
+static inline u64 readq(const volatile void __iomem *addr)
+{
+	u64 val;
+
+	log_read_mmio(64, addr, _THIS_IP_, _RET_IP_);
+	__io_br();
+	val = __le64_to_cpu((__le64 __force)__raw_readq(addr));
+	__io_ar(val);
+	log_post_read_mmio(val, 64, addr, _THIS_IP_, _RET_IP_);
+	return val;
+}
+#endif
+
+#ifndef writeb
+#define writeb writeb
+static inline void writeb(u8 value, volatile void __iomem *addr)
+{
+	log_write_mmio(value, 8, addr, _THIS_IP_, _RET_IP_);
+	__io_bw();
+	__raw_writeb(value, addr);
+	__io_aw();
+	log_post_write_mmio(value, 8, addr, _THIS_IP_, _RET_IP_);
+}
+#endif
+
+#ifndef writew
+#define writew writew
+static inline void writew(u16 value, volatile void __iomem *addr)
+{
+	log_write_mmio(value, 16, addr, _THIS_IP_, _RET_IP_);
+	__io_bw();
+	__raw_writew((u16 __force)cpu_to_le16(value), addr);
+	__io_aw();
+	log_post_write_mmio(value, 16, addr, _THIS_IP_, _RET_IP_);
+}
+#endif
+
+#ifndef writel
+#define writel writel
+static inline void writel(u32 value, volatile void __iomem *addr)
+{
+	log_write_mmio(value, 32, addr, _THIS_IP_, _RET_IP_);
+	__io_bw();
+	__raw_writel((u32 __force)__cpu_to_le32(value), addr);
+	__io_aw();
+	log_post_write_mmio(value, 32, addr, _THIS_IP_, _RET_IP_);
+}
+#endif
+
+#ifndef writeq
+#define writeq writeq
+static inline void writeq(u64 value, volatile void __iomem *addr)
+{
+	log_write_mmio(value, 64, addr, _THIS_IP_, _RET_IP_);
+	__io_bw();
+	__raw_writeq((u64 __force)__cpu_to_le64(value), addr);
+	__io_aw();
+	log_post_write_mmio(value, 64, addr, _THIS_IP_, _RET_IP_);
+}
+#endif
+
+/*
+ * {read,write}{b,w,l,q}_relaxed() are like the regular version, but
+ * are not guaranteed to provide ordering against spinlocks or memory
+ * accesses.
+ */
+#ifndef readb_relaxed
+#define readb_relaxed readb_relaxed
+static inline u8 readb_relaxed(const volatile void __iomem *addr)
+{
+	u8 val;
+
+	log_read_mmio(8, addr, _THIS_IP_, _RET_IP_);
+	val = __raw_readb(addr);
+	log_post_read_mmio(val, 8, addr, _THIS_IP_, _RET_IP_);
+	return val;
+}
+#endif
+
+#ifndef readw_relaxed
+#define readw_relaxed readw_relaxed
+static inline u16 readw_relaxed(const volatile void __iomem *addr)
+{
+	u16 val;
+
+	log_read_mmio(16, addr, _THIS_IP_, _RET_IP_);
+	val = __le16_to_cpu((__le16 __force)__raw_readw(addr));
+	log_post_read_mmio(val, 16, addr, _THIS_IP_, _RET_IP_);
+	return val;
+}
+#endif
+
+#ifndef readl_relaxed
+#define readl_relaxed readl_relaxed
+static inline u32 readl_relaxed(const volatile void __iomem *addr)
+{
+	u32 val;
+
+	log_read_mmio(32, addr, _THIS_IP_, _RET_IP_);
+	val = __le32_to_cpu((__le32 __force)__raw_readl(addr));
+	log_post_read_mmio(val, 32, addr, _THIS_IP_, _RET_IP_);
+	return val;
+}
+#endif
+
+#if defined(readq) && !defined(readq_relaxed)
+#define readq_relaxed readq_relaxed
+static inline u64 readq_relaxed(const volatile void __iomem *addr)
+{
+	u64 val;
+
+	log_read_mmio(64, addr, _THIS_IP_, _RET_IP_);
+	val = __le64_to_cpu((__le64 __force)__raw_readq(addr));
+	log_post_read_mmio(val, 64, addr, _THIS_IP_, _RET_IP_);
+	return val;
+}
+#endif
+
+#ifndef writeb_relaxed
+#define writeb_relaxed writeb_relaxed
+static inline void writeb_relaxed(u8 value, volatile void __iomem *addr)
+{
+	log_write_mmio(value, 8, addr, _THIS_IP_, _RET_IP_);
+	__raw_writeb(value, addr);
+	log_post_write_mmio(value, 8, addr, _THIS_IP_, _RET_IP_);
+}
+#endif
+
+#ifndef writew_relaxed
+#define writew_relaxed writew_relaxed
+static inline void writew_relaxed(u16 value, volatile void __iomem *addr)
+{
+	log_write_mmio(value, 16, addr, _THIS_IP_, _RET_IP_);
+	__raw_writew((u16 __force)cpu_to_le16(value), addr);
+	log_post_write_mmio(value, 16, addr, _THIS_IP_, _RET_IP_);
+}
+#endif
+
+#ifndef writel_relaxed
+#define writel_relaxed writel_relaxed
+static inline void writel_relaxed(u32 value, volatile void __iomem *addr)
+{
+	log_write_mmio(value, 32, addr, _THIS_IP_, _RET_IP_);
+	__raw_writel((u32 __force)__cpu_to_le32(value), addr);
+	log_post_write_mmio(value, 32, addr, _THIS_IP_, _RET_IP_);
+}
+#endif
+
+#if defined(writeq) && !defined(writeq_relaxed)
+#define writeq_relaxed writeq_relaxed
+static inline void writeq_relaxed(u64 value, volatile void __iomem *addr)
+{
+	log_write_mmio(value, 64, addr, _THIS_IP_, _RET_IP_);
+	__raw_writeq((u64 __force)__cpu_to_le64(value), addr);
+	log_post_write_mmio(value, 64, addr, _THIS_IP_, _RET_IP_);
+}
+#endif
+
+/*
+ * {read,write}s{b,w,l,q}() repeatedly access the same memory address in
+ * native endianness in 8-, 16-, 32- or 64-bit chunks (@count times).
+ */
+#ifndef readsb
+#define readsb readsb
+static inline void readsb(const volatile void __iomem *addr, void *buffer,
+			  unsigned int count)
+{
+	if (count) {
+		u8 *buf = buffer;
+
+		do {
+			u8 x = __raw_readb(addr);
+			*buf++ = x;
+		} while (--count);
+	}
+}
+#endif
+
+#ifndef readsw
+#define readsw readsw
+static inline void readsw(const volatile void __iomem *addr, void *buffer,
+			  unsigned int count)
+{
+	if (count) {
+		u16 *buf = buffer;
+
+		do {
+			u16 x = __raw_readw(addr);
+			*buf++ = x;
+		} while (--count);
+	}
+}
+#endif
+
+#ifndef readsl
+#define readsl readsl
+static inline void readsl(const volatile void __iomem *addr, void *buffer,
+			  unsigned int count)
+{
+	if (count) {
+		u32 *buf = buffer;
+
+		do {
+			u32 x = __raw_readl(addr);
+			*buf++ = x;
+		} while (--count);
+	}
+}
+#endif
+
+#ifndef readsq
+#define readsq readsq
+static inline void readsq(const volatile void __iomem *addr, void *buffer,
+			  unsigned int count)
+{
+	if (count) {
+		u64 *buf = buffer;
+
+		do {
+			u64 x = __raw_readq(addr);
+			*buf++ = x;
+		} while (--count);
+	}
+}
+#endif
+
+#ifndef writesb
+#define writesb writesb
+static inline void writesb(volatile void __iomem *addr, const void *buffer,
+			   unsigned int count)
+{
+	if (count) {
+		const u8 *buf = buffer;
+
+		do {
+			__raw_writeb(*buf++, addr);
+		} while (--count);
+	}
+}
+#endif
+
+#ifndef writesw
+#define writesw writesw
+static inline void writesw(volatile void __iomem *addr, const void *buffer,
+			   unsigned int count)
+{
+	if (count) {
+		const u16 *buf = buffer;
+
+		do {
+			__raw_writew(*buf++, addr);
+		} while (--count);
+	}
+}
+#endif
+
+#ifndef writesl
+#define writesl writesl
+static inline void writesl(volatile void __iomem *addr, const void *buffer,
+			   unsigned int count)
+{
+	if (count) {
+		const u32 *buf = buffer;
+
+		do {
+			__raw_writel(*buf++, addr);
+		} while (--count);
+	}
+}
+#endif
+
+#ifndef writesq
+#define writesq writesq
+static inline void writesq(volatile void __iomem *addr, const void *buffer,
+			   unsigned int count)
+{
+	if (count) {
+		const u64 *buf = buffer;
+
+		do {
+			__raw_writeq(*buf++, addr);
+		} while (--count);
+	}
+}
+#endif
+
+#endif /* _TOOLS_ASM_GENERIC_IO_H */
diff --git a/tools/include/asm/io.h b/tools/include/asm/io.h
new file mode 100644
index 000000000000..9ae219b12604
--- /dev/null
+++ b/tools/include/asm/io.h
@@ -0,0 +1,7 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _TOOLS_ASM_IO_H
+#define _TOOLS_ASM_IO_H
+
+#include <asm-generic/io.h>
+
+#endif /* _TOOLS_ASM_IO_H */
diff --git a/tools/include/linux/io.h b/tools/include/linux/io.h
index e129871fe661..4b94b84160b8 100644
--- a/tools/include/linux/io.h
+++ b/tools/include/linux/io.h
@@ -2,4 +2,6 @@
 #ifndef _TOOLS_IO_H
 #define _TOOLS_IO_H
 
-#endif
+#include <asm/io.h>
+
+#endif /* _TOOLS_IO_H */
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 11/33] tools headers: Import x86 MMIO helper overrides

[David Matlack]

Import the x86-specific overrides for <asm-generic/io.h> from the kernel
headers into tools/include/.

Changes made when importing:
 - Replace CONFIG_X86_64 with __x86_64__.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 tools/arch/x86/include/asm/io.h | 75 +++++++++++++++++++++++++++++++++
 tools/include/asm/io.h          |  4 ++
 2 files changed, 79 insertions(+)
 create mode 100644 tools/arch/x86/include/asm/io.h

diff --git a/tools/arch/x86/include/asm/io.h b/tools/arch/x86/include/asm/io.h
new file mode 100644
index 000000000000..4c787a2363de
--- /dev/null
+++ b/tools/arch/x86/include/asm/io.h
@@ -0,0 +1,75 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _TOOLS_ASM_X86_IO_H
+#define _TOOLS_ASM_X86_IO_H
+
+#include <linux/compiler.h>
+#include <linux/types.h>
+
+#define build_mmio_read(name, size, type, reg, barrier) \
+static inline type name(const volatile void __iomem *addr) \
+{ type ret; asm volatile("mov" size " %1,%0":reg (ret) \
+:"m" (*(volatile type __force *)addr) barrier); return ret; }
+
+#define build_mmio_write(name, size, type, reg, barrier) \
+static inline void name(type val, volatile void __iomem *addr) \
+{ asm volatile("mov" size " %0,%1": :reg (val), \
+"m" (*(volatile type __force *)addr) barrier); }
+
+build_mmio_read(readb, "b", unsigned char, "=q", :"memory")
+build_mmio_read(readw, "w", unsigned short, "=r", :"memory")
+build_mmio_read(readl, "l", unsigned int, "=r", :"memory")
+
+build_mmio_read(__readb, "b", unsigned char, "=q", )
+build_mmio_read(__readw, "w", unsigned short, "=r", )
+build_mmio_read(__readl, "l", unsigned int, "=r", )
+
+build_mmio_write(writeb, "b", unsigned char, "q", :"memory")
+build_mmio_write(writew, "w", unsigned short, "r", :"memory")
+build_mmio_write(writel, "l", unsigned int, "r", :"memory")
+
+build_mmio_write(__writeb, "b", unsigned char, "q", )
+build_mmio_write(__writew, "w", unsigned short, "r", )
+build_mmio_write(__writel, "l", unsigned int, "r", )
+
+#define readb readb
+#define readw readw
+#define readl readl
+#define readb_relaxed(a) __readb(a)
+#define readw_relaxed(a) __readw(a)
+#define readl_relaxed(a) __readl(a)
+#define __raw_readb __readb
+#define __raw_readw __readw
+#define __raw_readl __readl
+
+#define writeb writeb
+#define writew writew
+#define writel writel
+#define writeb_relaxed(v, a) __writeb(v, a)
+#define writew_relaxed(v, a) __writew(v, a)
+#define writel_relaxed(v, a) __writel(v, a)
+#define __raw_writeb __writeb
+#define __raw_writew __writew
+#define __raw_writel __writel
+
+#ifdef __x86_64__
+
+build_mmio_read(readq, "q", u64, "=r", :"memory")
+build_mmio_read(__readq, "q", u64, "=r", )
+build_mmio_write(writeq, "q", u64, "r", :"memory")
+build_mmio_write(__writeq, "q", u64, "r", )
+
+#define readq_relaxed(a)	__readq(a)
+#define writeq_relaxed(v, a)	__writeq(v, a)
+
+#define __raw_readq		__readq
+#define __raw_writeq		__writeq
+
+/* Let people know that we have them */
+#define readq			readq
+#define writeq			writeq
+
+#endif /* __x86_64__ */
+
+#include <asm-generic/io.h>
+
+#endif /* _TOOLS_ASM_X86_IO_H */
diff --git a/tools/include/asm/io.h b/tools/include/asm/io.h
index 9ae219b12604..eed5066f25c4 100644
--- a/tools/include/asm/io.h
+++ b/tools/include/asm/io.h
@@ -2,6 +2,10 @@
 #ifndef _TOOLS_ASM_IO_H
 #define _TOOLS_ASM_IO_H
 
+#if defined(__i386__) || defined(__x86_64__)
+#include "../../arch/x86/include/asm/io.h"
+#else
 #include <asm-generic/io.h>
+#endif
 
 #endif /* _TOOLS_ASM_IO_H */
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 12/33] tools headers: Import iosubmit_cmds512()

[David Matlack]

Import iosubmit_cmds512() from arch/x86/include/asm/io.h into tools/ so
it can be used by VFIO selftests to interact with Intel DSA devices.

Also pull in movdir64b() from arch/x86/include/asm/special_insns.h into
tools/, which is the underlying instruction used by iosubmit_cmds512().

Changes made when importing: None

Signed-off-by: David Matlack <dmatlack@google.com>
---
 tools/arch/x86/include/asm/io.h            | 26 +++++++++++++++++++++
 tools/arch/x86/include/asm/special_insns.h | 27 ++++++++++++++++++++++
 2 files changed, 53 insertions(+)
 create mode 100644 tools/arch/x86/include/asm/special_insns.h

diff --git a/tools/arch/x86/include/asm/io.h b/tools/arch/x86/include/asm/io.h
index 4c787a2363de..ecad61a3ea52 100644
--- a/tools/arch/x86/include/asm/io.h
+++ b/tools/arch/x86/include/asm/io.h
@@ -4,6 +4,7 @@
 
 #include <linux/compiler.h>
 #include <linux/types.h>
+#include "special_insns.h"
 
 #define build_mmio_read(name, size, type, reg, barrier) \
 static inline type name(const volatile void __iomem *addr) \
@@ -72,4 +73,29 @@ build_mmio_write(__writeq, "q", u64, "r", )
 
 #include <asm-generic/io.h>
 
+/**
+ * iosubmit_cmds512 - copy data to single MMIO location, in 512-bit units
+ * @dst: destination, in MMIO space (must be 512-bit aligned)
+ * @src: source
+ * @count: number of 512 bits quantities to submit
+ *
+ * Submit data from kernel space to MMIO space, in units of 512 bits at a
+ * time.  Order of access is not guaranteed, nor is a memory barrier
+ * performed afterwards.
+ *
+ * Warning: Do not use this helper unless your driver has checked that the CPU
+ * instruction is supported on the platform.
+ */
+static inline void iosubmit_cmds512(void __iomem *dst, const void *src,
+				    size_t count)
+{
+	const u8 *from = src;
+	const u8 *end = from + count * 64;
+
+	while (from < end) {
+		movdir64b(dst, from);
+		from += 64;
+	}
+}
+
 #endif /* _TOOLS_ASM_X86_IO_H */
diff --git a/tools/arch/x86/include/asm/special_insns.h b/tools/arch/x86/include/asm/special_insns.h
new file mode 100644
index 000000000000..04af42a99c38
--- /dev/null
+++ b/tools/arch/x86/include/asm/special_insns.h
@@ -0,0 +1,27 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _TOOLS_ASM_X86_SPECIAL_INSNS_H
+#define _TOOLS_ASM_X86_SPECIAL_INSNS_H
+
+/* The dst parameter must be 64-bytes aligned */
+static inline void movdir64b(void *dst, const void *src)
+{
+	const struct { char _[64]; } *__src = src;
+	struct { char _[64]; } *__dst = dst;
+
+	/*
+	 * MOVDIR64B %(rdx), rax.
+	 *
+	 * Both __src and __dst must be memory constraints in order to tell the
+	 * compiler that no other memory accesses should be reordered around
+	 * this one.
+	 *
+	 * Also, both must be supplied as lvalues because this tells
+	 * the compiler what the object is (its size) the instruction accesses.
+	 * I.e., not the pointers but what they point to, thus the deref'ing '*'.
+	 */
+	asm volatile(".byte 0x66, 0x0f, 0x38, 0xf8, 0x02"
+		     : "+m" (*__dst)
+		     :  "m" (*__src), "a" (__dst), "d" (__src));
+}
+
+#endif /* _TOOLS_ASM_X86_SPECIAL_INSNS_H */
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 13/33] tools headers: Import drivers/dma/ioat/{hw.h,registers.h}

[David Matlack]

Import drivers/dma/ioat/{hw.h,registers.h} into tools/include/ so that
they can be used in VFIO selftests to interact with Intel CBDMA devices.

Changes made when importing:
 - Drop system_has_dca_enabled() prototype from hw.h

Cc: Dave Jiang <dave.jiang@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: David Matlack <dmatlack@google.com>
---
 tools/include/drivers/dma/ioat/hw.h        | 270 +++++++++++++++++++++
 tools/include/drivers/dma/ioat/registers.h | 251 +++++++++++++++++++
 2 files changed, 521 insertions(+)
 create mode 100644 tools/include/drivers/dma/ioat/hw.h
 create mode 100644 tools/include/drivers/dma/ioat/registers.h

diff --git a/tools/include/drivers/dma/ioat/hw.h b/tools/include/drivers/dma/ioat/hw.h
new file mode 100644
index 000000000000..62c7a6644376
--- /dev/null
+++ b/tools/include/drivers/dma/ioat/hw.h
@@ -0,0 +1,270 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright(c) 2004 - 2009 Intel Corporation. All rights reserved.
+ */
+#ifndef _IOAT_HW_H_
+#define _IOAT_HW_H_
+
+#include <linux/types.h>
+
+/* PCI Configuration Space Values */
+#define IOAT_MMIO_BAR		0
+
+/* CB device ID's */
+#define PCI_DEVICE_ID_INTEL_IOAT_IVB0	0x0e20
+#define PCI_DEVICE_ID_INTEL_IOAT_IVB1	0x0e21
+#define PCI_DEVICE_ID_INTEL_IOAT_IVB2	0x0e22
+#define PCI_DEVICE_ID_INTEL_IOAT_IVB3	0x0e23
+#define PCI_DEVICE_ID_INTEL_IOAT_IVB4	0x0e24
+#define PCI_DEVICE_ID_INTEL_IOAT_IVB5	0x0e25
+#define PCI_DEVICE_ID_INTEL_IOAT_IVB6	0x0e26
+#define PCI_DEVICE_ID_INTEL_IOAT_IVB7	0x0e27
+#define PCI_DEVICE_ID_INTEL_IOAT_IVB8	0x0e2e
+#define PCI_DEVICE_ID_INTEL_IOAT_IVB9	0x0e2f
+
+#define PCI_DEVICE_ID_INTEL_IOAT_HSW0	0x2f20
+#define PCI_DEVICE_ID_INTEL_IOAT_HSW1	0x2f21
+#define PCI_DEVICE_ID_INTEL_IOAT_HSW2	0x2f22
+#define PCI_DEVICE_ID_INTEL_IOAT_HSW3	0x2f23
+#define PCI_DEVICE_ID_INTEL_IOAT_HSW4	0x2f24
+#define PCI_DEVICE_ID_INTEL_IOAT_HSW5	0x2f25
+#define PCI_DEVICE_ID_INTEL_IOAT_HSW6	0x2f26
+#define PCI_DEVICE_ID_INTEL_IOAT_HSW7	0x2f27
+#define PCI_DEVICE_ID_INTEL_IOAT_HSW8	0x2f2e
+#define PCI_DEVICE_ID_INTEL_IOAT_HSW9	0x2f2f
+
+#define PCI_DEVICE_ID_INTEL_IOAT_BWD0	0x0C50
+#define PCI_DEVICE_ID_INTEL_IOAT_BWD1	0x0C51
+#define PCI_DEVICE_ID_INTEL_IOAT_BWD2	0x0C52
+#define PCI_DEVICE_ID_INTEL_IOAT_BWD3	0x0C53
+
+#define PCI_DEVICE_ID_INTEL_IOAT_BDXDE0	0x6f50
+#define PCI_DEVICE_ID_INTEL_IOAT_BDXDE1	0x6f51
+#define PCI_DEVICE_ID_INTEL_IOAT_BDXDE2	0x6f52
+#define PCI_DEVICE_ID_INTEL_IOAT_BDXDE3	0x6f53
+
+#define PCI_DEVICE_ID_INTEL_IOAT_BDX0	0x6f20
+#define PCI_DEVICE_ID_INTEL_IOAT_BDX1	0x6f21
+#define PCI_DEVICE_ID_INTEL_IOAT_BDX2	0x6f22
+#define PCI_DEVICE_ID_INTEL_IOAT_BDX3	0x6f23
+#define PCI_DEVICE_ID_INTEL_IOAT_BDX4	0x6f24
+#define PCI_DEVICE_ID_INTEL_IOAT_BDX5	0x6f25
+#define PCI_DEVICE_ID_INTEL_IOAT_BDX6	0x6f26
+#define PCI_DEVICE_ID_INTEL_IOAT_BDX7	0x6f27
+#define PCI_DEVICE_ID_INTEL_IOAT_BDX8	0x6f2e
+#define PCI_DEVICE_ID_INTEL_IOAT_BDX9	0x6f2f
+
+#define PCI_DEVICE_ID_INTEL_IOAT_SKX	0x2021
+
+#define PCI_DEVICE_ID_INTEL_IOAT_ICX	0x0b00
+
+#define IOAT_VER_1_2            0x12    /* Version 1.2 */
+#define IOAT_VER_2_0            0x20    /* Version 2.0 */
+#define IOAT_VER_3_0            0x30    /* Version 3.0 */
+#define IOAT_VER_3_2            0x32    /* Version 3.2 */
+#define IOAT_VER_3_3            0x33    /* Version 3.3 */
+#define IOAT_VER_3_4		0x34	/* Version 3.4 */
+
+#define IOAT_DESC_SZ	64
+
+struct ioat_dma_descriptor {
+	uint32_t	size;
+	union {
+		uint32_t ctl;
+		struct {
+			unsigned int int_en:1;
+			unsigned int src_snoop_dis:1;
+			unsigned int dest_snoop_dis:1;
+			unsigned int compl_write:1;
+			unsigned int fence:1;
+			unsigned int null:1;
+			unsigned int src_brk:1;
+			unsigned int dest_brk:1;
+			unsigned int bundle:1;
+			unsigned int dest_dca:1;
+			unsigned int hint:1;
+			unsigned int rsvd2:13;
+			#define IOAT_OP_COPY 0x00
+			unsigned int op:8;
+		} ctl_f;
+	};
+	uint64_t	src_addr;
+	uint64_t	dst_addr;
+	uint64_t	next;
+	uint64_t	rsv1;
+	uint64_t	rsv2;
+	/* store some driver data in an unused portion of the descriptor */
+	union {
+		uint64_t	user1;
+		uint64_t	tx_cnt;
+	};
+	uint64_t	user2;
+};
+
+struct ioat_xor_descriptor {
+	uint32_t	size;
+	union {
+		uint32_t ctl;
+		struct {
+			unsigned int int_en:1;
+			unsigned int src_snoop_dis:1;
+			unsigned int dest_snoop_dis:1;
+			unsigned int compl_write:1;
+			unsigned int fence:1;
+			unsigned int src_cnt:3;
+			unsigned int bundle:1;
+			unsigned int dest_dca:1;
+			unsigned int hint:1;
+			unsigned int rsvd:13;
+			#define IOAT_OP_XOR 0x87
+			#define IOAT_OP_XOR_VAL 0x88
+			unsigned int op:8;
+		} ctl_f;
+	};
+	uint64_t	src_addr;
+	uint64_t	dst_addr;
+	uint64_t	next;
+	uint64_t	src_addr2;
+	uint64_t	src_addr3;
+	uint64_t	src_addr4;
+	uint64_t	src_addr5;
+};
+
+struct ioat_xor_ext_descriptor {
+	uint64_t	src_addr6;
+	uint64_t	src_addr7;
+	uint64_t	src_addr8;
+	uint64_t	next;
+	uint64_t	rsvd[4];
+};
+
+struct ioat_pq_descriptor {
+	union {
+		uint32_t	size;
+		uint32_t	dwbes;
+		struct {
+			unsigned int rsvd:25;
+			unsigned int p_val_err:1;
+			unsigned int q_val_err:1;
+			unsigned int rsvd1:4;
+			unsigned int wbes:1;
+		} dwbes_f;
+	};
+	union {
+		uint32_t ctl;
+		struct {
+			unsigned int int_en:1;
+			unsigned int src_snoop_dis:1;
+			unsigned int dest_snoop_dis:1;
+			unsigned int compl_write:1;
+			unsigned int fence:1;
+			unsigned int src_cnt:3;
+			unsigned int bundle:1;
+			unsigned int dest_dca:1;
+			unsigned int hint:1;
+			unsigned int p_disable:1;
+			unsigned int q_disable:1;
+			unsigned int rsvd2:2;
+			unsigned int wb_en:1;
+			unsigned int prl_en:1;
+			unsigned int rsvd3:7;
+			#define IOAT_OP_PQ 0x89
+			#define IOAT_OP_PQ_VAL 0x8a
+			#define IOAT_OP_PQ_16S 0xa0
+			#define IOAT_OP_PQ_VAL_16S 0xa1
+			unsigned int op:8;
+		} ctl_f;
+	};
+	uint64_t	src_addr;
+	uint64_t	p_addr;
+	uint64_t	next;
+	uint64_t	src_addr2;
+	union {
+		uint64_t	src_addr3;
+		uint64_t	sed_addr;
+	};
+	uint8_t		coef[8];
+	uint64_t	q_addr;
+};
+
+struct ioat_pq_ext_descriptor {
+	uint64_t	src_addr4;
+	uint64_t	src_addr5;
+	uint64_t	src_addr6;
+	uint64_t	next;
+	uint64_t	src_addr7;
+	uint64_t	src_addr8;
+	uint64_t	rsvd[2];
+};
+
+struct ioat_pq_update_descriptor {
+	uint32_t	size;
+	union {
+		uint32_t ctl;
+		struct {
+			unsigned int int_en:1;
+			unsigned int src_snoop_dis:1;
+			unsigned int dest_snoop_dis:1;
+			unsigned int compl_write:1;
+			unsigned int fence:1;
+			unsigned int src_cnt:3;
+			unsigned int bundle:1;
+			unsigned int dest_dca:1;
+			unsigned int hint:1;
+			unsigned int p_disable:1;
+			unsigned int q_disable:1;
+			unsigned int rsvd:3;
+			unsigned int coef:8;
+			#define IOAT_OP_PQ_UP 0x8b
+			unsigned int op:8;
+		} ctl_f;
+	};
+	uint64_t	src_addr;
+	uint64_t	p_addr;
+	uint64_t	next;
+	uint64_t	src_addr2;
+	uint64_t	p_src;
+	uint64_t	q_src;
+	uint64_t	q_addr;
+};
+
+struct ioat_raw_descriptor {
+	uint64_t	field[8];
+};
+
+struct ioat_pq16a_descriptor {
+	uint8_t coef[8];
+	uint64_t src_addr3;
+	uint64_t src_addr4;
+	uint64_t src_addr5;
+	uint64_t src_addr6;
+	uint64_t src_addr7;
+	uint64_t src_addr8;
+	uint64_t src_addr9;
+};
+
+struct ioat_pq16b_descriptor {
+	uint64_t src_addr10;
+	uint64_t src_addr11;
+	uint64_t src_addr12;
+	uint64_t src_addr13;
+	uint64_t src_addr14;
+	uint64_t src_addr15;
+	uint64_t src_addr16;
+	uint64_t rsvd;
+};
+
+union ioat_sed_pq_descriptor {
+	struct ioat_pq16a_descriptor a;
+	struct ioat_pq16b_descriptor b;
+};
+
+#define SED_SIZE	64
+
+struct ioat_sed_raw_descriptor {
+	uint64_t	a[8];
+	uint64_t	b[8];
+	uint64_t	c[8];
+};
+
+#endif
diff --git a/tools/include/drivers/dma/ioat/registers.h b/tools/include/drivers/dma/ioat/registers.h
new file mode 100644
index 000000000000..54cf0ad39887
--- /dev/null
+++ b/tools/include/drivers/dma/ioat/registers.h
@@ -0,0 +1,251 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * Copyright(c) 2004 - 2009 Intel Corporation. All rights reserved.
+ */
+#ifndef _IOAT_REGISTERS_H_
+#define _IOAT_REGISTERS_H_
+
+#define IOAT_PCI_DMACTRL_OFFSET			0x48
+#define IOAT_PCI_DMACTRL_DMA_EN			0x00000001
+#define IOAT_PCI_DMACTRL_MSI_EN			0x00000002
+
+#define IOAT_PCI_DEVICE_ID_OFFSET		0x02
+#define IOAT_PCI_DMAUNCERRSTS_OFFSET		0x148
+#define IOAT_PCI_CHANERR_INT_OFFSET		0x180
+#define IOAT_PCI_CHANERRMASK_INT_OFFSET		0x184
+
+/* MMIO Device Registers */
+#define IOAT_CHANCNT_OFFSET			0x00	/*  8-bit */
+
+#define IOAT_XFERCAP_OFFSET			0x01	/*  8-bit */
+#define IOAT_XFERCAP_4KB			12
+#define IOAT_XFERCAP_8KB			13
+#define IOAT_XFERCAP_16KB			14
+#define IOAT_XFERCAP_32KB			15
+#define IOAT_XFERCAP_32GB			0
+
+#define IOAT_GENCTRL_OFFSET			0x02	/*  8-bit */
+#define IOAT_GENCTRL_DEBUG_EN			0x01
+
+#define IOAT_INTRCTRL_OFFSET			0x03	/*  8-bit */
+#define IOAT_INTRCTRL_MASTER_INT_EN		0x01	/* Master Interrupt Enable */
+#define IOAT_INTRCTRL_INT_STATUS		0x02	/* ATTNSTATUS -or- Channel Int */
+#define IOAT_INTRCTRL_INT			0x04	/* INT_STATUS -and- MASTER_INT_EN */
+#define IOAT_INTRCTRL_MSIX_VECTOR_CONTROL	0x08	/* Enable all MSI-X vectors */
+
+#define IOAT_ATTNSTATUS_OFFSET			0x04	/* Each bit is a channel */
+
+#define IOAT_VER_OFFSET				0x08	/*  8-bit */
+#define IOAT_VER_MAJOR_MASK			0xF0
+#define IOAT_VER_MINOR_MASK			0x0F
+#define GET_IOAT_VER_MAJOR(x)			(((x) & IOAT_VER_MAJOR_MASK) >> 4)
+#define GET_IOAT_VER_MINOR(x)			((x) & IOAT_VER_MINOR_MASK)
+
+#define IOAT_PERPORTOFFSET_OFFSET		0x0A	/* 16-bit */
+
+#define IOAT_INTRDELAY_OFFSET			0x0C	/* 16-bit */
+#define IOAT_INTRDELAY_MASK			0x3FFF	/* Interrupt Delay Time */
+#define IOAT_INTRDELAY_COALESE_SUPPORT		0x8000	/* Interrupt Coalescing Supported */
+
+#define IOAT_DEVICE_STATUS_OFFSET		0x0E	/* 16-bit */
+#define IOAT_DEVICE_STATUS_DEGRADED_MODE	0x0001
+#define IOAT_DEVICE_MMIO_RESTRICTED		0x0002
+#define IOAT_DEVICE_MEMORY_BYPASS		0x0004
+#define IOAT_DEVICE_ADDRESS_REMAPPING		0x0008
+
+#define IOAT_DMA_CAP_OFFSET			0x10	/* 32-bit */
+#define IOAT_CAP_PAGE_BREAK			0x00000001
+#define IOAT_CAP_CRC				0x00000002
+#define IOAT_CAP_SKIP_MARKER			0x00000004
+#define IOAT_CAP_DCA				0x00000010
+#define IOAT_CAP_CRC_MOVE			0x00000020
+#define IOAT_CAP_FILL_BLOCK			0x00000040
+#define IOAT_CAP_APIC				0x00000080
+#define IOAT_CAP_XOR				0x00000100
+#define IOAT_CAP_PQ				0x00000200
+#define IOAT_CAP_DWBES				0x00002000
+#define IOAT_CAP_RAID16SS			0x00020000
+#define IOAT_CAP_DPS				0x00800000
+
+#define IOAT_PREFETCH_LIMIT_OFFSET		0x4C	/* CHWPREFLMT */
+
+#define IOAT_CHANNEL_MMIO_SIZE			0x80	/* Each Channel MMIO space is this size */
+
+/* DMA Channel Registers */
+#define IOAT_CHANCTRL_OFFSET			0x00	/* 16-bit Channel Control Register */
+#define IOAT_CHANCTRL_CHANNEL_PRIORITY_MASK	0xF000
+#define IOAT3_CHANCTRL_COMPL_DCA_EN		0x0200
+#define IOAT_CHANCTRL_CHANNEL_IN_USE		0x0100
+#define IOAT_CHANCTRL_DESCRIPTOR_ADDR_SNOOP_CONTROL	0x0020
+#define IOAT_CHANCTRL_ERR_INT_EN		0x0010
+#define IOAT_CHANCTRL_ANY_ERR_ABORT_EN		0x0008
+#define IOAT_CHANCTRL_ERR_COMPLETION_EN		0x0004
+#define IOAT_CHANCTRL_INT_REARM			0x0001
+#define IOAT_CHANCTRL_RUN			(IOAT_CHANCTRL_INT_REARM |\
+						 IOAT_CHANCTRL_ERR_INT_EN |\
+						 IOAT_CHANCTRL_ERR_COMPLETION_EN |\
+						 IOAT_CHANCTRL_ANY_ERR_ABORT_EN)
+
+#define IOAT_DMA_COMP_OFFSET			0x02	/* 16-bit DMA channel compatibility */
+#define IOAT_DMA_COMP_V1			0x0001	/* Compatibility with DMA version 1 */
+#define IOAT_DMA_COMP_V2			0x0002	/* Compatibility with DMA version 2 */
+
+#define IOAT_CHANSTS_OFFSET		0x08	/* 64-bit Channel Status Register */
+#define IOAT_CHANSTS_COMPLETED_DESCRIPTOR_ADDR	(~0x3fULL)
+#define IOAT_CHANSTS_SOFT_ERR			0x10ULL
+#define IOAT_CHANSTS_UNAFFILIATED_ERR		0x8ULL
+#define IOAT_CHANSTS_STATUS	0x7ULL
+#define IOAT_CHANSTS_ACTIVE	0x0
+#define IOAT_CHANSTS_DONE	0x1
+#define IOAT_CHANSTS_SUSPENDED	0x2
+#define IOAT_CHANSTS_HALTED	0x3
+
+
+
+#define IOAT_CHAN_DMACOUNT_OFFSET	0x06    /* 16-bit DMA Count register */
+
+#define IOAT_DCACTRL_OFFSET         0x30   /* 32 bit Direct Cache Access Control Register */
+#define IOAT_DCACTRL_CMPL_WRITE_ENABLE 0x10000
+#define IOAT_DCACTRL_TARGET_CPU_MASK   0xFFFF /* APIC ID */
+
+/* CB DCA Memory Space Registers */
+#define IOAT_DCAOFFSET_OFFSET       0x14
+/* CB_BAR + IOAT_DCAOFFSET value */
+#define IOAT_DCA_VER_OFFSET         0x00
+#define IOAT_DCA_VER_MAJOR_MASK     0xF0
+#define IOAT_DCA_VER_MINOR_MASK     0x0F
+
+#define IOAT_DCA_COMP_OFFSET        0x02
+#define IOAT_DCA_COMP_V1            0x1
+
+#define IOAT_FSB_CAPABILITY_OFFSET  0x04
+#define IOAT_FSB_CAPABILITY_PREFETCH    0x1
+
+#define IOAT_PCI_CAPABILITY_OFFSET  0x06
+#define IOAT_PCI_CAPABILITY_MEMWR   0x1
+
+#define IOAT_FSB_CAP_ENABLE_OFFSET  0x08
+#define IOAT_FSB_CAP_ENABLE_PREFETCH    0x1
+
+#define IOAT_PCI_CAP_ENABLE_OFFSET  0x0A
+#define IOAT_PCI_CAP_ENABLE_MEMWR   0x1
+
+#define IOAT_APICID_TAG_MAP_OFFSET  0x0C
+#define IOAT_APICID_TAG_MAP_TAG0    0x0000000F
+#define IOAT_APICID_TAG_MAP_TAG0_SHIFT 0
+#define IOAT_APICID_TAG_MAP_TAG1    0x000000F0
+#define IOAT_APICID_TAG_MAP_TAG1_SHIFT 4
+#define IOAT_APICID_TAG_MAP_TAG2    0x00000F00
+#define IOAT_APICID_TAG_MAP_TAG2_SHIFT 8
+#define IOAT_APICID_TAG_MAP_TAG3    0x0000F000
+#define IOAT_APICID_TAG_MAP_TAG3_SHIFT 12
+#define IOAT_APICID_TAG_MAP_TAG4    0x000F0000
+#define IOAT_APICID_TAG_MAP_TAG4_SHIFT 16
+#define IOAT_APICID_TAG_CB2_VALID   0x8080808080
+
+#define IOAT_DCA_GREQID_OFFSET      0x10
+#define IOAT_DCA_GREQID_SIZE        0x04
+#define IOAT_DCA_GREQID_MASK        0xFFFF
+#define IOAT_DCA_GREQID_IGNOREFUN   0x10000000
+#define IOAT_DCA_GREQID_VALID       0x20000000
+#define IOAT_DCA_GREQID_LASTID      0x80000000
+
+#define IOAT3_CSI_CAPABILITY_OFFSET 0x08
+#define IOAT3_CSI_CAPABILITY_PREFETCH    0x1
+
+#define IOAT3_PCI_CAPABILITY_OFFSET 0x0A
+#define IOAT3_PCI_CAPABILITY_MEMWR  0x1
+
+#define IOAT3_CSI_CONTROL_OFFSET    0x0C
+#define IOAT3_CSI_CONTROL_PREFETCH  0x1
+
+#define IOAT3_PCI_CONTROL_OFFSET    0x0E
+#define IOAT3_PCI_CONTROL_MEMWR     0x1
+
+#define IOAT3_APICID_TAG_MAP_OFFSET 0x10
+#define IOAT3_APICID_TAG_MAP_OFFSET_LOW  0x10
+#define IOAT3_APICID_TAG_MAP_OFFSET_HIGH 0x14
+
+#define IOAT3_DCA_GREQID_OFFSET     0x02
+
+#define IOAT1_CHAINADDR_OFFSET		0x0C	/* 64-bit Descriptor Chain Address Register */
+#define IOAT2_CHAINADDR_OFFSET		0x10	/* 64-bit Descriptor Chain Address Register */
+#define IOAT_CHAINADDR_OFFSET(ver)		((ver) < IOAT_VER_2_0 \
+						? IOAT1_CHAINADDR_OFFSET : IOAT2_CHAINADDR_OFFSET)
+#define IOAT1_CHAINADDR_OFFSET_LOW	0x0C
+#define IOAT2_CHAINADDR_OFFSET_LOW	0x10
+#define IOAT_CHAINADDR_OFFSET_LOW(ver)		((ver) < IOAT_VER_2_0 \
+						? IOAT1_CHAINADDR_OFFSET_LOW : IOAT2_CHAINADDR_OFFSET_LOW)
+#define IOAT1_CHAINADDR_OFFSET_HIGH	0x10
+#define IOAT2_CHAINADDR_OFFSET_HIGH	0x14
+#define IOAT_CHAINADDR_OFFSET_HIGH(ver)		((ver) < IOAT_VER_2_0 \
+						? IOAT1_CHAINADDR_OFFSET_HIGH : IOAT2_CHAINADDR_OFFSET_HIGH)
+
+#define IOAT1_CHANCMD_OFFSET		0x14	/*  8-bit DMA Channel Command Register */
+#define IOAT2_CHANCMD_OFFSET		0x04	/*  8-bit DMA Channel Command Register */
+#define IOAT_CHANCMD_OFFSET(ver)		((ver) < IOAT_VER_2_0 \
+						? IOAT1_CHANCMD_OFFSET : IOAT2_CHANCMD_OFFSET)
+#define IOAT_CHANCMD_RESET			0x20
+#define IOAT_CHANCMD_RESUME			0x10
+#define IOAT_CHANCMD_ABORT			0x08
+#define IOAT_CHANCMD_SUSPEND			0x04
+#define IOAT_CHANCMD_APPEND			0x02
+#define IOAT_CHANCMD_START			0x01
+
+#define IOAT_CHANCMP_OFFSET			0x18	/* 64-bit Channel Completion Address Register */
+#define IOAT_CHANCMP_OFFSET_LOW			0x18
+#define IOAT_CHANCMP_OFFSET_HIGH		0x1C
+
+#define IOAT_CDAR_OFFSET			0x20	/* 64-bit Current Descriptor Address Register */
+#define IOAT_CDAR_OFFSET_LOW			0x20
+#define IOAT_CDAR_OFFSET_HIGH			0x24
+
+#define IOAT_CHANERR_OFFSET			0x28	/* 32-bit Channel Error Register */
+#define IOAT_CHANERR_SRC_ADDR_ERR	0x0001
+#define IOAT_CHANERR_DEST_ADDR_ERR	0x0002
+#define IOAT_CHANERR_NEXT_ADDR_ERR	0x0004
+#define IOAT_CHANERR_NEXT_DESC_ALIGN_ERR	0x0008
+#define IOAT_CHANERR_CHAIN_ADDR_VALUE_ERR	0x0010
+#define IOAT_CHANERR_CHANCMD_ERR		0x0020
+#define IOAT_CHANERR_CHIPSET_UNCORRECTABLE_DATA_INTEGRITY_ERR	0x0040
+#define IOAT_CHANERR_DMA_UNCORRECTABLE_DATA_INTEGRITY_ERR	0x0080
+#define IOAT_CHANERR_READ_DATA_ERR		0x0100
+#define IOAT_CHANERR_WRITE_DATA_ERR		0x0200
+#define IOAT_CHANERR_CONTROL_ERR	0x0400
+#define IOAT_CHANERR_LENGTH_ERR	0x0800
+#define IOAT_CHANERR_COMPLETION_ADDR_ERR	0x1000
+#define IOAT_CHANERR_INT_CONFIGURATION_ERR	0x2000
+#define IOAT_CHANERR_SOFT_ERR			0x4000
+#define IOAT_CHANERR_UNAFFILIATED_ERR		0x8000
+#define IOAT_CHANERR_XOR_P_OR_CRC_ERR		0x10000
+#define IOAT_CHANERR_XOR_Q_ERR			0x20000
+#define IOAT_CHANERR_DESCRIPTOR_COUNT_ERR	0x40000
+
+#define IOAT_CHANERR_HANDLE_MASK (IOAT_CHANERR_XOR_P_OR_CRC_ERR | IOAT_CHANERR_XOR_Q_ERR)
+#define IOAT_CHANERR_RECOVER_MASK (IOAT_CHANERR_READ_DATA_ERR | \
+				   IOAT_CHANERR_WRITE_DATA_ERR)
+
+#define IOAT_CHANERR_MASK_OFFSET		0x2C	/* 32-bit Channel Error Register */
+
+#define IOAT_CHAN_DRSCTL_OFFSET			0xB6
+#define IOAT_CHAN_DRSZ_4KB			0x0000
+#define IOAT_CHAN_DRSZ_8KB			0x0001
+#define IOAT_CHAN_DRSZ_2MB			0x0009
+#define IOAT_CHAN_DRS_EN			0x0100
+#define IOAT_CHAN_DRS_AUTOWRAP			0x0200
+
+#define IOAT_CHAN_LTR_SWSEL_OFFSET		0xBC
+#define IOAT_CHAN_LTR_SWSEL_ACTIVE		0x0
+#define IOAT_CHAN_LTR_SWSEL_IDLE		0x1
+
+#define IOAT_CHAN_LTR_ACTIVE_OFFSET		0xC0
+#define IOAT_CHAN_LTR_ACTIVE_SNVAL		0x0000	/* 0 us */
+#define IOAT_CHAN_LTR_ACTIVE_SNLATSCALE		0x0800	/* 1us scale */
+#define IOAT_CHAN_LTR_ACTIVE_SNREQMNT		0x8000	/* snoop req enable */
+
+#define IOAT_CHAN_LTR_IDLE_OFFSET		0xC4
+#define IOAT_CHAN_LTR_IDLE_SNVAL		0x0258	/* 600 us */
+#define IOAT_CHAN_LTR_IDLE_SNLATSCALE		0x0800	/* 1us scale */
+#define IOAT_CHAN_LTR_IDLE_SNREQMNT		0x8000	/* snoop req enable */
+
+#endif /* _IOAT_REGISTERS_H_ */
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 14/33] tools headers: Import drivers/dma/idxd/registers.h

[David Matlack]

Import drivers/dma/idxd/registers.h into tools/include/ so that it can
be used in VFIO selftests to interact with Intel DSA devices.

Changes made when importing:
 - Drop macro definitions that access struct idxd_device (e.g.
   WQCFG_OFFSET()).

Cc: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: David Matlack <dmatlack@google.com>
---
 tools/include/drivers/dma/idxd/registers.h | 601 +++++++++++++++++++++
 1 file changed, 601 insertions(+)
 create mode 100644 tools/include/drivers/dma/idxd/registers.h

diff --git a/tools/include/drivers/dma/idxd/registers.h b/tools/include/drivers/dma/idxd/registers.h
new file mode 100644
index 000000000000..7f4699e6a5c3
--- /dev/null
+++ b/tools/include/drivers/dma/idxd/registers.h
@@ -0,0 +1,601 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/* Copyright(c) 2019 Intel Corporation. All rights rsvd. */
+#ifndef _IDXD_REGISTERS_H_
+#define _IDXD_REGISTERS_H_
+
+#include <linux/compiler.h>
+#include <linux/types.h>
+#include <linux/idxd.h>
+
+/* PCI Config */
+#define PCI_DEVICE_ID_INTEL_DSA_SPR0	0x0b25
+#define PCI_DEVICE_ID_INTEL_IAX_SPR0	0x0cfe
+
+#define DEVICE_VERSION_1		0x100
+#define DEVICE_VERSION_2		0x200
+
+#define IDXD_MMIO_BAR		0
+#define IDXD_WQ_BAR		2
+#define IDXD_PORTAL_SIZE	PAGE_SIZE
+
+/* MMIO Device BAR0 Registers */
+#define IDXD_VER_OFFSET			0x00
+#define IDXD_VER_MAJOR_MASK		0xf0
+#define IDXD_VER_MINOR_MASK		0x0f
+#define GET_IDXD_VER_MAJOR(x)		(((x) & IDXD_VER_MAJOR_MASK) >> 4)
+#define GET_IDXD_VER_MINOR(x)		((x) & IDXD_VER_MINOR_MASK)
+
+union gen_cap_reg {
+	struct {
+		u64 block_on_fault:1;
+		u64 overlap_copy:1;
+		u64 cache_control_mem:1;
+		u64 cache_control_cache:1;
+		u64 cmd_cap:1;
+		u64 rsvd:3;
+		u64 dest_readback:1;
+		u64 drain_readback:1;
+		u64 rsvd2:3;
+		u64 evl_support:2;
+		u64 batch_continuation:1;
+		u64 max_xfer_shift:5;
+		u64 max_batch_shift:4;
+		u64 max_ims_mult:6;
+		u64 config_en:1;
+		u64 rsvd3:32;
+	};
+	u64 bits;
+} __packed;
+#define IDXD_GENCAP_OFFSET		0x10
+
+union wq_cap_reg {
+	struct {
+		u64 total_wq_size:16;
+		u64 num_wqs:8;
+		u64 wqcfg_size:4;
+		u64 rsvd:20;
+		u64 shared_mode:1;
+		u64 dedicated_mode:1;
+		u64 wq_ats_support:1;
+		u64 priority:1;
+		u64 occupancy:1;
+		u64 occupancy_int:1;
+		u64 op_config:1;
+		u64 wq_prs_support:1;
+		u64 rsvd4:8;
+	};
+	u64 bits;
+} __packed;
+#define IDXD_WQCAP_OFFSET		0x20
+#define IDXD_WQCFG_MIN			5
+
+union group_cap_reg {
+	struct {
+		u64 num_groups:8;
+		u64 total_rdbufs:8;	/* formerly total_tokens */
+		u64 rdbuf_ctrl:1;	/* formerly token_en */
+		u64 rdbuf_limit:1;	/* formerly token_limit */
+		u64 progress_limit:1;	/* descriptor and batch descriptor */
+		u64 rsvd:45;
+	};
+	u64 bits;
+} __packed;
+#define IDXD_GRPCAP_OFFSET		0x30
+
+union engine_cap_reg {
+	struct {
+		u64 num_engines:8;
+		u64 rsvd:56;
+	};
+	u64 bits;
+} __packed;
+
+#define IDXD_ENGCAP_OFFSET		0x38
+
+#define IDXD_OPCAP_NOOP			0x0001
+#define IDXD_OPCAP_BATCH			0x0002
+#define IDXD_OPCAP_MEMMOVE		0x0008
+struct opcap {
+	u64 bits[4];
+};
+
+#define IDXD_MAX_OPCAP_BITS		256U
+
+#define IDXD_OPCAP_OFFSET		0x40
+
+#define IDXD_TABLE_OFFSET		0x60
+union offsets_reg {
+	struct {
+		u64 grpcfg:16;
+		u64 wqcfg:16;
+		u64 msix_perm:16;
+		u64 ims:16;
+		u64 perfmon:16;
+		u64 rsvd:48;
+	};
+	u64 bits[2];
+} __packed;
+
+#define IDXD_TABLE_MULT			0x100
+
+#define IDXD_GENCFG_OFFSET		0x80
+union gencfg_reg {
+	struct {
+		u32 rdbuf_limit:8;
+		u32 rsvd:4;
+		u32 user_int_en:1;
+		u32 evl_en:1;
+		u32 rsvd2:18;
+	};
+	u32 bits;
+} __packed;
+
+#define IDXD_GENCTRL_OFFSET		0x88
+union genctrl_reg {
+	struct {
+		u32 softerr_int_en:1;
+		u32 halt_int_en:1;
+		u32 evl_int_en:1;
+		u32 rsvd:29;
+	};
+	u32 bits;
+} __packed;
+
+#define IDXD_GENSTATS_OFFSET		0x90
+union gensts_reg {
+	struct {
+		u32 state:2;
+		u32 reset_type:2;
+		u32 rsvd:28;
+	};
+	u32 bits;
+} __packed;
+
+enum idxd_device_status_state {
+	IDXD_DEVICE_STATE_DISABLED = 0,
+	IDXD_DEVICE_STATE_ENABLED,
+	IDXD_DEVICE_STATE_DRAIN,
+	IDXD_DEVICE_STATE_HALT,
+};
+
+enum idxd_device_reset_type {
+	IDXD_DEVICE_RESET_SOFTWARE = 0,
+	IDXD_DEVICE_RESET_FLR,
+	IDXD_DEVICE_RESET_WARM,
+	IDXD_DEVICE_RESET_COLD,
+};
+
+#define IDXD_INTCAUSE_OFFSET		0x98
+#define IDXD_INTC_ERR			0x01
+#define IDXD_INTC_CMD			0x02
+#define IDXD_INTC_OCCUPY			0x04
+#define IDXD_INTC_PERFMON_OVFL		0x08
+#define IDXD_INTC_HALT_STATE		0x10
+#define IDXD_INTC_EVL			0x20
+#define IDXD_INTC_INT_HANDLE_REVOKED	0x80000000
+
+#define IDXD_CMD_OFFSET			0xa0
+union idxd_command_reg {
+	struct {
+		u32 operand:20;
+		u32 cmd:5;
+		u32 rsvd:6;
+		u32 int_req:1;
+	};
+	u32 bits;
+} __packed;
+
+enum idxd_cmd {
+	IDXD_CMD_ENABLE_DEVICE = 1,
+	IDXD_CMD_DISABLE_DEVICE,
+	IDXD_CMD_DRAIN_ALL,
+	IDXD_CMD_ABORT_ALL,
+	IDXD_CMD_RESET_DEVICE,
+	IDXD_CMD_ENABLE_WQ,
+	IDXD_CMD_DISABLE_WQ,
+	IDXD_CMD_DRAIN_WQ,
+	IDXD_CMD_ABORT_WQ,
+	IDXD_CMD_RESET_WQ,
+	IDXD_CMD_DRAIN_PASID,
+	IDXD_CMD_ABORT_PASID,
+	IDXD_CMD_REQUEST_INT_HANDLE,
+	IDXD_CMD_RELEASE_INT_HANDLE,
+};
+
+#define CMD_INT_HANDLE_IMS		0x10000
+
+#define IDXD_CMDSTS_OFFSET		0xa8
+union cmdsts_reg {
+	struct {
+		u8 err;
+		u16 result;
+		u8 rsvd:7;
+		u8 active:1;
+	};
+	u32 bits;
+} __packed;
+#define IDXD_CMDSTS_ACTIVE		0x80000000
+#define IDXD_CMDSTS_ERR_MASK		0xff
+#define IDXD_CMDSTS_RES_SHIFT		8
+
+enum idxd_cmdsts_err {
+	IDXD_CMDSTS_SUCCESS = 0,
+	IDXD_CMDSTS_INVAL_CMD,
+	IDXD_CMDSTS_INVAL_WQIDX,
+	IDXD_CMDSTS_HW_ERR,
+	/* enable device errors */
+	IDXD_CMDSTS_ERR_DEV_ENABLED = 0x10,
+	IDXD_CMDSTS_ERR_CONFIG,
+	IDXD_CMDSTS_ERR_BUSMASTER_EN,
+	IDXD_CMDSTS_ERR_PASID_INVAL,
+	IDXD_CMDSTS_ERR_WQ_SIZE_ERANGE,
+	IDXD_CMDSTS_ERR_GRP_CONFIG,
+	IDXD_CMDSTS_ERR_GRP_CONFIG2,
+	IDXD_CMDSTS_ERR_GRP_CONFIG3,
+	IDXD_CMDSTS_ERR_GRP_CONFIG4,
+	/* enable wq errors */
+	IDXD_CMDSTS_ERR_DEV_NOTEN = 0x20,
+	IDXD_CMDSTS_ERR_WQ_ENABLED,
+	IDXD_CMDSTS_ERR_WQ_SIZE,
+	IDXD_CMDSTS_ERR_WQ_PRIOR,
+	IDXD_CMDSTS_ERR_WQ_MODE,
+	IDXD_CMDSTS_ERR_BOF_EN,
+	IDXD_CMDSTS_ERR_PASID_EN,
+	IDXD_CMDSTS_ERR_MAX_BATCH_SIZE,
+	IDXD_CMDSTS_ERR_MAX_XFER_SIZE,
+	/* disable device errors */
+	IDXD_CMDSTS_ERR_DIS_DEV_EN = 0x31,
+	/* disable WQ, drain WQ, abort WQ, reset WQ */
+	IDXD_CMDSTS_ERR_DEV_NOT_EN,
+	/* request interrupt handle */
+	IDXD_CMDSTS_ERR_INVAL_INT_IDX = 0x41,
+	IDXD_CMDSTS_ERR_NO_HANDLE,
+};
+
+#define IDXD_CMDCAP_OFFSET		0xb0
+
+#define IDXD_SWERR_OFFSET		0xc0
+#define IDXD_SWERR_VALID		0x00000001
+#define IDXD_SWERR_OVERFLOW		0x00000002
+#define IDXD_SWERR_ACK			(IDXD_SWERR_VALID | IDXD_SWERR_OVERFLOW)
+union sw_err_reg {
+	struct {
+		u64 valid:1;
+		u64 overflow:1;
+		u64 desc_valid:1;
+		u64 wq_idx_valid:1;
+		u64 batch:1;
+		u64 fault_rw:1;
+		u64 priv:1;
+		u64 rsvd:1;
+		u64 error:8;
+		u64 wq_idx:8;
+		u64 rsvd2:8;
+		u64 operation:8;
+		u64 pasid:20;
+		u64 rsvd3:4;
+
+		u64 batch_idx:16;
+		u64 rsvd4:16;
+		u64 invalid_flags:32;
+
+		u64 fault_addr;
+
+		u64 rsvd5;
+	};
+	u64 bits[4];
+} __packed;
+
+union iaa_cap_reg {
+	struct {
+		u64 dec_aecs_format_ver:1;
+		u64 drop_init_bits:1;
+		u64 chaining:1;
+		u64 force_array_output_mod:1;
+		u64 load_part_aecs:1;
+		u64 comp_early_abort:1;
+		u64 nested_comp:1;
+		u64 diction_comp:1;
+		u64 header_gen:1;
+		u64 crypto_gcm:1;
+		u64 crypto_cfb:1;
+		u64 crypto_xts:1;
+		u64 rsvd:52;
+	};
+	u64 bits;
+} __packed;
+
+#define IDXD_IAACAP_OFFSET	0x180
+
+#define IDXD_EVLCFG_OFFSET	0xe0
+union evlcfg_reg {
+	struct {
+		u64 pasid_en:1;
+		u64 priv:1;
+		u64 rsvd:10;
+		u64 base_addr:52;
+
+		u64 size:16;
+		u64 pasid:20;
+		u64 rsvd2:28;
+	};
+	u64 bits[2];
+} __packed;
+
+#define IDXD_EVL_SIZE_MIN	0x0040
+#define IDXD_EVL_SIZE_MAX	0xffff
+
+union msix_perm {
+	struct {
+		u32 rsvd:2;
+		u32 ignore:1;
+		u32 pasid_en:1;
+		u32 rsvd2:8;
+		u32 pasid:20;
+	};
+	u32 bits;
+} __packed;
+
+union group_flags {
+	struct {
+		u64 tc_a:3;
+		u64 tc_b:3;
+		u64 rsvd:1;
+		u64 use_rdbuf_limit:1;
+		u64 rdbufs_reserved:8;
+		u64 rsvd2:4;
+		u64 rdbufs_allowed:8;
+		u64 rsvd3:4;
+		u64 desc_progress_limit:2;
+		u64 rsvd4:2;
+		u64 batch_progress_limit:2;
+		u64 rsvd5:26;
+	};
+	u64 bits;
+} __packed;
+
+struct grpcfg {
+	u64 wqs[4];
+	u64 engines;
+	union group_flags flags;
+} __packed;
+
+union wqcfg {
+	struct {
+		/* bytes 0-3 */
+		u16 wq_size;
+		u16 rsvd;
+
+		/* bytes 4-7 */
+		u16 wq_thresh;
+		u16 rsvd1;
+
+		/* bytes 8-11 */
+		u32 mode:1;	/* shared or dedicated */
+		u32 bof:1;	/* block on fault */
+		u32 wq_ats_disable:1;
+		u32 wq_prs_disable:1;
+		u32 priority:4;
+		u32 pasid:20;
+		u32 pasid_en:1;
+		u32 priv:1;
+		u32 rsvd3:2;
+
+		/* bytes 12-15 */
+		u32 max_xfer_shift:5;
+		u32 max_batch_shift:4;
+		u32 rsvd4:23;
+
+		/* bytes 16-19 */
+		u16 occupancy_inth;
+		u16 occupancy_table_sel:1;
+		u16 rsvd5:15;
+
+		/* bytes 20-23 */
+		u16 occupancy_limit;
+		u16 occupancy_int_en:1;
+		u16 rsvd6:15;
+
+		/* bytes 24-27 */
+		u16 occupancy;
+		u16 occupancy_int:1;
+		u16 rsvd7:12;
+		u16 mode_support:1;
+		u16 wq_state:2;
+
+		/* bytes 28-31 */
+		u32 rsvd8;
+
+		/* bytes 32-63 */
+		u64 op_config[4];
+	};
+	u32 bits[16];
+} __packed;
+
+#define WQCFG_PASID_IDX                2
+#define WQCFG_PRIVL_IDX		2
+#define WQCFG_OCCUP_IDX		6
+
+#define WQCFG_OCCUP_MASK	0xffff
+
+#define GRPCFG_SIZE		64
+#define GRPWQCFG_STRIDES	4
+
+/* Following is performance monitor registers */
+#define IDXD_PERFCAP_OFFSET		0x0
+union idxd_perfcap {
+	struct {
+		u64 num_perf_counter:6;
+		u64 rsvd1:2;
+		u64 counter_width:8;
+		u64 num_event_category:4;
+		u64 global_event_category:16;
+		u64 filter:8;
+		u64 rsvd2:8;
+		u64 cap_per_counter:1;
+		u64 writeable_counter:1;
+		u64 counter_freeze:1;
+		u64 overflow_interrupt:1;
+		u64 rsvd3:8;
+	};
+	u64 bits;
+} __packed;
+
+#define IDXD_EVNTCAP_OFFSET		0x80
+union idxd_evntcap {
+	struct {
+		u64 events:28;
+		u64 rsvd:36;
+	};
+	u64 bits;
+} __packed;
+
+struct idxd_event {
+	union {
+		struct {
+			u32 event_category:4;
+			u32 events:28;
+		};
+		u32 val;
+	};
+} __packed;
+
+#define IDXD_CNTRCAP_OFFSET		0x800
+struct idxd_cntrcap {
+	union {
+		struct {
+			u32 counter_width:8;
+			u32 rsvd:20;
+			u32 num_events:4;
+		};
+		u32 val;
+	};
+	struct idxd_event events[];
+} __packed;
+
+#define IDXD_PERFRST_OFFSET		0x10
+union idxd_perfrst {
+	struct {
+		u32 perfrst_config:1;
+		u32 perfrst_counter:1;
+		u32 rsvd:30;
+	};
+	u32 val;
+} __packed;
+
+#define IDXD_OVFSTATUS_OFFSET		0x30
+#define IDXD_PERFFRZ_OFFSET		0x20
+#define IDXD_CNTRCFG_OFFSET		0x100
+union idxd_cntrcfg {
+	struct {
+		u64 enable:1;
+		u64 interrupt_ovf:1;
+		u64 global_freeze_ovf:1;
+		u64 rsvd1:5;
+		u64 event_category:4;
+		u64 rsvd2:20;
+		u64 events:28;
+		u64 rsvd3:4;
+	};
+	u64 val;
+} __packed;
+
+#define IDXD_FLTCFG_OFFSET		0x300
+
+#define IDXD_CNTRDATA_OFFSET		0x200
+union idxd_cntrdata {
+	struct {
+		u64 event_count_value;
+	};
+	u64 val;
+} __packed;
+
+union event_cfg {
+	struct {
+		u64 event_cat:4;
+		u64 event_enc:28;
+	};
+	u64 val;
+} __packed;
+
+union filter_cfg {
+	struct {
+		u64 wq:32;
+		u64 tc:8;
+		u64 pg_sz:4;
+		u64 xfer_sz:8;
+		u64 eng:8;
+	};
+	u64 val;
+} __packed;
+
+#define IDXD_EVLSTATUS_OFFSET		0xf0
+
+union evl_status_reg {
+	struct {
+		u32 head:16;
+		u32 rsvd:16;
+		u32 tail:16;
+		u32 rsvd2:14;
+		u32 int_pending:1;
+		u32 rsvd3:1;
+	};
+	struct {
+		u32 bits_lower32;
+		u32 bits_upper32;
+	};
+	u64 bits;
+} __packed;
+
+#define IDXD_MAX_BATCH_IDENT	256
+
+struct __evl_entry {
+	u64 rsvd:2;
+	u64 desc_valid:1;
+	u64 wq_idx_valid:1;
+	u64 batch:1;
+	u64 fault_rw:1;
+	u64 priv:1;
+	u64 err_info_valid:1;
+	u64 error:8;
+	u64 wq_idx:8;
+	u64 batch_id:8;
+	u64 operation:8;
+	u64 pasid:20;
+	u64 rsvd2:4;
+
+	u16 batch_idx;
+	u16 rsvd3;
+	union {
+		/* Invalid Flags 0x11 */
+		u32 invalid_flags;
+		/* Invalid Int Handle 0x19 */
+		/* Page fault 0x1a */
+		/* Page fault 0x06, 0x1f, only operand_id */
+		/* Page fault before drain or in batch, 0x26, 0x27 */
+		struct {
+			u16 int_handle;
+			u16 rci:1;
+			u16 ims:1;
+			u16 rcr:1;
+			u16 first_err_in_batch:1;
+			u16 rsvd4_2:9;
+			u16 operand_id:3;
+		};
+	};
+	u64 fault_addr;
+	u64 rsvd5;
+} __packed;
+
+struct dsa_evl_entry {
+	struct __evl_entry e;
+	struct dsa_completion_record cr;
+} __packed;
+
+struct iax_evl_entry {
+	struct __evl_entry e;
+	u64 rsvd[4];
+	struct iax_completion_record cr;
+} __packed;
+
+#endif
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 15/33] tools headers: Import linux/pci_ids.h

[David Matlack]

Import include/linux/pci_ids.h into tools/include/. This will be used in
subsequent commits to interact with PCI devices from VFIO selftests.

Changes made when importing: None.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 tools/include/linux/pci_ids.h | 3212 +++++++++++++++++++++++++++++++++
 1 file changed, 3212 insertions(+)
 create mode 100644 tools/include/linux/pci_ids.h

diff --git a/tools/include/linux/pci_ids.h b/tools/include/linux/pci_ids.h
new file mode 100644
index 000000000000..5fb3d4c393a9
--- /dev/null
+++ b/tools/include/linux/pci_ids.h
@@ -0,0 +1,3212 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+/*
+ *	PCI Class, Vendor and Device IDs
+ *
+ *	Please keep sorted by numeric Vendor ID and Device ID.
+ *
+ *	Do not add new entries to this file unless the definitions
+ *	are shared between multiple drivers.
+ */
+#ifndef _LINUX_PCI_IDS_H
+#define _LINUX_PCI_IDS_H
+
+/* Device classes and subclasses */
+
+#define PCI_CLASS_NOT_DEFINED		0x0000
+#define PCI_CLASS_NOT_DEFINED_VGA	0x0001
+
+#define PCI_BASE_CLASS_STORAGE		0x01
+#define PCI_CLASS_STORAGE_SCSI		0x0100
+#define PCI_CLASS_STORAGE_IDE		0x0101
+#define PCI_CLASS_STORAGE_FLOPPY	0x0102
+#define PCI_CLASS_STORAGE_IPI		0x0103
+#define PCI_CLASS_STORAGE_RAID		0x0104
+#define PCI_CLASS_STORAGE_SATA		0x0106
+#define PCI_CLASS_STORAGE_SATA_AHCI	0x010601
+#define PCI_CLASS_STORAGE_SAS		0x0107
+#define PCI_CLASS_STORAGE_EXPRESS	0x010802
+#define PCI_CLASS_STORAGE_OTHER		0x0180
+
+
+#define PCI_BASE_CLASS_NETWORK		0x02
+#define PCI_CLASS_NETWORK_ETHERNET	0x0200
+#define PCI_CLASS_NETWORK_TOKEN_RING	0x0201
+#define PCI_CLASS_NETWORK_FDDI		0x0202
+#define PCI_CLASS_NETWORK_ATM		0x0203
+#define PCI_CLASS_NETWORK_OTHER		0x0280
+
+#define PCI_BASE_CLASS_DISPLAY		0x03
+#define PCI_CLASS_DISPLAY_VGA		0x0300
+#define PCI_CLASS_DISPLAY_XGA		0x0301
+#define PCI_CLASS_DISPLAY_3D		0x0302
+#define PCI_CLASS_DISPLAY_OTHER		0x0380
+
+#define PCI_BASE_CLASS_MULTIMEDIA	0x04
+#define PCI_CLASS_MULTIMEDIA_VIDEO	0x0400
+#define PCI_CLASS_MULTIMEDIA_AUDIO	0x0401
+#define PCI_CLASS_MULTIMEDIA_PHONE	0x0402
+#define PCI_CLASS_MULTIMEDIA_HD_AUDIO	0x0403
+#define PCI_CLASS_MULTIMEDIA_OTHER	0x0480
+
+#define PCI_BASE_CLASS_MEMORY		0x05
+#define PCI_CLASS_MEMORY_RAM		0x0500
+#define PCI_CLASS_MEMORY_FLASH		0x0501
+#define PCI_CLASS_MEMORY_CXL		0x0502
+#define PCI_CLASS_MEMORY_OTHER		0x0580
+
+#define PCI_BASE_CLASS_BRIDGE		0x06
+#define PCI_CLASS_BRIDGE_HOST		0x0600
+#define PCI_CLASS_BRIDGE_ISA		0x0601
+#define PCI_CLASS_BRIDGE_EISA		0x0602
+#define PCI_CLASS_BRIDGE_MC		0x0603
+#define PCI_CLASS_BRIDGE_PCI		0x0604
+#define PCI_CLASS_BRIDGE_PCI_NORMAL		0x060400
+#define PCI_CLASS_BRIDGE_PCI_SUBTRACTIVE	0x060401
+#define PCI_CLASS_BRIDGE_PCMCIA		0x0605
+#define PCI_CLASS_BRIDGE_NUBUS		0x0606
+#define PCI_CLASS_BRIDGE_CARDBUS	0x0607
+#define PCI_CLASS_BRIDGE_RACEWAY	0x0608
+#define PCI_CLASS_BRIDGE_OTHER		0x0680
+
+#define PCI_BASE_CLASS_COMMUNICATION	0x07
+#define PCI_CLASS_COMMUNICATION_SERIAL	0x0700
+#define PCI_CLASS_COMMUNICATION_PARALLEL 0x0701
+#define PCI_CLASS_COMMUNICATION_MULTISERIAL 0x0702
+#define PCI_CLASS_COMMUNICATION_MODEM	0x0703
+#define PCI_CLASS_COMMUNICATION_OTHER	0x0780
+
+/* Interface for SERIAL/MODEM */
+#define PCI_SERIAL_16550_COMPATIBLE	0x02
+
+#define PCI_BASE_CLASS_SYSTEM		0x08
+#define PCI_CLASS_SYSTEM_PIC		0x0800
+#define PCI_CLASS_SYSTEM_PIC_IOAPIC	0x080010
+#define PCI_CLASS_SYSTEM_PIC_IOXAPIC	0x080020
+#define PCI_CLASS_SYSTEM_DMA		0x0801
+#define PCI_CLASS_SYSTEM_TIMER		0x0802
+#define PCI_CLASS_SYSTEM_RTC		0x0803
+#define PCI_CLASS_SYSTEM_PCI_HOTPLUG	0x0804
+#define PCI_CLASS_SYSTEM_SDHCI		0x0805
+#define PCI_CLASS_SYSTEM_RCEC		0x0807
+#define PCI_CLASS_SYSTEM_OTHER		0x0880
+
+#define PCI_BASE_CLASS_INPUT		0x09
+#define PCI_CLASS_INPUT_KEYBOARD	0x0900
+#define PCI_CLASS_INPUT_PEN		0x0901
+#define PCI_CLASS_INPUT_MOUSE		0x0902
+#define PCI_CLASS_INPUT_SCANNER		0x0903
+#define PCI_CLASS_INPUT_GAMEPORT	0x0904
+#define PCI_CLASS_INPUT_OTHER		0x0980
+
+#define PCI_BASE_CLASS_DOCKING		0x0a
+#define PCI_CLASS_DOCKING_GENERIC	0x0a00
+#define PCI_CLASS_DOCKING_OTHER		0x0a80
+
+#define PCI_BASE_CLASS_PROCESSOR	0x0b
+#define PCI_CLASS_PROCESSOR_386		0x0b00
+#define PCI_CLASS_PROCESSOR_486		0x0b01
+#define PCI_CLASS_PROCESSOR_PENTIUM	0x0b02
+#define PCI_CLASS_PROCESSOR_ALPHA	0x0b10
+#define PCI_CLASS_PROCESSOR_POWERPC	0x0b20
+#define PCI_CLASS_PROCESSOR_MIPS	0x0b30
+#define PCI_CLASS_PROCESSOR_CO		0x0b40
+
+#define PCI_BASE_CLASS_SERIAL		0x0c
+#define PCI_CLASS_SERIAL_FIREWIRE	0x0c00
+#define PCI_CLASS_SERIAL_FIREWIRE_OHCI	0x0c0010
+#define PCI_CLASS_SERIAL_ACCESS		0x0c01
+#define PCI_CLASS_SERIAL_SSA		0x0c02
+#define PCI_CLASS_SERIAL_USB		0x0c03
+#define PCI_CLASS_SERIAL_USB_UHCI	0x0c0300
+#define PCI_CLASS_SERIAL_USB_OHCI	0x0c0310
+#define PCI_CLASS_SERIAL_USB_EHCI	0x0c0320
+#define PCI_CLASS_SERIAL_USB_XHCI	0x0c0330
+#define PCI_CLASS_SERIAL_USB_DEVICE	0x0c03fe
+#define PCI_CLASS_SERIAL_FIBER		0x0c04
+#define PCI_CLASS_SERIAL_SMBUS		0x0c05
+#define PCI_CLASS_SERIAL_IPMI		0x0c07
+#define PCI_CLASS_SERIAL_IPMI_SMIC	0x0c0700
+#define PCI_CLASS_SERIAL_IPMI_KCS	0x0c0701
+#define PCI_CLASS_SERIAL_IPMI_BT	0x0c0702
+
+#define PCI_BASE_CLASS_WIRELESS			0x0d
+#define PCI_CLASS_WIRELESS_RF_CONTROLLER	0x0d10
+#define PCI_CLASS_WIRELESS_WHCI			0x0d1010
+
+#define PCI_BASE_CLASS_INTELLIGENT	0x0e
+#define PCI_CLASS_INTELLIGENT_I2O	0x0e00
+
+#define PCI_BASE_CLASS_SATELLITE	0x0f
+#define PCI_CLASS_SATELLITE_TV		0x0f00
+#define PCI_CLASS_SATELLITE_AUDIO	0x0f01
+#define PCI_CLASS_SATELLITE_VOICE	0x0f03
+#define PCI_CLASS_SATELLITE_DATA	0x0f04
+
+#define PCI_BASE_CLASS_CRYPT		0x10
+#define PCI_CLASS_CRYPT_NETWORK		0x1000
+#define PCI_CLASS_CRYPT_ENTERTAINMENT	0x1001
+#define PCI_CLASS_CRYPT_OTHER		0x1080
+
+#define PCI_BASE_CLASS_SIGNAL_PROCESSING 0x11
+#define PCI_CLASS_SP_DPIO		0x1100
+#define PCI_CLASS_SP_OTHER		0x1180
+
+#define PCI_BASE_CLASS_ACCELERATOR	0x12
+#define PCI_CLASS_ACCELERATOR_PROCESSING	0x1200
+
+#define PCI_CLASS_OTHERS		0xff
+
+/* Vendors and devices.  Sort key: vendor first, device next. */
+#define PCI_VENDOR_ID_PCI_SIG		0x0001
+
+#define PCI_VENDOR_ID_LOONGSON		0x0014
+
+#define PCI_DEVICE_ID_LOONGSON_HDA      0x7a07
+#define PCI_DEVICE_ID_LOONGSON_HDMI     0x7a37
+
+#define PCI_VENDOR_ID_SOLIDIGM		0x025e
+
+#define PCI_VENDOR_ID_TTTECH		0x0357
+#define PCI_DEVICE_ID_TTTECH_MC322	0x000a
+
+#define PCI_VENDOR_ID_DYNALINK		0x0675
+#define PCI_DEVICE_ID_DYNALINK_IS64PH	0x1702
+
+#define PCI_VENDOR_ID_UBIQUITI		0x0777
+
+#define PCI_VENDOR_ID_BERKOM			0x0871
+#define PCI_DEVICE_ID_BERKOM_A1T		0xffa1
+#define PCI_DEVICE_ID_BERKOM_T_CONCEPT		0xffa2
+#define PCI_DEVICE_ID_BERKOM_A4T		0xffa4
+#define PCI_DEVICE_ID_BERKOM_SCITEL_QUADRO	0xffa8
+
+#define PCI_VENDOR_ID_COMPAQ		0x0e11
+#define PCI_DEVICE_ID_COMPAQ_TOKENRING	0x0508
+#define PCI_DEVICE_ID_COMPAQ_TACHYON	0xa0fc
+#define PCI_DEVICE_ID_COMPAQ_SMART2P	0xae10
+#define PCI_DEVICE_ID_COMPAQ_NETEL100	0xae32
+#define PCI_DEVICE_ID_COMPAQ_NETEL10	0xae34
+#define PCI_DEVICE_ID_COMPAQ_TRIFLEX_IDE 0xae33
+#define PCI_DEVICE_ID_COMPAQ_NETFLEX3I	0xae35
+#define PCI_DEVICE_ID_COMPAQ_NETEL100D	0xae40
+#define PCI_DEVICE_ID_COMPAQ_NETEL100PI	0xae43
+#define PCI_DEVICE_ID_COMPAQ_NETEL100I	0xb011
+#define PCI_DEVICE_ID_COMPAQ_CISS	0xb060
+#define PCI_DEVICE_ID_COMPAQ_CISSB	0xb178
+#define PCI_DEVICE_ID_COMPAQ_CISSC	0x46
+#define PCI_DEVICE_ID_COMPAQ_THUNDER	0xf130
+#define PCI_DEVICE_ID_COMPAQ_NETFLEX3B	0xf150
+
+#define PCI_VENDOR_ID_NCR		0x1000
+#define PCI_VENDOR_ID_LSI_LOGIC		0x1000
+#define PCI_DEVICE_ID_NCR_53C810	0x0001
+#define PCI_DEVICE_ID_NCR_53C820	0x0002
+#define PCI_DEVICE_ID_NCR_53C825	0x0003
+#define PCI_DEVICE_ID_NCR_53C815	0x0004
+#define PCI_DEVICE_ID_LSI_53C810AP	0x0005
+#define PCI_DEVICE_ID_NCR_53C860	0x0006
+#define PCI_DEVICE_ID_LSI_53C1510	0x000a
+#define PCI_DEVICE_ID_NCR_53C896	0x000b
+#define PCI_DEVICE_ID_NCR_53C895	0x000c
+#define PCI_DEVICE_ID_NCR_53C885	0x000d
+#define PCI_DEVICE_ID_NCR_53C875	0x000f
+#define PCI_DEVICE_ID_NCR_53C1510	0x0010
+#define PCI_DEVICE_ID_LSI_53C895A	0x0012
+#define PCI_DEVICE_ID_LSI_53C875A	0x0013
+#define PCI_DEVICE_ID_LSI_53C1010_33	0x0020
+#define PCI_DEVICE_ID_LSI_53C1010_66	0x0021
+#define PCI_DEVICE_ID_LSI_53C1030	0x0030
+#define PCI_DEVICE_ID_LSI_1030_53C1035	0x0032
+#define PCI_DEVICE_ID_LSI_53C1035	0x0040
+#define PCI_DEVICE_ID_NCR_53C875J	0x008f
+#define PCI_DEVICE_ID_LSI_FC909		0x0621
+#define PCI_DEVICE_ID_LSI_FC929		0x0622
+#define PCI_DEVICE_ID_LSI_FC929_LAN	0x0623
+#define PCI_DEVICE_ID_LSI_FC919		0x0624
+#define PCI_DEVICE_ID_LSI_FC919_LAN	0x0625
+#define PCI_DEVICE_ID_LSI_FC929X	0x0626
+#define PCI_DEVICE_ID_LSI_FC939X	0x0642
+#define PCI_DEVICE_ID_LSI_FC949X	0x0640
+#define PCI_DEVICE_ID_LSI_FC949ES	0x0646
+#define PCI_DEVICE_ID_LSI_FC919X	0x0628
+#define PCI_DEVICE_ID_NCR_YELLOWFIN	0x0701
+#define PCI_DEVICE_ID_LSI_61C102	0x0901
+#define PCI_DEVICE_ID_LSI_63C815	0x1000
+#define PCI_DEVICE_ID_LSI_SAS1064	0x0050
+#define PCI_DEVICE_ID_LSI_SAS1064R	0x0411
+#define PCI_DEVICE_ID_LSI_SAS1066	0x005E
+#define PCI_DEVICE_ID_LSI_SAS1068	0x0054
+#define PCI_DEVICE_ID_LSI_SAS1064A	0x005C
+#define PCI_DEVICE_ID_LSI_SAS1064E	0x0056
+#define PCI_DEVICE_ID_LSI_SAS1066E	0x005A
+#define PCI_DEVICE_ID_LSI_SAS1068E	0x0058
+#define PCI_DEVICE_ID_LSI_SAS1078	0x0060
+
+#define PCI_VENDOR_ID_ATI		0x1002
+/* Mach64 */
+#define PCI_DEVICE_ID_ATI_68800		0x4158
+#define PCI_DEVICE_ID_ATI_215CT222	0x4354
+#define PCI_DEVICE_ID_ATI_210888CX	0x4358
+#define PCI_DEVICE_ID_ATI_215ET222	0x4554
+/* Mach64 / Rage */
+#define PCI_DEVICE_ID_ATI_215GB		0x4742
+#define PCI_DEVICE_ID_ATI_215GD		0x4744
+#define PCI_DEVICE_ID_ATI_215GI		0x4749
+#define PCI_DEVICE_ID_ATI_215GP		0x4750
+#define PCI_DEVICE_ID_ATI_215GQ		0x4751
+#define PCI_DEVICE_ID_ATI_215XL		0x4752
+#define PCI_DEVICE_ID_ATI_215GT		0x4754
+#define PCI_DEVICE_ID_ATI_215GTB	0x4755
+#define PCI_DEVICE_ID_ATI_215_IV	0x4756
+#define PCI_DEVICE_ID_ATI_215_IW	0x4757
+#define PCI_DEVICE_ID_ATI_215_IZ	0x475A
+#define PCI_DEVICE_ID_ATI_210888GX	0x4758
+#define PCI_DEVICE_ID_ATI_215_LB	0x4c42
+#define PCI_DEVICE_ID_ATI_215_LD	0x4c44
+#define PCI_DEVICE_ID_ATI_215_LG	0x4c47
+#define PCI_DEVICE_ID_ATI_215_LI	0x4c49
+#define PCI_DEVICE_ID_ATI_215_LM	0x4c4D
+#define PCI_DEVICE_ID_ATI_215_LN	0x4c4E
+#define PCI_DEVICE_ID_ATI_215_LR	0x4c52
+#define PCI_DEVICE_ID_ATI_215_LS	0x4c53
+#define PCI_DEVICE_ID_ATI_264_LT	0x4c54
+/* Mach64 VT */
+#define PCI_DEVICE_ID_ATI_264VT		0x5654
+#define PCI_DEVICE_ID_ATI_264VU		0x5655
+#define PCI_DEVICE_ID_ATI_264VV		0x5656
+/* Rage128 GL */
+#define PCI_DEVICE_ID_ATI_RAGE128_RE	0x5245
+#define PCI_DEVICE_ID_ATI_RAGE128_RF	0x5246
+#define PCI_DEVICE_ID_ATI_RAGE128_RG	0x5247
+/* Rage128 VR */
+#define PCI_DEVICE_ID_ATI_RAGE128_RK	0x524b
+#define PCI_DEVICE_ID_ATI_RAGE128_RL	0x524c
+#define PCI_DEVICE_ID_ATI_RAGE128_SE	0x5345
+#define PCI_DEVICE_ID_ATI_RAGE128_SF	0x5346
+#define PCI_DEVICE_ID_ATI_RAGE128_SG	0x5347
+#define PCI_DEVICE_ID_ATI_RAGE128_SH	0x5348
+#define PCI_DEVICE_ID_ATI_RAGE128_SK	0x534b
+#define PCI_DEVICE_ID_ATI_RAGE128_SL	0x534c
+#define PCI_DEVICE_ID_ATI_RAGE128_SM	0x534d
+#define PCI_DEVICE_ID_ATI_RAGE128_SN	0x534e
+/* Rage128 Ultra */
+#define PCI_DEVICE_ID_ATI_RAGE128_TF	0x5446
+#define PCI_DEVICE_ID_ATI_RAGE128_TL	0x544c
+#define PCI_DEVICE_ID_ATI_RAGE128_TR	0x5452
+#define PCI_DEVICE_ID_ATI_RAGE128_TS	0x5453
+#define PCI_DEVICE_ID_ATI_RAGE128_TT	0x5454
+#define PCI_DEVICE_ID_ATI_RAGE128_TU	0x5455
+/* Rage128 M3 */
+#define PCI_DEVICE_ID_ATI_RAGE128_LE	0x4c45
+#define PCI_DEVICE_ID_ATI_RAGE128_LF	0x4c46
+/* Rage128 M4 */
+#define PCI_DEVICE_ID_ATI_RAGE128_MF    0x4d46
+#define PCI_DEVICE_ID_ATI_RAGE128_ML    0x4d4c
+/* Rage128 Pro GL */
+#define PCI_DEVICE_ID_ATI_RAGE128_PA	0x5041
+#define PCI_DEVICE_ID_ATI_RAGE128_PB	0x5042
+#define PCI_DEVICE_ID_ATI_RAGE128_PC	0x5043
+#define PCI_DEVICE_ID_ATI_RAGE128_PD	0x5044
+#define PCI_DEVICE_ID_ATI_RAGE128_PE	0x5045
+#define PCI_DEVICE_ID_ATI_RAGE128_PF	0x5046
+/* Rage128 Pro VR */
+#define PCI_DEVICE_ID_ATI_RAGE128_PG	0x5047
+#define PCI_DEVICE_ID_ATI_RAGE128_PH	0x5048
+#define PCI_DEVICE_ID_ATI_RAGE128_PI	0x5049
+#define PCI_DEVICE_ID_ATI_RAGE128_PJ	0x504A
+#define PCI_DEVICE_ID_ATI_RAGE128_PK	0x504B
+#define PCI_DEVICE_ID_ATI_RAGE128_PL	0x504C
+#define PCI_DEVICE_ID_ATI_RAGE128_PM	0x504D
+#define PCI_DEVICE_ID_ATI_RAGE128_PN	0x504E
+#define PCI_DEVICE_ID_ATI_RAGE128_PO	0x504F
+#define PCI_DEVICE_ID_ATI_RAGE128_PP	0x5050
+#define PCI_DEVICE_ID_ATI_RAGE128_PQ	0x5051
+#define PCI_DEVICE_ID_ATI_RAGE128_PR	0x5052
+#define PCI_DEVICE_ID_ATI_RAGE128_PS	0x5053
+#define PCI_DEVICE_ID_ATI_RAGE128_PT	0x5054
+#define PCI_DEVICE_ID_ATI_RAGE128_PU	0x5055
+#define PCI_DEVICE_ID_ATI_RAGE128_PV	0x5056
+#define PCI_DEVICE_ID_ATI_RAGE128_PW	0x5057
+#define PCI_DEVICE_ID_ATI_RAGE128_PX	0x5058
+/* Rage128 M4 */
+/* Radeon R100 */
+#define PCI_DEVICE_ID_ATI_RADEON_QD	0x5144
+#define PCI_DEVICE_ID_ATI_RADEON_QE	0x5145
+#define PCI_DEVICE_ID_ATI_RADEON_QF	0x5146
+#define PCI_DEVICE_ID_ATI_RADEON_QG	0x5147
+/* Radeon RV100 (VE) */
+#define PCI_DEVICE_ID_ATI_RADEON_QY	0x5159
+#define PCI_DEVICE_ID_ATI_RADEON_QZ	0x515a
+/* Radeon R200 (8500) */
+#define PCI_DEVICE_ID_ATI_RADEON_QL	0x514c
+#define PCI_DEVICE_ID_ATI_RADEON_QN	0x514e
+#define PCI_DEVICE_ID_ATI_RADEON_QO	0x514f
+#define PCI_DEVICE_ID_ATI_RADEON_Ql	0x516c
+#define PCI_DEVICE_ID_ATI_RADEON_BB	0x4242
+/* Radeon R200 (9100) */
+#define PCI_DEVICE_ID_ATI_RADEON_QM	0x514d
+/* Radeon RV200 (7500) */
+#define PCI_DEVICE_ID_ATI_RADEON_QW	0x5157
+#define PCI_DEVICE_ID_ATI_RADEON_QX	0x5158
+/* Radeon NV-100 */
+/* Radeon RV250 (9000) */
+#define PCI_DEVICE_ID_ATI_RADEON_Id	0x4964
+#define PCI_DEVICE_ID_ATI_RADEON_Ie	0x4965
+#define PCI_DEVICE_ID_ATI_RADEON_If	0x4966
+#define PCI_DEVICE_ID_ATI_RADEON_Ig	0x4967
+/* Radeon RV280 (9200) */
+#define PCI_DEVICE_ID_ATI_RADEON_Ya	0x5961
+#define PCI_DEVICE_ID_ATI_RADEON_Yd	0x5964
+/* Radeon R300 (9500) */
+/* Radeon R300 (9700) */
+#define PCI_DEVICE_ID_ATI_RADEON_ND	0x4e44
+#define PCI_DEVICE_ID_ATI_RADEON_NE	0x4e45
+#define PCI_DEVICE_ID_ATI_RADEON_NF	0x4e46
+#define PCI_DEVICE_ID_ATI_RADEON_NG	0x4e47
+/* Radeon R350 (9800) */
+/* Radeon RV350 (9600) */
+/* Radeon M6 */
+#define PCI_DEVICE_ID_ATI_RADEON_LY	0x4c59
+#define PCI_DEVICE_ID_ATI_RADEON_LZ	0x4c5a
+/* Radeon M7 */
+#define PCI_DEVICE_ID_ATI_RADEON_LW	0x4c57
+#define PCI_DEVICE_ID_ATI_RADEON_LX	0x4c58
+/* Radeon M9 */
+#define PCI_DEVICE_ID_ATI_RADEON_Ld	0x4c64
+#define PCI_DEVICE_ID_ATI_RADEON_Le	0x4c65
+#define PCI_DEVICE_ID_ATI_RADEON_Lf	0x4c66
+#define PCI_DEVICE_ID_ATI_RADEON_Lg	0x4c67
+/* Radeon */
+/* RadeonIGP */
+#define PCI_DEVICE_ID_ATI_RS100		0xcab0
+#define PCI_DEVICE_ID_ATI_RS200		0xcab2
+#define PCI_DEVICE_ID_ATI_RS200_B	0xcbb2
+#define PCI_DEVICE_ID_ATI_RS250		0xcab3
+#define PCI_DEVICE_ID_ATI_RS300_100	0x5830
+#define PCI_DEVICE_ID_ATI_RS300_133	0x5831
+#define PCI_DEVICE_ID_ATI_RS300_166	0x5832
+#define PCI_DEVICE_ID_ATI_RS300_200	0x5833
+#define PCI_DEVICE_ID_ATI_RS350_100     0x7830
+#define PCI_DEVICE_ID_ATI_RS350_133     0x7831
+#define PCI_DEVICE_ID_ATI_RS350_166     0x7832
+#define PCI_DEVICE_ID_ATI_RS350_200     0x7833
+#define PCI_DEVICE_ID_ATI_RS400_100     0x5a30
+#define PCI_DEVICE_ID_ATI_RS400_133     0x5a31
+#define PCI_DEVICE_ID_ATI_RS400_166     0x5a32
+#define PCI_DEVICE_ID_ATI_RS400_200     0x5a33
+#define PCI_DEVICE_ID_ATI_RS480         0x5950
+/* ATI IXP Chipset */
+#define PCI_DEVICE_ID_ATI_IXP200_IDE	0x4349
+#define PCI_DEVICE_ID_ATI_IXP200_SMBUS	0x4353
+#define PCI_DEVICE_ID_ATI_IXP300_SMBUS	0x4363
+#define PCI_DEVICE_ID_ATI_IXP300_IDE	0x4369
+#define PCI_DEVICE_ID_ATI_IXP300_SATA   0x436e
+#define PCI_DEVICE_ID_ATI_IXP400_SMBUS	0x4372
+#define PCI_DEVICE_ID_ATI_IXP400_IDE	0x4376
+#define PCI_DEVICE_ID_ATI_IXP400_SATA   0x4379
+#define PCI_DEVICE_ID_ATI_IXP400_SATA2	0x437a
+#define PCI_DEVICE_ID_ATI_IXP600_SATA	0x4380
+#define PCI_DEVICE_ID_ATI_SBX00_SMBUS	0x4385
+#define PCI_DEVICE_ID_ATI_IXP600_IDE	0x438c
+#define PCI_DEVICE_ID_ATI_IXP700_SATA	0x4390
+#define PCI_DEVICE_ID_ATI_IXP700_IDE	0x439c
+
+#define PCI_VENDOR_ID_VLSI		0x1004
+#define PCI_DEVICE_ID_VLSI_82C592	0x0005
+#define PCI_DEVICE_ID_VLSI_82C593	0x0006
+#define PCI_DEVICE_ID_VLSI_82C594	0x0007
+#define PCI_DEVICE_ID_VLSI_82C597	0x0009
+#define PCI_DEVICE_ID_VLSI_82C541	0x000c
+#define PCI_DEVICE_ID_VLSI_82C543	0x000d
+#define PCI_DEVICE_ID_VLSI_82C532	0x0101
+#define PCI_DEVICE_ID_VLSI_82C534	0x0102
+#define PCI_DEVICE_ID_VLSI_82C535	0x0104
+#define PCI_DEVICE_ID_VLSI_82C147	0x0105
+#define PCI_DEVICE_ID_VLSI_VAS96011	0x0702
+
+/* AMD RD890 Chipset */
+#define PCI_DEVICE_ID_RD890_IOMMU	0x5a23
+
+#define PCI_VENDOR_ID_ADL		0x1005
+#define PCI_DEVICE_ID_ADL_2301		0x2301
+
+#define PCI_VENDOR_ID_NS		0x100b
+#define PCI_DEVICE_ID_NS_87415		0x0002
+#define PCI_DEVICE_ID_NS_87560_LIO	0x000e
+#define PCI_DEVICE_ID_NS_87560_USB	0x0012
+#define PCI_DEVICE_ID_NS_83815		0x0020
+#define PCI_DEVICE_ID_NS_83820		0x0022
+#define PCI_DEVICE_ID_NS_CS5535_ISA	0x002b
+#define PCI_DEVICE_ID_NS_CS5535_IDE	0x002d
+#define PCI_DEVICE_ID_NS_CS5535_AUDIO	0x002e
+#define PCI_DEVICE_ID_NS_CS5535_USB	0x002f
+#define PCI_DEVICE_ID_NS_GX_VIDEO	0x0030
+#define PCI_DEVICE_ID_NS_SATURN		0x0035
+#define PCI_DEVICE_ID_NS_SCx200_BRIDGE	0x0500
+#define PCI_DEVICE_ID_NS_SCx200_SMI	0x0501
+#define PCI_DEVICE_ID_NS_SCx200_IDE	0x0502
+#define PCI_DEVICE_ID_NS_SCx200_AUDIO	0x0503
+#define PCI_DEVICE_ID_NS_SCx200_VIDEO	0x0504
+#define PCI_DEVICE_ID_NS_SCx200_XBUS	0x0505
+#define PCI_DEVICE_ID_NS_SC1100_BRIDGE	0x0510
+#define PCI_DEVICE_ID_NS_SC1100_SMI	0x0511
+#define PCI_DEVICE_ID_NS_SC1100_XBUS	0x0515
+#define PCI_DEVICE_ID_NS_87410		0xd001
+
+#define PCI_DEVICE_ID_NS_GX_HOST_BRIDGE  0x0028
+
+#define PCI_VENDOR_ID_TSENG		0x100c
+#define PCI_DEVICE_ID_TSENG_W32P_2	0x3202
+#define PCI_DEVICE_ID_TSENG_W32P_b	0x3205
+#define PCI_DEVICE_ID_TSENG_W32P_c	0x3206
+#define PCI_DEVICE_ID_TSENG_W32P_d	0x3207
+#define PCI_DEVICE_ID_TSENG_ET6000	0x3208
+
+#define PCI_VENDOR_ID_WEITEK		0x100e
+#define PCI_DEVICE_ID_WEITEK_P9000	0x9001
+#define PCI_DEVICE_ID_WEITEK_P9100	0x9100
+
+#define PCI_VENDOR_ID_DEC		0x1011
+#define PCI_DEVICE_ID_DEC_BRD		0x0001
+#define PCI_DEVICE_ID_DEC_TULIP		0x0002
+#define PCI_DEVICE_ID_DEC_TGA		0x0004
+#define PCI_DEVICE_ID_DEC_TULIP_FAST	0x0009
+#define PCI_DEVICE_ID_DEC_TGA2		0x000D
+#define PCI_DEVICE_ID_DEC_FDDI		0x000F
+#define PCI_DEVICE_ID_DEC_TULIP_PLUS	0x0014
+#define PCI_DEVICE_ID_DEC_21142		0x0019
+#define PCI_DEVICE_ID_DEC_21052		0x0021
+#define PCI_DEVICE_ID_DEC_21150		0x0022
+#define PCI_DEVICE_ID_DEC_21152		0x0024
+#define PCI_DEVICE_ID_DEC_21153		0x0025
+#define PCI_DEVICE_ID_DEC_21154		0x0026
+#define PCI_DEVICE_ID_DEC_21285		0x1065
+#define PCI_DEVICE_ID_COMPAQ_42XX	0x0046
+
+#define PCI_VENDOR_ID_CIRRUS		0x1013
+#define PCI_DEVICE_ID_CIRRUS_7548	0x0038
+#define PCI_DEVICE_ID_CIRRUS_5430	0x00a0
+#define PCI_DEVICE_ID_CIRRUS_5434_4	0x00a4
+#define PCI_DEVICE_ID_CIRRUS_5434_8	0x00a8
+#define PCI_DEVICE_ID_CIRRUS_5436	0x00ac
+#define PCI_DEVICE_ID_CIRRUS_5446	0x00b8
+#define PCI_DEVICE_ID_CIRRUS_5480	0x00bc
+#define PCI_DEVICE_ID_CIRRUS_5462	0x00d0
+#define PCI_DEVICE_ID_CIRRUS_5464	0x00d4
+#define PCI_DEVICE_ID_CIRRUS_5465	0x00d6
+#define PCI_DEVICE_ID_CIRRUS_6729	0x1100
+#define PCI_DEVICE_ID_CIRRUS_6832	0x1110
+#define PCI_DEVICE_ID_CIRRUS_7543	0x1202
+#define PCI_DEVICE_ID_CIRRUS_4610	0x6001
+#define PCI_DEVICE_ID_CIRRUS_4612	0x6003
+#define PCI_DEVICE_ID_CIRRUS_4615	0x6004
+
+#define PCI_VENDOR_ID_IBM		0x1014
+#define PCI_DEVICE_ID_IBM_TR		0x0018
+#define PCI_DEVICE_ID_IBM_TR_WAKE	0x003e
+#define PCI_DEVICE_ID_IBM_CPC710_PCI64	0x00fc
+#define PCI_DEVICE_ID_IBM_SNIPE		0x0180
+#define PCI_DEVICE_ID_IBM_CITRINE		0x028C
+#define PCI_DEVICE_ID_IBM_GEMSTONE		0xB166
+#define PCI_DEVICE_ID_IBM_OBSIDIAN		0x02BD
+#define PCI_DEVICE_ID_IBM_ICOM_DEV_ID_1	0x0031
+#define PCI_DEVICE_ID_IBM_ICOM_DEV_ID_2	0x0219
+#define PCI_DEVICE_ID_IBM_ICOM_V2_TWO_PORTS_RVX		0x021A
+#define PCI_DEVICE_ID_IBM_ICOM_V2_ONE_PORT_RVX_ONE_PORT_MDM	0x0251
+#define PCI_DEVICE_ID_IBM_ICOM_V2_ONE_PORT_RVX_ONE_PORT_MDM_PCIE 0x0361
+#define PCI_DEVICE_ID_IBM_ICOM_FOUR_PORT_MODEL	0x252
+
+#define PCI_SUBVENDOR_ID_IBM		0x1014
+#define PCI_SUBDEVICE_ID_IBM_SATURN_SERIAL_ONE_PORT	0x03d4
+
+#define PCI_VENDOR_ID_UNISYS		0x1018
+#define PCI_DEVICE_ID_UNISYS_DMA_DIRECTOR 0x001C
+
+#define PCI_VENDOR_ID_COMPEX2		0x101a /* pci.ids says "AT&T GIS (NCR)" */
+#define PCI_DEVICE_ID_COMPEX2_100VG	0x0005
+
+#define PCI_VENDOR_ID_WD		0x101c
+#define PCI_DEVICE_ID_WD_90C		0xc24a
+
+#define PCI_VENDOR_ID_AMI		0x101e
+#define PCI_DEVICE_ID_AMI_MEGARAID3	0x1960
+#define PCI_DEVICE_ID_AMI_MEGARAID	0x9010
+#define PCI_DEVICE_ID_AMI_MEGARAID2	0x9060
+
+#define PCI_VENDOR_ID_AMD		0x1022
+#define PCI_DEVICE_ID_AMD_K8_NB		0x1100
+#define PCI_DEVICE_ID_AMD_K8_NB_ADDRMAP	0x1101
+#define PCI_DEVICE_ID_AMD_K8_NB_MEMCTL	0x1102
+#define PCI_DEVICE_ID_AMD_K8_NB_MISC	0x1103
+#define PCI_DEVICE_ID_AMD_10H_NB_HT	0x1200
+#define PCI_DEVICE_ID_AMD_10H_NB_MAP	0x1201
+#define PCI_DEVICE_ID_AMD_10H_NB_DRAM	0x1202
+#define PCI_DEVICE_ID_AMD_10H_NB_MISC	0x1203
+#define PCI_DEVICE_ID_AMD_10H_NB_LINK	0x1204
+#define PCI_DEVICE_ID_AMD_11H_NB_HT	0x1300
+#define PCI_DEVICE_ID_AMD_11H_NB_MAP	0x1301
+#define PCI_DEVICE_ID_AMD_11H_NB_DRAM	0x1302
+#define PCI_DEVICE_ID_AMD_11H_NB_MISC	0x1303
+#define PCI_DEVICE_ID_AMD_11H_NB_LINK	0x1304
+#define PCI_DEVICE_ID_AMD_15H_M10H_F3	0x1403
+#define PCI_DEVICE_ID_AMD_15H_M30H_NB_F3 0x141d
+#define PCI_DEVICE_ID_AMD_15H_M30H_NB_F4 0x141e
+#define PCI_DEVICE_ID_AMD_15H_M60H_NB_F3 0x1573
+#define PCI_DEVICE_ID_AMD_15H_M60H_NB_F4 0x1574
+#define PCI_DEVICE_ID_AMD_15H_NB_F0	0x1600
+#define PCI_DEVICE_ID_AMD_15H_NB_F1	0x1601
+#define PCI_DEVICE_ID_AMD_15H_NB_F2	0x1602
+#define PCI_DEVICE_ID_AMD_15H_NB_F3	0x1603
+#define PCI_DEVICE_ID_AMD_15H_NB_F4	0x1604
+#define PCI_DEVICE_ID_AMD_15H_NB_F5	0x1605
+#define PCI_DEVICE_ID_AMD_16H_NB_F3	0x1533
+#define PCI_DEVICE_ID_AMD_16H_NB_F4	0x1534
+#define PCI_DEVICE_ID_AMD_16H_M30H_NB_F3 0x1583
+#define PCI_DEVICE_ID_AMD_16H_M30H_NB_F4 0x1584
+#define PCI_DEVICE_ID_AMD_17H_DF_F3	0x1463
+#define PCI_DEVICE_ID_AMD_17H_M10H_DF_F3 0x15eb
+#define PCI_DEVICE_ID_AMD_17H_M30H_DF_F3 0x1493
+#define PCI_DEVICE_ID_AMD_17H_M60H_DF_F3 0x144b
+#define PCI_DEVICE_ID_AMD_17H_M70H_DF_F3 0x1443
+#define PCI_DEVICE_ID_AMD_17H_MA0H_DF_F3 0x1727
+#define PCI_DEVICE_ID_AMD_19H_DF_F3	0x1653
+#define PCI_DEVICE_ID_AMD_19H_M10H_DF_F3 0x14b0
+#define PCI_DEVICE_ID_AMD_19H_M40H_DF_F3 0x167c
+#define PCI_DEVICE_ID_AMD_19H_M50H_DF_F3 0x166d
+#define PCI_DEVICE_ID_AMD_19H_M60H_DF_F3 0x14e3
+#define PCI_DEVICE_ID_AMD_19H_M70H_DF_F3 0x14f3
+#define PCI_DEVICE_ID_AMD_19H_M78H_DF_F3 0x12fb
+#define PCI_DEVICE_ID_AMD_1AH_M00H_DF_F3 0x12c3
+#define PCI_DEVICE_ID_AMD_1AH_M20H_DF_F3 0x16fb
+#define PCI_DEVICE_ID_AMD_MI200_DF_F3	0x14d3
+#define PCI_DEVICE_ID_AMD_CNB17H_F3	0x1703
+#define PCI_DEVICE_ID_AMD_LANCE		0x2000
+#define PCI_DEVICE_ID_AMD_LANCE_HOME	0x2001
+#define PCI_DEVICE_ID_AMD_SCSI		0x2020
+#define PCI_DEVICE_ID_AMD_SERENADE	0x36c0
+#define PCI_DEVICE_ID_AMD_FE_GATE_7006	0x7006
+#define PCI_DEVICE_ID_AMD_FE_GATE_7007	0x7007
+#define PCI_DEVICE_ID_AMD_FE_GATE_700C	0x700C
+#define PCI_DEVICE_ID_AMD_FE_GATE_700E	0x700E
+#define PCI_DEVICE_ID_AMD_COBRA_7401	0x7401
+#define PCI_DEVICE_ID_AMD_VIPER_7409	0x7409
+#define PCI_DEVICE_ID_AMD_VIPER_740B	0x740B
+#define PCI_DEVICE_ID_AMD_VIPER_7410	0x7410
+#define PCI_DEVICE_ID_AMD_VIPER_7411	0x7411
+#define PCI_DEVICE_ID_AMD_VIPER_7413	0x7413
+#define PCI_DEVICE_ID_AMD_VIPER_7440	0x7440
+#define PCI_DEVICE_ID_AMD_OPUS_7441	0x7441
+#define PCI_DEVICE_ID_AMD_OPUS_7443	0x7443
+#define PCI_DEVICE_ID_AMD_VIPER_7443	0x7443
+#define PCI_DEVICE_ID_AMD_OPUS_7445	0x7445
+#define PCI_DEVICE_ID_AMD_GOLAM_7450	0x7450
+#define PCI_DEVICE_ID_AMD_8111_PCI	0x7460
+#define PCI_DEVICE_ID_AMD_8111_LPC	0x7468
+#define PCI_DEVICE_ID_AMD_8111_IDE	0x7469
+#define PCI_DEVICE_ID_AMD_8111_SMBUS2	0x746a
+#define PCI_DEVICE_ID_AMD_8111_SMBUS	0x746b
+#define PCI_DEVICE_ID_AMD_8111_AUDIO	0x746d
+#define PCI_DEVICE_ID_AMD_8151_0	0x7454
+#define PCI_DEVICE_ID_AMD_8131_BRIDGE	0x7450
+#define PCI_DEVICE_ID_AMD_8131_APIC	0x7451
+#define PCI_DEVICE_ID_AMD_8132_BRIDGE	0x7458
+#define PCI_DEVICE_ID_AMD_NL_USB	0x7912
+#define PCI_DEVICE_ID_AMD_CS5535_IDE    0x208F
+#define PCI_DEVICE_ID_AMD_CS5536_ISA    0x2090
+#define PCI_DEVICE_ID_AMD_CS5536_FLASH  0x2091
+#define PCI_DEVICE_ID_AMD_CS5536_AUDIO  0x2093
+#define PCI_DEVICE_ID_AMD_CS5536_OHC    0x2094
+#define PCI_DEVICE_ID_AMD_CS5536_EHC    0x2095
+#define PCI_DEVICE_ID_AMD_CS5536_UDC    0x2096
+#define PCI_DEVICE_ID_AMD_CS5536_UOC    0x2097
+#define PCI_DEVICE_ID_AMD_CS5536_DEV_IDE    0x2092
+#define PCI_DEVICE_ID_AMD_CS5536_IDE    0x209A
+#define PCI_DEVICE_ID_AMD_LX_VIDEO  0x2081
+#define PCI_DEVICE_ID_AMD_LX_AES    0x2082
+#define PCI_DEVICE_ID_AMD_HUDSON2_SATA_IDE	0x7800
+#define PCI_DEVICE_ID_AMD_HUDSON2_SMBUS		0x780b
+#define PCI_DEVICE_ID_AMD_HUDSON2_IDE		0x780c
+#define PCI_DEVICE_ID_AMD_KERNCZ_SMBUS  0x790b
+
+#define PCI_VENDOR_ID_TRIDENT		0x1023
+#define PCI_DEVICE_ID_TRIDENT_4DWAVE_DX	0x2000
+#define PCI_DEVICE_ID_TRIDENT_4DWAVE_NX	0x2001
+#define PCI_DEVICE_ID_TRIDENT_9320	0x9320
+#define PCI_DEVICE_ID_TRIDENT_9388	0x9388
+#define PCI_DEVICE_ID_TRIDENT_9397	0x9397
+#define PCI_DEVICE_ID_TRIDENT_939A	0x939A
+#define PCI_DEVICE_ID_TRIDENT_9520	0x9520
+#define PCI_DEVICE_ID_TRIDENT_9525	0x9525
+#define PCI_DEVICE_ID_TRIDENT_9420	0x9420
+#define PCI_DEVICE_ID_TRIDENT_9440	0x9440
+#define PCI_DEVICE_ID_TRIDENT_9660	0x9660
+#define PCI_DEVICE_ID_TRIDENT_9750	0x9750
+#define PCI_DEVICE_ID_TRIDENT_9850	0x9850
+#define PCI_DEVICE_ID_TRIDENT_9880	0x9880
+#define PCI_DEVICE_ID_TRIDENT_8400	0x8400
+#define PCI_DEVICE_ID_TRIDENT_8420	0x8420
+#define PCI_DEVICE_ID_TRIDENT_8500	0x8500
+
+#define PCI_VENDOR_ID_AI		0x1025
+#define PCI_DEVICE_ID_AI_M1435		0x1435
+
+#define PCI_VENDOR_ID_DELL		0x1028
+#define PCI_DEVICE_ID_DELL_RACIII	0x0008
+#define PCI_DEVICE_ID_DELL_RAC4		0x0012
+#define PCI_DEVICE_ID_DELL_PERC5	0x0015
+
+#define PCI_SUBVENDOR_ID_DELL		0x1028
+
+#define PCI_VENDOR_ID_MATROX		0x102B
+#define PCI_DEVICE_ID_MATROX_MGA_2	0x0518
+#define PCI_DEVICE_ID_MATROX_MIL	0x0519
+#define PCI_DEVICE_ID_MATROX_MYS	0x051A
+#define PCI_DEVICE_ID_MATROX_MIL_2	0x051b
+#define PCI_DEVICE_ID_MATROX_MYS_AGP	0x051e
+#define PCI_DEVICE_ID_MATROX_MIL_2_AGP	0x051f
+#define PCI_DEVICE_ID_MATROX_MGA_IMP	0x0d10
+#define PCI_DEVICE_ID_MATROX_G100_MM	0x1000
+#define PCI_DEVICE_ID_MATROX_G100_AGP	0x1001
+#define PCI_DEVICE_ID_MATROX_G200_PCI	0x0520
+#define PCI_DEVICE_ID_MATROX_G200_AGP	0x0521
+#define	PCI_DEVICE_ID_MATROX_G400	0x0525
+#define	PCI_DEVICE_ID_MATROX_G200EV_PCI	0x0530
+#define PCI_DEVICE_ID_MATROX_G550	0x2527
+#define PCI_DEVICE_ID_MATROX_VIA	0x4536
+
+#define PCI_VENDOR_ID_MOBILITY_ELECTRONICS	0x14f2
+
+#define PCI_VENDOR_ID_CT		0x102c
+#define PCI_DEVICE_ID_CT_69000		0x00c0
+#define PCI_DEVICE_ID_CT_65545		0x00d8
+#define PCI_DEVICE_ID_CT_65548		0x00dc
+#define PCI_DEVICE_ID_CT_65550		0x00e0
+#define PCI_DEVICE_ID_CT_65554		0x00e4
+#define PCI_DEVICE_ID_CT_65555		0x00e5
+
+#define PCI_VENDOR_ID_MIRO		0x1031
+#define PCI_DEVICE_ID_MIRO_36050	0x5601
+#define PCI_DEVICE_ID_MIRO_DC10PLUS	0x7efe
+#define PCI_DEVICE_ID_MIRO_DC30PLUS	0xd801
+
+#define PCI_VENDOR_ID_NEC		0x1033
+#define PCI_DEVICE_ID_NEC_CBUS_1	0x0001 /* PCI-Cbus Bridge */
+#define PCI_DEVICE_ID_NEC_LOCAL		0x0002 /* Local Bridge */
+#define PCI_DEVICE_ID_NEC_ATM		0x0003 /* ATM LAN Controller */
+#define PCI_DEVICE_ID_NEC_R4000		0x0004 /* R4000 Bridge */
+#define PCI_DEVICE_ID_NEC_486		0x0005 /* 486 Like Peripheral Bus Bridge */
+#define PCI_DEVICE_ID_NEC_ACCEL_1	0x0006 /* Graphic Accelerator */
+#define PCI_DEVICE_ID_NEC_UXBUS		0x0007 /* UX-Bus Bridge */
+#define PCI_DEVICE_ID_NEC_ACCEL_2	0x0008 /* Graphic Accelerator */
+#define PCI_DEVICE_ID_NEC_GRAPH		0x0009 /* PCI-CoreGraph Bridge */
+#define PCI_DEVICE_ID_NEC_VL		0x0016 /* PCI-VL Bridge */
+#define PCI_DEVICE_ID_NEC_STARALPHA2	0x002c /* STAR ALPHA2 */
+#define PCI_DEVICE_ID_NEC_CBUS_2	0x002d /* PCI-Cbus Bridge */
+#define PCI_DEVICE_ID_NEC_USB		0x0035 /* PCI-USB Host */
+#define PCI_DEVICE_ID_NEC_CBUS_3	0x003b
+#define PCI_DEVICE_ID_NEC_NAPCCARD	0x003e
+#define PCI_DEVICE_ID_NEC_PCX2		0x0046 /* PowerVR */
+#define PCI_DEVICE_ID_NEC_VRC5476       0x009b
+#define PCI_DEVICE_ID_NEC_VRC4173	0x00a5
+#define PCI_DEVICE_ID_NEC_VRC5477_AC97  0x00a6
+#define PCI_DEVICE_ID_NEC_PC9821CS01    0x800c /* PC-9821-CS01 */
+#define PCI_DEVICE_ID_NEC_PC9821NRB06   0x800d /* PC-9821NR-B06 */
+
+#define PCI_VENDOR_ID_FD		0x1036
+#define PCI_DEVICE_ID_FD_36C70		0x0000
+
+#define PCI_VENDOR_ID_SI		0x1039
+#define PCI_DEVICE_ID_SI_5591_AGP	0x0001
+#define PCI_DEVICE_ID_SI_6202		0x0002
+#define PCI_DEVICE_ID_SI_503		0x0008
+#define PCI_DEVICE_ID_SI_ACPI		0x0009
+#define PCI_DEVICE_ID_SI_SMBUS		0x0016
+#define PCI_DEVICE_ID_SI_LPC		0x0018
+#define PCI_DEVICE_ID_SI_5597_VGA	0x0200
+#define PCI_DEVICE_ID_SI_6205		0x0205
+#define PCI_DEVICE_ID_SI_501		0x0406
+#define PCI_DEVICE_ID_SI_496		0x0496
+#define PCI_DEVICE_ID_SI_300		0x0300
+#define PCI_DEVICE_ID_SI_315H		0x0310
+#define PCI_DEVICE_ID_SI_315		0x0315
+#define PCI_DEVICE_ID_SI_315PRO		0x0325
+#define PCI_DEVICE_ID_SI_530		0x0530
+#define PCI_DEVICE_ID_SI_540		0x0540
+#define PCI_DEVICE_ID_SI_550		0x0550
+#define PCI_DEVICE_ID_SI_540_VGA	0x5300
+#define PCI_DEVICE_ID_SI_550_VGA	0x5315
+#define PCI_DEVICE_ID_SI_620		0x0620
+#define PCI_DEVICE_ID_SI_630		0x0630
+#define PCI_DEVICE_ID_SI_633		0x0633
+#define PCI_DEVICE_ID_SI_635		0x0635
+#define PCI_DEVICE_ID_SI_640		0x0640
+#define PCI_DEVICE_ID_SI_645		0x0645
+#define PCI_DEVICE_ID_SI_646		0x0646
+#define PCI_DEVICE_ID_SI_648		0x0648
+#define PCI_DEVICE_ID_SI_650		0x0650
+#define PCI_DEVICE_ID_SI_651		0x0651
+#define PCI_DEVICE_ID_SI_655		0x0655
+#define PCI_DEVICE_ID_SI_661		0x0661
+#define PCI_DEVICE_ID_SI_730		0x0730
+#define PCI_DEVICE_ID_SI_733		0x0733
+#define PCI_DEVICE_ID_SI_630_VGA	0x6300
+#define PCI_DEVICE_ID_SI_735		0x0735
+#define PCI_DEVICE_ID_SI_740		0x0740
+#define PCI_DEVICE_ID_SI_741		0x0741
+#define PCI_DEVICE_ID_SI_745		0x0745
+#define PCI_DEVICE_ID_SI_746		0x0746
+#define PCI_DEVICE_ID_SI_755		0x0755
+#define PCI_DEVICE_ID_SI_760		0x0760
+#define PCI_DEVICE_ID_SI_900		0x0900
+#define PCI_DEVICE_ID_SI_961		0x0961
+#define PCI_DEVICE_ID_SI_962		0x0962
+#define PCI_DEVICE_ID_SI_963		0x0963
+#define PCI_DEVICE_ID_SI_965		0x0965
+#define PCI_DEVICE_ID_SI_966		0x0966
+#define PCI_DEVICE_ID_SI_968		0x0968
+#define PCI_DEVICE_ID_SI_1180		0x1180
+#define PCI_DEVICE_ID_SI_5511		0x5511
+#define PCI_DEVICE_ID_SI_5513		0x5513
+#define PCI_DEVICE_ID_SI_5517		0x5517
+#define PCI_DEVICE_ID_SI_5518		0x5518
+#define PCI_DEVICE_ID_SI_5571		0x5571
+#define PCI_DEVICE_ID_SI_5581		0x5581
+#define PCI_DEVICE_ID_SI_5582		0x5582
+#define PCI_DEVICE_ID_SI_5591		0x5591
+#define PCI_DEVICE_ID_SI_5596		0x5596
+#define PCI_DEVICE_ID_SI_5597		0x5597
+#define PCI_DEVICE_ID_SI_5598		0x5598
+#define PCI_DEVICE_ID_SI_5600		0x5600
+#define PCI_DEVICE_ID_SI_7012		0x7012
+#define PCI_DEVICE_ID_SI_7013		0x7013
+#define PCI_DEVICE_ID_SI_7016		0x7016
+#define PCI_DEVICE_ID_SI_7018		0x7018
+
+#define PCI_VENDOR_ID_HP		0x103c
+#define PCI_VENDOR_ID_HP_3PAR		0x1590
+#define PCI_DEVICE_ID_HP_VISUALIZE_EG	0x1005
+#define PCI_DEVICE_ID_HP_VISUALIZE_FX6	0x1006
+#define PCI_DEVICE_ID_HP_VISUALIZE_FX4	0x1008
+#define PCI_DEVICE_ID_HP_VISUALIZE_FX2	0x100a
+#define PCI_DEVICE_ID_HP_TACHYON	0x1028
+#define PCI_DEVICE_ID_HP_TACHLITE	0x1029
+#define PCI_DEVICE_ID_HP_J2585A		0x1030
+#define PCI_DEVICE_ID_HP_J2585B		0x1031
+#define PCI_DEVICE_ID_HP_J2973A		0x1040
+#define PCI_DEVICE_ID_HP_J2970A		0x1042
+#define PCI_DEVICE_ID_HP_DIVA		0x1048
+#define PCI_DEVICE_ID_HP_DIVA_TOSCA1	0x1049
+#define PCI_DEVICE_ID_HP_DIVA_TOSCA2	0x104A
+#define PCI_DEVICE_ID_HP_DIVA_MAESTRO	0x104B
+#define PCI_DEVICE_ID_HP_REO_IOC	0x10f1
+#define PCI_DEVICE_ID_HP_VISUALIZE_FXE	0x108b
+#define PCI_DEVICE_ID_HP_DIVA_HALFDOME	0x1223
+#define PCI_DEVICE_ID_HP_DIVA_KEYSTONE	0x1226
+#define PCI_DEVICE_ID_HP_DIVA_POWERBAR	0x1227
+#define PCI_DEVICE_ID_HP_ZX1_IOC	0x122a
+#define PCI_DEVICE_ID_HP_PCIX_LBA	0x122e
+#define PCI_DEVICE_ID_HP_SX1000_IOC	0x127c
+#define PCI_DEVICE_ID_HP_DIVA_EVEREST	0x1282
+#define PCI_DEVICE_ID_HP_DIVA_AUX	0x1290
+#define PCI_DEVICE_ID_HP_DIVA_RMP3	0x1301
+#define PCI_DEVICE_ID_HP_DIVA_HURRICANE	0x132a
+#define PCI_DEVICE_ID_HP_CISSA		0x3220
+#define PCI_DEVICE_ID_HP_CISSC		0x3230
+#define PCI_DEVICE_ID_HP_CISSD		0x3238
+#define PCI_DEVICE_ID_HP_CISSE		0x323a
+#define PCI_DEVICE_ID_HP_CISSF		0x323b
+#define PCI_DEVICE_ID_HP_CISSH		0x323c
+#define PCI_DEVICE_ID_HP_CISSI		0x3239
+#define PCI_DEVICE_ID_HP_ZX2_IOC	0x4031
+
+#define PCI_VENDOR_ID_PCTECH		0x1042
+#define PCI_DEVICE_ID_PCTECH_RZ1000	0x1000
+#define PCI_DEVICE_ID_PCTECH_RZ1001	0x1001
+#define PCI_DEVICE_ID_PCTECH_SAMURAI_IDE 0x3020
+
+#define PCI_VENDOR_ID_ASUSTEK		0x1043
+#define PCI_DEVICE_ID_ASUSTEK_0675	0x0675
+
+#define PCI_VENDOR_ID_DPT		0x1044
+#define PCI_DEVICE_ID_DPT		0xa400
+
+#define PCI_VENDOR_ID_OPTI		0x1045
+#define PCI_DEVICE_ID_OPTI_82C558	0xc558
+#define PCI_DEVICE_ID_OPTI_82C621	0xc621
+#define PCI_DEVICE_ID_OPTI_82C700	0xc700
+#define PCI_DEVICE_ID_OPTI_82C825	0xd568
+
+#define PCI_VENDOR_ID_ELSA		0x1048
+#define PCI_DEVICE_ID_ELSA_MICROLINK	0x1000
+#define PCI_DEVICE_ID_ELSA_QS3000	0x3000
+
+#define PCI_VENDOR_ID_STMICRO		0x104A
+#define PCI_DEVICE_ID_STMICRO_USB_HOST	0xCC00
+#define PCI_DEVICE_ID_STMICRO_USB_OHCI	0xCC01
+#define PCI_DEVICE_ID_STMICRO_USB_OTG	0xCC02
+#define PCI_DEVICE_ID_STMICRO_UART_HWFC 0xCC03
+#define PCI_DEVICE_ID_STMICRO_UART_NO_HWFC	0xCC04
+#define PCI_DEVICE_ID_STMICRO_SOC_DMA	0xCC05
+#define PCI_DEVICE_ID_STMICRO_SATA	0xCC06
+#define PCI_DEVICE_ID_STMICRO_I2C	0xCC07
+#define PCI_DEVICE_ID_STMICRO_SPI_HS	0xCC08
+#define PCI_DEVICE_ID_STMICRO_MAC	0xCC09
+#define PCI_DEVICE_ID_STMICRO_SDIO_EMMC 0xCC0A
+#define PCI_DEVICE_ID_STMICRO_SDIO	0xCC0B
+#define PCI_DEVICE_ID_STMICRO_GPIO	0xCC0C
+#define PCI_DEVICE_ID_STMICRO_VIP	0xCC0D
+#define PCI_DEVICE_ID_STMICRO_AUDIO_ROUTER_DMA	0xCC0E
+#define PCI_DEVICE_ID_STMICRO_AUDIO_ROUTER_SRCS 0xCC0F
+#define PCI_DEVICE_ID_STMICRO_AUDIO_ROUTER_MSPS 0xCC10
+#define PCI_DEVICE_ID_STMICRO_CAN	0xCC11
+#define PCI_DEVICE_ID_STMICRO_MLB	0xCC12
+#define PCI_DEVICE_ID_STMICRO_DBP	0xCC13
+#define PCI_DEVICE_ID_STMICRO_SATA_PHY	0xCC14
+#define PCI_DEVICE_ID_STMICRO_ESRAM	0xCC15
+#define PCI_DEVICE_ID_STMICRO_VIC	0xCC16
+
+#define PCI_VENDOR_ID_BUSLOGIC		      0x104B
+#define PCI_DEVICE_ID_BUSLOGIC_MULTIMASTER_NC 0x0140
+#define PCI_DEVICE_ID_BUSLOGIC_MULTIMASTER    0x1040
+#define PCI_DEVICE_ID_BUSLOGIC_FLASHPOINT     0x8130
+
+#define PCI_VENDOR_ID_TI		0x104c
+#define PCI_DEVICE_ID_TI_TVP4020	0x3d07
+#define PCI_DEVICE_ID_TI_4450		0x8011
+#define PCI_DEVICE_ID_TI_XX21_XX11	0x8031
+#define PCI_DEVICE_ID_TI_XX21_XX11_FM	0x8033
+#define PCI_DEVICE_ID_TI_XX21_XX11_SD	0x8034
+#define PCI_DEVICE_ID_TI_X515		0x8036
+#define PCI_DEVICE_ID_TI_XX12		0x8039
+#define PCI_DEVICE_ID_TI_XX12_FM	0x803b
+#define PCI_DEVICE_ID_TI_XIO2000A	0x8231
+#define PCI_DEVICE_ID_TI_1130		0xac12
+#define PCI_DEVICE_ID_TI_1031		0xac13
+#define PCI_DEVICE_ID_TI_1131		0xac15
+#define PCI_DEVICE_ID_TI_1250		0xac16
+#define PCI_DEVICE_ID_TI_1220		0xac17
+#define PCI_DEVICE_ID_TI_1221		0xac19
+#define PCI_DEVICE_ID_TI_1210		0xac1a
+#define PCI_DEVICE_ID_TI_1450		0xac1b
+#define PCI_DEVICE_ID_TI_1225		0xac1c
+#define PCI_DEVICE_ID_TI_1251A		0xac1d
+#define PCI_DEVICE_ID_TI_1211		0xac1e
+#define PCI_DEVICE_ID_TI_1251B		0xac1f
+#define PCI_DEVICE_ID_TI_4410		0xac41
+#define PCI_DEVICE_ID_TI_4451		0xac42
+#define PCI_DEVICE_ID_TI_4510		0xac44
+#define PCI_DEVICE_ID_TI_4520		0xac46
+#define PCI_DEVICE_ID_TI_7510		0xac47
+#define PCI_DEVICE_ID_TI_7610		0xac48
+#define PCI_DEVICE_ID_TI_7410		0xac49
+#define PCI_DEVICE_ID_TI_1410		0xac50
+#define PCI_DEVICE_ID_TI_1420		0xac51
+#define PCI_DEVICE_ID_TI_1451A		0xac52
+#define PCI_DEVICE_ID_TI_1620		0xac54
+#define PCI_DEVICE_ID_TI_1520		0xac55
+#define PCI_DEVICE_ID_TI_1510		0xac56
+#define PCI_DEVICE_ID_TI_X620		0xac8d
+#define PCI_DEVICE_ID_TI_X420		0xac8e
+#define PCI_DEVICE_ID_TI_XX20_FM	0xac8f
+#define PCI_DEVICE_ID_TI_J721E		0xb00d
+#define PCI_DEVICE_ID_TI_DRA74x		0xb500
+#define PCI_DEVICE_ID_TI_DRA72x		0xb501
+
+#define PCI_VENDOR_ID_SONY		0x104d
+
+/* Winbond have two vendor IDs! See 0x10ad as well */
+#define PCI_VENDOR_ID_WINBOND2		0x1050
+#define PCI_DEVICE_ID_WINBOND2_89C940F	0x5a5a
+#define PCI_DEVICE_ID_WINBOND2_6692	0x6692
+
+#define PCI_VENDOR_ID_ANIGMA		0x1051
+#define PCI_DEVICE_ID_ANIGMA_MC145575	0x0100
+
+#define PCI_VENDOR_ID_EFAR		0x1055
+#define PCI_DEVICE_ID_EFAR_SLC90E66_1	0x9130
+#define PCI_DEVICE_ID_EFAR_SLC90E66_3	0x9463
+
+#define PCI_VENDOR_ID_MOTOROLA		0x1057
+#define PCI_DEVICE_ID_MOTOROLA_MPC105	0x0001
+#define PCI_DEVICE_ID_MOTOROLA_MPC106	0x0002
+#define PCI_DEVICE_ID_MOTOROLA_MPC107	0x0004
+#define PCI_DEVICE_ID_MOTOROLA_RAVEN	0x4801
+#define PCI_DEVICE_ID_MOTOROLA_FALCON	0x4802
+#define PCI_DEVICE_ID_MOTOROLA_HAWK	0x4803
+#define PCI_DEVICE_ID_MOTOROLA_HARRIER	0x480b
+#define PCI_DEVICE_ID_MOTOROLA_MPC5200	0x5803
+#define PCI_DEVICE_ID_MOTOROLA_MPC5200B	0x5809
+
+#define PCI_VENDOR_ID_PROMISE		0x105a
+#define PCI_DEVICE_ID_PROMISE_20265	0x0d30
+#define PCI_DEVICE_ID_PROMISE_20267	0x4d30
+#define PCI_DEVICE_ID_PROMISE_20246	0x4d33
+#define PCI_DEVICE_ID_PROMISE_20262	0x4d38
+#define PCI_DEVICE_ID_PROMISE_20263	0x0D38
+#define PCI_DEVICE_ID_PROMISE_20268	0x4d68
+#define PCI_DEVICE_ID_PROMISE_20269	0x4d69
+#define PCI_DEVICE_ID_PROMISE_20270	0x6268
+#define PCI_DEVICE_ID_PROMISE_20271	0x6269
+#define PCI_DEVICE_ID_PROMISE_20275	0x1275
+#define PCI_DEVICE_ID_PROMISE_20276	0x5275
+#define PCI_DEVICE_ID_PROMISE_20277	0x7275
+
+#define PCI_VENDOR_ID_FOXCONN		0x105b
+
+#define PCI_VENDOR_ID_UMC		0x1060
+#define PCI_DEVICE_ID_UMC_UM8673F	0x0101
+#define PCI_DEVICE_ID_UMC_UM8886BF	0x673a
+#define PCI_DEVICE_ID_UMC_UM8886A	0x886a
+
+#define PCI_VENDOR_ID_PICOPOWER		0x1066
+#define PCI_DEVICE_ID_PICOPOWER_PT86C523	0x0002
+#define PCI_DEVICE_ID_PICOPOWER_PT86C523BBP	0x8002
+
+#define PCI_VENDOR_ID_MYLEX		0x1069
+#define PCI_DEVICE_ID_MYLEX_DAC960_P	0x0001
+#define PCI_DEVICE_ID_MYLEX_DAC960_PD	0x0002
+#define PCI_DEVICE_ID_MYLEX_DAC960_PG	0x0010
+#define PCI_DEVICE_ID_MYLEX_DAC960_LA	0x0020
+#define PCI_DEVICE_ID_MYLEX_DAC960_LP	0x0050
+#define PCI_DEVICE_ID_MYLEX_DAC960_BA	0xBA56
+#define PCI_DEVICE_ID_MYLEX_DAC960_GEM	0xB166
+
+#define PCI_VENDOR_ID_APPLE		0x106b
+#define PCI_DEVICE_ID_APPLE_BANDIT	0x0001
+#define PCI_DEVICE_ID_APPLE_HYDRA	0x000e
+#define PCI_DEVICE_ID_APPLE_UNI_N_FW	0x0018
+#define PCI_DEVICE_ID_APPLE_UNI_N_AGP	0x0020
+#define PCI_DEVICE_ID_APPLE_UNI_N_GMAC	0x0021
+#define PCI_DEVICE_ID_APPLE_UNI_N_GMACP	0x0024
+#define PCI_DEVICE_ID_APPLE_UNI_N_AGP_P	0x0027
+#define PCI_DEVICE_ID_APPLE_UNI_N_AGP15	0x002d
+#define PCI_DEVICE_ID_APPLE_UNI_N_PCI15	0x002e
+#define PCI_DEVICE_ID_APPLE_UNI_N_GMAC2	0x0032
+#define PCI_DEVICE_ID_APPLE_UNI_N_ATA	0x0033
+#define PCI_DEVICE_ID_APPLE_UNI_N_AGP2	0x0034
+#define PCI_DEVICE_ID_APPLE_IPID_ATA100	0x003b
+#define PCI_DEVICE_ID_APPLE_K2_ATA100	0x0043
+#define PCI_DEVICE_ID_APPLE_U3_AGP	0x004b
+#define PCI_DEVICE_ID_APPLE_K2_GMAC	0x004c
+#define PCI_DEVICE_ID_APPLE_SH_ATA      0x0050
+#define PCI_DEVICE_ID_APPLE_SH_SUNGEM   0x0051
+#define PCI_DEVICE_ID_APPLE_U3L_AGP	0x0058
+#define PCI_DEVICE_ID_APPLE_U3H_AGP	0x0059
+#define PCI_DEVICE_ID_APPLE_U4_PCIE	0x005b
+#define PCI_DEVICE_ID_APPLE_IPID2_AGP	0x0066
+#define PCI_DEVICE_ID_APPLE_IPID2_ATA	0x0069
+#define PCI_DEVICE_ID_APPLE_IPID2_FW	0x006a
+#define PCI_DEVICE_ID_APPLE_IPID2_GMAC	0x006b
+#define PCI_DEVICE_ID_APPLE_TIGON3	0x1645
+
+#define PCI_VENDOR_ID_YAMAHA		0x1073
+#define PCI_DEVICE_ID_YAMAHA_724	0x0004
+#define PCI_DEVICE_ID_YAMAHA_724F	0x000d
+#define PCI_DEVICE_ID_YAMAHA_740	0x000a
+#define PCI_DEVICE_ID_YAMAHA_740C	0x000c
+#define PCI_DEVICE_ID_YAMAHA_744	0x0010
+#define PCI_DEVICE_ID_YAMAHA_754	0x0012
+
+#define PCI_VENDOR_ID_QLOGIC		0x1077
+#define PCI_DEVICE_ID_QLOGIC_ISP10160	0x1016
+#define PCI_DEVICE_ID_QLOGIC_ISP1020	0x1020
+#define PCI_DEVICE_ID_QLOGIC_ISP1080	0x1080
+#define PCI_DEVICE_ID_QLOGIC_ISP12160	0x1216
+#define PCI_DEVICE_ID_QLOGIC_ISP1240	0x1240
+#define PCI_DEVICE_ID_QLOGIC_ISP1280	0x1280
+#define PCI_DEVICE_ID_QLOGIC_ISP2100	0x2100
+#define PCI_DEVICE_ID_QLOGIC_ISP2200	0x2200
+#define PCI_DEVICE_ID_QLOGIC_ISP2300	0x2300
+#define PCI_DEVICE_ID_QLOGIC_ISP2312	0x2312
+#define PCI_DEVICE_ID_QLOGIC_ISP2322	0x2322
+#define PCI_DEVICE_ID_QLOGIC_ISP6312	0x6312
+#define PCI_DEVICE_ID_QLOGIC_ISP6322	0x6322
+#define PCI_DEVICE_ID_QLOGIC_ISP2422	0x2422
+#define PCI_DEVICE_ID_QLOGIC_ISP2432	0x2432
+#define PCI_DEVICE_ID_QLOGIC_ISP2512	0x2512
+#define PCI_DEVICE_ID_QLOGIC_ISP2522	0x2522
+#define PCI_DEVICE_ID_QLOGIC_ISP5422	0x5422
+#define PCI_DEVICE_ID_QLOGIC_ISP5432	0x5432
+
+#define PCI_VENDOR_ID_CYRIX		0x1078
+#define PCI_DEVICE_ID_CYRIX_5510	0x0000
+#define PCI_DEVICE_ID_CYRIX_PCI_MASTER	0x0001
+#define PCI_DEVICE_ID_CYRIX_5520	0x0002
+#define PCI_DEVICE_ID_CYRIX_5530_LEGACY	0x0100
+#define PCI_DEVICE_ID_CYRIX_5530_IDE	0x0102
+#define PCI_DEVICE_ID_CYRIX_5530_AUDIO	0x0103
+#define PCI_DEVICE_ID_CYRIX_5530_VIDEO	0x0104
+
+#define PCI_VENDOR_ID_CONTAQ		0x1080
+#define PCI_DEVICE_ID_CONTAQ_82C693	0xc693
+
+#define PCI_VENDOR_ID_OLICOM		0x108d
+#define PCI_DEVICE_ID_OLICOM_OC2325	0x0012
+#define PCI_DEVICE_ID_OLICOM_OC2183	0x0013
+#define PCI_DEVICE_ID_OLICOM_OC2326	0x0014
+
+#define PCI_VENDOR_ID_SUN		0x108e
+#define PCI_DEVICE_ID_SUN_EBUS		0x1000
+#define PCI_DEVICE_ID_SUN_HAPPYMEAL	0x1001
+#define PCI_DEVICE_ID_SUN_RIO_EBUS	0x1100
+#define PCI_DEVICE_ID_SUN_RIO_GEM	0x1101
+#define PCI_DEVICE_ID_SUN_RIO_1394	0x1102
+#define PCI_DEVICE_ID_SUN_RIO_USB	0x1103
+#define PCI_DEVICE_ID_SUN_GEM		0x2bad
+#define PCI_DEVICE_ID_SUN_SIMBA		0x5000
+#define PCI_DEVICE_ID_SUN_PBM		0x8000
+#define PCI_DEVICE_ID_SUN_SCHIZO	0x8001
+#define PCI_DEVICE_ID_SUN_SABRE		0xa000
+#define PCI_DEVICE_ID_SUN_HUMMINGBIRD	0xa001
+#define PCI_DEVICE_ID_SUN_TOMATILLO	0xa801
+#define PCI_DEVICE_ID_SUN_CASSINI	0xabba
+
+#define PCI_VENDOR_ID_NI		0x1093
+#define PCI_DEVICE_ID_NI_PCI2322	0xd130
+#define PCI_DEVICE_ID_NI_PCI2324	0xd140
+#define PCI_DEVICE_ID_NI_PCI2328	0xd150
+#define PCI_DEVICE_ID_NI_PXI8422_2322	0xd190
+#define PCI_DEVICE_ID_NI_PXI8422_2324	0xd1a0
+#define PCI_DEVICE_ID_NI_PXI8420_2322	0xd1d0
+#define PCI_DEVICE_ID_NI_PXI8420_2324	0xd1e0
+#define PCI_DEVICE_ID_NI_PXI8420_2328	0xd1f0
+#define PCI_DEVICE_ID_NI_PXI8420_23216	0xd1f1
+#define PCI_DEVICE_ID_NI_PCI2322I	0xd250
+#define PCI_DEVICE_ID_NI_PCI2324I	0xd270
+#define PCI_DEVICE_ID_NI_PCI23216	0xd2b0
+#define PCI_DEVICE_ID_NI_PXI8430_2322	0x7080
+#define PCI_DEVICE_ID_NI_PCI8430_2322	0x70db
+#define PCI_DEVICE_ID_NI_PXI8430_2324	0x70dd
+#define PCI_DEVICE_ID_NI_PCI8430_2324	0x70df
+#define PCI_DEVICE_ID_NI_PXI8430_2328	0x70e2
+#define PCI_DEVICE_ID_NI_PCI8430_2328	0x70e4
+#define PCI_DEVICE_ID_NI_PXI8430_23216	0x70e6
+#define PCI_DEVICE_ID_NI_PCI8430_23216	0x70e7
+#define PCI_DEVICE_ID_NI_PXI8432_2322	0x70e8
+#define PCI_DEVICE_ID_NI_PCI8432_2322	0x70ea
+#define PCI_DEVICE_ID_NI_PXI8432_2324	0x70ec
+#define PCI_DEVICE_ID_NI_PCI8432_2324	0x70ee
+
+#define PCI_VENDOR_ID_CMD		0x1095
+#define PCI_DEVICE_ID_CMD_643		0x0643
+#define PCI_DEVICE_ID_CMD_646		0x0646
+#define PCI_DEVICE_ID_CMD_648		0x0648
+#define PCI_DEVICE_ID_CMD_649		0x0649
+
+#define PCI_DEVICE_ID_SII_680		0x0680
+#define PCI_DEVICE_ID_SII_3112		0x3112
+#define PCI_DEVICE_ID_SII_1210SA	0x0240
+
+#define PCI_VENDOR_ID_BROOKTREE		0x109e
+#define PCI_DEVICE_ID_BROOKTREE_878	0x0878
+#define PCI_DEVICE_ID_BROOKTREE_879	0x0879
+
+#define PCI_VENDOR_ID_SGI		0x10a9
+#define PCI_DEVICE_ID_SGI_IOC3		0x0003
+#define PCI_DEVICE_ID_SGI_LITHIUM	0x1002
+
+#define PCI_VENDOR_ID_WINBOND		0x10ad
+#define PCI_DEVICE_ID_WINBOND_82C105	0x0105
+#define PCI_DEVICE_ID_WINBOND_83C553	0x0565
+
+#define PCI_VENDOR_ID_PLX		0x10b5
+#define PCI_DEVICE_ID_PLX_R685		0x1030
+#define PCI_DEVICE_ID_PLX_ROMULUS	0x106a
+#define PCI_DEVICE_ID_PLX_SPCOM800	0x1076
+#define PCI_DEVICE_ID_PLX_1077		0x1077
+#define PCI_DEVICE_ID_PLX_SPCOM200	0x1103
+#define PCI_DEVICE_ID_PLX_DJINN_ITOO	0x1151
+#define PCI_DEVICE_ID_PLX_R753		0x1152
+#define PCI_DEVICE_ID_PLX_OLITEC	0x1187
+#define PCI_DEVICE_ID_PLX_PCI200SYN	0x3196
+#define PCI_DEVICE_ID_PLX_9030          0x9030
+#define PCI_DEVICE_ID_PLX_9050		0x9050
+#define PCI_DEVICE_ID_PLX_9056		0x9056
+#define PCI_DEVICE_ID_PLX_9080		0x9080
+#define PCI_DEVICE_ID_PLX_GTEK_SERIAL2	0xa001
+
+#define PCI_VENDOR_ID_MADGE		0x10b6
+#define PCI_DEVICE_ID_MADGE_MK2		0x0002
+
+#define PCI_VENDOR_ID_3COM		0x10b7
+#define PCI_DEVICE_ID_3COM_3C985	0x0001
+#define PCI_DEVICE_ID_3COM_3C940	0x1700
+#define PCI_DEVICE_ID_3COM_3C339	0x3390
+#define PCI_DEVICE_ID_3COM_3C359	0x3590
+#define PCI_DEVICE_ID_3COM_3C940B	0x80eb
+#define PCI_DEVICE_ID_3COM_3CR990	0x9900
+#define PCI_DEVICE_ID_3COM_3CR990_TX_95	0x9902
+#define PCI_DEVICE_ID_3COM_3CR990_TX_97	0x9903
+#define PCI_DEVICE_ID_3COM_3CR990B	0x9904
+#define PCI_DEVICE_ID_3COM_3CR990_FX	0x9905
+#define PCI_DEVICE_ID_3COM_3CR990SVR95	0x9908
+#define PCI_DEVICE_ID_3COM_3CR990SVR97	0x9909
+#define PCI_DEVICE_ID_3COM_3CR990SVR	0x990a
+
+#define PCI_VENDOR_ID_AL		0x10b9
+#define PCI_DEVICE_ID_AL_M1489		0x1489
+#define PCI_DEVICE_ID_AL_M1533		0x1533
+#define PCI_DEVICE_ID_AL_M1535		0x1535
+#define PCI_DEVICE_ID_AL_M1541		0x1541
+#define PCI_DEVICE_ID_AL_M1563		0x1563
+#define PCI_DEVICE_ID_AL_M1621		0x1621
+#define PCI_DEVICE_ID_AL_M1631		0x1631
+#define PCI_DEVICE_ID_AL_M1632		0x1632
+#define PCI_DEVICE_ID_AL_M1641		0x1641
+#define PCI_DEVICE_ID_AL_M1644		0x1644
+#define PCI_DEVICE_ID_AL_M1647		0x1647
+#define PCI_DEVICE_ID_AL_M1651		0x1651
+#define PCI_DEVICE_ID_AL_M1671		0x1671
+#define PCI_DEVICE_ID_AL_M1681		0x1681
+#define PCI_DEVICE_ID_AL_M1683		0x1683
+#define PCI_DEVICE_ID_AL_M1689		0x1689
+#define PCI_DEVICE_ID_AL_M5219		0x5219
+#define PCI_DEVICE_ID_AL_M5228		0x5228
+#define PCI_DEVICE_ID_AL_M5229		0x5229
+#define PCI_DEVICE_ID_AL_M5451		0x5451
+#define PCI_DEVICE_ID_AL_M7101		0x7101
+
+#define PCI_VENDOR_ID_NEOMAGIC		0x10c8
+#define PCI_DEVICE_ID_NEOMAGIC_NM256AV_AUDIO 0x8005
+#define PCI_DEVICE_ID_NEOMAGIC_NM256ZX_AUDIO 0x8006
+#define PCI_DEVICE_ID_NEOMAGIC_NM256XL_PLUS_AUDIO 0x8016
+
+#define PCI_VENDOR_ID_TCONRAD		0x10da
+#define PCI_DEVICE_ID_TCONRAD_TOKENRING	0x0508
+
+#define PCI_VENDOR_ID_ROHM		0x10db
+
+#define PCI_VENDOR_ID_NVIDIA			0x10de
+#define PCI_DEVICE_ID_NVIDIA_TNT		0x0020
+#define PCI_DEVICE_ID_NVIDIA_TNT2		0x0028
+#define PCI_DEVICE_ID_NVIDIA_UTNT2		0x0029
+#define PCI_DEVICE_ID_NVIDIA_TNT_UNKNOWN        0x002a
+#define PCI_DEVICE_ID_NVIDIA_VTNT2		0x002C
+#define PCI_DEVICE_ID_NVIDIA_UVTNT2		0x002D
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP04_SMBUS	0x0034
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP04_IDE	0x0035
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP04_SATA	0x0036
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP04_SATA2	0x003e
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_6800_ULTRA 0x0040
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_6800       0x0041
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_6800_LE    0x0042
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_6800_GT    0x0045
+#define PCI_DEVICE_ID_NVIDIA_QUADRO_FX_4000     0x004E
+#define PCI_DEVICE_ID_NVIDIA_NFORCE4_SMBUS	0x0052
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_CK804_IDE	0x0053
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_CK804_SATA	0x0054
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_CK804_SATA2	0x0055
+#define PCI_DEVICE_ID_NVIDIA_CK804_AUDIO	0x0059
+#define PCI_DEVICE_ID_NVIDIA_CK804_PCIE		0x005d
+#define PCI_DEVICE_ID_NVIDIA_NFORCE2_SMBUS	0x0064
+#define PCI_DEVICE_ID_NVIDIA_NFORCE2_IDE	0x0065
+#define PCI_DEVICE_ID_NVIDIA_MCP2_MODEM		0x0069
+#define PCI_DEVICE_ID_NVIDIA_MCP2_AUDIO		0x006a
+#define PCI_DEVICE_ID_NVIDIA_NFORCE2S_SMBUS	0x0084
+#define PCI_DEVICE_ID_NVIDIA_NFORCE2S_IDE	0x0085
+#define PCI_DEVICE_ID_NVIDIA_MCP2S_MODEM	0x0089
+#define PCI_DEVICE_ID_NVIDIA_CK8_AUDIO		0x008a
+#define PCI_DEVICE_ID_NVIDIA_NFORCE2S_SATA	0x008e
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_7800_GT   0x0090
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_7800_GTX	0x0091
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_GO_7800   0x0098
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_GO_7800_GTX 0x0099
+#define PCI_DEVICE_ID_NVIDIA_ITNT2		0x00A0
+#define PCI_DEVICE_ID_GEFORCE_6800A             0x00c1
+#define PCI_DEVICE_ID_GEFORCE_6800A_LE          0x00c2
+#define PCI_DEVICE_ID_GEFORCE_GO_6800           0x00c8
+#define PCI_DEVICE_ID_GEFORCE_GO_6800_ULTRA     0x00c9
+#define PCI_DEVICE_ID_QUADRO_FX_GO1400          0x00cc
+#define PCI_DEVICE_ID_QUADRO_FX_1400            0x00ce
+#define PCI_DEVICE_ID_NVIDIA_NFORCE3		0x00d1
+#define PCI_DEVICE_ID_NVIDIA_NFORCE3_SMBUS	0x00d4
+#define PCI_DEVICE_ID_NVIDIA_NFORCE3_IDE	0x00d5
+#define PCI_DEVICE_ID_NVIDIA_MCP3_MODEM		0x00d9
+#define PCI_DEVICE_ID_NVIDIA_MCP3_AUDIO		0x00da
+#define PCI_DEVICE_ID_NVIDIA_NFORCE3S		0x00e1
+#define PCI_DEVICE_ID_NVIDIA_NFORCE3S_SATA	0x00e3
+#define PCI_DEVICE_ID_NVIDIA_NFORCE3S_SMBUS	0x00e4
+#define PCI_DEVICE_ID_NVIDIA_NFORCE3S_IDE	0x00e5
+#define PCI_DEVICE_ID_NVIDIA_CK8S_AUDIO		0x00ea
+#define PCI_DEVICE_ID_NVIDIA_NFORCE3S_SATA2	0x00ee
+#define PCIE_DEVICE_ID_NVIDIA_GEFORCE_6800_ALT1 0x00f0
+#define PCIE_DEVICE_ID_NVIDIA_GEFORCE_6600_ALT1 0x00f1
+#define PCIE_DEVICE_ID_NVIDIA_GEFORCE_6600_ALT2 0x00f2
+#define PCIE_DEVICE_ID_NVIDIA_GEFORCE_6200_ALT1 0x00f3
+#define PCIE_DEVICE_ID_NVIDIA_GEFORCE_6800_GT   0x00f9
+#define PCIE_DEVICE_ID_NVIDIA_QUADRO_NVS280	0x00fd
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_SDR	0x0100
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_DDR	0x0101
+#define PCI_DEVICE_ID_NVIDIA_QUADRO		0x0103
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE2_MX	0x0110
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE2_MX2	0x0111
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE2_GO	0x0112
+#define PCI_DEVICE_ID_NVIDIA_QUADRO2_MXR	0x0113
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_6600_GT	0x0140
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_6600	0x0141
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_6610_XL	0x0145
+#define PCI_DEVICE_ID_NVIDIA_QUADRO_FX_540	0x014E
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_6200	0x014F
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE2_GTS	0x0150
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE2_GTS2	0x0151
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE2_ULTRA	0x0152
+#define PCI_DEVICE_ID_NVIDIA_QUADRO2_PRO	0x0153
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_6200_TURBOCACHE 0x0161
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_GO_6200    0x0164
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_GO_6250    0x0166
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_GO_6200_1  0x0167
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_GO_6250_1  0x0168
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_MX_460	0x0170
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_MX_440	0x0171
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_MX_420	0x0172
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_MX_440_SE	0x0173
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_440_GO	0x0174
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_420_GO	0x0175
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_420_GO_M32 0x0176
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_460_GO    0x0177
+#define PCI_DEVICE_ID_NVIDIA_QUADRO4_500XGL	0x0178
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_440_GO_M64 0x0179
+#define PCI_DEVICE_ID_NVIDIA_QUADRO4_200	0x017A
+#define PCI_DEVICE_ID_NVIDIA_QUADRO4_550XGL	0x017B
+#define PCI_DEVICE_ID_NVIDIA_QUADRO4_500_GOGL	0x017C
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_410_GO_M16 0x017D
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_MX_440_8X 0x0181
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_MX_440SE_8X 0x0182
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_MX_420_8X 0x0183
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_MX_4000   0x0185
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_448_GO    0x0186
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_488_GO    0x0187
+#define PCI_DEVICE_ID_NVIDIA_QUADRO4_580_XGL    0x0188
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_MX_MAC    0x0189
+#define PCI_DEVICE_ID_NVIDIA_QUADRO4_280_NVS    0x018A
+#define PCI_DEVICE_ID_NVIDIA_QUADRO4_380_XGL    0x018B
+#define PCI_DEVICE_ID_NVIDIA_IGEFORCE2		0x01a0
+#define PCI_DEVICE_ID_NVIDIA_NFORCE		0x01a4
+#define PCI_DEVICE_ID_NVIDIA_MCP1_AUDIO		0x01b1
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_SMBUS	0x01b4
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_IDE		0x01bc
+#define PCI_DEVICE_ID_NVIDIA_MCP1_MODEM		0x01c1
+#define PCI_DEVICE_ID_NVIDIA_NFORCE2		0x01e0
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE3		0x0200
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE3_1		0x0201
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE3_2		0x0202
+#define PCI_DEVICE_ID_NVIDIA_QUADRO_DDC		0x0203
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_6800B      0x0211
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_6800B_LE   0x0212
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_6800B_GT   0x0215
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_TI_4600	0x0250
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_TI_4400	0x0251
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_TI_4200	0x0253
+#define PCI_DEVICE_ID_NVIDIA_QUADRO4_900XGL	0x0258
+#define PCI_DEVICE_ID_NVIDIA_QUADRO4_750XGL	0x0259
+#define PCI_DEVICE_ID_NVIDIA_QUADRO4_700XGL	0x025B
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP51_SMBUS	0x0264
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP51_IDE	0x0265
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP51_SATA	0x0266
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP51_SATA2	0x0267
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP55_SMBUS	0x0368
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP55_IDE	0x036E
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP55_SATA	0x037E
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP55_SATA2	0x037F
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_TI_4800	0x0280
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_TI_4800_8X    0x0281
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_TI_4800SE     0x0282
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE4_4200_GO       0x0286
+#define PCI_DEVICE_ID_NVIDIA_QUADRO4_980_XGL        0x0288
+#define PCI_DEVICE_ID_NVIDIA_QUADRO4_780_XGL        0x0289
+#define PCI_DEVICE_ID_NVIDIA_QUADRO4_700_GOGL       0x028C
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_5800_ULTRA  0x0301
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_5800        0x0302
+#define PCI_DEVICE_ID_NVIDIA_QUADRO_FX_2000         0x0308
+#define PCI_DEVICE_ID_NVIDIA_QUADRO_FX_1000         0x0309
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_5600_ULTRA  0x0311
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_5600        0x0312
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_5600SE      0x0314
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_GO5600      0x031A
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_GO5650      0x031B
+#define PCI_DEVICE_ID_NVIDIA_QUADRO_FX_GO700        0x031C
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_5200        0x0320
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_5200_ULTRA  0x0321
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_5200_1      0x0322
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_5200SE      0x0323
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_GO5200      0x0324
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_GO5250      0x0325
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_5500        0x0326
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_5100        0x0327
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_GO5250_32   0x0328
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_GO_5200	    0x0329
+#define PCI_DEVICE_ID_NVIDIA_QUADRO_NVS_280_PCI     0x032A
+#define PCI_DEVICE_ID_NVIDIA_QUADRO_FX_500          0x032B
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_GO5300      0x032C
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_GO5100      0x032D
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_5900_ULTRA  0x0330
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_5900        0x0331
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_5900XT      0x0332
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_5950_ULTRA  0x0333
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_5900ZT      0x0334
+#define PCI_DEVICE_ID_NVIDIA_QUADRO_FX_3000         0x0338
+#define PCI_DEVICE_ID_NVIDIA_QUADRO_FX_700          0x033F
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_5700_ULTRA  0x0341
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_5700        0x0342
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_5700LE      0x0343
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_5700VE      0x0344
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_GO5700_1    0x0347
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_FX_GO5700_2    0x0348
+#define PCI_DEVICE_ID_NVIDIA_QUADRO_FX_GO1000       0x034C
+#define PCI_DEVICE_ID_NVIDIA_QUADRO_FX_1100         0x034E
+#define PCI_DEVICE_ID_NVIDIA_MCP55_BRIDGE_V0	    0x0360
+#define PCI_DEVICE_ID_NVIDIA_MCP55_BRIDGE_V4	    0x0364
+#define PCI_DEVICE_ID_NVIDIA_NVENET_15              0x0373
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP61_SATA      0x03E7
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP61_SMBUS	    0x03EB
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP61_IDE       0x03EC
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP61_SATA2     0x03F6
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP61_SATA3     0x03F7
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP65_SMBUS	    0x0446
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP65_IDE	    0x0448
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP67_SMBUS     0x0542
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP67_IDE       0x0560
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP73_IDE       0x056C
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP78S_SMBUS    0x0752
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP77_IDE       0x0759
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP73_SMBUS     0x07D8
+#define PCI_DEVICE_ID_NVIDIA_GEFORCE_320M           0x08A0
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP79_SMBUS     0x0AA2
+#define PCI_DEVICE_ID_NVIDIA_NFORCE_MCP89_SATA	    0x0D85
+
+#define PCI_VENDOR_ID_IMS		0x10e0
+#define PCI_DEVICE_ID_IMS_TT128		0x9128
+#define PCI_DEVICE_ID_IMS_TT3D		0x9135
+
+#define PCI_VENDOR_ID_AMCC		0x10e8
+#define PCI_VENDOR_ID_AMPERE		0x1def
+
+#define PCI_VENDOR_ID_INTERG		0x10ea
+#define PCI_DEVICE_ID_INTERG_1682	0x1682
+#define PCI_DEVICE_ID_INTERG_2000	0x2000
+#define PCI_DEVICE_ID_INTERG_2010	0x2010
+#define PCI_DEVICE_ID_INTERG_5000	0x5000
+#define PCI_DEVICE_ID_INTERG_5050	0x5050
+
+#define PCI_VENDOR_ID_REALTEK		0x10ec
+#define PCI_DEVICE_ID_REALTEK_8139	0x8139
+
+#define PCI_VENDOR_ID_XILINX		0x10ee
+#define PCI_DEVICE_ID_RME_DIGI96	0x3fc0
+#define PCI_DEVICE_ID_RME_DIGI96_8	0x3fc1
+#define PCI_DEVICE_ID_RME_DIGI96_8_PRO	0x3fc2
+#define PCI_DEVICE_ID_RME_DIGI96_8_PAD_OR_PST 0x3fc3
+#define PCI_DEVICE_ID_XILINX_HAMMERFALL_DSP 0x3fc5
+#define PCI_DEVICE_ID_XILINX_HAMMERFALL_DSP_MADI 0x3fc6
+
+#define PCI_VENDOR_ID_INIT		0x1101
+
+#define PCI_VENDOR_ID_CREATIVE		0x1102 /* duplicate: ECTIVA */
+#define PCI_DEVICE_ID_CREATIVE_EMU10K1	0x0002
+#define PCI_DEVICE_ID_CREATIVE_20K1	0x0005
+#define PCI_DEVICE_ID_CREATIVE_20K2	0x000b
+#define PCI_SUBDEVICE_ID_CREATIVE_SB0760	0x0024
+#define PCI_SUBDEVICE_ID_CREATIVE_SB08801	0x0041
+#define PCI_SUBDEVICE_ID_CREATIVE_SB08802	0x0042
+#define PCI_SUBDEVICE_ID_CREATIVE_SB08803	0x0043
+#define PCI_SUBDEVICE_ID_CREATIVE_SB1270	0x0062
+#define PCI_SUBDEVICE_ID_CREATIVE_HENDRIX	0x6000
+
+#define PCI_VENDOR_ID_ECTIVA		0x1102 /* duplicate: CREATIVE */
+#define PCI_DEVICE_ID_ECTIVA_EV1938	0x8938
+
+#define PCI_VENDOR_ID_TTI		0x1103
+#define PCI_DEVICE_ID_TTI_HPT343	0x0003
+#define PCI_DEVICE_ID_TTI_HPT366	0x0004
+#define PCI_DEVICE_ID_TTI_HPT372	0x0005
+#define PCI_DEVICE_ID_TTI_HPT302	0x0006
+#define PCI_DEVICE_ID_TTI_HPT371	0x0007
+#define PCI_DEVICE_ID_TTI_HPT374	0x0008
+#define PCI_DEVICE_ID_TTI_HPT372N	0x0009	/* apparently a 372N variant? */
+
+#define PCI_VENDOR_ID_SIGMA		0x1105
+
+#define PCI_VENDOR_ID_VIA		0x1106
+#define PCI_DEVICE_ID_VIA_8763_0	0x0198
+#define PCI_DEVICE_ID_VIA_8380_0	0x0204
+#define PCI_DEVICE_ID_VIA_3238_0	0x0238
+#define PCI_DEVICE_ID_VIA_PT880		0x0258
+#define PCI_DEVICE_ID_VIA_PT880ULTRA	0x0308
+#define PCI_DEVICE_ID_VIA_PX8X0_0	0x0259
+#define PCI_DEVICE_ID_VIA_3269_0	0x0269
+#define PCI_DEVICE_ID_VIA_K8T800PRO_0	0x0282
+#define PCI_DEVICE_ID_VIA_3296_0	0x0296
+#define PCI_DEVICE_ID_VIA_8363_0	0x0305
+#define PCI_DEVICE_ID_VIA_P4M800CE	0x0314
+#define PCI_DEVICE_ID_VIA_P4M890	0x0327
+#define PCI_DEVICE_ID_VIA_VT3324	0x0324
+#define PCI_DEVICE_ID_VIA_VT3336	0x0336
+#define PCI_DEVICE_ID_VIA_VT3351	0x0351
+#define PCI_DEVICE_ID_VIA_VT3364	0x0364
+#define PCI_DEVICE_ID_VIA_8371_0	0x0391
+#define PCI_DEVICE_ID_VIA_6415		0x0415
+#define PCI_DEVICE_ID_VIA_8501_0	0x0501
+#define PCI_DEVICE_ID_VIA_82C561	0x0561
+#define PCI_DEVICE_ID_VIA_82C586_1	0x0571
+#define PCI_DEVICE_ID_VIA_82C576	0x0576
+#define PCI_DEVICE_ID_VIA_82C586_0	0x0586
+#define PCI_DEVICE_ID_VIA_82C596	0x0596
+#define PCI_DEVICE_ID_VIA_82C597_0	0x0597
+#define PCI_DEVICE_ID_VIA_82C598_0	0x0598
+#define PCI_DEVICE_ID_VIA_8601_0	0x0601
+#define PCI_DEVICE_ID_VIA_8605_0	0x0605
+#define PCI_DEVICE_ID_VIA_82C686	0x0686
+#define PCI_DEVICE_ID_VIA_82C691_0	0x0691
+#define PCI_DEVICE_ID_VIA_82C576_1	0x1571
+#define PCI_DEVICE_ID_VIA_82C586_2	0x3038
+#define PCI_DEVICE_ID_VIA_82C586_3	0x3040
+#define PCI_DEVICE_ID_VIA_82C596_3	0x3050
+#define PCI_DEVICE_ID_VIA_82C596B_3	0x3051
+#define PCI_DEVICE_ID_VIA_82C686_4	0x3057
+#define PCI_DEVICE_ID_VIA_82C686_5	0x3058
+#define PCI_DEVICE_ID_VIA_8233_5	0x3059
+#define PCI_DEVICE_ID_VIA_8233_0	0x3074
+#define PCI_DEVICE_ID_VIA_8633_0	0x3091
+#define PCI_DEVICE_ID_VIA_8367_0	0x3099
+#define PCI_DEVICE_ID_VIA_8653_0	0x3101
+#define PCI_DEVICE_ID_VIA_8622		0x3102
+#define PCI_DEVICE_ID_VIA_8235_USB_2	0x3104
+#define PCI_DEVICE_ID_VIA_8233C_0	0x3109
+#define PCI_DEVICE_ID_VIA_8361		0x3112
+#define PCI_DEVICE_ID_VIA_XM266		0x3116
+#define PCI_DEVICE_ID_VIA_612X		0x3119
+#define PCI_DEVICE_ID_VIA_862X_0	0x3123
+#define PCI_DEVICE_ID_VIA_8753_0	0x3128
+#define PCI_DEVICE_ID_VIA_8233A		0x3147
+#define PCI_DEVICE_ID_VIA_8703_51_0	0x3148
+#define PCI_DEVICE_ID_VIA_8237_SATA	0x3149
+#define PCI_DEVICE_ID_VIA_XN266		0x3156
+#define PCI_DEVICE_ID_VIA_6410		0x3164
+#define PCI_DEVICE_ID_VIA_8754C_0	0x3168
+#define PCI_DEVICE_ID_VIA_8235		0x3177
+#define PCI_DEVICE_ID_VIA_8385_0	0x3188
+#define PCI_DEVICE_ID_VIA_8377_0	0x3189
+#define PCI_DEVICE_ID_VIA_8378_0	0x3205
+#define PCI_DEVICE_ID_VIA_8783_0	0x3208
+#define PCI_DEVICE_ID_VIA_8237		0x3227
+#define PCI_DEVICE_ID_VIA_8251		0x3287
+#define PCI_DEVICE_ID_VIA_8261		0x3402
+#define PCI_DEVICE_ID_VIA_8237A		0x3337
+#define PCI_DEVICE_ID_VIA_8237S		0x3372
+#define PCI_DEVICE_ID_VIA_SATA_EIDE	0x5324
+#define PCI_DEVICE_ID_VIA_8231		0x8231
+#define PCI_DEVICE_ID_VIA_8231_4	0x8235
+#define PCI_DEVICE_ID_VIA_8365_1	0x8305
+#define PCI_DEVICE_ID_VIA_CX700		0x8324
+#define PCI_DEVICE_ID_VIA_CX700_IDE	0x0581
+#define PCI_DEVICE_ID_VIA_VX800		0x8353
+#define PCI_DEVICE_ID_VIA_VX855		0x8409
+#define PCI_DEVICE_ID_VIA_VX900		0x8410
+#define PCI_DEVICE_ID_VIA_8371_1	0x8391
+#define PCI_DEVICE_ID_VIA_82C598_1	0x8598
+#define PCI_DEVICE_ID_VIA_838X_1	0xB188
+#define PCI_DEVICE_ID_VIA_83_87XX_1	0xB198
+#define PCI_DEVICE_ID_VIA_VX855_IDE	0xC409
+#define PCI_DEVICE_ID_VIA_ANON		0xFFFF
+
+#define PCI_VENDOR_ID_SIEMENS           0x110A
+#define PCI_DEVICE_ID_SIEMENS_DSCC4     0x2102
+
+#define PCI_VENDOR_ID_VORTEX		0x1119
+#define PCI_DEVICE_ID_VORTEX_GDT60x0	0x0000
+#define PCI_DEVICE_ID_VORTEX_GDT6000B	0x0001
+#define PCI_DEVICE_ID_VORTEX_GDT6x10	0x0002
+#define PCI_DEVICE_ID_VORTEX_GDT6x20	0x0003
+#define PCI_DEVICE_ID_VORTEX_GDT6530	0x0004
+#define PCI_DEVICE_ID_VORTEX_GDT6550	0x0005
+#define PCI_DEVICE_ID_VORTEX_GDT6x17	0x0006
+#define PCI_DEVICE_ID_VORTEX_GDT6x27	0x0007
+#define PCI_DEVICE_ID_VORTEX_GDT6537	0x0008
+#define PCI_DEVICE_ID_VORTEX_GDT6557	0x0009
+#define PCI_DEVICE_ID_VORTEX_GDT6x15	0x000a
+#define PCI_DEVICE_ID_VORTEX_GDT6x25	0x000b
+#define PCI_DEVICE_ID_VORTEX_GDT6535	0x000c
+#define PCI_DEVICE_ID_VORTEX_GDT6555	0x000d
+#define PCI_DEVICE_ID_VORTEX_GDT6x17RP	0x0100
+#define PCI_DEVICE_ID_VORTEX_GDT6x27RP	0x0101
+#define PCI_DEVICE_ID_VORTEX_GDT6537RP	0x0102
+#define PCI_DEVICE_ID_VORTEX_GDT6557RP	0x0103
+#define PCI_DEVICE_ID_VORTEX_GDT6x11RP	0x0104
+#define PCI_DEVICE_ID_VORTEX_GDT6x21RP	0x0105
+
+#define PCI_VENDOR_ID_EF		0x111a
+#define PCI_DEVICE_ID_EF_ATM_FPGA	0x0000
+#define PCI_DEVICE_ID_EF_ATM_ASIC	0x0002
+#define PCI_DEVICE_ID_EF_ATM_LANAI2	0x0003
+#define PCI_DEVICE_ID_EF_ATM_LANAIHB	0x0005
+
+#define PCI_VENDOR_ID_IDT		0x111d
+#define PCI_DEVICE_ID_IDT_IDT77201	0x0001
+
+#define PCI_VENDOR_ID_FORE		0x1127
+#define PCI_DEVICE_ID_FORE_PCA200E	0x0300
+
+#define PCI_VENDOR_ID_PHILIPS		0x1131
+#define PCI_DEVICE_ID_PHILIPS_SAA7146	0x7146
+#define PCI_DEVICE_ID_PHILIPS_SAA9730	0x9730
+
+#define PCI_VENDOR_ID_EICON		0x1133
+#define PCI_DEVICE_ID_EICON_DIVA20	0xe002
+#define PCI_DEVICE_ID_EICON_DIVA20_U	0xe004
+#define PCI_DEVICE_ID_EICON_DIVA201	0xe005
+#define PCI_DEVICE_ID_EICON_DIVA202	0xe00b
+#define PCI_DEVICE_ID_EICON_MAESTRA	0xe010
+#define PCI_DEVICE_ID_EICON_MAESTRAQ	0xe012
+#define PCI_DEVICE_ID_EICON_MAESTRAQ_U	0xe013
+#define PCI_DEVICE_ID_EICON_MAESTRAP	0xe014
+
+#define PCI_VENDOR_ID_CISCO		0x1137
+
+#define PCI_VENDOR_ID_ZIATECH		0x1138
+#define PCI_DEVICE_ID_ZIATECH_5550_HC	0x5550
+
+
+#define PCI_VENDOR_ID_SYSKONNECT	0x1148
+#define PCI_DEVICE_ID_SYSKONNECT_TR	0x4200
+#define PCI_DEVICE_ID_SYSKONNECT_GE	0x4300
+#define PCI_DEVICE_ID_SYSKONNECT_YU	0x4320
+#define PCI_DEVICE_ID_SYSKONNECT_9DXX	0x4400
+#define PCI_DEVICE_ID_SYSKONNECT_9MXX	0x4500
+
+#define PCI_VENDOR_ID_DIGI		0x114f
+#define PCI_DEVICE_ID_DIGI_DF_M_IOM2_E	0x0070
+#define PCI_DEVICE_ID_DIGI_DF_M_E	0x0071
+#define PCI_DEVICE_ID_DIGI_DF_M_IOM2_A	0x0072
+#define PCI_DEVICE_ID_DIGI_DF_M_A	0x0073
+#define PCI_DEVICE_ID_DIGI_NEO_8	0x00B1
+#define PCI_DEVICE_ID_NEO_2DB9          0x00C8
+#define PCI_DEVICE_ID_NEO_2DB9PRI       0x00C9
+#define PCI_DEVICE_ID_NEO_2RJ45         0x00CA
+#define PCI_DEVICE_ID_NEO_2RJ45PRI      0x00CB
+#define PCIE_DEVICE_ID_NEO_4_IBM        0x00F4
+
+#define PCI_VENDOR_ID_XIRCOM		0x115d
+#define PCI_DEVICE_ID_XIRCOM_RBM56G	0x0101
+#define PCI_DEVICE_ID_XIRCOM_X3201_MDM	0x0103
+
+#define PCI_VENDOR_ID_SERVERWORKS	  0x1166
+#define PCI_DEVICE_ID_SERVERWORKS_HE	  0x0008
+#define PCI_DEVICE_ID_SERVERWORKS_LE	  0x0009
+#define PCI_DEVICE_ID_SERVERWORKS_GCNB_LE 0x0017
+#define PCI_DEVICE_ID_SERVERWORKS_HT1000_PXB	0x0036
+#define PCI_DEVICE_ID_SERVERWORKS_EPB	  0x0103
+#define PCI_DEVICE_ID_SERVERWORKS_HT2000_PCIE	0x0132
+#define PCI_DEVICE_ID_SERVERWORKS_OSB4	  0x0200
+#define PCI_DEVICE_ID_SERVERWORKS_CSB5	  0x0201
+#define PCI_DEVICE_ID_SERVERWORKS_CSB6    0x0203
+#define PCI_DEVICE_ID_SERVERWORKS_HT1000SB 0x0205
+#define PCI_DEVICE_ID_SERVERWORKS_OSB4IDE 0x0211
+#define PCI_DEVICE_ID_SERVERWORKS_CSB5IDE 0x0212
+#define PCI_DEVICE_ID_SERVERWORKS_CSB6IDE 0x0213
+#define PCI_DEVICE_ID_SERVERWORKS_HT1000IDE 0x0214
+#define PCI_DEVICE_ID_SERVERWORKS_CSB6IDE2 0x0217
+#define PCI_DEVICE_ID_SERVERWORKS_CSB6LPC 0x0227
+#define PCI_DEVICE_ID_SERVERWORKS_HT1100LD 0x0408
+
+#define PCI_VENDOR_ID_ALTERA		0x1172
+
+#define PCI_VENDOR_ID_SBE		0x1176
+#define PCI_DEVICE_ID_SBE_WANXL100	0x0301
+#define PCI_DEVICE_ID_SBE_WANXL200	0x0302
+#define PCI_DEVICE_ID_SBE_WANXL400	0x0104
+#define PCI_SUBDEVICE_ID_SBE_T3E3	0x0009
+#define PCI_SUBDEVICE_ID_SBE_2T3E3_P0	0x0901
+#define PCI_SUBDEVICE_ID_SBE_2T3E3_P1	0x0902
+
+#define PCI_VENDOR_ID_TOSHIBA		0x1179
+#define PCI_DEVICE_ID_TOSHIBA_PICCOLO_1	0x0101
+#define PCI_DEVICE_ID_TOSHIBA_PICCOLO_2	0x0102
+#define PCI_DEVICE_ID_TOSHIBA_PICCOLO_3	0x0103
+#define PCI_DEVICE_ID_TOSHIBA_PICCOLO_5	0x0105
+#define PCI_DEVICE_ID_TOSHIBA_TOPIC95	0x060a
+#define PCI_DEVICE_ID_TOSHIBA_TOPIC97	0x060f
+#define PCI_DEVICE_ID_TOSHIBA_TOPIC100	0x0617
+
+#define PCI_VENDOR_ID_TOSHIBA_2		0x102f
+#define PCI_DEVICE_ID_TOSHIBA_TC35815CF	0x0030
+#define PCI_DEVICE_ID_TOSHIBA_TC35815_NWU	0x0031
+#define PCI_DEVICE_ID_TOSHIBA_TC35815_TX4939	0x0032
+#define PCI_DEVICE_ID_TOSHIBA_TC86C001_IDE	0x0105
+#define PCI_DEVICE_ID_TOSHIBA_TC86C001_MISC	0x0108
+#define PCI_DEVICE_ID_TOSHIBA_SPIDER_NET 0x01b3
+
+#define PCI_VENDOR_ID_ATTO		0x117c
+
+#define PCI_VENDOR_ID_RICOH		0x1180
+#define PCI_DEVICE_ID_RICOH_RL5C465	0x0465
+#define PCI_DEVICE_ID_RICOH_RL5C466	0x0466
+#define PCI_DEVICE_ID_RICOH_RL5C475	0x0475
+#define PCI_DEVICE_ID_RICOH_RL5C476	0x0476
+#define PCI_DEVICE_ID_RICOH_RL5C478	0x0478
+#define PCI_DEVICE_ID_RICOH_R5C822	0x0822
+#define PCI_DEVICE_ID_RICOH_R5CE822	0xe822
+#define PCI_DEVICE_ID_RICOH_R5CE823	0xe823
+#define PCI_DEVICE_ID_RICOH_R5C832	0x0832
+#define PCI_DEVICE_ID_RICOH_R5C843	0x0843
+
+#define PCI_VENDOR_ID_DLINK		0x1186
+#define PCI_DEVICE_ID_DLINK_DGE510T	0x4c00
+
+#define PCI_VENDOR_ID_ARTOP		0x1191
+#define PCI_DEVICE_ID_ARTOP_ATP850UF	0x0005
+#define PCI_DEVICE_ID_ARTOP_ATP860	0x0006
+#define PCI_DEVICE_ID_ARTOP_ATP860R	0x0007
+#define PCI_DEVICE_ID_ARTOP_ATP865	0x0008
+#define PCI_DEVICE_ID_ARTOP_ATP865R	0x0009
+#define PCI_DEVICE_ID_ARTOP_ATP867A	0x000A
+#define PCI_DEVICE_ID_ARTOP_ATP867B	0x000B
+#define PCI_DEVICE_ID_ARTOP_AEC7610	0x8002
+#define PCI_DEVICE_ID_ARTOP_AEC7612UW	0x8010
+#define PCI_DEVICE_ID_ARTOP_AEC7612U	0x8020
+#define PCI_DEVICE_ID_ARTOP_AEC7612S	0x8030
+#define PCI_DEVICE_ID_ARTOP_AEC7612D	0x8040
+#define PCI_DEVICE_ID_ARTOP_AEC7612SUW	0x8050
+#define PCI_DEVICE_ID_ARTOP_8060	0x8060
+
+#define PCI_VENDOR_ID_ZEITNET		0x1193
+#define PCI_DEVICE_ID_ZEITNET_1221	0x0001
+#define PCI_DEVICE_ID_ZEITNET_1225	0x0002
+
+#define PCI_VENDOR_ID_FUJITSU_ME	0x119e
+#define PCI_DEVICE_ID_FUJITSU_FS155	0x0001
+#define PCI_DEVICE_ID_FUJITSU_FS50	0x0003
+
+#define PCI_SUBVENDOR_ID_KEYSPAN	0x11a9
+#define PCI_SUBDEVICE_ID_KEYSPAN_SX2	0x5334
+
+#define PCI_VENDOR_ID_MARVELL		0x11ab
+#define PCI_VENDOR_ID_MARVELL_EXT	0x1b4b
+#define PCI_DEVICE_ID_MARVELL_GT64111	0x4146
+#define PCI_DEVICE_ID_MARVELL_GT64260	0x6430
+#define PCI_DEVICE_ID_MARVELL_MV64360	0x6460
+#define PCI_DEVICE_ID_MARVELL_MV64460	0x6480
+#define PCI_DEVICE_ID_MARVELL_88ALP01_NAND	0x4100
+#define PCI_DEVICE_ID_MARVELL_88ALP01_SD	0x4101
+#define PCI_DEVICE_ID_MARVELL_88ALP01_CCIC	0x4102
+
+#define PCI_VENDOR_ID_V3		0x11b0
+#define PCI_DEVICE_ID_V3_V960		0x0001
+#define PCI_DEVICE_ID_V3_V351		0x0002
+
+#define PCI_VENDOR_ID_ATT		0x11c1
+#define PCI_DEVICE_ID_ATT_VENUS_MODEM	0x480
+
+#define PCI_VENDOR_ID_SPECIALIX		0x11cb
+#define PCI_SUBDEVICE_ID_SPECIALIX_SPEED4 0xa004
+
+#define PCI_VENDOR_ID_ANALOG_DEVICES	0x11d4
+#define PCI_DEVICE_ID_AD1889JS		0x1889
+
+#define PCI_DEVICE_ID_SEGA_BBA		0x1234
+
+#define PCI_VENDOR_ID_ZORAN		0x11de
+#define PCI_DEVICE_ID_ZORAN_36057	0x6057
+#define PCI_DEVICE_ID_ZORAN_36120	0x6120
+
+#define PCI_VENDOR_ID_COMPEX		0x11f6
+#define PCI_DEVICE_ID_COMPEX_ENET100VG4	0x0112
+
+#define PCI_VENDOR_ID_PMC_Sierra	0x11f8
+#define PCI_VENDOR_ID_MICROSEMI		0x11f8
+
+#define PCI_VENDOR_ID_RP		0x11fe
+
+#define PCI_VENDOR_ID_CYCLADES		0x120e
+#define PCI_DEVICE_ID_PC300_RX_2	0x0300
+#define PCI_DEVICE_ID_PC300_RX_1	0x0301
+#define PCI_DEVICE_ID_PC300_TE_2	0x0310
+#define PCI_DEVICE_ID_PC300_TE_1	0x0311
+#define PCI_DEVICE_ID_PC300_TE_M_2	0x0320
+#define PCI_DEVICE_ID_PC300_TE_M_1	0x0321
+
+#define PCI_VENDOR_ID_ESSENTIAL		0x120f
+#define PCI_DEVICE_ID_ESSENTIAL_ROADRUNNER	0x0001
+
+#define PCI_VENDOR_ID_O2		0x1217
+#define PCI_DEVICE_ID_O2_6729		0x6729
+#define PCI_DEVICE_ID_O2_6730		0x673a
+#define PCI_DEVICE_ID_O2_6832		0x6832
+#define PCI_DEVICE_ID_O2_6836		0x6836
+#define PCI_DEVICE_ID_O2_6812		0x6872
+#define PCI_DEVICE_ID_O2_6933		0x6933
+#define PCI_DEVICE_ID_O2_8120		0x8120
+#define PCI_DEVICE_ID_O2_8220		0x8220
+#define PCI_DEVICE_ID_O2_8221		0x8221
+#define PCI_DEVICE_ID_O2_8320		0x8320
+#define PCI_DEVICE_ID_O2_8321		0x8321
+
+#define PCI_VENDOR_ID_3DFX		0x121a
+#define PCI_DEVICE_ID_3DFX_VOODOO	0x0001
+#define PCI_DEVICE_ID_3DFX_VOODOO2	0x0002
+#define PCI_DEVICE_ID_3DFX_BANSHEE	0x0003
+#define PCI_DEVICE_ID_3DFX_VOODOO3	0x0005
+#define PCI_DEVICE_ID_3DFX_VOODOO5	0x0009
+
+#define PCI_VENDOR_ID_AVM		0x1244
+#define PCI_DEVICE_ID_AVM_B1		0x0700
+#define PCI_DEVICE_ID_AVM_C4		0x0800
+#define PCI_DEVICE_ID_AVM_A1		0x0a00
+#define PCI_DEVICE_ID_AVM_A1_V2		0x0e00
+#define PCI_DEVICE_ID_AVM_C2		0x1100
+#define PCI_DEVICE_ID_AVM_T1		0x1200
+
+#define PCI_VENDOR_ID_STALLION		0x124d
+
+/* Allied Telesyn */
+#define PCI_VENDOR_ID_AT		0x1259
+#define PCI_SUBDEVICE_ID_AT_2700FX	0x2701
+#define PCI_SUBDEVICE_ID_AT_2701FX	0x2703
+
+#define PCI_VENDOR_ID_ASIX		0x125b
+#define PCI_DEVICE_ID_ASIX_AX99100	0x9100
+#define PCI_DEVICE_ID_ASIX_AX99100_LB	0x9110
+
+#define PCI_VENDOR_ID_ESS		0x125d
+#define PCI_DEVICE_ID_ESS_ESS1968	0x1968
+#define PCI_DEVICE_ID_ESS_ESS1978	0x1978
+#define PCI_DEVICE_ID_ESS_ALLEGRO_1	0x1988
+#define PCI_DEVICE_ID_ESS_ALLEGRO	0x1989
+#define PCI_DEVICE_ID_ESS_CANYON3D_2LE	0x1990
+#define PCI_DEVICE_ID_ESS_CANYON3D_2	0x1992
+#define PCI_DEVICE_ID_ESS_MAESTRO3	0x1998
+#define PCI_DEVICE_ID_ESS_MAESTRO3_1	0x1999
+#define PCI_DEVICE_ID_ESS_MAESTRO3_HW	0x199a
+#define PCI_DEVICE_ID_ESS_MAESTRO3_2	0x199b
+
+#define PCI_VENDOR_ID_SATSAGEM		0x1267
+#define PCI_DEVICE_ID_SATSAGEM_NICCY	0x1016
+
+#define PCI_VENDOR_ID_ENSONIQ		0x1274
+#define PCI_DEVICE_ID_ENSONIQ_CT5880	0x5880
+#define PCI_DEVICE_ID_ENSONIQ_ES1370	0x5000
+#define PCI_DEVICE_ID_ENSONIQ_ES1371	0x1371
+
+#define PCI_VENDOR_ID_TRANSMETA		0x1279
+#define PCI_DEVICE_ID_EFFICEON		0x0060
+
+#define PCI_VENDOR_ID_ROCKWELL		0x127A
+
+#define PCI_VENDOR_ID_ITE		0x1283
+#define PCI_DEVICE_ID_ITE_8172		0x8172
+#define PCI_DEVICE_ID_ITE_8211		0x8211
+#define PCI_DEVICE_ID_ITE_8212		0x8212
+#define PCI_DEVICE_ID_ITE_8213		0x8213
+#define PCI_DEVICE_ID_ITE_8152		0x8152
+#define PCI_DEVICE_ID_ITE_8872		0x8872
+#define PCI_DEVICE_ID_ITE_IT8330G_0	0xe886
+
+/* formerly Platform Tech */
+#define PCI_DEVICE_ID_ESS_ESS0100	0x0100
+
+#define PCI_VENDOR_ID_ALTEON		0x12ae
+
+#define PCI_SUBVENDOR_ID_CONNECT_TECH			0x12c4
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_BH8_232		0x0001
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_BH4_232		0x0002
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_BH2_232		0x0003
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_BH8_485		0x0004
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_BH8_485_4_4	0x0005
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_BH4_485		0x0006
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_BH4_485_2_2	0x0007
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_BH2_485		0x0008
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_BH8_485_2_6	0x0009
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_BH081101V1	0x000A
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_BH041101V1	0x000B
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_BH2_20MHZ		0x000C
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_BH2_PTM		0x000D
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_NT960PCI		0x0100
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_TITAN_2		0x0201
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_TITAN_4		0x0202
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_PCI_UART_2_232	0x0300
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_PCI_UART_4_232	0x0301
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_PCI_UART_8_232	0x0302
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_PCI_UART_1_1	0x0310
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_PCI_UART_2_2	0x0311
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_PCI_UART_4_4	0x0312
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_PCI_UART_2	0x0320
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_PCI_UART_4	0x0321
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_PCI_UART_8	0x0322
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_PCI_UART_2_485	0x0330
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_PCI_UART_4_485	0x0331
+#define PCI_SUBDEVICE_ID_CONNECT_TECH_PCI_UART_8_485	0x0332
+
+#define PCI_VENDOR_ID_NVIDIA_SGS	0x12d2
+#define PCI_DEVICE_ID_NVIDIA_SGS_RIVA128 0x0018
+
+#define PCI_VENDOR_ID_PERICOM			0x12D8
+#define PCI_DEVICE_ID_PERICOM_PI7C9X7951	0x7951
+#define PCI_DEVICE_ID_PERICOM_PI7C9X7952	0x7952
+#define PCI_DEVICE_ID_PERICOM_PI7C9X7954	0x7954
+#define PCI_DEVICE_ID_PERICOM_PI7C9X7958	0x7958
+
+#define PCI_SUBVENDOR_ID_CHASE_PCIFAST		0x12E0
+#define PCI_SUBDEVICE_ID_CHASE_PCIFAST4		0x0031
+#define PCI_SUBDEVICE_ID_CHASE_PCIFAST8		0x0021
+#define PCI_SUBDEVICE_ID_CHASE_PCIFAST16	0x0011
+#define PCI_SUBDEVICE_ID_CHASE_PCIFAST16FMC	0x0041
+#define PCI_SUBVENDOR_ID_CHASE_PCIRAS		0x124D
+#define PCI_SUBDEVICE_ID_CHASE_PCIRAS4		0xF001
+#define PCI_SUBDEVICE_ID_CHASE_PCIRAS8		0xF010
+
+#define PCI_VENDOR_ID_AUREAL		0x12eb
+#define PCI_DEVICE_ID_AUREAL_VORTEX_1	0x0001
+#define PCI_DEVICE_ID_AUREAL_VORTEX_2	0x0002
+#define PCI_DEVICE_ID_AUREAL_ADVANTAGE	0x0003
+
+#define PCI_VENDOR_ID_ELECTRONICDESIGNGMBH 0x12f8
+#define PCI_DEVICE_ID_LML_33R10		0x8a02
+
+#define PCI_VENDOR_ID_ESDGMBH		0x12fe
+#define PCI_DEVICE_ID_ESDGMBH_CPCIASIO4 0x0111
+
+#define PCI_VENDOR_ID_CB		0x1307	/* Measurement Computing */
+
+#define PCI_VENDOR_ID_SIIG		0x131f
+#define PCI_SUBVENDOR_ID_SIIG		0x131f
+#define PCI_DEVICE_ID_SIIG_1S_10x_550	0x1000
+#define PCI_DEVICE_ID_SIIG_1S_10x_650	0x1001
+#define PCI_DEVICE_ID_SIIG_1S_10x_850	0x1002
+#define PCI_DEVICE_ID_SIIG_1S1P_10x_550	0x1010
+#define PCI_DEVICE_ID_SIIG_1S1P_10x_650	0x1011
+#define PCI_DEVICE_ID_SIIG_1S1P_10x_850	0x1012
+#define PCI_DEVICE_ID_SIIG_1P_10x	0x1020
+#define PCI_DEVICE_ID_SIIG_2P_10x	0x1021
+#define PCI_DEVICE_ID_SIIG_2S_10x_550	0x1030
+#define PCI_DEVICE_ID_SIIG_2S_10x_650	0x1031
+#define PCI_DEVICE_ID_SIIG_2S_10x_850	0x1032
+#define PCI_DEVICE_ID_SIIG_2S1P_10x_550	0x1034
+#define PCI_DEVICE_ID_SIIG_2S1P_10x_650	0x1035
+#define PCI_DEVICE_ID_SIIG_2S1P_10x_850	0x1036
+#define PCI_DEVICE_ID_SIIG_4S_10x_550	0x1050
+#define PCI_DEVICE_ID_SIIG_4S_10x_650	0x1051
+#define PCI_DEVICE_ID_SIIG_4S_10x_850	0x1052
+#define PCI_DEVICE_ID_SIIG_1S_20x_550	0x2000
+#define PCI_DEVICE_ID_SIIG_1S_20x_650	0x2001
+#define PCI_DEVICE_ID_SIIG_1S_20x_850	0x2002
+#define PCI_DEVICE_ID_SIIG_1P_20x	0x2020
+#define PCI_DEVICE_ID_SIIG_2P_20x	0x2021
+#define PCI_DEVICE_ID_SIIG_2S_20x_550	0x2030
+#define PCI_DEVICE_ID_SIIG_2S_20x_650	0x2031
+#define PCI_DEVICE_ID_SIIG_2S_20x_850	0x2032
+#define PCI_DEVICE_ID_SIIG_2P1S_20x_550	0x2040
+#define PCI_DEVICE_ID_SIIG_2P1S_20x_650	0x2041
+#define PCI_DEVICE_ID_SIIG_2P1S_20x_850	0x2042
+#define PCI_DEVICE_ID_SIIG_1S1P_20x_550	0x2010
+#define PCI_DEVICE_ID_SIIG_1S1P_20x_650	0x2011
+#define PCI_DEVICE_ID_SIIG_1S1P_20x_850	0x2012
+#define PCI_DEVICE_ID_SIIG_4S_20x_550	0x2050
+#define PCI_DEVICE_ID_SIIG_4S_20x_650	0x2051
+#define PCI_DEVICE_ID_SIIG_4S_20x_850	0x2052
+#define PCI_DEVICE_ID_SIIG_2S1P_20x_550	0x2060
+#define PCI_DEVICE_ID_SIIG_2S1P_20x_650	0x2061
+#define PCI_DEVICE_ID_SIIG_2S1P_20x_850	0x2062
+#define PCI_DEVICE_ID_SIIG_8S_20x_550	0x2080
+#define PCI_DEVICE_ID_SIIG_8S_20x_650	0x2081
+#define PCI_DEVICE_ID_SIIG_8S_20x_850	0x2082
+#define PCI_SUBDEVICE_ID_SIIG_QUARTET_SERIAL	0x2050
+
+#define PCI_VENDOR_ID_RADISYS		0x1331
+
+#define PCI_VENDOR_ID_MICRO_MEMORY		0x1332
+#define PCI_DEVICE_ID_MICRO_MEMORY_5415CN	0x5415
+#define PCI_DEVICE_ID_MICRO_MEMORY_5425CN	0x5425
+#define PCI_DEVICE_ID_MICRO_MEMORY_6155		0x6155
+
+#define PCI_VENDOR_ID_DOMEX		0x134a
+#define PCI_DEVICE_ID_DOMEX_DMX3191D	0x0001
+
+#define PCI_VENDOR_ID_INTASHIELD	0x135a
+#define PCI_DEVICE_ID_INTASHIELD_IS200	0x0d80
+#define PCI_DEVICE_ID_INTASHIELD_IS400	0x0dc0
+
+#define PCI_VENDOR_ID_QUATECH		0x135C
+#define PCI_DEVICE_ID_QUATECH_QSC100	0x0010
+#define PCI_DEVICE_ID_QUATECH_DSC100	0x0020
+#define PCI_DEVICE_ID_QUATECH_DSC200	0x0030
+#define PCI_DEVICE_ID_QUATECH_QSC200	0x0040
+#define PCI_DEVICE_ID_QUATECH_ESC100D	0x0050
+#define PCI_DEVICE_ID_QUATECH_ESC100M	0x0060
+#define PCI_DEVICE_ID_QUATECH_QSCP100	0x0120
+#define PCI_DEVICE_ID_QUATECH_DSCP100	0x0130
+#define PCI_DEVICE_ID_QUATECH_QSCP200	0x0140
+#define PCI_DEVICE_ID_QUATECH_DSCP200	0x0150
+#define PCI_DEVICE_ID_QUATECH_QSCLP100	0x0170
+#define PCI_DEVICE_ID_QUATECH_DSCLP100	0x0180
+#define PCI_DEVICE_ID_QUATECH_DSC100E	0x0181
+#define PCI_DEVICE_ID_QUATECH_SSCLP100	0x0190
+#define PCI_DEVICE_ID_QUATECH_QSCLP200	0x01A0
+#define PCI_DEVICE_ID_QUATECH_DSCLP200	0x01B0
+#define PCI_DEVICE_ID_QUATECH_DSC200E	0x01B1
+#define PCI_DEVICE_ID_QUATECH_SSCLP200	0x01C0
+#define PCI_DEVICE_ID_QUATECH_ESCLP100	0x01E0
+#define PCI_DEVICE_ID_QUATECH_SPPXP_100 0x0278
+
+#define PCI_VENDOR_ID_SEALEVEL		0x135e
+#define PCI_DEVICE_ID_SEALEVEL_U530	0x7101
+#define PCI_DEVICE_ID_SEALEVEL_UCOMM2	0x7201
+#define PCI_DEVICE_ID_SEALEVEL_UCOMM422	0x7402
+#define PCI_DEVICE_ID_SEALEVEL_UCOMM232	0x7202
+#define PCI_DEVICE_ID_SEALEVEL_COMM4	0x7401
+#define PCI_DEVICE_ID_SEALEVEL_COMM8	0x7801
+#define PCI_DEVICE_ID_SEALEVEL_7803	0x7803
+#define PCI_DEVICE_ID_SEALEVEL_UCOMM8	0x7804
+
+#define PCI_VENDOR_ID_HYPERCOPE		0x1365
+#define PCI_DEVICE_ID_HYPERCOPE_PLX	0x9050
+#define PCI_SUBDEVICE_ID_HYPERCOPE_OLD_ERGO	0x0104
+#define PCI_SUBDEVICE_ID_HYPERCOPE_ERGO		0x0106
+#define PCI_SUBDEVICE_ID_HYPERCOPE_METRO	0x0107
+#define PCI_SUBDEVICE_ID_HYPERCOPE_CHAMP2	0x0108
+
+#define PCI_VENDOR_ID_DIGIGRAM		0x1369
+#define PCI_SUBDEVICE_ID_DIGIGRAM_LX6464ES_SERIAL_SUBSYSTEM	0xc001
+#define PCI_SUBDEVICE_ID_DIGIGRAM_LX6464ES_CAE_SERIAL_SUBSYSTEM	0xc002
+#define PCI_SUBDEVICE_ID_DIGIGRAM_LX6464ESE_SERIAL_SUBSYSTEM		0xc021
+#define PCI_SUBDEVICE_ID_DIGIGRAM_LX6464ESE_CAE_SERIAL_SUBSYSTEM	0xc022
+
+#define PCI_VENDOR_ID_KAWASAKI		0x136b
+#define PCI_DEVICE_ID_MCHIP_KL5A72002	0xff01
+
+#define PCI_VENDOR_ID_CNET		0x1371
+#define PCI_DEVICE_ID_CNET_GIGACARD	0x434e
+
+#define PCI_VENDOR_ID_LMC		0x1376
+#define PCI_DEVICE_ID_LMC_HSSI		0x0003
+#define PCI_DEVICE_ID_LMC_DS3		0x0004
+#define PCI_DEVICE_ID_LMC_SSI		0x0005
+#define PCI_DEVICE_ID_LMC_T1		0x0006
+
+#define PCI_VENDOR_ID_NETGEAR		0x1385
+#define PCI_DEVICE_ID_NETGEAR_GA620	0x620a
+
+#define PCI_VENDOR_ID_APPLICOM		0x1389
+#define PCI_DEVICE_ID_APPLICOM_PCIGENERIC 0x0001
+#define PCI_DEVICE_ID_APPLICOM_PCI2000IBS_CAN 0x0002
+#define PCI_DEVICE_ID_APPLICOM_PCI2000PFB 0x0003
+
+#define PCI_VENDOR_ID_MOXA		0x1393
+#define PCI_DEVICE_ID_MOXA_CP204J	0x2040
+#define PCI_DEVICE_ID_MOXA_C218		0x2180
+#define PCI_DEVICE_ID_MOXA_C320		0x3200
+
+#define PCI_VENDOR_ID_CCD		0x1397
+#define PCI_DEVICE_ID_CCD_HFC4S		0x08B4
+#define PCI_SUBDEVICE_ID_CCD_PMX2S	0x1234
+#define PCI_DEVICE_ID_CCD_HFC8S		0x16B8
+#define PCI_DEVICE_ID_CCD_2BD0		0x2bd0
+#define PCI_DEVICE_ID_CCD_HFCE1		0x30B1
+#define PCI_SUBDEVICE_ID_CCD_SPD4S	0x3136
+#define PCI_SUBDEVICE_ID_CCD_SPDE1	0x3137
+#define PCI_DEVICE_ID_CCD_B000		0xb000
+#define PCI_DEVICE_ID_CCD_B006		0xb006
+#define PCI_DEVICE_ID_CCD_B007		0xb007
+#define PCI_DEVICE_ID_CCD_B008		0xb008
+#define PCI_DEVICE_ID_CCD_B009		0xb009
+#define PCI_DEVICE_ID_CCD_B00A		0xb00a
+#define PCI_DEVICE_ID_CCD_B00B		0xb00b
+#define PCI_DEVICE_ID_CCD_B00C		0xb00c
+#define PCI_DEVICE_ID_CCD_B100		0xb100
+#define PCI_SUBDEVICE_ID_CCD_IOB4ST	0xB520
+#define PCI_SUBDEVICE_ID_CCD_IOB8STR	0xB521
+#define PCI_SUBDEVICE_ID_CCD_IOB8ST	0xB522
+#define PCI_SUBDEVICE_ID_CCD_IOB1E1	0xB523
+#define PCI_SUBDEVICE_ID_CCD_SWYX4S	0xB540
+#define PCI_SUBDEVICE_ID_CCD_JH4S20	0xB550
+#define PCI_SUBDEVICE_ID_CCD_IOB8ST_1	0xB552
+#define PCI_SUBDEVICE_ID_CCD_JHSE1	0xB553
+#define PCI_SUBDEVICE_ID_CCD_JH8S	0xB55B
+#define PCI_SUBDEVICE_ID_CCD_BN4S	0xB560
+#define PCI_SUBDEVICE_ID_CCD_BN8S	0xB562
+#define PCI_SUBDEVICE_ID_CCD_BNE1	0xB563
+#define PCI_SUBDEVICE_ID_CCD_BNE1D	0xB564
+#define PCI_SUBDEVICE_ID_CCD_BNE1DP	0xB565
+#define PCI_SUBDEVICE_ID_CCD_BN2S	0xB566
+#define PCI_SUBDEVICE_ID_CCD_BN1SM	0xB567
+#define PCI_SUBDEVICE_ID_CCD_BN4SM	0xB568
+#define PCI_SUBDEVICE_ID_CCD_BN2SM	0xB569
+#define PCI_SUBDEVICE_ID_CCD_BNE1M	0xB56A
+#define PCI_SUBDEVICE_ID_CCD_BN8SP	0xB56B
+#define PCI_SUBDEVICE_ID_CCD_HFC4S	0xB620
+#define PCI_SUBDEVICE_ID_CCD_HFC8S	0xB622
+#define PCI_DEVICE_ID_CCD_B700		0xb700
+#define PCI_DEVICE_ID_CCD_B701		0xb701
+#define PCI_SUBDEVICE_ID_CCD_HFCE1	0xC523
+#define PCI_SUBDEVICE_ID_CCD_OV2S	0xE884
+#define PCI_SUBDEVICE_ID_CCD_OV4S	0xE888
+#define PCI_SUBDEVICE_ID_CCD_OV8S	0xE998
+
+#define PCI_VENDOR_ID_EXAR		0x13a8
+#define PCI_DEVICE_ID_EXAR_XR17C152	0x0152
+#define PCI_DEVICE_ID_EXAR_XR17C154	0x0154
+#define PCI_DEVICE_ID_EXAR_XR17C158	0x0158
+#define PCI_DEVICE_ID_EXAR_XR17V352	0x0352
+#define PCI_DEVICE_ID_EXAR_XR17V354	0x0354
+#define PCI_DEVICE_ID_EXAR_XR17V358	0x0358
+
+#define PCI_VENDOR_ID_MICROGATE		0x13c0
+
+#define PCI_VENDOR_ID_3WARE		0x13C1
+#define PCI_DEVICE_ID_3WARE_1000	0x1000
+#define PCI_DEVICE_ID_3WARE_7000	0x1001
+#define PCI_DEVICE_ID_3WARE_9000	0x1002
+
+#define PCI_VENDOR_ID_IOMEGA		0x13ca
+#define PCI_DEVICE_ID_IOMEGA_BUZ	0x4231
+
+#define PCI_VENDOR_ID_ABOCOM		0x13D1
+#define PCI_DEVICE_ID_ABOCOM_2BD1       0x2BD1
+
+#define PCI_VENDOR_ID_SUNDANCE		0x13f0
+
+#define PCI_VENDOR_ID_CMEDIA		0x13f6
+#define PCI_DEVICE_ID_CMEDIA_CM8338A	0x0100
+#define PCI_DEVICE_ID_CMEDIA_CM8338B	0x0101
+#define PCI_DEVICE_ID_CMEDIA_CM8738	0x0111
+#define PCI_DEVICE_ID_CMEDIA_CM8738B	0x0112
+
+#define PCI_VENDOR_ID_ADVANTECH		0x13fe
+
+#define PCI_VENDOR_ID_MEILHAUS		0x1402
+
+#define PCI_VENDOR_ID_LAVA		0x1407
+#define PCI_DEVICE_ID_LAVA_DSERIAL	0x0100 /* 2x 16550 */
+#define PCI_DEVICE_ID_LAVA_QUATRO_A	0x0101 /* 2x 16550, half of 4 port */
+#define PCI_DEVICE_ID_LAVA_QUATRO_B	0x0102 /* 2x 16550, half of 4 port */
+#define PCI_DEVICE_ID_LAVA_QUATTRO_A	0x0120 /* 2x 16550A, half of 4 port */
+#define PCI_DEVICE_ID_LAVA_QUATTRO_B	0x0121 /* 2x 16550A, half of 4 port */
+#define PCI_DEVICE_ID_LAVA_OCTO_A	0x0180 /* 4x 16550A, half of 8 port */
+#define PCI_DEVICE_ID_LAVA_OCTO_B	0x0181 /* 4x 16550A, half of 8 port */
+#define PCI_DEVICE_ID_LAVA_PORT_PLUS	0x0200 /* 2x 16650 */
+#define PCI_DEVICE_ID_LAVA_QUAD_A	0x0201 /* 2x 16650, half of 4 port */
+#define PCI_DEVICE_ID_LAVA_QUAD_B	0x0202 /* 2x 16650, half of 4 port */
+#define PCI_DEVICE_ID_LAVA_SSERIAL	0x0500 /* 1x 16550 */
+#define PCI_DEVICE_ID_LAVA_PORT_650	0x0600 /* 1x 16650 */
+#define PCI_DEVICE_ID_LAVA_PARALLEL	0x8000
+#define PCI_DEVICE_ID_LAVA_DUAL_PAR_A	0x8002 /* The Lava Dual Parallel is */
+#define PCI_DEVICE_ID_LAVA_DUAL_PAR_B	0x8003 /* two PCI devices on a card */
+#define PCI_DEVICE_ID_LAVA_BOCA_IOPPAR	0x8800
+
+#define PCI_VENDOR_ID_TIMEDIA		0x1409
+#define PCI_DEVICE_ID_TIMEDIA_1889	0x7168
+
+#define PCI_VENDOR_ID_ICE		0x1412
+#define PCI_DEVICE_ID_ICE_1712		0x1712
+#define PCI_DEVICE_ID_VT1724		0x1724
+
+#define PCI_VENDOR_ID_MICROSOFT		0x1414
+#define PCI_DEVICE_ID_HYPERV_VIDEO	0x5353
+
+#define PCI_VENDOR_ID_OXSEMI		0x1415
+#define PCI_DEVICE_ID_OXSEMI_12PCI840	0x8403
+#define PCI_DEVICE_ID_OXSEMI_PCIe840		0xC000
+#define PCI_DEVICE_ID_OXSEMI_PCIe840_G		0xC004
+#define PCI_DEVICE_ID_OXSEMI_PCIe952_0		0xC100
+#define PCI_DEVICE_ID_OXSEMI_PCIe952_0_G	0xC104
+#define PCI_DEVICE_ID_OXSEMI_PCIe952_1		0xC110
+#define PCI_DEVICE_ID_OXSEMI_PCIe952_1_G	0xC114
+#define PCI_DEVICE_ID_OXSEMI_PCIe952_1_U	0xC118
+#define PCI_DEVICE_ID_OXSEMI_PCIe952_1_GU	0xC11C
+#define PCI_DEVICE_ID_OXSEMI_16PCI954	0x9501
+#define PCI_DEVICE_ID_OXSEMI_C950	0x950B
+#define PCI_DEVICE_ID_OXSEMI_16PCI95N	0x9511
+#define PCI_DEVICE_ID_OXSEMI_16PCI954PP	0x9513
+#define PCI_DEVICE_ID_OXSEMI_16PCI952	0x9521
+#define PCI_DEVICE_ID_OXSEMI_16PCI952PP	0x9523
+#define PCI_SUBDEVICE_ID_OXSEMI_C950	0x0001
+
+#define PCI_VENDOR_ID_CHELSIO		0x1425
+
+#define PCI_VENDOR_ID_ADLINK		0x144a
+
+#define PCI_VENDOR_ID_SAMSUNG		0x144d
+
+#define PCI_VENDOR_ID_GIGABYTE		0x1458
+
+#define PCI_VENDOR_ID_AMBIT		0x1468
+
+#define PCI_VENDOR_ID_MYRICOM		0x14c1
+
+#define PCI_VENDOR_ID_MEDIATEK		0x14c3
+#define PCI_DEVICE_ID_MEDIATEK_7629	0x7629
+
+#define PCI_VENDOR_ID_TITAN		0x14D2
+#define PCI_DEVICE_ID_TITAN_010L	0x8001
+#define PCI_DEVICE_ID_TITAN_100L	0x8010
+#define PCI_DEVICE_ID_TITAN_110L	0x8011
+#define PCI_DEVICE_ID_TITAN_200L	0x8020
+#define PCI_DEVICE_ID_TITAN_210L	0x8021
+#define PCI_DEVICE_ID_TITAN_400L	0x8040
+#define PCI_DEVICE_ID_TITAN_800L	0x8080
+#define PCI_DEVICE_ID_TITAN_100		0xA001
+#define PCI_DEVICE_ID_TITAN_200		0xA005
+#define PCI_DEVICE_ID_TITAN_400		0xA003
+#define PCI_DEVICE_ID_TITAN_800B	0xA004
+
+#define PCI_VENDOR_ID_PANACOM		0x14d4
+#define PCI_DEVICE_ID_PANACOM_QUADMODEM	0x0400
+#define PCI_DEVICE_ID_PANACOM_DUALMODEM	0x0402
+
+#define PCI_VENDOR_ID_SIPACKETS		0x14d9
+#define PCI_DEVICE_ID_SP1011		0x0010
+
+#define PCI_VENDOR_ID_AFAVLAB		0x14db
+#define PCI_DEVICE_ID_AFAVLAB_P028	0x2180
+#define PCI_DEVICE_ID_AFAVLAB_P030	0x2182
+#define PCI_SUBDEVICE_ID_AFAVLAB_P061		0x2150
+
+#define PCI_VENDOR_ID_AMPLICON		0x14dc
+
+#define PCI_VENDOR_ID_BCM_GVC          0x14a4
+#define PCI_VENDOR_ID_BROADCOM		0x14e4
+#define PCI_DEVICE_ID_TIGON3_5752	0x1600
+#define PCI_DEVICE_ID_TIGON3_5752M	0x1601
+#define PCI_DEVICE_ID_NX2_5709		0x1639
+#define PCI_DEVICE_ID_NX2_5709S		0x163a
+#define PCI_DEVICE_ID_TIGON3_5700	0x1644
+#define PCI_DEVICE_ID_TIGON3_5701	0x1645
+#define PCI_DEVICE_ID_TIGON3_5702	0x1646
+#define PCI_DEVICE_ID_TIGON3_5703	0x1647
+#define PCI_DEVICE_ID_TIGON3_5704	0x1648
+#define PCI_DEVICE_ID_TIGON3_5704S_2	0x1649
+#define PCI_DEVICE_ID_NX2_5706		0x164a
+#define PCI_DEVICE_ID_NX2_5708		0x164c
+#define PCI_DEVICE_ID_TIGON3_5702FE	0x164d
+#define PCI_DEVICE_ID_NX2_57710		0x164e
+#define PCI_DEVICE_ID_NX2_57711		0x164f
+#define PCI_DEVICE_ID_NX2_57711E	0x1650
+#define PCI_DEVICE_ID_TIGON3_5705	0x1653
+#define PCI_DEVICE_ID_TIGON3_5705_2	0x1654
+#define PCI_DEVICE_ID_TIGON3_5719	0x1657
+#define PCI_DEVICE_ID_TIGON3_5721	0x1659
+#define PCI_DEVICE_ID_TIGON3_5722	0x165a
+#define PCI_DEVICE_ID_TIGON3_5723	0x165b
+#define PCI_DEVICE_ID_TIGON3_5705M	0x165d
+#define PCI_DEVICE_ID_TIGON3_5705M_2	0x165e
+#define PCI_DEVICE_ID_NX2_57712		0x1662
+#define PCI_DEVICE_ID_NX2_57712E	0x1663
+#define PCI_DEVICE_ID_NX2_57712_MF	0x1663
+#define PCI_DEVICE_ID_TIGON3_5714	0x1668
+#define PCI_DEVICE_ID_TIGON3_5714S	0x1669
+#define PCI_DEVICE_ID_TIGON3_5780	0x166a
+#define PCI_DEVICE_ID_TIGON3_5780S	0x166b
+#define PCI_DEVICE_ID_TIGON3_5705F	0x166e
+#define PCI_DEVICE_ID_NX2_57712_VF	0x166f
+#define PCI_DEVICE_ID_TIGON3_5754M	0x1672
+#define PCI_DEVICE_ID_TIGON3_5755M	0x1673
+#define PCI_DEVICE_ID_TIGON3_5756	0x1674
+#define PCI_DEVICE_ID_TIGON3_5750	0x1676
+#define PCI_DEVICE_ID_TIGON3_5751	0x1677
+#define PCI_DEVICE_ID_TIGON3_5715	0x1678
+#define PCI_DEVICE_ID_TIGON3_5715S	0x1679
+#define PCI_DEVICE_ID_TIGON3_5754	0x167a
+#define PCI_DEVICE_ID_TIGON3_5755	0x167b
+#define PCI_DEVICE_ID_TIGON3_5751M	0x167d
+#define PCI_DEVICE_ID_TIGON3_5751F	0x167e
+#define PCI_DEVICE_ID_TIGON3_5787F	0x167f
+#define PCI_DEVICE_ID_TIGON3_5761E	0x1680
+#define PCI_DEVICE_ID_TIGON3_5761	0x1681
+#define PCI_DEVICE_ID_TIGON3_5764	0x1684
+#define PCI_DEVICE_ID_NX2_57800		0x168a
+#define PCI_DEVICE_ID_NX2_57840		0x168d
+#define PCI_DEVICE_ID_NX2_57810		0x168e
+#define PCI_DEVICE_ID_TIGON3_5787M	0x1693
+#define PCI_DEVICE_ID_TIGON3_5782	0x1696
+#define PCI_DEVICE_ID_TIGON3_5784	0x1698
+#define PCI_DEVICE_ID_TIGON3_5786	0x169a
+#define PCI_DEVICE_ID_TIGON3_5787	0x169b
+#define PCI_DEVICE_ID_TIGON3_5788	0x169c
+#define PCI_DEVICE_ID_TIGON3_5789	0x169d
+#define PCI_DEVICE_ID_NX2_57840_4_10	0x16a1
+#define PCI_DEVICE_ID_NX2_57840_2_20	0x16a2
+#define PCI_DEVICE_ID_NX2_57840_MF	0x16a4
+#define PCI_DEVICE_ID_NX2_57800_MF	0x16a5
+#define PCI_DEVICE_ID_TIGON3_5702X	0x16a6
+#define PCI_DEVICE_ID_TIGON3_5703X	0x16a7
+#define PCI_DEVICE_ID_TIGON3_5704S	0x16a8
+#define PCI_DEVICE_ID_NX2_57800_VF	0x16a9
+#define PCI_DEVICE_ID_NX2_5706S		0x16aa
+#define PCI_DEVICE_ID_NX2_5708S		0x16ac
+#define PCI_DEVICE_ID_NX2_57840_VF	0x16ad
+#define PCI_DEVICE_ID_NX2_57810_MF	0x16ae
+#define PCI_DEVICE_ID_NX2_57810_VF	0x16af
+#define PCI_DEVICE_ID_TIGON3_5702A3	0x16c6
+#define PCI_DEVICE_ID_TIGON3_5703A3	0x16c7
+#define PCI_DEVICE_ID_TIGON3_5781	0x16dd
+#define PCI_DEVICE_ID_TIGON3_5753	0x16f7
+#define PCI_DEVICE_ID_TIGON3_5753M	0x16fd
+#define PCI_DEVICE_ID_TIGON3_5753F	0x16fe
+#define PCI_DEVICE_ID_TIGON3_5901	0x170d
+#define PCI_DEVICE_ID_BCM4401B1		0x170c
+#define PCI_DEVICE_ID_TIGON3_5901_2	0x170e
+#define PCI_DEVICE_ID_TIGON3_5906	0x1712
+#define PCI_DEVICE_ID_TIGON3_5906M	0x1713
+#define PCI_DEVICE_ID_BCM4401		0x4401
+#define PCI_DEVICE_ID_BCM4401B0		0x4402
+
+#define PCI_VENDOR_ID_TOPIC		0x151f
+#define PCI_DEVICE_ID_TOPIC_TP560	0x0000
+
+#define PCI_VENDOR_ID_MAINPINE		0x1522
+#define PCI_DEVICE_ID_MAINPINE_PBRIDGE	0x0100
+#define PCI_VENDOR_ID_ENE		0x1524
+#define PCI_DEVICE_ID_ENE_CB710_FLASH	0x0510
+#define PCI_DEVICE_ID_ENE_CB712_SD	0x0550
+#define PCI_DEVICE_ID_ENE_CB712_SD_2	0x0551
+#define PCI_DEVICE_ID_ENE_CB714_SD	0x0750
+#define PCI_DEVICE_ID_ENE_CB714_SD_2	0x0751
+#define PCI_DEVICE_ID_ENE_1211		0x1211
+#define PCI_DEVICE_ID_ENE_1225		0x1225
+#define PCI_DEVICE_ID_ENE_1410		0x1410
+#define PCI_DEVICE_ID_ENE_710		0x1411
+#define PCI_DEVICE_ID_ENE_712		0x1412
+#define PCI_DEVICE_ID_ENE_1420		0x1420
+#define PCI_DEVICE_ID_ENE_720		0x1421
+#define PCI_DEVICE_ID_ENE_722		0x1422
+
+#define PCI_SUBVENDOR_ID_PERLE          0x155f
+#define PCI_SUBDEVICE_ID_PCI_RAS4       0xf001
+#define PCI_SUBDEVICE_ID_PCI_RAS8       0xf010
+
+#define PCI_VENDOR_ID_SYBA		0x1592
+#define PCI_DEVICE_ID_SYBA_2P_EPP	0x0782
+#define PCI_DEVICE_ID_SYBA_1P_ECP	0x0783
+
+#define PCI_VENDOR_ID_MORETON		0x15aa
+#define PCI_DEVICE_ID_RASTEL_2PORT	0x2000
+
+#define PCI_VENDOR_ID_VMWARE		0x15ad
+#define PCI_DEVICE_ID_VMWARE_VMXNET3	0x07b0
+
+#define PCI_VENDOR_ID_ZOLTRIX		0x15b0
+#define PCI_DEVICE_ID_ZOLTRIX_2BD0	0x2bd0
+
+#define PCI_VENDOR_ID_MELLANOX		0x15b3
+#define PCI_DEVICE_ID_MELLANOX_CONNECTX3	0x1003
+#define PCI_DEVICE_ID_MELLANOX_CONNECTX3_PRO	0x1007
+#define PCI_DEVICE_ID_MELLANOX_CONNECTIB	0x1011
+#define PCI_DEVICE_ID_MELLANOX_CONNECTX4	0x1013
+#define PCI_DEVICE_ID_MELLANOX_CONNECTX4_LX	0x1015
+#define PCI_DEVICE_ID_MELLANOX_TAVOR		0x5a44
+#define PCI_DEVICE_ID_MELLANOX_TAVOR_BRIDGE	0x5a46
+#define PCI_DEVICE_ID_MELLANOX_SINAI_OLD	0x5e8c
+#define PCI_DEVICE_ID_MELLANOX_SINAI		0x6274
+#define PCI_DEVICE_ID_MELLANOX_ARBEL_COMPAT	0x6278
+#define PCI_DEVICE_ID_MELLANOX_ARBEL		0x6282
+#define PCI_DEVICE_ID_MELLANOX_HERMON_SDR	0x6340
+#define PCI_DEVICE_ID_MELLANOX_HERMON_DDR	0x634a
+#define PCI_DEVICE_ID_MELLANOX_HERMON_QDR	0x6354
+#define PCI_DEVICE_ID_MELLANOX_HERMON_EN	0x6368
+#define PCI_DEVICE_ID_MELLANOX_CONNECTX_EN	0x6372
+#define PCI_DEVICE_ID_MELLANOX_HERMON_DDR_GEN2	0x6732
+#define PCI_DEVICE_ID_MELLANOX_HERMON_QDR_GEN2	0x673c
+#define PCI_DEVICE_ID_MELLANOX_CONNECTX_EN_5_GEN2 0x6746
+#define PCI_DEVICE_ID_MELLANOX_HERMON_EN_GEN2	0x6750
+#define PCI_DEVICE_ID_MELLANOX_CONNECTX_EN_T_GEN2 0x675a
+#define PCI_DEVICE_ID_MELLANOX_CONNECTX_EN_GEN2	0x6764
+#define PCI_DEVICE_ID_MELLANOX_CONNECTX2	0x676e
+
+#define PCI_VENDOR_ID_DFI		0x15bd
+
+#define PCI_VENDOR_ID_QUICKNET		0x15e2
+#define PCI_DEVICE_ID_QUICKNET_XJ	0x0500
+
+/*
+ * ADDI-DATA GmbH communication cards <info@addi-data.com>
+ */
+#define PCI_VENDOR_ID_ADDIDATA                 0x15B8
+#define PCI_DEVICE_ID_ADDIDATA_APCI7500        0x7000
+#define PCI_DEVICE_ID_ADDIDATA_APCI7420        0x7001
+#define PCI_DEVICE_ID_ADDIDATA_APCI7300        0x7002
+#define PCI_DEVICE_ID_ADDIDATA_APCI7500_2      0x7009
+#define PCI_DEVICE_ID_ADDIDATA_APCI7420_2      0x700A
+#define PCI_DEVICE_ID_ADDIDATA_APCI7300_2      0x700B
+#define PCI_DEVICE_ID_ADDIDATA_APCI7500_3      0x700C
+#define PCI_DEVICE_ID_ADDIDATA_APCI7420_3      0x700D
+#define PCI_DEVICE_ID_ADDIDATA_APCI7300_3      0x700E
+#define PCI_DEVICE_ID_ADDIDATA_APCI7800_3      0x700F
+#define PCI_DEVICE_ID_ADDIDATA_APCIe7300       0x7010
+#define PCI_DEVICE_ID_ADDIDATA_APCIe7420       0x7011
+#define PCI_DEVICE_ID_ADDIDATA_APCIe7500       0x7012
+#define PCI_DEVICE_ID_ADDIDATA_APCIe7800       0x7013
+
+#define PCI_VENDOR_ID_PDC		0x15e9
+
+#define PCI_VENDOR_ID_FARSITE           0x1619
+#define PCI_DEVICE_ID_FARSITE_T2P       0x0400
+#define PCI_DEVICE_ID_FARSITE_T4P       0x0440
+#define PCI_DEVICE_ID_FARSITE_T1U       0x0610
+#define PCI_DEVICE_ID_FARSITE_T2U       0x0620
+#define PCI_DEVICE_ID_FARSITE_T4U       0x0640
+#define PCI_DEVICE_ID_FARSITE_TE1       0x1610
+#define PCI_DEVICE_ID_FARSITE_TE1C      0x1612
+
+#define PCI_VENDOR_ID_ARIMA		0x161f
+
+#define PCI_VENDOR_ID_BROCADE		0x1657
+#define PCI_DEVICE_ID_BROCADE_CT	0x0014
+#define PCI_DEVICE_ID_BROCADE_FC_8G1P	0x0017
+#define PCI_DEVICE_ID_BROCADE_CT_FC	0x0021
+
+#define PCI_VENDOR_ID_SIBYTE		0x166d
+#define PCI_DEVICE_ID_BCM1250_PCI	0x0001
+#define PCI_DEVICE_ID_BCM1250_HT	0x0002
+
+#define PCI_VENDOR_ID_ATHEROS		0x168c
+
+#define PCI_VENDOR_ID_NETCELL		0x169c
+#define PCI_DEVICE_ID_REVOLUTION	0x0044
+
+#define PCI_VENDOR_ID_CENATEK		0x16CA
+#define PCI_DEVICE_ID_CENATEK_IDE	0x0001
+
+#define PCI_VENDOR_ID_SYNOPSYS		0x16c3
+#define PCI_DEVICE_ID_SYNOPSYS_HAPSUSB3		0xabcd
+#define PCI_DEVICE_ID_SYNOPSYS_HAPSUSB3_AXI	0xabce
+#define PCI_DEVICE_ID_SYNOPSYS_HAPSUSB31	0xabcf
+#define PCI_DEVICE_ID_SYNOPSYS_EDDA	0xedda
+
+#define PCI_VENDOR_ID_USR		0x16ec
+
+#define PCI_VENDOR_ID_VITESSE		0x1725
+#define PCI_DEVICE_ID_VITESSE_VSC7174	0x7174
+
+#define PCI_VENDOR_ID_LINKSYS		0x1737
+#define PCI_DEVICE_ID_LINKSYS_EG1064	0x1064
+
+#define PCI_VENDOR_ID_ALTIMA		0x173b
+#define PCI_DEVICE_ID_ALTIMA_AC1000	0x03e8
+#define PCI_DEVICE_ID_ALTIMA_AC1001	0x03e9
+#define PCI_DEVICE_ID_ALTIMA_AC9100	0x03ea
+#define PCI_DEVICE_ID_ALTIMA_AC1003	0x03eb
+
+#define PCI_VENDOR_ID_CAVIUM		0x177d
+
+#define PCI_VENDOR_ID_TECHWELL		0x1797
+#define PCI_DEVICE_ID_TECHWELL_6800	0x6800
+#define PCI_DEVICE_ID_TECHWELL_6801	0x6801
+#define PCI_DEVICE_ID_TECHWELL_6804	0x6804
+#define PCI_DEVICE_ID_TECHWELL_6816_1	0x6810
+#define PCI_DEVICE_ID_TECHWELL_6816_2	0x6811
+#define PCI_DEVICE_ID_TECHWELL_6816_3	0x6812
+#define PCI_DEVICE_ID_TECHWELL_6816_4	0x6813
+
+#define PCI_VENDOR_ID_BELKIN		0x1799
+#define PCI_DEVICE_ID_BELKIN_F5D7010V7	0x701f
+
+#define PCI_VENDOR_ID_RDC		0x17f3
+#define PCI_DEVICE_ID_RDC_R6020		0x6020
+#define PCI_DEVICE_ID_RDC_R6030		0x6030
+#define PCI_DEVICE_ID_RDC_R6040		0x6040
+#define PCI_DEVICE_ID_RDC_R6060		0x6060
+#define PCI_DEVICE_ID_RDC_R6061		0x6061
+#define PCI_DEVICE_ID_RDC_D1010		0x1010
+
+#define PCI_VENDOR_ID_GLI		0x17a0
+
+#define PCI_VENDOR_ID_LENOVO		0x17aa
+
+#define PCI_VENDOR_ID_QCOM		0x17cb
+
+#define PCI_VENDOR_ID_CDNS		0x17cd
+
+#define PCI_VENDOR_ID_ARECA		0x17d3
+#define PCI_DEVICE_ID_ARECA_1110	0x1110
+#define PCI_DEVICE_ID_ARECA_1120	0x1120
+#define PCI_DEVICE_ID_ARECA_1130	0x1130
+#define PCI_DEVICE_ID_ARECA_1160	0x1160
+#define PCI_DEVICE_ID_ARECA_1170	0x1170
+#define PCI_DEVICE_ID_ARECA_1200	0x1200
+#define PCI_DEVICE_ID_ARECA_1201	0x1201
+#define PCI_DEVICE_ID_ARECA_1202	0x1202
+#define PCI_DEVICE_ID_ARECA_1210	0x1210
+#define PCI_DEVICE_ID_ARECA_1220	0x1220
+#define PCI_DEVICE_ID_ARECA_1230	0x1230
+#define PCI_DEVICE_ID_ARECA_1260	0x1260
+#define PCI_DEVICE_ID_ARECA_1270	0x1270
+#define PCI_DEVICE_ID_ARECA_1280	0x1280
+#define PCI_DEVICE_ID_ARECA_1380	0x1380
+#define PCI_DEVICE_ID_ARECA_1381	0x1381
+#define PCI_DEVICE_ID_ARECA_1680	0x1680
+#define PCI_DEVICE_ID_ARECA_1681	0x1681
+
+#define PCI_VENDOR_ID_S2IO		0x17d5
+#define	PCI_DEVICE_ID_S2IO_WIN		0x5731
+#define	PCI_DEVICE_ID_S2IO_UNI		0x5831
+#define PCI_DEVICE_ID_HERC_WIN		0x5732
+#define PCI_DEVICE_ID_HERC_UNI		0x5832
+
+#define PCI_VENDOR_ID_SITECOM		0x182d
+#define PCI_DEVICE_ID_SITECOM_DC105V2	0x3069
+
+#define PCI_VENDOR_ID_TOPSPIN		0x1867
+
+#define PCI_VENDOR_ID_COMMTECH		0x18f7
+
+#define PCI_VENDOR_ID_SILAN		0x1904
+
+#define PCI_VENDOR_ID_RENESAS		0x1912
+#define PCI_DEVICE_ID_RENESAS_SH7781	0x0001
+#define PCI_DEVICE_ID_RENESAS_SH7780	0x0002
+#define PCI_DEVICE_ID_RENESAS_SH7763	0x0004
+#define PCI_DEVICE_ID_RENESAS_SH7785	0x0007
+#define PCI_DEVICE_ID_RENESAS_SH7786	0x0010
+
+#define PCI_VENDOR_ID_SOLARFLARE	0x1924
+#define PCI_DEVICE_ID_SOLARFLARE_SFC4000A_0	0x0703
+#define PCI_DEVICE_ID_SOLARFLARE_SFC4000A_1	0x6703
+#define PCI_DEVICE_ID_SOLARFLARE_SFC4000B	0x0710
+
+#define PCI_VENDOR_ID_TDI               0x192E
+#define PCI_DEVICE_ID_TDI_EHCI          0x0101
+
+#define PCI_VENDOR_ID_FREESCALE		0x1957	/* duplicate: NXP */
+#define PCI_VENDOR_ID_NXP		0x1957	/* duplicate: FREESCALE */
+#define PCI_DEVICE_ID_MPC8308		0xc006
+#define PCI_DEVICE_ID_MPC8315E		0x00b4
+#define PCI_DEVICE_ID_MPC8315		0x00b5
+#define PCI_DEVICE_ID_MPC8314E		0x00b6
+#define PCI_DEVICE_ID_MPC8314		0x00b7
+#define PCI_DEVICE_ID_MPC8378E		0x00c4
+#define PCI_DEVICE_ID_MPC8378		0x00c5
+#define PCI_DEVICE_ID_MPC8377E		0x00c6
+#define PCI_DEVICE_ID_MPC8377		0x00c7
+#define PCI_DEVICE_ID_MPC8548E		0x0012
+#define PCI_DEVICE_ID_MPC8548		0x0013
+#define PCI_DEVICE_ID_MPC8543E		0x0014
+#define PCI_DEVICE_ID_MPC8543		0x0015
+#define PCI_DEVICE_ID_MPC8547E		0x0018
+#define PCI_DEVICE_ID_MPC8545E		0x0019
+#define PCI_DEVICE_ID_MPC8545		0x001a
+#define PCI_DEVICE_ID_MPC8569E		0x0061
+#define PCI_DEVICE_ID_MPC8569		0x0060
+#define PCI_DEVICE_ID_MPC8568E		0x0020
+#define PCI_DEVICE_ID_MPC8568		0x0021
+#define PCI_DEVICE_ID_MPC8567E		0x0022
+#define PCI_DEVICE_ID_MPC8567		0x0023
+#define PCI_DEVICE_ID_MPC8533E		0x0030
+#define PCI_DEVICE_ID_MPC8533		0x0031
+#define PCI_DEVICE_ID_MPC8544E		0x0032
+#define PCI_DEVICE_ID_MPC8544		0x0033
+#define PCI_DEVICE_ID_MPC8572E		0x0040
+#define PCI_DEVICE_ID_MPC8572		0x0041
+#define PCI_DEVICE_ID_MPC8536E		0x0050
+#define PCI_DEVICE_ID_MPC8536		0x0051
+#define PCI_DEVICE_ID_P2020E		0x0070
+#define PCI_DEVICE_ID_P2020		0x0071
+#define PCI_DEVICE_ID_P2010E		0x0078
+#define PCI_DEVICE_ID_P2010		0x0079
+#define PCI_DEVICE_ID_P1020E		0x0100
+#define PCI_DEVICE_ID_P1020		0x0101
+#define PCI_DEVICE_ID_P1021E		0x0102
+#define PCI_DEVICE_ID_P1021		0x0103
+#define PCI_DEVICE_ID_P1011E		0x0108
+#define PCI_DEVICE_ID_P1011		0x0109
+#define PCI_DEVICE_ID_P1022E		0x0110
+#define PCI_DEVICE_ID_P1022		0x0111
+#define PCI_DEVICE_ID_P1013E		0x0118
+#define PCI_DEVICE_ID_P1013		0x0119
+#define PCI_DEVICE_ID_P4080E		0x0400
+#define PCI_DEVICE_ID_P4080		0x0401
+#define PCI_DEVICE_ID_P4040E		0x0408
+#define PCI_DEVICE_ID_P4040		0x0409
+#define PCI_DEVICE_ID_P2040E		0x0410
+#define PCI_DEVICE_ID_P2040		0x0411
+#define PCI_DEVICE_ID_P3041E		0x041E
+#define PCI_DEVICE_ID_P3041		0x041F
+#define PCI_DEVICE_ID_P5020E		0x0420
+#define PCI_DEVICE_ID_P5020		0x0421
+#define PCI_DEVICE_ID_P5010E		0x0428
+#define PCI_DEVICE_ID_P5010		0x0429
+#define PCI_DEVICE_ID_MPC8641		0x7010
+#define PCI_DEVICE_ID_MPC8641D		0x7011
+#define PCI_DEVICE_ID_MPC8610		0x7018
+
+#define PCI_VENDOR_ID_PASEMI		0x1959
+
+#define PCI_VENDOR_ID_ATTANSIC		0x1969
+#define PCI_DEVICE_ID_ATTANSIC_L1	0x1048
+#define PCI_DEVICE_ID_ATTANSIC_L2	0x2048
+
+#define PCI_VENDOR_ID_JMICRON		0x197B
+#define PCI_DEVICE_ID_JMICRON_JMB360	0x2360
+#define PCI_DEVICE_ID_JMICRON_JMB361	0x2361
+#define PCI_DEVICE_ID_JMICRON_JMB362	0x2362
+#define PCI_DEVICE_ID_JMICRON_JMB363	0x2363
+#define PCI_DEVICE_ID_JMICRON_JMB364	0x2364
+#define PCI_DEVICE_ID_JMICRON_JMB365	0x2365
+#define PCI_DEVICE_ID_JMICRON_JMB366	0x2366
+#define PCI_DEVICE_ID_JMICRON_JMB368	0x2368
+#define PCI_DEVICE_ID_JMICRON_JMB369	0x2369
+#define PCI_DEVICE_ID_JMICRON_JMB38X_SD	0x2381
+#define PCI_DEVICE_ID_JMICRON_JMB38X_MMC 0x2382
+#define PCI_DEVICE_ID_JMICRON_JMB38X_MS	0x2383
+#define PCI_DEVICE_ID_JMICRON_JMB385_MS	0x2388
+#define PCI_DEVICE_ID_JMICRON_JMB388_SD	0x2391
+#define PCI_DEVICE_ID_JMICRON_JMB388_ESD 0x2392
+#define PCI_DEVICE_ID_JMICRON_JMB390_MS	0x2393
+
+#define PCI_VENDOR_ID_KORENIX		0x1982
+#define PCI_DEVICE_ID_KORENIX_JETCARDF0	0x1600
+#define PCI_DEVICE_ID_KORENIX_JETCARDF1	0x16ff
+#define PCI_DEVICE_ID_KORENIX_JETCARDF2	0x1700
+#define PCI_DEVICE_ID_KORENIX_JETCARDF3	0x17ff
+
+#define PCI_VENDOR_ID_HUAWEI		0x19e5
+#define PCI_DEVICE_ID_HUAWEI_ZIP_VF	0xa251
+#define PCI_DEVICE_ID_HUAWEI_SEC_VF	0xa256
+#define PCI_DEVICE_ID_HUAWEI_HPRE_VF	0xa259
+
+#define PCI_VENDOR_ID_NETRONOME		0x19ee
+#define PCI_DEVICE_ID_NETRONOME_NFP3800	0x3800
+#define PCI_DEVICE_ID_NETRONOME_NFP4000	0x4000
+#define PCI_DEVICE_ID_NETRONOME_NFP5000	0x5000
+#define PCI_DEVICE_ID_NETRONOME_NFP6000	0x6000
+#define PCI_DEVICE_ID_NETRONOME_NFP3800_VF	0x3803
+#define PCI_DEVICE_ID_NETRONOME_NFP6000_VF	0x6003
+
+#define PCI_VENDOR_ID_QMI		0x1a32
+
+#define PCI_VENDOR_ID_AZWAVE		0x1a3b
+
+#define PCI_VENDOR_ID_REDHAT_QUMRANET    0x1af4
+#define PCI_SUBVENDOR_ID_REDHAT_QUMRANET 0x1af4
+#define PCI_SUBDEVICE_ID_QEMU            0x1100
+
+#define PCI_VENDOR_ID_ASMEDIA		0x1b21
+
+#define PCI_VENDOR_ID_REDHAT		0x1b36
+
+#define PCI_VENDOR_ID_SILICOM_DENMARK	0x1c2c
+
+#define PCI_VENDOR_ID_AMAZON_ANNAPURNA_LABS	0x1c36
+
+#define PCI_VENDOR_ID_CIRCUITCO		0x1cc8
+#define PCI_SUBSYSTEM_ID_CIRCUITCO_MINNOWBOARD	0x0001
+
+#define PCI_VENDOR_ID_AMAZON		0x1d0f
+
+#define PCI_VENDOR_ID_ZHAOXIN		0x1d17
+
+#define PCI_VENDOR_ID_HYGON		0x1d94
+
+#define PCI_VENDOR_ID_FUNGIBLE		0x1dad
+
+#define PCI_VENDOR_ID_HXT		0x1dbf
+
+#define PCI_VENDOR_ID_TEKRAM		0x1de1
+#define PCI_DEVICE_ID_TEKRAM_DC290	0xdc29
+
+#define PCI_VENDOR_ID_TEHUTI		0x1fc9
+#define PCI_DEVICE_ID_TEHUTI_3009	0x3009
+#define PCI_DEVICE_ID_TEHUTI_3010	0x3010
+#define PCI_DEVICE_ID_TEHUTI_3014	0x3014
+
+#define PCI_VENDOR_ID_SUNIX		0x1fd4
+#define PCI_DEVICE_ID_SUNIX_1999	0x1999
+
+#define PCI_VENDOR_ID_HINT             0x3388
+#define PCI_DEVICE_ID_HINT_VXPROII_IDE 0x8013
+
+#define PCI_VENDOR_ID_3DLABS		0x3d3d
+#define PCI_DEVICE_ID_3DLABS_PERMEDIA2	0x0007
+#define PCI_DEVICE_ID_3DLABS_PERMEDIA2V	0x0009
+
+#define PCI_VENDOR_ID_NETXEN		0x4040
+#define PCI_DEVICE_ID_NX2031_10GXSR	0x0001
+#define PCI_DEVICE_ID_NX2031_10GCX4	0x0002
+#define PCI_DEVICE_ID_NX2031_4GCU	0x0003
+#define PCI_DEVICE_ID_NX2031_IMEZ	0x0004
+#define PCI_DEVICE_ID_NX2031_HMEZ	0x0005
+#define PCI_DEVICE_ID_NX2031_XG_MGMT	0x0024
+#define PCI_DEVICE_ID_NX2031_XG_MGMT2	0x0025
+#define PCI_DEVICE_ID_NX3031		0x0100
+
+#define PCI_VENDOR_ID_AKS		0x416c
+#define PCI_DEVICE_ID_AKS_ALADDINCARD	0x0100
+
+#define PCI_VENDOR_ID_ACCESSIO		0x494f
+#define PCI_DEVICE_ID_ACCESSIO_WDG_CSM	0x22c0
+
+#define PCI_VENDOR_ID_S3		0x5333
+#define PCI_DEVICE_ID_S3_TRIO		0x8811
+#define PCI_DEVICE_ID_S3_868		0x8880
+#define PCI_DEVICE_ID_S3_968		0x88f0
+#define PCI_DEVICE_ID_S3_SAVAGE4	0x8a25
+#define PCI_DEVICE_ID_S3_PROSAVAGE8	0x8d04
+#define PCI_DEVICE_ID_S3_SONICVIBES	0xca00
+
+#define PCI_VENDOR_ID_DUNORD		0x5544
+#define PCI_DEVICE_ID_DUNORD_I3000	0x0001
+
+#define PCI_VENDOR_ID_DCI		0x6666
+#define PCI_DEVICE_ID_DCI_PCCOM4	0x0001
+#define PCI_DEVICE_ID_DCI_PCCOM8	0x0002
+#define PCI_DEVICE_ID_DCI_PCCOM2	0x0004
+
+#define PCI_VENDOR_ID_INTEL		0x8086
+#define PCI_DEVICE_ID_INTEL_EESSC	0x0008
+#define PCI_DEVICE_ID_INTEL_HDA_CML_LP	0x02c8
+#define PCI_DEVICE_ID_INTEL_PXHD_0	0x0320
+#define PCI_DEVICE_ID_INTEL_PXHD_1	0x0321
+#define PCI_DEVICE_ID_INTEL_PXH_0	0x0329
+#define PCI_DEVICE_ID_INTEL_PXH_1	0x032a
+#define PCI_DEVICE_ID_INTEL_PXHV	0x032c
+#define PCI_DEVICE_ID_INTEL_80332_0	0x0330
+#define PCI_DEVICE_ID_INTEL_80332_1	0x0332
+#define PCI_DEVICE_ID_INTEL_80333_0	0x0370
+#define PCI_DEVICE_ID_INTEL_80333_1	0x0372
+#define PCI_DEVICE_ID_INTEL_QAT_DH895XCC	0x0435
+#define PCI_DEVICE_ID_INTEL_QAT_DH895XCC_VF	0x0443
+#define PCI_DEVICE_ID_INTEL_82375	0x0482
+#define PCI_DEVICE_ID_INTEL_82424	0x0483
+#define PCI_DEVICE_ID_INTEL_82378	0x0484
+#define PCI_DEVICE_ID_INTEL_82425	0x0486
+#define PCI_DEVICE_ID_INTEL_HDA_CML_H	0x06c8
+#define PCI_DEVICE_ID_INTEL_MRST_SD0	0x0807
+#define PCI_DEVICE_ID_INTEL_MRST_SD1	0x0808
+#define PCI_DEVICE_ID_INTEL_HDA_OAKTRAIL	0x080a
+#define PCI_DEVICE_ID_INTEL_MFD_SD	0x0820
+#define PCI_DEVICE_ID_INTEL_MFD_SDIO1	0x0821
+#define PCI_DEVICE_ID_INTEL_MFD_SDIO2	0x0822
+#define PCI_DEVICE_ID_INTEL_MFD_EMMC0	0x0823
+#define PCI_DEVICE_ID_INTEL_MFD_EMMC1	0x0824
+#define PCI_DEVICE_ID_INTEL_MRST_SD2	0x084f
+#define PCI_DEVICE_ID_INTEL_QUARK_X1000_ILB	0x095e
+#define PCI_DEVICE_ID_INTEL_I960	0x0960
+#define PCI_DEVICE_ID_INTEL_I960RM	0x0962
+#define PCI_DEVICE_ID_INTEL_HDA_HSW_0	0x0a0c
+#define PCI_DEVICE_ID_INTEL_HDA_HSW_2	0x0c0c
+#define PCI_DEVICE_ID_INTEL_CENTERTON_ILB	0x0c60
+#define PCI_DEVICE_ID_INTEL_HDA_HSW_3	0x0d0c
+#define PCI_DEVICE_ID_INTEL_HDA_BYT	0x0f04
+#define PCI_DEVICE_ID_INTEL_SST_BYT	0x0f28
+#define PCI_DEVICE_ID_INTEL_8257X_SOL	0x1062
+#define PCI_DEVICE_ID_INTEL_82573E_SOL	0x1085
+#define PCI_DEVICE_ID_INTEL_82573L_SOL	0x108f
+#define PCI_DEVICE_ID_INTEL_82815_MC	0x1130
+#define PCI_DEVICE_ID_INTEL_82815_CGC	0x1132
+#define PCI_DEVICE_ID_INTEL_SST_TNG	0x119a
+#define PCI_DEVICE_ID_INTEL_82092AA_0	0x1221
+#define PCI_DEVICE_ID_INTEL_82437	0x122d
+#define PCI_DEVICE_ID_INTEL_82371FB_0	0x122e
+#define PCI_DEVICE_ID_INTEL_82371FB_1	0x1230
+#define PCI_DEVICE_ID_INTEL_82371MX	0x1234
+#define PCI_DEVICE_ID_INTEL_82441	0x1237
+#define PCI_DEVICE_ID_INTEL_82380FB	0x124b
+#define PCI_DEVICE_ID_INTEL_82439	0x1250
+#define PCI_DEVICE_ID_INTEL_LIGHT_RIDGE             0x1513 /* Tbt 1 Gen 1 */
+#define PCI_DEVICE_ID_INTEL_EAGLE_RIDGE             0x151a
+#define PCI_DEVICE_ID_INTEL_LIGHT_PEAK              0x151b
+#define PCI_DEVICE_ID_INTEL_CACTUS_RIDGE_4C         0x1547 /* Tbt 1 Gen 2 */
+#define PCI_DEVICE_ID_INTEL_CACTUS_RIDGE_2C         0x1548
+#define PCI_DEVICE_ID_INTEL_PORT_RIDGE              0x1549
+#define PCI_DEVICE_ID_INTEL_REDWOOD_RIDGE_2C_NHI    0x1566 /* Tbt 1 Gen 3 */
+#define PCI_DEVICE_ID_INTEL_REDWOOD_RIDGE_2C_BRIDGE 0x1567
+#define PCI_DEVICE_ID_INTEL_REDWOOD_RIDGE_4C_NHI    0x1568
+#define PCI_DEVICE_ID_INTEL_REDWOOD_RIDGE_4C_BRIDGE 0x1569
+#define PCI_DEVICE_ID_INTEL_FALCON_RIDGE_2C_NHI     0x156a /* Thunderbolt 2 */
+#define PCI_DEVICE_ID_INTEL_FALCON_RIDGE_2C_BRIDGE  0x156b
+#define PCI_DEVICE_ID_INTEL_FALCON_RIDGE_4C_NHI     0x156c
+#define PCI_DEVICE_ID_INTEL_FALCON_RIDGE_4C_BRIDGE  0x156d
+#define PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_2C_NHI     0x1575 /* Thunderbolt 3 */
+#define PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_2C_BRIDGE  0x1576
+#define PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_4C_NHI     0x1577
+#define PCI_DEVICE_ID_INTEL_ALPINE_RIDGE_4C_BRIDGE  0x1578
+#define PCI_DEVICE_ID_INTEL_HDA_BDW	0x160c
+#define PCI_DEVICE_ID_INTEL_80960_RP	0x1960
+#define PCI_DEVICE_ID_INTEL_QAT_C3XXX	0x19e2
+#define PCI_DEVICE_ID_INTEL_QAT_C3XXX_VF	0x19e3
+#define PCI_DEVICE_ID_INTEL_82840_HB	0x1a21
+#define PCI_DEVICE_ID_INTEL_82845_HB	0x1a30
+#define PCI_DEVICE_ID_INTEL_IOAT	0x1a38
+#define PCI_DEVICE_ID_INTEL_HDA_CPT	0x1c20
+#define PCI_DEVICE_ID_INTEL_COUGARPOINT_LPC_MIN	0x1c41
+#define PCI_DEVICE_ID_INTEL_COUGARPOINT_LPC_MAX	0x1c5f
+#define PCI_DEVICE_ID_INTEL_HDA_PBG	0x1d20
+#define PCI_DEVICE_ID_INTEL_PATSBURG_LPC_0	0x1d40
+#define PCI_DEVICE_ID_INTEL_PATSBURG_LPC_1	0x1d41
+#define PCI_DEVICE_ID_INTEL_HDA_PPT	0x1e20
+#define PCI_DEVICE_ID_INTEL_PANTHERPOINT_XHCI	0x1e31
+#define PCI_DEVICE_ID_INTEL_PANTHERPOINT_LPC_MIN	0x1e40
+#define PCI_DEVICE_ID_INTEL_PANTHERPOINT_LPC_MAX	0x1e5f
+#define PCI_DEVICE_ID_INTEL_VMD_201D	0x201d
+#define PCI_DEVICE_ID_INTEL_HDA_BSW	0x2284
+#define PCI_DEVICE_ID_INTEL_SST_BSW	0x22a8
+#define PCI_DEVICE_ID_INTEL_DH89XXCC_LPC_MIN	0x2310
+#define PCI_DEVICE_ID_INTEL_DH89XXCC_LPC_MAX	0x231f
+#define PCI_DEVICE_ID_INTEL_82801AA_0	0x2410
+#define PCI_DEVICE_ID_INTEL_82801AA_1	0x2411
+#define PCI_DEVICE_ID_INTEL_82801AA_3	0x2413
+#define PCI_DEVICE_ID_INTEL_82801AA_5	0x2415
+#define PCI_DEVICE_ID_INTEL_82801AA_6	0x2416
+#define PCI_DEVICE_ID_INTEL_82801AA_8	0x2418
+#define PCI_DEVICE_ID_INTEL_82801AB_0	0x2420
+#define PCI_DEVICE_ID_INTEL_82801AB_1	0x2421
+#define PCI_DEVICE_ID_INTEL_82801AB_3	0x2423
+#define PCI_DEVICE_ID_INTEL_82801AB_5	0x2425
+#define PCI_DEVICE_ID_INTEL_82801AB_6	0x2426
+#define PCI_DEVICE_ID_INTEL_82801AB_8	0x2428
+#define PCI_DEVICE_ID_INTEL_82801BA_0	0x2440
+#define PCI_DEVICE_ID_INTEL_82801BA_2	0x2443
+#define PCI_DEVICE_ID_INTEL_82801BA_4	0x2445
+#define PCI_DEVICE_ID_INTEL_82801BA_6	0x2448
+#define PCI_DEVICE_ID_INTEL_82801BA_8	0x244a
+#define PCI_DEVICE_ID_INTEL_82801BA_9	0x244b
+#define PCI_DEVICE_ID_INTEL_82801BA_10	0x244c
+#define PCI_DEVICE_ID_INTEL_82801BA_11	0x244e
+#define PCI_DEVICE_ID_INTEL_82801E_0	0x2450
+#define PCI_DEVICE_ID_INTEL_82801E_11	0x245b
+#define PCI_DEVICE_ID_INTEL_82801CA_0	0x2480
+#define PCI_DEVICE_ID_INTEL_82801CA_3	0x2483
+#define PCI_DEVICE_ID_INTEL_82801CA_5	0x2485
+#define PCI_DEVICE_ID_INTEL_82801CA_6	0x2486
+#define PCI_DEVICE_ID_INTEL_82801CA_10	0x248a
+#define PCI_DEVICE_ID_INTEL_82801CA_11	0x248b
+#define PCI_DEVICE_ID_INTEL_82801CA_12	0x248c
+#define PCI_DEVICE_ID_INTEL_82801DB_0	0x24c0
+#define PCI_DEVICE_ID_INTEL_82801DB_1	0x24c1
+#define PCI_DEVICE_ID_INTEL_82801DB_2	0x24c2
+#define PCI_DEVICE_ID_INTEL_82801DB_3	0x24c3
+#define PCI_DEVICE_ID_INTEL_82801DB_5	0x24c5
+#define PCI_DEVICE_ID_INTEL_82801DB_6	0x24c6
+#define PCI_DEVICE_ID_INTEL_82801DB_9	0x24c9
+#define PCI_DEVICE_ID_INTEL_82801DB_10	0x24ca
+#define PCI_DEVICE_ID_INTEL_82801DB_11	0x24cb
+#define PCI_DEVICE_ID_INTEL_82801DB_12  0x24cc
+#define PCI_DEVICE_ID_INTEL_82801EB_0	0x24d0
+#define PCI_DEVICE_ID_INTEL_82801EB_1	0x24d1
+#define PCI_DEVICE_ID_INTEL_82801EB_3	0x24d3
+#define PCI_DEVICE_ID_INTEL_82801EB_5	0x24d5
+#define PCI_DEVICE_ID_INTEL_82801EB_6	0x24d6
+#define PCI_DEVICE_ID_INTEL_82801EB_11	0x24db
+#define PCI_DEVICE_ID_INTEL_82801EB_12	0x24dc
+#define PCI_DEVICE_ID_INTEL_82801EB_13	0x24dd
+#define PCI_DEVICE_ID_INTEL_82820_HB	0x2500
+#define PCI_DEVICE_ID_INTEL_82820_UP_HB	0x2501
+#define PCI_DEVICE_ID_INTEL_82850_HB	0x2530
+#define PCI_DEVICE_ID_INTEL_82860_HB	0x2531
+#define PCI_DEVICE_ID_INTEL_E7501_MCH	0x254c
+#define PCI_DEVICE_ID_INTEL_7505_0	0x2550
+#define PCI_DEVICE_ID_INTEL_7205_0	0x255d
+#define PCI_DEVICE_ID_INTEL_82845G_HB	0x2560
+#define PCI_DEVICE_ID_INTEL_82845G_IG	0x2562
+#define PCI_DEVICE_ID_INTEL_82865_HB	0x2570
+#define PCI_DEVICE_ID_INTEL_82865_IG	0x2572
+#define PCI_DEVICE_ID_INTEL_82875_HB	0x2578
+#define PCI_DEVICE_ID_INTEL_82915G_HB	0x2580
+#define PCI_DEVICE_ID_INTEL_82915G_IG	0x2582
+#define PCI_DEVICE_ID_INTEL_82915GM_HB	0x2590
+#define PCI_DEVICE_ID_INTEL_82915GM_IG	0x2592
+#define PCI_DEVICE_ID_INTEL_ESB_1	0x25a1
+#define PCI_DEVICE_ID_INTEL_ESB_2	0x25a2
+#define PCI_DEVICE_ID_INTEL_ESB_4	0x25a4
+#define PCI_DEVICE_ID_INTEL_ESB_5	0x25a6
+#define PCI_DEVICE_ID_INTEL_ESB_9	0x25ab
+#define PCI_DEVICE_ID_INTEL_ESB_10	0x25ac
+#define PCI_DEVICE_ID_INTEL_5000_ERR	0x25f0
+#define PCI_DEVICE_ID_INTEL_5000_FBD0	0x25f5
+#define PCI_DEVICE_ID_INTEL_5000_FBD1	0x25f6
+#define PCI_DEVICE_ID_INTEL_ICH6_0	0x2640
+#define PCI_DEVICE_ID_INTEL_ICH6_1	0x2641
+#define PCI_DEVICE_ID_INTEL_ICH6_2	0x2642
+#define PCI_DEVICE_ID_INTEL_HDA_ICH6	0x2668
+#define PCI_DEVICE_ID_INTEL_ICH6_16	0x266a
+#define PCI_DEVICE_ID_INTEL_ICH6_17	0x266d
+#define PCI_DEVICE_ID_INTEL_ICH6_18	0x266e
+#define PCI_DEVICE_ID_INTEL_ICH6_19	0x266f
+#define PCI_DEVICE_ID_INTEL_ESB2_0	0x2670
+#define PCI_DEVICE_ID_INTEL_ESB2_14	0x2698
+#define PCI_DEVICE_ID_INTEL_HDA_ESB2	0x269a
+#define PCI_DEVICE_ID_INTEL_ESB2_17	0x269b
+#define PCI_DEVICE_ID_INTEL_ESB2_18	0x269e
+#define PCI_DEVICE_ID_INTEL_82945G_HB	0x2770
+#define PCI_DEVICE_ID_INTEL_82945G_IG	0x2772
+#define PCI_DEVICE_ID_INTEL_3000_HB	0x2778
+#define PCI_DEVICE_ID_INTEL_82945GM_HB	0x27a0
+#define PCI_DEVICE_ID_INTEL_82945GM_IG	0x27a2
+#define PCI_DEVICE_ID_INTEL_ICH7_30	0x27b0
+#define PCI_DEVICE_ID_INTEL_ICH7_0	0x27b8
+#define PCI_DEVICE_ID_INTEL_ICH7_1	0x27b9
+#define PCI_DEVICE_ID_INTEL_TGP_LPC	0x27bc
+#define PCI_DEVICE_ID_INTEL_ICH7_31	0x27bd
+#define PCI_DEVICE_ID_INTEL_HDA_ICH7	0x27d8
+#define PCI_DEVICE_ID_INTEL_ICH7_17	0x27da
+#define PCI_DEVICE_ID_INTEL_ICH7_19	0x27dd
+#define PCI_DEVICE_ID_INTEL_ICH7_20	0x27de
+#define PCI_DEVICE_ID_INTEL_ICH7_21	0x27df
+#define PCI_DEVICE_ID_INTEL_ICH8_0	0x2810
+#define PCI_DEVICE_ID_INTEL_ICH8_1	0x2811
+#define PCI_DEVICE_ID_INTEL_ICH8_2	0x2812
+#define PCI_DEVICE_ID_INTEL_ICH8_3	0x2814
+#define PCI_DEVICE_ID_INTEL_ICH8_4	0x2815
+#define PCI_DEVICE_ID_INTEL_ICH8_5	0x283e
+#define PCI_DEVICE_ID_INTEL_HDA_ICH8	0x284b
+#define PCI_DEVICE_ID_INTEL_ICH8_6	0x2850
+#define PCI_DEVICE_ID_INTEL_VMD_28C0	0x28c0
+#define PCI_DEVICE_ID_INTEL_ICH9_0	0x2910
+#define PCI_DEVICE_ID_INTEL_ICH9_2	0x2912
+#define PCI_DEVICE_ID_INTEL_ICH9_3	0x2913
+#define PCI_DEVICE_ID_INTEL_ICH9_4	0x2914
+#define PCI_DEVICE_ID_INTEL_ICH9_7	0x2916
+#define PCI_DEVICE_ID_INTEL_ICH9_1	0x2917
+#define PCI_DEVICE_ID_INTEL_ICH9_8	0x2918
+#define PCI_DEVICE_ID_INTEL_ICH9_5	0x2919
+#define PCI_DEVICE_ID_INTEL_ICH9_6	0x2930
+#define PCI_DEVICE_ID_INTEL_HDA_ICH9_0	0x293e
+#define PCI_DEVICE_ID_INTEL_HDA_ICH9_1	0x293f
+#define PCI_DEVICE_ID_INTEL_I7_MCR	0x2c18
+#define PCI_DEVICE_ID_INTEL_I7_MC_TAD	0x2c19
+#define PCI_DEVICE_ID_INTEL_I7_MC_RAS	0x2c1a
+#define PCI_DEVICE_ID_INTEL_I7_MC_TEST	0x2c1c
+#define PCI_DEVICE_ID_INTEL_I7_MC_CH0_CTRL  0x2c20
+#define PCI_DEVICE_ID_INTEL_I7_MC_CH0_ADDR  0x2c21
+#define PCI_DEVICE_ID_INTEL_I7_MC_CH0_RANK  0x2c22
+#define PCI_DEVICE_ID_INTEL_I7_MC_CH0_TC    0x2c23
+#define PCI_DEVICE_ID_INTEL_I7_MC_CH1_CTRL  0x2c28
+#define PCI_DEVICE_ID_INTEL_I7_MC_CH1_ADDR  0x2c29
+#define PCI_DEVICE_ID_INTEL_I7_MC_CH1_RANK  0x2c2a
+#define PCI_DEVICE_ID_INTEL_I7_MC_CH1_TC    0x2c2b
+#define PCI_DEVICE_ID_INTEL_I7_MC_CH2_CTRL  0x2c30
+#define PCI_DEVICE_ID_INTEL_I7_MC_CH2_ADDR  0x2c31
+#define PCI_DEVICE_ID_INTEL_I7_MC_CH2_RANK  0x2c32
+#define PCI_DEVICE_ID_INTEL_I7_MC_CH2_TC    0x2c33
+#define PCI_DEVICE_ID_INTEL_I7_NONCORE_ALT 0x2c40
+#define PCI_DEVICE_ID_INTEL_I7_NONCORE	0x2c41
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_NONCORE     0x2c50
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_NONCORE_ALT 0x2c51
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_NONCORE_REV2 0x2c70
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_SAD         0x2c81
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_QPI_LINK0   0x2c90
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_QPI_PHY0    0x2c91
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MCR         0x2c98
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_TAD      0x2c99
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_TEST     0x2c9c
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_CH0_CTRL 0x2ca0
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_CH0_ADDR 0x2ca1
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_CH0_RANK 0x2ca2
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_CH0_TC   0x2ca3
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_CH1_CTRL 0x2ca8
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_CH1_ADDR 0x2ca9
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_CH1_RANK 0x2caa
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_CH1_TC   0x2cab
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MCR_REV2          0x2d98
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_TAD_REV2       0x2d99
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_RAS_REV2       0x2d9a
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_TEST_REV2      0x2d9c
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_CH0_CTRL_REV2  0x2da0
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_CH0_ADDR_REV2  0x2da1
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_CH0_RANK_REV2  0x2da2
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_CH0_TC_REV2    0x2da3
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_CH1_CTRL_REV2  0x2da8
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_CH1_ADDR_REV2  0x2da9
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_CH1_RANK_REV2  0x2daa
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_CH1_TC_REV2    0x2dab
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_CH2_CTRL_REV2  0x2db0
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_CH2_ADDR_REV2  0x2db1
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_CH2_RANK_REV2  0x2db2
+#define PCI_DEVICE_ID_INTEL_LYNNFIELD_MC_CH2_TC_REV2    0x2db3
+#define PCI_DEVICE_ID_INTEL_HDA_GML	0x3198
+#define PCI_DEVICE_ID_INTEL_82855PM_HB	0x3340
+#define PCI_DEVICE_ID_INTEL_IOAT_TBG4	0x3429
+#define PCI_DEVICE_ID_INTEL_IOAT_TBG5	0x342a
+#define PCI_DEVICE_ID_INTEL_IOAT_TBG6	0x342b
+#define PCI_DEVICE_ID_INTEL_IOAT_TBG7	0x342c
+#define PCI_DEVICE_ID_INTEL_X58_HUB_MGMT 0x342e
+#define PCI_DEVICE_ID_INTEL_IOAT_TBG0	0x3430
+#define PCI_DEVICE_ID_INTEL_IOAT_TBG1	0x3431
+#define PCI_DEVICE_ID_INTEL_IOAT_TBG2	0x3432
+#define PCI_DEVICE_ID_INTEL_IOAT_TBG3	0x3433
+#define PCI_DEVICE_ID_INTEL_HDA_ICL_LP	0x34c8
+#define PCI_DEVICE_ID_INTEL_82830_HB	0x3575
+#define PCI_DEVICE_ID_INTEL_82830_CGC	0x3577
+#define PCI_DEVICE_ID_INTEL_82855GM_HB	0x3580
+#define PCI_DEVICE_ID_INTEL_82855GM_IG	0x3582
+#define PCI_DEVICE_ID_INTEL_82854_HB	0x358c
+#define PCI_DEVICE_ID_INTEL_82854_IG	0x358e
+#define PCI_DEVICE_ID_INTEL_E7520_MCH	0x3590
+#define PCI_DEVICE_ID_INTEL_E7320_MCH	0x3592
+#define PCI_DEVICE_ID_INTEL_MCH_PA	0x3595
+#define PCI_DEVICE_ID_INTEL_MCH_PA1	0x3596
+#define PCI_DEVICE_ID_INTEL_MCH_PB	0x3597
+#define PCI_DEVICE_ID_INTEL_MCH_PB1	0x3598
+#define PCI_DEVICE_ID_INTEL_MCH_PC	0x3599
+#define PCI_DEVICE_ID_INTEL_MCH_PC1	0x359a
+#define PCI_DEVICE_ID_INTEL_E7525_MCH	0x359e
+#define PCI_DEVICE_ID_INTEL_IOAT_CNB	0x360b
+#define PCI_DEVICE_ID_INTEL_FBD_CNB	0x360c
+#define PCI_DEVICE_ID_INTEL_I7300_MCH_ERR 0x360c
+#define PCI_DEVICE_ID_INTEL_I7300_MCH_FB0 0x360f
+#define PCI_DEVICE_ID_INTEL_I7300_MCH_FB1 0x3610
+#define PCI_DEVICE_ID_INTEL_IOAT_JSF0	0x3710
+#define PCI_DEVICE_ID_INTEL_IOAT_JSF1	0x3711
+#define PCI_DEVICE_ID_INTEL_IOAT_JSF2	0x3712
+#define PCI_DEVICE_ID_INTEL_IOAT_JSF3	0x3713
+#define PCI_DEVICE_ID_INTEL_IOAT_JSF4	0x3714
+#define PCI_DEVICE_ID_INTEL_IOAT_JSF5	0x3715
+#define PCI_DEVICE_ID_INTEL_IOAT_JSF6	0x3716
+#define PCI_DEVICE_ID_INTEL_IOAT_JSF7	0x3717
+#define PCI_DEVICE_ID_INTEL_IOAT_JSF8	0x3718
+#define PCI_DEVICE_ID_INTEL_IOAT_JSF9	0x3719
+#define PCI_DEVICE_ID_INTEL_QAT_C62X	0x37c8
+#define PCI_DEVICE_ID_INTEL_QAT_C62X_VF	0x37c9
+#define PCI_DEVICE_ID_INTEL_HDA_ICL_N	0x38c8
+#define PCI_DEVICE_ID_INTEL_ICH10_0	0x3a14
+#define PCI_DEVICE_ID_INTEL_ICH10_1	0x3a16
+#define PCI_DEVICE_ID_INTEL_ICH10_2	0x3a18
+#define PCI_DEVICE_ID_INTEL_ICH10_3	0x3a1a
+#define PCI_DEVICE_ID_INTEL_ICH10_4	0x3a30
+#define PCI_DEVICE_ID_INTEL_HDA_ICH10_0	0x3a3e
+#define PCI_DEVICE_ID_INTEL_ICH10_5	0x3a60
+#define PCI_DEVICE_ID_INTEL_HDA_ICH10_1 0x3a6e
+#define PCI_DEVICE_ID_INTEL_5_3400_SERIES_LPC_MIN	0x3b00
+#define PCI_DEVICE_ID_INTEL_5_3400_SERIES_LPC_MAX	0x3b1f
+#define PCI_DEVICE_ID_INTEL_HDA_5_3400_SERIES_0	0x3b56
+#define PCI_DEVICE_ID_INTEL_HDA_5_3400_SERIES_1	0x3b57
+#define PCI_DEVICE_ID_INTEL_IOAT_SNB0	0x3c20
+#define PCI_DEVICE_ID_INTEL_IOAT_SNB1	0x3c21
+#define PCI_DEVICE_ID_INTEL_IOAT_SNB2	0x3c22
+#define PCI_DEVICE_ID_INTEL_IOAT_SNB3	0x3c23
+#define PCI_DEVICE_ID_INTEL_IOAT_SNB4	0x3c24
+#define PCI_DEVICE_ID_INTEL_IOAT_SNB5	0x3c25
+#define PCI_DEVICE_ID_INTEL_IOAT_SNB6	0x3c26
+#define PCI_DEVICE_ID_INTEL_IOAT_SNB7	0x3c27
+#define PCI_DEVICE_ID_INTEL_IOAT_SNB8	0x3c2e
+#define PCI_DEVICE_ID_INTEL_IOAT_SNB9	0x3c2f
+#define PCI_DEVICE_ID_INTEL_UNC_QPI0	0x3c41
+#define PCI_DEVICE_ID_INTEL_UNC_QPI1	0x3c42
+#define PCI_DEVICE_ID_INTEL_UNC_R2PCIE	0x3c43
+#define PCI_DEVICE_ID_INTEL_UNC_R3QPI0	0x3c44
+#define PCI_DEVICE_ID_INTEL_UNC_R3QPI1	0x3c45
+#define PCI_DEVICE_ID_INTEL_UNC_HA	0x3c46
+#define PCI_DEVICE_ID_INTEL_SBRIDGE_IMC_RAS	0x3c71	/* 15.1 */
+#define PCI_DEVICE_ID_INTEL_SBRIDGE_IMC_ERR0	0x3c72	/* 16.2 */
+#define PCI_DEVICE_ID_INTEL_SBRIDGE_IMC_ERR1	0x3c73	/* 16.3 */
+#define PCI_DEVICE_ID_INTEL_SBRIDGE_IMC_ERR2	0x3c76	/* 16.6 */
+#define PCI_DEVICE_ID_INTEL_SBRIDGE_IMC_ERR3	0x3c77	/* 16.7 */
+#define PCI_DEVICE_ID_INTEL_SBRIDGE_IMC_HA0	0x3ca0	/* 14.0 */
+#define PCI_DEVICE_ID_INTEL_SBRIDGE_IMC_TA	0x3ca8	/* 15.0 */
+#define PCI_DEVICE_ID_INTEL_SBRIDGE_IMC_TAD0	0x3caa	/* 15.2 */
+#define PCI_DEVICE_ID_INTEL_SBRIDGE_IMC_TAD1	0x3cab	/* 15.3 */
+#define PCI_DEVICE_ID_INTEL_SBRIDGE_IMC_TAD2	0x3cac	/* 15.4 */
+#define PCI_DEVICE_ID_INTEL_SBRIDGE_IMC_TAD3	0x3cad	/* 15.5 */
+#define PCI_DEVICE_ID_INTEL_UNC_IMC0	0x3cb0
+#define PCI_DEVICE_ID_INTEL_UNC_IMC1	0x3cb1
+#define PCI_DEVICE_ID_INTEL_UNC_IMC2	0x3cb4
+#define PCI_DEVICE_ID_INTEL_UNC_IMC3	0x3cb5
+#define PCI_DEVICE_ID_INTEL_SBRIDGE_IMC_DDRIO	0x3cb8	/* 17.0 */
+#define PCI_DEVICE_ID_INTEL_JAKETOWN_UBOX	0x3ce0
+#define PCI_DEVICE_ID_INTEL_SBRIDGE_SAD0	0x3cf4	/* 12.6 */
+#define PCI_DEVICE_ID_INTEL_SBRIDGE_BR		0x3cf5	/* 13.6 */
+#define PCI_DEVICE_ID_INTEL_SBRIDGE_SAD1	0x3cf6	/* 12.7 */
+#define PCI_DEVICE_ID_INTEL_HDA_ICL_H	0x3dc8
+#define PCI_DEVICE_ID_INTEL_IOAT_SNB	0x402f
+#define PCI_DEVICE_ID_INTEL_5400_ERR	0x4030
+#define PCI_DEVICE_ID_INTEL_5400_FBD0	0x4035
+#define PCI_DEVICE_ID_INTEL_5400_FBD1	0x4036
+#define PCI_DEVICE_ID_INTEL_HDA_TGL_H	0x43c8
+#define PCI_DEVICE_ID_INTEL_HDA_DG1	0x490d
+#define PCI_DEVICE_ID_INTEL_HDA_EHL_0	0x4b55
+#define PCI_DEVICE_ID_INTEL_HDA_EHL_3	0x4b58
+#define PCI_DEVICE_ID_INTEL_HDA_JSL_N	0x4dc8
+#define PCI_DEVICE_ID_INTEL_HDA_DG2_0	0x4f90
+#define PCI_DEVICE_ID_INTEL_HDA_DG2_1	0x4f91
+#define PCI_DEVICE_ID_INTEL_HDA_DG2_2	0x4f92
+#define PCI_DEVICE_ID_INTEL_EP80579_0	0x5031
+#define PCI_DEVICE_ID_INTEL_EP80579_1	0x5032
+#define PCI_DEVICE_ID_INTEL_HDA_ADL_P	0x51c8
+#define PCI_DEVICE_ID_INTEL_HDA_ADL_PS	0x51c9
+#define PCI_DEVICE_ID_INTEL_HDA_RPL_P_0	0x51ca
+#define PCI_DEVICE_ID_INTEL_HDA_RPL_P_1	0x51cb
+#define PCI_DEVICE_ID_INTEL_HDA_ADL_M	0x51cc
+#define PCI_DEVICE_ID_INTEL_HDA_ADL_PX	0x51cd
+#define PCI_DEVICE_ID_INTEL_HDA_RPL_M	0x51ce
+#define PCI_DEVICE_ID_INTEL_HDA_RPL_PX	0x51cf
+#define PCI_DEVICE_ID_INTEL_HDA_ADL_N	0x54c8
+#define PCI_DEVICE_ID_INTEL_HDA_APL	0x5a98
+#define PCI_DEVICE_ID_INTEL_5100_16	0x65f0
+#define PCI_DEVICE_ID_INTEL_5100_19	0x65f3
+#define PCI_DEVICE_ID_INTEL_5100_21	0x65f5
+#define PCI_DEVICE_ID_INTEL_5100_22	0x65f6
+#define PCI_DEVICE_ID_INTEL_IOAT_SCNB	0x65ff
+#define PCI_DEVICE_ID_INTEL_82371SB_0	0x7000
+#define PCI_DEVICE_ID_INTEL_82371SB_1	0x7010
+#define PCI_DEVICE_ID_INTEL_82371SB_2	0x7020
+#define PCI_DEVICE_ID_INTEL_82437VX	0x7030
+#define PCI_DEVICE_ID_INTEL_82439TX	0x7100
+#define PCI_DEVICE_ID_INTEL_82371AB_0	0x7110
+#define PCI_DEVICE_ID_INTEL_82371AB	0x7111
+#define PCI_DEVICE_ID_INTEL_82371AB_2	0x7112
+#define PCI_DEVICE_ID_INTEL_82371AB_3	0x7113
+#define PCI_DEVICE_ID_INTEL_82810_MC1	0x7120
+#define PCI_DEVICE_ID_INTEL_82810_IG1	0x7121
+#define PCI_DEVICE_ID_INTEL_82810_MC3	0x7122
+#define PCI_DEVICE_ID_INTEL_82810_IG3	0x7123
+#define PCI_DEVICE_ID_INTEL_82810E_MC	0x7124
+#define PCI_DEVICE_ID_INTEL_82810E_IG	0x7125
+#define PCI_DEVICE_ID_INTEL_82443LX_0	0x7180
+#define PCI_DEVICE_ID_INTEL_82443LX_1	0x7181
+#define PCI_DEVICE_ID_INTEL_82443BX_0	0x7190
+#define PCI_DEVICE_ID_INTEL_82443BX_1	0x7191
+#define PCI_DEVICE_ID_INTEL_82443BX_2	0x7192
+#define PCI_DEVICE_ID_INTEL_440MX	0x7195
+#define PCI_DEVICE_ID_INTEL_440MX_6	0x7196
+#define PCI_DEVICE_ID_INTEL_82443MX_0	0x7198
+#define PCI_DEVICE_ID_INTEL_82443MX_1	0x7199
+#define PCI_DEVICE_ID_INTEL_82443MX_3	0x719b
+#define PCI_DEVICE_ID_INTEL_82443GX_0	0x71a0
+#define PCI_DEVICE_ID_INTEL_82443GX_2	0x71a2
+#define PCI_DEVICE_ID_INTEL_82372FB_1	0x7601
+#define PCI_DEVICE_ID_INTEL_HDA_RPL_S	0x7a50
+#define PCI_DEVICE_ID_INTEL_HDA_ADL_S	0x7ad0
+#define PCI_DEVICE_ID_INTEL_HDA_MTL	0x7e28
+#define PCI_DEVICE_ID_INTEL_HDA_ARL_S	0x7f50
+#define PCI_DEVICE_ID_INTEL_SCH_LPC	0x8119
+#define PCI_DEVICE_ID_INTEL_SCH_IDE	0x811a
+#define PCI_DEVICE_ID_INTEL_HDA_POULSBO	0x811b
+#define PCI_DEVICE_ID_INTEL_E6XX_CU	0x8183
+#define PCI_DEVICE_ID_INTEL_ITC_LPC	0x8186
+#define PCI_DEVICE_ID_INTEL_82454GX	0x84c4
+#define PCI_DEVICE_ID_INTEL_82450GX	0x84c5
+#define PCI_DEVICE_ID_INTEL_82451NX	0x84ca
+#define PCI_DEVICE_ID_INTEL_82454NX     0x84cb
+#define PCI_DEVICE_ID_INTEL_84460GX	0x84ea
+#define PCI_DEVICE_ID_INTEL_IXP4XX	0x8500
+#define PCI_DEVICE_ID_INTEL_HDA_LPT	0x8c20
+#define PCI_DEVICE_ID_INTEL_HDA_9_SERIES	0x8ca0
+#define PCI_DEVICE_ID_INTEL_HDA_WBG_0	0x8d20
+#define PCI_DEVICE_ID_INTEL_HDA_WBG_1	0x8d21
+#define PCI_DEVICE_ID_INTEL_IXP2800	0x9004
+#define PCI_DEVICE_ID_INTEL_HDA_LKF	0x98c8
+#define PCI_DEVICE_ID_INTEL_VMD_9A0B	0x9a0b
+#define PCI_DEVICE_ID_INTEL_HDA_LPT_LP_0	0x9c20
+#define PCI_DEVICE_ID_INTEL_HDA_LPT_LP_1	0x9c21
+#define PCI_DEVICE_ID_INTEL_HDA_WPT_LP	0x9ca0
+#define PCI_DEVICE_ID_INTEL_HDA_SKL_LP	0x9d70
+#define PCI_DEVICE_ID_INTEL_HDA_KBL_LP	0x9d71
+#define PCI_DEVICE_ID_INTEL_HDA_CNL_LP	0x9dc8
+#define PCI_DEVICE_ID_INTEL_HDA_TGL_LP	0xa0c8
+#define PCI_DEVICE_ID_INTEL_HDA_SKL	0xa170
+#define PCI_DEVICE_ID_INTEL_HDA_KBL	0xa171
+#define PCI_DEVICE_ID_INTEL_HDA_LBG_0	0xa1f0
+#define PCI_DEVICE_ID_INTEL_HDA_LBG_1	0xa270
+#define PCI_DEVICE_ID_INTEL_HDA_KBL_H	0xa2f0
+#define PCI_DEVICE_ID_INTEL_HDA_CNL_H	0xa348
+#define PCI_DEVICE_ID_INTEL_HDA_CML_S	0xa3f0
+#define PCI_DEVICE_ID_INTEL_HDA_LNL_P	0xa828
+#define PCI_DEVICE_ID_INTEL_S21152BB	0xb152
+#define PCI_DEVICE_ID_INTEL_HDA_CML_R	0xf0c8
+#define PCI_DEVICE_ID_INTEL_HDA_RKL_S	0xf1c8
+
+#define PCI_VENDOR_ID_WANGXUN		0x8088
+
+#define PCI_VENDOR_ID_SCALEMP		0x8686
+#define PCI_DEVICE_ID_SCALEMP_VSMP_CTL	0x1010
+
+#define PCI_VENDOR_ID_COMPUTONE		0x8e0e
+#define PCI_DEVICE_ID_COMPUTONE_PG	0x0302
+#define PCI_SUBVENDOR_ID_COMPUTONE	0x8e0e
+#define PCI_SUBDEVICE_ID_COMPUTONE_PG4	0x0001
+#define PCI_SUBDEVICE_ID_COMPUTONE_PG8	0x0002
+#define PCI_SUBDEVICE_ID_COMPUTONE_PG6	0x0003
+
+#define PCI_VENDOR_ID_KTI		0x8e2e
+
+#define PCI_VENDOR_ID_ADAPTEC		0x9004
+#define PCI_DEVICE_ID_ADAPTEC_7810	0x1078
+#define PCI_DEVICE_ID_ADAPTEC_7821	0x2178
+#define PCI_DEVICE_ID_ADAPTEC_38602	0x3860
+#define PCI_DEVICE_ID_ADAPTEC_7850	0x5078
+#define PCI_DEVICE_ID_ADAPTEC_7855	0x5578
+#define PCI_DEVICE_ID_ADAPTEC_3860	0x6038
+#define PCI_DEVICE_ID_ADAPTEC_1480A	0x6075
+#define PCI_DEVICE_ID_ADAPTEC_7860	0x6078
+#define PCI_DEVICE_ID_ADAPTEC_7861	0x6178
+#define PCI_DEVICE_ID_ADAPTEC_7870	0x7078
+#define PCI_DEVICE_ID_ADAPTEC_7871	0x7178
+#define PCI_DEVICE_ID_ADAPTEC_7872	0x7278
+#define PCI_DEVICE_ID_ADAPTEC_7873	0x7378
+#define PCI_DEVICE_ID_ADAPTEC_7874	0x7478
+#define PCI_DEVICE_ID_ADAPTEC_7895	0x7895
+#define PCI_DEVICE_ID_ADAPTEC_7880	0x8078
+#define PCI_DEVICE_ID_ADAPTEC_7881	0x8178
+#define PCI_DEVICE_ID_ADAPTEC_7882	0x8278
+#define PCI_DEVICE_ID_ADAPTEC_7883	0x8378
+#define PCI_DEVICE_ID_ADAPTEC_7884	0x8478
+#define PCI_DEVICE_ID_ADAPTEC_7885	0x8578
+#define PCI_DEVICE_ID_ADAPTEC_7886	0x8678
+#define PCI_DEVICE_ID_ADAPTEC_7887	0x8778
+#define PCI_DEVICE_ID_ADAPTEC_7888	0x8878
+
+#define PCI_VENDOR_ID_ADAPTEC2		0x9005
+#define PCI_DEVICE_ID_ADAPTEC2_2940U2	0x0010
+#define PCI_DEVICE_ID_ADAPTEC2_2930U2	0x0011
+#define PCI_DEVICE_ID_ADAPTEC2_7890B	0x0013
+#define PCI_DEVICE_ID_ADAPTEC2_7890	0x001f
+#define PCI_DEVICE_ID_ADAPTEC2_3940U2	0x0050
+#define PCI_DEVICE_ID_ADAPTEC2_3950U2D	0x0051
+#define PCI_DEVICE_ID_ADAPTEC2_7896	0x005f
+#define PCI_DEVICE_ID_ADAPTEC2_7892A	0x0080
+#define PCI_DEVICE_ID_ADAPTEC2_7892B	0x0081
+#define PCI_DEVICE_ID_ADAPTEC2_7892D	0x0083
+#define PCI_DEVICE_ID_ADAPTEC2_7892P	0x008f
+#define PCI_DEVICE_ID_ADAPTEC2_7899A	0x00c0
+#define PCI_DEVICE_ID_ADAPTEC2_7899B	0x00c1
+#define PCI_DEVICE_ID_ADAPTEC2_7899D	0x00c3
+#define PCI_DEVICE_ID_ADAPTEC2_7899P	0x00cf
+#define PCI_DEVICE_ID_ADAPTEC2_OBSIDIAN   0x0500
+#define PCI_DEVICE_ID_ADAPTEC2_SCAMP	0x0503
+
+#define PCI_VENDOR_ID_HOLTEK		0x9412
+#define PCI_DEVICE_ID_HOLTEK_6565	0x6565
+
+#define PCI_VENDOR_ID_NETMOS		0x9710
+#define PCI_DEVICE_ID_NETMOS_9705	0x9705
+#define PCI_DEVICE_ID_NETMOS_9715	0x9715
+#define PCI_DEVICE_ID_NETMOS_9735	0x9735
+#define PCI_DEVICE_ID_NETMOS_9745	0x9745
+#define PCI_DEVICE_ID_NETMOS_9755	0x9755
+#define PCI_DEVICE_ID_NETMOS_9805	0x9805
+#define PCI_DEVICE_ID_NETMOS_9815	0x9815
+#define PCI_DEVICE_ID_NETMOS_9835	0x9835
+#define PCI_DEVICE_ID_NETMOS_9845	0x9845
+#define PCI_DEVICE_ID_NETMOS_9855	0x9855
+#define PCI_DEVICE_ID_NETMOS_9865	0x9865
+#define PCI_DEVICE_ID_NETMOS_9900	0x9900
+#define PCI_DEVICE_ID_NETMOS_9901	0x9901
+#define PCI_DEVICE_ID_NETMOS_9904	0x9904
+#define PCI_DEVICE_ID_NETMOS_9912	0x9912
+#define PCI_DEVICE_ID_NETMOS_9922	0x9922
+
+#define PCI_VENDOR_ID_3COM_2		0xa727
+
+#define PCI_VENDOR_ID_SOLIDRUN		0xd063
+
+#define PCI_VENDOR_ID_DIGIUM		0xd161
+#define PCI_DEVICE_ID_DIGIUM_HFC4S	0xb410
+
+#define PCI_SUBVENDOR_ID_EXSYS		0xd84d
+#define PCI_SUBDEVICE_ID_EXSYS_4014	0x4014
+#define PCI_SUBDEVICE_ID_EXSYS_4055	0x4055
+
+#define PCI_VENDOR_ID_TIGERJET		0xe159
+#define PCI_DEVICE_ID_TIGERJET_300	0x0001
+#define PCI_DEVICE_ID_TIGERJET_100	0x0002
+
+#define PCI_VENDOR_ID_XILINX_RME	0xea60
+#define PCI_DEVICE_ID_RME_DIGI32	0x9896
+#define PCI_DEVICE_ID_RME_DIGI32_PRO	0x9897
+#define PCI_DEVICE_ID_RME_DIGI32_8	0x9898
+
+#define PCI_VENDOR_ID_XEN		0x5853
+#define PCI_DEVICE_ID_XEN_PLATFORM	0x0001
+
+#define PCI_VENDOR_ID_OCZ		0x1b85
+
+#define PCI_VENDOR_ID_NCUBE		0x10ff
+
+#endif /* _LINUX_PCI_IDS_H */
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 16/33] vfio: selftests: Keep track of DMA regions mapped into the device

[David Matlack]

Keep track of the list of DMA regions that are mapped into the device
using a linked list and a new struct vfio_dma_region and use that to add
{__,}to_iova() for converting host virtual addresses into IOVAs.

This will be used in a subsequent commit to map multiple DMA regions
into a device that are then used by drivers.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 .../selftests/vfio/lib/include/vfio_util.h    | 23 +++++++--
 .../selftests/vfio/lib/vfio_pci_device.c      | 49 ++++++++++++++++---
 .../selftests/vfio/vfio_dma_mapping_test.c    | 20 +++++---
 3 files changed, 75 insertions(+), 17 deletions(-)

diff --git a/tools/testing/selftests/vfio/lib/include/vfio_util.h b/tools/testing/selftests/vfio/lib/include/vfio_util.h
index 4e5c6d395a12..ab96a6628f0e 100644
--- a/tools/testing/selftests/vfio/lib/include/vfio_util.h
+++ b/tools/testing/selftests/vfio/lib/include/vfio_util.h
@@ -51,6 +51,17 @@ struct vfio_pci_bar {
 	void *vaddr;
 };
 
+typedef u64 iova_t;
+
+#define INVALID_IOVA UINT64_MAX
+
+struct vfio_dma_region {
+	struct list_head link;
+	void *vaddr;
+	iova_t iova;
+	u64 size;
+};
+
 struct vfio_pci_device {
 	int fd;
 	int group_fd;
@@ -63,6 +74,8 @@ struct vfio_pci_device {
 	struct vfio_irq_info msi_info;
 	struct vfio_irq_info msix_info;
 
+	struct list_head dma_regions;
+
 	/* eventfds for MSI and MSI-x interrupts */
 	int msi_eventfds[PCI_MSIX_FLAGS_QSIZE + 1];
 };
@@ -71,9 +84,10 @@ struct vfio_pci_device *vfio_pci_device_init(const char *bdf, int iommu_type);
 void vfio_pci_device_cleanup(struct vfio_pci_device *device);
 void vfio_pci_device_reset(struct vfio_pci_device *device);
 
-void vfio_pci_dma_map(struct vfio_pci_device *device, u64 iova, u64 size,
-		      void *vaddr);
-void vfio_pci_dma_unmap(struct vfio_pci_device *device, u64 iova, u64 size);
+void vfio_pci_dma_map(struct vfio_pci_device *device,
+		      struct vfio_dma_region *region);
+void vfio_pci_dma_unmap(struct vfio_pci_device *device,
+			struct vfio_dma_region *region);
 
 void vfio_pci_config_access(struct vfio_pci_device *device, bool write,
 			    size_t config, size_t size, void *data);
@@ -124,4 +138,7 @@ static inline void vfio_pci_msix_disable(struct vfio_pci_device *device)
 	vfio_pci_irq_disable(device, VFIO_PCI_MSIX_IRQ_INDEX);
 }
 
+iova_t __to_iova(struct vfio_pci_device *device, void *vaddr);
+iova_t to_iova(struct vfio_pci_device *device, void *vaddr);
+
 #endif /* SELFTESTS_VFIO_LIB_INCLUDE_VFIO_UTIL_H */
diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_device.c b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
index a2747de9ad69..e823d3ff7f45 100644
--- a/tools/testing/selftests/vfio/lib/vfio_pci_device.c
+++ b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
@@ -27,6 +27,33 @@
 		    "ioctl(%s, %s, %s) returned %d\n", #_fd, #_op, #_arg, __ret); \
 } while (0)
 
+iova_t __to_iova(struct vfio_pci_device *device, void *vaddr)
+{
+	struct vfio_dma_region *region;
+
+	list_for_each_entry(region, &device->dma_regions, link) {
+		if (vaddr < region->vaddr)
+			continue;
+
+		if (vaddr >= region->vaddr + region->size)
+			continue;
+
+		return region->iova + (vaddr - region->vaddr);
+	}
+
+	return INVALID_IOVA;
+}
+
+iova_t to_iova(struct vfio_pci_device *device, void *vaddr)
+{
+	iova_t iova;
+
+	iova = __to_iova(device, vaddr);
+	VFIO_ASSERT_NE(iova, INVALID_IOVA, "%p is not mapped into device.\n", vaddr);
+
+	return iova;
+}
+
 static void vfio_pci_irq_set(struct vfio_pci_device *device,
 			     u32 index, u32 vector, u32 count, int *fds)
 {
@@ -113,28 +140,34 @@ static void vfio_pci_irq_get(struct vfio_pci_device *device, u32 index,
 	ioctl_assert(device->fd, VFIO_DEVICE_GET_IRQ_INFO, irq_info);
 }
 
-void vfio_pci_dma_map(struct vfio_pci_device *device, u64 iova, u64 size, void *vaddr)
+void vfio_pci_dma_map(struct vfio_pci_device *device,
+		      struct vfio_dma_region *region)
 {
 	struct vfio_iommu_type1_dma_map map = {
 		.argsz = sizeof(map),
 		.flags = VFIO_DMA_MAP_FLAG_READ | VFIO_DMA_MAP_FLAG_WRITE,
-		.vaddr = (u64)vaddr,
-		.iova = iova,
-		.size = size,
+		.vaddr = (u64)region->vaddr,
+		.iova = region->iova,
+		.size = region->size,
 	};
 
 	ioctl_assert(device->container_fd, VFIO_IOMMU_MAP_DMA, &map);
+
+	list_add(&region->link, &device->dma_regions);
 }
 
-void vfio_pci_dma_unmap(struct vfio_pci_device *device, u64 iova, u64 size)
+void vfio_pci_dma_unmap(struct vfio_pci_device *device,
+			struct vfio_dma_region *region)
 {
 	struct vfio_iommu_type1_dma_unmap unmap = {
 		.argsz = sizeof(unmap),
-		.iova = iova,
-		.size = size,
+		.iova = region->iova,
+		.size = region->size,
 	};
 
 	ioctl_assert(device->container_fd, VFIO_IOMMU_UNMAP_DMA, &unmap);
+
+	list_del(&region->link);
 }
 
 static void vfio_pci_region_get(struct vfio_pci_device *device, int index,
@@ -256,6 +289,8 @@ static void vfio_pci_iommu_setup(struct vfio_pci_device *device, unsigned long i
 {
 	int ret;
 
+	INIT_LIST_HEAD(&device->dma_regions);
+
 	ret = ioctl(device->container_fd, VFIO_CHECK_EXTENSION, iommu_type);
 	VFIO_ASSERT_GT(ret, 0, "VFIO IOMMU type %lu not supported\n", iommu_type);
 
diff --git a/tools/testing/selftests/vfio/vfio_dma_mapping_test.c b/tools/testing/selftests/vfio/vfio_dma_mapping_test.c
index c4920267ae42..d85621d165d9 100644
--- a/tools/testing/selftests/vfio/vfio_dma_mapping_test.c
+++ b/tools/testing/selftests/vfio/vfio_dma_mapping_test.c
@@ -117,15 +117,20 @@ TEST_F(vfio_dma_mapping_test, dma_map_unmap)
 	const int prot = PROT_READ | PROT_WRITE;
 	const u64 iova = test_config.iova;
 	const u64 size = test_config.size;
+	struct vfio_dma_region region = {
+		.iova = iova,
+		.size = size,
+	};
 	struct iommu_mapping mapping;
-	void *mem;
 	int rc;
 
-	mem = mmap(NULL, size, prot, test_config.mmap_flags, -1, 0);
-	ASSERT_NE(mem, MAP_FAILED);
+	region.vaddr = mmap(NULL, size, prot, test_config.mmap_flags, -1, 0);
+	ASSERT_NE(region.vaddr, MAP_FAILED);
 
-	vfio_pci_dma_map(self->device, iova, size, mem);
-	printf("Mapped HVA %p (size 0x%lx) at IOVA 0x%lx\n", mem, size, iova);
+	vfio_pci_dma_map(self->device, &region);
+	printf("Mapped HVA %p (size 0x%lx) at IOVA 0x%lx\n", region.vaddr, size, iova);
+
+	ASSERT_EQ(iova, to_iova(self->device, region.vaddr));
 
 	rc = iommu_mapping_get(test_config.bdf, iova, &mapping);
 	if (rc == -EOPNOTSUPP)
@@ -157,11 +162,12 @@ TEST_F(vfio_dma_mapping_test, dma_map_unmap)
 	}
 
 unmap:
-	vfio_pci_dma_unmap(self->device, iova, size);
+	vfio_pci_dma_unmap(self->device, &region);
 	printf("Unmapped IOVA 0x%lx\n", iova);
+	ASSERT_EQ(INVALID_IOVA, __to_iova(self->device, region.vaddr));
 	ASSERT_NE(0, iommu_mapping_get(test_config.bdf, iova, &mapping));
 
-	ASSERT_TRUE(!munmap(mem, size));
+	ASSERT_TRUE(!munmap(region.vaddr, size));
 }
 
 static void help(const char *name)
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 17/33] vfio: selftests: Enable asserting MSI eventfds not firing

[David Matlack]

Make it possible to assert that a given MSI eventfd did _not_ fire by
adding a helper to mark an eventfd non-blocking. Demonstrate this in
vfio_pci_device_test by asserting the MSI eventfd did not fire before
vfio_pci_irq_trigger().

Signed-off-by: David Matlack <dmatlack@google.com>
---
 tools/testing/selftests/vfio/lib/include/vfio_util.h | 12 ++++++++++++
 tools/testing/selftests/vfio/vfio_pci_device_test.c  | 10 +++++++++-
 2 files changed, 21 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/vfio/lib/include/vfio_util.h b/tools/testing/selftests/vfio/lib/include/vfio_util.h
index ab96a6628f0e..2b96be07f182 100644
--- a/tools/testing/selftests/vfio/lib/include/vfio_util.h
+++ b/tools/testing/selftests/vfio/lib/include/vfio_util.h
@@ -2,6 +2,7 @@
 #ifndef SELFTESTS_VFIO_LIB_INCLUDE_VFIO_UTIL_H
 #define SELFTESTS_VFIO_LIB_INCLUDE_VFIO_UTIL_H
 
+#include <fcntl.h>
 #include <string.h>
 #include <linux/vfio.h>
 #include <linux/list.h>
@@ -116,6 +117,17 @@ void vfio_pci_irq_enable(struct vfio_pci_device *device, u32 index,
 void vfio_pci_irq_disable(struct vfio_pci_device *device, u32 index);
 void vfio_pci_irq_trigger(struct vfio_pci_device *device, u32 index, u32 vector);
 
+static inline void fcntl_set_nonblock(int fd)
+{
+	int r;
+
+	r = fcntl(fd, F_GETFL, 0);
+	VFIO_ASSERT_NE(r, -1, "F_GETFL failed for fd %d\n", fd);
+
+	r = fcntl(fd, F_SETFL, r | O_NONBLOCK);
+	VFIO_ASSERT_NE(r, -1, "F_SETFL O_NONBLOCK failed for fd %d\n", fd);
+}
+
 static inline void vfio_pci_msi_enable(struct vfio_pci_device *device,
 				       u32 vector, int count)
 {
diff --git a/tools/testing/selftests/vfio/vfio_pci_device_test.c b/tools/testing/selftests/vfio/vfio_pci_device_test.c
index b969ecdfb042..27f60cccd3a7 100644
--- a/tools/testing/selftests/vfio/vfio_pci_device_test.c
+++ b/tools/testing/selftests/vfio/vfio_pci_device_test.c
@@ -125,6 +125,7 @@ FIXTURE_TEARDOWN(vfio_pci_irq_test) {
 TEST_F(vfio_pci_irq_test, enable_trigger_disable)
 {
 	bool msix = variant->irq_index == VFIO_PCI_MSIX_IRQ_INDEX;
+	int msi_eventfd;
 	u32 count;
 	u64 value;
 	int i;
@@ -143,8 +144,15 @@ TEST_F(vfio_pci_irq_test, enable_trigger_disable)
 	printf("MSI%s: enabled %d interrupts\n", msix ? "-x" : "", count);
 
 	for (i = 0; i < count; i++) {
+		msi_eventfd = self->device->msi_eventfds[i];
+
+		fcntl_set_nonblock(msi_eventfd);
+		ASSERT_EQ(-1, read(msi_eventfd, &value, 8));
+		ASSERT_EQ(EAGAIN, errno);
+
 		vfio_pci_irq_trigger(self->device, variant->irq_index, i);
-		ASSERT_EQ(8, read(self->device->msi_eventfds[i], &value, 8));
+
+		ASSERT_EQ(8, read(msi_eventfd, &value, 8));
 		ASSERT_EQ(1, value);
 	}
 
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 18/33] vfio: selftests: Add a helper for matching vendor+device IDs

[David Matlack]

Add a helper function for matching a device against a given vendor and
device ID. This will be used in a subsequent commit to match devices
against drivers.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 tools/testing/selftests/vfio/lib/include/vfio_util.h | 7 +++++++
 tools/testing/selftests/vfio/vfio_pci_device_test.c  | 4 +---
 2 files changed, 8 insertions(+), 3 deletions(-)

diff --git a/tools/testing/selftests/vfio/lib/include/vfio_util.h b/tools/testing/selftests/vfio/lib/include/vfio_util.h
index 2b96be07f182..df56c84273e8 100644
--- a/tools/testing/selftests/vfio/lib/include/vfio_util.h
+++ b/tools/testing/selftests/vfio/lib/include/vfio_util.h
@@ -153,4 +153,11 @@ static inline void vfio_pci_msix_disable(struct vfio_pci_device *device)
 iova_t __to_iova(struct vfio_pci_device *device, void *vaddr);
 iova_t to_iova(struct vfio_pci_device *device, void *vaddr);
 
+static inline bool vfio_pci_device_match(struct vfio_pci_device *device,
+					 u16 vendor_id, u16 device_id)
+{
+	return (vendor_id == vfio_pci_config_readw(device, PCI_VENDOR_ID)) &&
+		(device_id == vfio_pci_config_readw(device, PCI_DEVICE_ID));
+}
+
 #endif /* SELFTESTS_VFIO_LIB_INCLUDE_VFIO_UTIL_H */
diff --git a/tools/testing/selftests/vfio/vfio_pci_device_test.c b/tools/testing/selftests/vfio/vfio_pci_device_test.c
index 27f60cccd3a7..1a26df0f2280 100644
--- a/tools/testing/selftests/vfio/vfio_pci_device_test.c
+++ b/tools/testing/selftests/vfio/vfio_pci_device_test.c
@@ -54,9 +54,7 @@ TEST_F(vfio_pci_device_test, config_space_read_write)
 	/* Check that Vendor and Device match what the kernel reports. */
 	vendor = read_pci_id_from_sysfs("vendor");
 	device = read_pci_id_from_sysfs("device");
-
-	ASSERT_EQ(vendor, vfio_pci_config_readw(self->device, PCI_VENDOR_ID));
-	ASSERT_EQ(device, vfio_pci_config_readw(self->device, PCI_DEVICE_ID));
+	ASSERT_TRUE(vfio_pci_device_match(self->device, vendor, device));
 
 	printf("Vendor: %04x, Device: %04x\n", vendor, device);
 
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 19/33] vfio: selftests: Add driver framework

[David Matlack]

Add a driver framework to VFIO selftests, so that devices can generate
DMA and interrupts in a common way that can be then utilized by tests.
This will enable VFIO selftests to exercise real hardware DMA and
interrupt paths, without needing any device-specific code in the test
itself.

Subsequent commits will introduce drivers for specific devices.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 .../selftests/vfio/lib/include/vfio_util.h    |  92 ++++++++++++++
 tools/testing/selftests/vfio/lib/libvfio.mk   |   1 +
 .../selftests/vfio/lib/vfio_pci_device.c      |   5 +
 .../selftests/vfio/lib/vfio_pci_driver.c      | 116 ++++++++++++++++++
 4 files changed, 214 insertions(+)
 create mode 100644 tools/testing/selftests/vfio/lib/vfio_pci_driver.c

diff --git a/tools/testing/selftests/vfio/lib/include/vfio_util.h b/tools/testing/selftests/vfio/lib/include/vfio_util.h
index df56c84273e8..c824c993d82e 100644
--- a/tools/testing/selftests/vfio/lib/include/vfio_util.h
+++ b/tools/testing/selftests/vfio/lib/include/vfio_util.h
@@ -63,6 +63,85 @@ struct vfio_dma_region {
 	u64 size;
 };
 
+struct vfio_pci_device;
+
+struct vfio_pci_driver_ops {
+	const char *name;
+
+	/**
+	 * @probe() - Check if the driver supports the given device.
+	 *
+	 * Return: 0 on success, non-0 on failure.
+	 */
+	int (*probe)(struct vfio_pci_device *device);
+
+	/**
+	 * @init() - Initialize the driver for @device.
+	 *
+	 * Must be called after device->driver.region has been initialized.
+	 */
+	void (*init)(struct vfio_pci_device *device);
+
+	/**
+	 * remove() - Deinitialize the driver for @device.
+	 */
+	void (*remove)(struct vfio_pci_device *device);
+
+	/**
+	 * memcpy_start() - Kick off @count repeated memcpy operations from
+	 * [@src, @src + @size) to [@dst, @dst + @size).
+	 *
+	 * Guarantees:
+	 *  - The device will attempt DMA reads on [src, src + size).
+	 *  - The device will attempt DMA writes on [dst, dst + size).
+	 *  - The device will not generate any interrupts.
+	 *
+	 * memcpy_start() returns immediately, it does not wait for the
+	 * copies to complete.
+	 */
+	void (*memcpy_start)(struct vfio_pci_device *device,
+			     iova_t src, iova_t dst, u64 size, u64 count);
+
+	/**
+	 * memcpy_wait() - Wait until the memcpy operations started by
+	 * memcpy_start() have finished.
+	 *
+	 * Guarantees:
+	 *  - All in-flight DMAs initiated by memcpy_start() are fully complete
+	 *    before memcpy_wait() returns.
+	 *
+	 * Returns non-0 if the driver detects that an error occurred during the
+	 * memcpy, 0 otherwise.
+	 */
+	int (*memcpy_wait)(struct vfio_pci_device *device);
+
+	/**
+	 * send_msi() - Make the device send the MSI device->driver.msi.
+	 *
+	 * Guarantees:
+	 *  - The device will send the MSI once.
+	 */
+	void (*send_msi)(struct vfio_pci_device *device);
+};
+
+struct vfio_pci_driver {
+	const struct vfio_pci_driver_ops *ops;
+	bool initialized;
+	bool memcpy_in_progress;
+
+	/* Region to be used by the driver (e.g. for in-memory descriptors) */
+	struct vfio_dma_region region;
+
+	/* The maximum size that can be passed to memcpy_start(). */
+	u64 max_memcpy_size;
+
+	/* The maximum count that can be passed to memcpy_start(). */
+	u64 max_memcpy_count;
+
+	/* The MSI vector the device will signal in ops->send_msi(). */
+	int msi;
+};
+
 struct vfio_pci_device {
 	int fd;
 	int group_fd;
@@ -79,6 +158,8 @@ struct vfio_pci_device {
 
 	/* eventfds for MSI and MSI-x interrupts */
 	int msi_eventfds[PCI_MSIX_FLAGS_QSIZE + 1];
+
+	struct vfio_pci_driver driver;
 };
 
 struct vfio_pci_device *vfio_pci_device_init(const char *bdf, int iommu_type);
@@ -160,4 +241,15 @@ static inline bool vfio_pci_device_match(struct vfio_pci_device *device,
 		(device_id == vfio_pci_config_readw(device, PCI_DEVICE_ID));
 }
 
+void vfio_pci_driver_probe(struct vfio_pci_device *device);
+void vfio_pci_driver_init(struct vfio_pci_device *device);
+void vfio_pci_driver_remove(struct vfio_pci_device *device);
+int vfio_pci_driver_memcpy(struct vfio_pci_device *device,
+			   iova_t src, iova_t dst, u64 size);
+void vfio_pci_driver_memcpy_start(struct vfio_pci_device *device,
+				  iova_t src, iova_t dst, u64 size,
+				  u64 count);
+int vfio_pci_driver_memcpy_wait(struct vfio_pci_device *device);
+void vfio_pci_driver_send_msi(struct vfio_pci_device *device);
+
 #endif /* SELFTESTS_VFIO_LIB_INCLUDE_VFIO_UTIL_H */
diff --git a/tools/testing/selftests/vfio/lib/libvfio.mk b/tools/testing/selftests/vfio/lib/libvfio.mk
index 388533f8d407..10a1a9316e4c 100644
--- a/tools/testing/selftests/vfio/lib/libvfio.mk
+++ b/tools/testing/selftests/vfio/lib/libvfio.mk
@@ -1,6 +1,7 @@
 VFIO_DIR := $(selfdir)/vfio
 
 LIBVFIO_C := lib/vfio_pci_device.c
+LIBVFIO_C += lib/vfio_pci_driver.c
 
 LIBVFIO_O := $(patsubst %.c, $(OUTPUT)/%.o, $(LIBVFIO_C))
 
diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_device.c b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
index e823d3ff7f45..63f033047b4c 100644
--- a/tools/testing/selftests/vfio/lib/vfio_pci_device.c
+++ b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
@@ -340,6 +340,8 @@ struct vfio_pci_device *vfio_pci_device_init(const char *bdf, int iommu_type)
 	vfio_pci_iommu_setup(device, iommu_type);
 	vfio_pci_device_setup(device, bdf);
 
+	vfio_pci_driver_probe(device);
+
 	return device;
 }
 
@@ -347,6 +349,9 @@ void vfio_pci_device_cleanup(struct vfio_pci_device *device)
 {
 	int i;
 
+	if (device->driver.initialized)
+		vfio_pci_driver_remove(device);
+
 	vfio_pci_bar_unmap_all(device);
 
 	VFIO_ASSERT_EQ(close(device->fd), 0);
diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_driver.c b/tools/testing/selftests/vfio/lib/vfio_pci_driver.c
new file mode 100644
index 000000000000..c98bd2d31d8a
--- /dev/null
+++ b/tools/testing/selftests/vfio/lib/vfio_pci_driver.c
@@ -0,0 +1,116 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include <stdio.h>
+
+#include "../../../kselftest.h"
+#include <vfio_util.h>
+
+static struct vfio_pci_driver_ops *driver_ops[] = {};
+
+void vfio_pci_driver_probe(struct vfio_pci_device *device)
+{
+	struct vfio_pci_driver_ops *ops;
+	int i;
+
+	VFIO_ASSERT_NULL(device->driver.ops);
+
+	for (i = 0; i < ARRAY_SIZE(driver_ops); i++) {
+		ops = driver_ops[i];
+
+		if (ops->probe(device))
+			continue;
+
+		printf("Driver found: %s\n", ops->name);
+		device->driver.ops = ops;
+	}
+}
+
+static void vfio_check_driver_op(struct vfio_pci_driver *driver, void *op,
+				 const char *op_name)
+{
+	VFIO_ASSERT_NOT_NULL(driver->ops);
+	VFIO_ASSERT_NOT_NULL(op, "Driver has no %s()\n", op_name);
+	VFIO_ASSERT_EQ(driver->initialized, op != driver->ops->init);
+	VFIO_ASSERT_EQ(driver->memcpy_in_progress, op == driver->ops->memcpy_wait);
+}
+
+#define VFIO_CHECK_DRIVER_OP(_driver, _op) do {				\
+	struct vfio_pci_driver *__driver = (_driver);			\
+	vfio_check_driver_op(__driver, __driver->ops->_op, #_op);	\
+} while (0)
+
+void vfio_pci_driver_init(struct vfio_pci_device *device)
+{
+	struct vfio_pci_driver *driver = &device->driver;
+
+	VFIO_ASSERT_NOT_NULL(driver->region.vaddr);
+	VFIO_CHECK_DRIVER_OP(driver, init);
+
+	driver->ops->init(device);
+
+	driver->initialized = true;
+
+	printf("%s: region: vaddr %p, iova 0x%lx, size 0x%lx\n",
+	       driver->ops->name,
+	       driver->region.vaddr,
+	       driver->region.iova,
+	       driver->region.size);
+
+	printf("%s: max_memcpy_size 0x%lx, max_memcpy_count 0x%lx\n",
+	       driver->ops->name,
+	       driver->max_memcpy_size,
+	       driver->max_memcpy_count);
+}
+
+void vfio_pci_driver_remove(struct vfio_pci_device *device)
+{
+	struct vfio_pci_driver *driver = &device->driver;
+
+	VFIO_CHECK_DRIVER_OP(driver, remove);
+
+	driver->ops->remove(device);
+	driver->initialized = false;
+}
+
+void vfio_pci_driver_send_msi(struct vfio_pci_device *device)
+{
+	struct vfio_pci_driver *driver = &device->driver;
+
+	VFIO_CHECK_DRIVER_OP(driver, send_msi);
+
+	driver->ops->send_msi(device);
+}
+
+void vfio_pci_driver_memcpy_start(struct vfio_pci_device *device,
+				  iova_t src, iova_t dst, u64 size,
+				  u64 count)
+{
+	struct vfio_pci_driver *driver = &device->driver;
+
+	VFIO_ASSERT_LE(size, driver->max_memcpy_size);
+	VFIO_ASSERT_LE(count, driver->max_memcpy_count);
+	VFIO_CHECK_DRIVER_OP(driver, memcpy_start);
+
+	driver->ops->memcpy_start(device, src, dst, size, count);
+	driver->memcpy_in_progress = true;
+}
+
+int vfio_pci_driver_memcpy_wait(struct vfio_pci_device *device)
+{
+	struct vfio_pci_driver *driver = &device->driver;
+	int r;
+
+	VFIO_CHECK_DRIVER_OP(driver, memcpy_wait);
+
+	r = driver->ops->memcpy_wait(device);
+	driver->memcpy_in_progress = false;
+
+	return r;
+}
+
+int vfio_pci_driver_memcpy(struct vfio_pci_device *device,
+			   iova_t src, iova_t dst, u64 size)
+{
+	vfio_pci_driver_memcpy_start(device, src, dst, size, 1);
+
+	return vfio_pci_driver_memcpy_wait(device);
+}
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 20/33] vfio: sefltests: Add vfio_pci_driver_test

[David Matlack]

Add a new selftest that tests all driver operations. This test serves
both as a demonstration of the driver framework, and also as a
correctness test for future drivers.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 tools/testing/selftests/vfio/Makefile         |   1 +
 .../selftests/vfio/vfio_pci_driver_test.c     | 239 ++++++++++++++++++
 2 files changed, 240 insertions(+)
 create mode 100644 tools/testing/selftests/vfio/vfio_pci_driver_test.c

diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftests/vfio/Makefile
index 2cc97176c6be..21fb1809035e 100644
--- a/tools/testing/selftests/vfio/Makefile
+++ b/tools/testing/selftests/vfio/Makefile
@@ -2,6 +2,7 @@ CFLAGS = $(KHDR_INCLUDES)
 TEST_GEN_PROGS_EXTENDED += vfio_dma_mapping_test
 TEST_GEN_PROGS_EXTENDED += vfio_iommufd_setup_test
 TEST_GEN_PROGS_EXTENDED += vfio_pci_device_test
+TEST_GEN_PROGS_EXTENDED += vfio_pci_driver_test
 include ../lib.mk
 include lib/libvfio.mk
 
diff --git a/tools/testing/selftests/vfio/vfio_pci_driver_test.c b/tools/testing/selftests/vfio/vfio_pci_driver_test.c
new file mode 100644
index 000000000000..5d14d892b796
--- /dev/null
+++ b/tools/testing/selftests/vfio/vfio_pci_driver_test.c
@@ -0,0 +1,239 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include <sys/ioctl.h>
+#include <sys/mman.h>
+
+#include <linux/pci_regs.h>
+#include <linux/sizes.h>
+#include <linux/vfio.h>
+#include <linux/io.h>
+
+#include <vfio_util.h>
+
+#include "../kselftest_harness.h"
+
+const char *device_bdf;
+
+#define ASSERT_NO_MSI(_eventfd) do {			\
+	u64 __value;					\
+							\
+	ASSERT_EQ(-1, read(_eventfd, &__value, 8));	\
+	ASSERT_EQ(EAGAIN, errno);			\
+} while (0)
+
+static void region_setup(struct vfio_pci_device *device,
+			 struct vfio_dma_region *region,
+			 iova_t iova, u64 size)
+{
+	const int flags = MAP_SHARED | MAP_ANONYMOUS;
+	const int prot = PROT_READ | PROT_WRITE;
+	void *vaddr;
+
+	vaddr = mmap(NULL, size, prot, flags, -1, 0);
+	VFIO_ASSERT_NE(vaddr, MAP_FAILED);
+
+	region->vaddr = vaddr;
+	region->iova = iova;
+	region->size = size;
+
+	vfio_pci_dma_map(device, region);
+}
+
+static void region_teardown(struct vfio_pci_device *device,
+			    struct vfio_dma_region *region)
+{
+	vfio_pci_dma_unmap(device, region);
+	VFIO_ASSERT_EQ(munmap(region->vaddr, region->size), 0);
+}
+
+FIXTURE(vfio_pci_driver_test) {
+	struct vfio_pci_device *device;
+	struct vfio_dma_region memcpy_region;
+	void *vaddr;
+	int msi_fd;
+
+	u64 size;
+	void *src;
+	void *dst;
+	iova_t src_iova;
+	iova_t dst_iova;
+	iova_t unmapped_iova;
+};
+
+FIXTURE_SETUP(vfio_pci_driver_test) {
+	struct vfio_pci_driver *driver;
+
+	self->device = vfio_pci_device_init(device_bdf, VFIO_TYPE1_IOMMU);
+
+	driver = &self->device->driver;
+
+	region_setup(self->device, &self->memcpy_region, SZ_1G, SZ_1G);
+	region_setup(self->device, &driver->region, SZ_4G, SZ_2M);
+
+	/* Any IOVA that doesn't overlap memcpy_region and driver->region. */
+	self->unmapped_iova = 8UL * SZ_1G;
+
+	vfio_pci_driver_init(self->device);
+	self->msi_fd = self->device->msi_eventfds[driver->msi];
+
+	/*
+	 * Use the maximum size supported by the device for memcpy operations,
+	 * slimmed down to fit into the memcpy region (divided by 2 so src and
+	 * dst regions do not overlap).
+	 */
+	self->size = self->device->driver.max_memcpy_size;
+	self->size = min(self->size, self->memcpy_region.size / 2);
+
+	self->src = self->memcpy_region.vaddr;
+	self->dst = self->src + self->size;
+
+	self->src_iova = to_iova(self->device, self->src);
+	self->dst_iova = to_iova(self->device, self->dst);
+}
+
+FIXTURE_TEARDOWN(vfio_pci_driver_test) {
+	struct vfio_pci_driver *driver = &self->device->driver;
+
+	vfio_pci_driver_remove(self->device);
+
+	region_teardown(self->device, &self->memcpy_region);
+	region_teardown(self->device, &driver->region);
+
+	vfio_pci_device_cleanup(self->device);
+}
+
+TEST_F(vfio_pci_driver_test, init_remove)
+{
+	int i;
+
+	for (i = 0; i < 10; i++) {
+		vfio_pci_driver_remove(self->device);
+		vfio_pci_driver_init(self->device);
+	}
+}
+
+TEST_F(vfio_pci_driver_test, memcpy_success)
+{
+	fcntl_set_nonblock(self->msi_fd);
+
+	memset(self->src, 'x', self->size);
+	memset(self->dst, 'y', self->size);
+
+	ASSERT_EQ(0, vfio_pci_driver_memcpy(self->device,
+					    self->src_iova,
+					    self->dst_iova,
+					    self->size));
+
+	ASSERT_EQ(0, memcmp(self->src, self->dst, self->size));
+	ASSERT_NO_MSI(self->msi_fd);
+}
+
+TEST_F(vfio_pci_driver_test, memcpy_from_unmapped_iova)
+{
+	fcntl_set_nonblock(self->msi_fd);
+
+	/*
+	 * Ignore the return value since not all devices will detect and report
+	 * accesses to unmapped IOVAs as errors.
+	 */
+	vfio_pci_driver_memcpy(self->device, self->unmapped_iova,
+			       self->dst_iova, self->size);
+
+	ASSERT_NO_MSI(self->msi_fd);
+}
+
+TEST_F(vfio_pci_driver_test, memcpy_to_unmapped_iova)
+{
+	fcntl_set_nonblock(self->msi_fd);
+
+	/*
+	 * Ignore the return value since not all devices will detect and report
+	 * accesses to unmapped IOVAs as errors.
+	 */
+	vfio_pci_driver_memcpy(self->device, self->src_iova,
+			       self->unmapped_iova, self->size);
+
+	ASSERT_NO_MSI(self->msi_fd);
+}
+
+TEST_F(vfio_pci_driver_test, send_msi)
+{
+	u64 value;
+
+	vfio_pci_driver_send_msi(self->device);
+	ASSERT_EQ(8, read(self->msi_fd, &value, 8));
+	ASSERT_EQ(1, value);
+}
+
+TEST_F(vfio_pci_driver_test, mix_and_match)
+{
+	u64 value;
+	int i;
+
+	for (i = 0; i < 10; i++) {
+		memset(self->src, 'x', self->size);
+		memset(self->dst, 'y', self->size);
+
+		ASSERT_EQ(0, vfio_pci_driver_memcpy(self->device,
+						    self->src_iova,
+						    self->dst_iova,
+						    self->size));
+
+		ASSERT_EQ(0, memcmp(self->src, self->dst, self->size));
+
+		vfio_pci_driver_memcpy(self->device,
+				       self->unmapped_iova,
+				       self->dst_iova,
+				       self->size);
+
+		vfio_pci_driver_send_msi(self->device);
+		ASSERT_EQ(8, read(self->msi_fd, &value, 8));
+		ASSERT_EQ(1, value);
+	}
+}
+
+TEST_F_TIMEOUT(vfio_pci_driver_test, memcpy_storm, 60)
+{
+	struct vfio_pci_driver *driver = &self->device->driver;
+	u64 total_size;
+	u64 count;
+
+	fcntl_set_nonblock(self->msi_fd);
+
+	/*
+	 * Perform up to 250GiB worth of DMA reads and writes across several
+	 * memcpy operations. Some devices can support even more but the test
+	 * will take too long.
+	 */
+	total_size = 250UL * SZ_1G;
+	count = min(total_size / self->size, driver->max_memcpy_count);
+
+	printf("Kicking off %lu memcpys of size 0x%lx\n", count, self->size);
+	vfio_pci_driver_memcpy_start(self->device,
+				     self->src_iova,
+				     self->dst_iova,
+				     self->size, count);
+
+	ASSERT_EQ(0, vfio_pci_driver_memcpy_wait(self->device));
+	ASSERT_NO_MSI(self->msi_fd);
+}
+
+int main(int argc, char *argv[])
+{
+	struct vfio_pci_device *device;
+
+	if (argc != 2) {
+		fprintf(stderr, "usage: %s segment:bus:device.function\n", argv[0]);
+		return KSFT_FAIL;
+	}
+
+	device_bdf = argv[1];
+
+	device = vfio_pci_device_init(device_bdf, VFIO_TYPE1_IOMMU);
+	if (!device->driver.ops) {
+		fprintf(stderr, "No driver found for device %s\n", device_bdf);
+		return KSFT_SKIP;
+	}
+	vfio_pci_device_cleanup(device);
+
+	return test_harness_run(1, argv);
+}
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 21/33] vfio: selftests: Add driver for Intel CBDMA

[David Matlack]

Add a driver for the Intel CBDMA device. This driver is based on and
named after the Linux driver for this device (drivers/dma/ioat/) and
also based on previous work from Peter Shier <pshier@google.com>.

The driver aims to be as simple as possible. It uses a single descriptor
to issue DMA operations, and only supports the copy operation. For "DMA
storms", the driver kicks off the maximum number of maximum-sized DMA
operations. On Skylake server parts, this was 2^16-1 copies of size 2M
and lasts about 15 seconds.

Cc: Dave Jiang <dave.jiang@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: David Matlack <dmatlack@google.com>
---
 .../testing/selftests/vfio/lib/drivers/ioat.c | 235 ++++++++++++++++++
 tools/testing/selftests/vfio/lib/libvfio.mk   |   7 +
 .../selftests/vfio/lib/vfio_pci_driver.c      |  10 +-
 3 files changed, 251 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/vfio/lib/drivers/ioat.c

diff --git a/tools/testing/selftests/vfio/lib/drivers/ioat.c b/tools/testing/selftests/vfio/lib/drivers/ioat.c
new file mode 100644
index 000000000000..bb6d054e6c4b
--- /dev/null
+++ b/tools/testing/selftests/vfio/lib/drivers/ioat.c
@@ -0,0 +1,235 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include <stdint.h>
+#include <unistd.h>
+
+#include <linux/errno.h>
+#include <linux/io.h>
+#include <linux/pci_ids.h>
+#include <linux/sizes.h>
+
+#include <vfio_util.h>
+
+#include <drivers/dma/ioat/hw.h>
+#include <drivers/dma/ioat/registers.h>
+
+#define IOAT_DMACOUNT_MAX UINT16_MAX
+
+struct ioat_state {
+	/* Single descriptor used to issue DMA memcpy operations */
+	struct ioat_dma_descriptor desc;
+
+	/* Copy buffers used by ioat_send_msi() to generate an interrupt. */
+	u64 send_msi_src;
+	u64 send_msi_dst;
+};
+
+static inline struct ioat_state *to_ioat_state(struct vfio_pci_device *device)
+{
+	return device->driver.region.vaddr;
+}
+
+static inline void *ioat_channel_registers(struct vfio_pci_device *device)
+{
+	return device->bars[0].vaddr + IOAT_CHANNEL_MMIO_SIZE;
+}
+
+static int ioat_probe(struct vfio_pci_device *device)
+{
+	u8 version;
+	int r;
+
+	if (!vfio_pci_device_match(device, PCI_VENDOR_ID_INTEL,
+				   PCI_DEVICE_ID_INTEL_IOAT_SKX))
+		return -EINVAL;
+
+	VFIO_ASSERT_NOT_NULL(device->bars[0].vaddr);
+
+	version = readb(device->bars[0].vaddr + IOAT_VER_OFFSET);
+	switch (version) {
+	case IOAT_VER_3_2:
+	case IOAT_VER_3_3:
+		r = 0;
+		break;
+	default:
+		printf("ioat: Unsupported version: 0x%x\n", version);
+		r = -EINVAL;
+	}
+	return r;
+}
+
+static u64 ioat_channel_status(void *bar)
+{
+	return readq(bar + IOAT_CHANSTS_OFFSET) & IOAT_CHANSTS_STATUS;
+}
+
+static void ioat_clear_errors(struct vfio_pci_device *device)
+{
+	void *registers = ioat_channel_registers(device);
+	u32 errors;
+
+	errors = vfio_pci_config_readl(device, IOAT_PCI_CHANERR_INT_OFFSET);
+	vfio_pci_config_writel(device, IOAT_PCI_CHANERR_INT_OFFSET, errors);
+
+	errors = vfio_pci_config_readl(device, IOAT_PCI_DMAUNCERRSTS_OFFSET);
+	vfio_pci_config_writel(device, IOAT_PCI_CHANERR_INT_OFFSET, errors);
+
+	errors = readl(registers + IOAT_CHANERR_OFFSET);
+	writel(errors, registers + IOAT_CHANERR_OFFSET);
+}
+
+static void ioat_reset(struct vfio_pci_device *device)
+{
+	void *registers = ioat_channel_registers(device);
+	u32 sleep_ms = 1, attempts = 5000 / sleep_ms;
+	u8 chancmd;
+
+	ioat_clear_errors(device);
+
+	writeb(IOAT_CHANCMD_RESET, registers + IOAT2_CHANCMD_OFFSET);
+
+	for (;;) {
+		chancmd = readb(registers + IOAT2_CHANCMD_OFFSET);
+		if (!(chancmd & IOAT_CHANCMD_RESET))
+			break;
+
+		VFIO_ASSERT_GT(--attempts, 0);
+		usleep(sleep_ms * 1000);
+	}
+
+	VFIO_ASSERT_EQ(ioat_channel_status(registers), IOAT_CHANSTS_HALTED);
+}
+
+static void ioat_init(struct vfio_pci_device *device)
+{
+	struct ioat_state *ioat = to_ioat_state(device);
+	u8 intrctrl;
+
+	VFIO_ASSERT_GE(device->driver.region.size, sizeof(*ioat));
+
+	vfio_pci_config_writew(device, PCI_COMMAND,
+			       PCI_COMMAND_MEMORY |
+			       PCI_COMMAND_MASTER |
+			       PCI_COMMAND_INTX_DISABLE);
+
+	ioat_reset(device);
+
+	/* Enable the use of MXI-x interrupts for channel interrupts. */
+	intrctrl = IOAT_INTRCTRL_MSIX_VECTOR_CONTROL;
+	writeb(intrctrl, device->bars[0].vaddr + IOAT_INTRCTRL_OFFSET);
+
+	vfio_pci_msix_enable(device, 0, device->msix_info.count);
+
+	device->driver.msi = 0;
+	device->driver.max_memcpy_size =
+		1UL << readb(device->bars[0].vaddr + IOAT_XFERCAP_OFFSET);
+	device->driver.max_memcpy_count = IOAT_DMACOUNT_MAX;
+}
+
+static void ioat_remove(struct vfio_pci_device *device)
+{
+	ioat_reset(device);
+	vfio_pci_msix_disable(device);
+}
+
+static void ioat_handle_error(struct vfio_pci_device *device)
+{
+	void *registers = ioat_channel_registers(device);
+
+	printf("Error detected during memcpy operation!\n"
+	       "  CHANERR: 0x%x\n"
+	       "  CHANERR_INT: 0x%x\n"
+	       "  DMAUNCERRSTS: 0x%x\n",
+	       readl(registers + IOAT_CHANERR_OFFSET),
+	       vfio_pci_config_readl(device, IOAT_PCI_CHANERR_INT_OFFSET),
+	       vfio_pci_config_readl(device, IOAT_PCI_DMAUNCERRSTS_OFFSET));
+
+	ioat_reset(device);
+}
+
+static int ioat_memcpy_wait(struct vfio_pci_device *device)
+{
+	void *registers = ioat_channel_registers(device);
+	u64 status;
+	int r = 0;
+
+	/* Wait until all operations complete. */
+	for (;;) {
+		status = ioat_channel_status(registers);
+		if (status == IOAT_CHANSTS_DONE)
+			break;
+
+		if (status == IOAT_CHANSTS_HALTED) {
+			ioat_handle_error(device);
+			return -1;
+		}
+	}
+
+	/* Put the channel into the SUSPENDED state. */
+	writeb(IOAT_CHANCMD_SUSPEND, registers + IOAT2_CHANCMD_OFFSET);
+	for (;;) {
+		status = ioat_channel_status(registers);
+		if (status == IOAT_CHANSTS_SUSPENDED)
+			break;
+	}
+
+	return r;
+}
+
+static void __ioat_memcpy_start(struct vfio_pci_device *device,
+				iova_t src, iova_t dst, u64 size,
+				u16 count, bool interrupt)
+{
+	void *registers = ioat_channel_registers(device);
+	struct ioat_state *ioat = to_ioat_state(device);
+	u64 desc_iova;
+	u16 chanctrl;
+
+	desc_iova = to_iova(device, &ioat->desc);
+	ioat->desc = (struct ioat_dma_descriptor) {
+		.ctl_f.op = IOAT_OP_COPY,
+		.ctl_f.int_en = interrupt,
+		.src_addr = src,
+		.dst_addr = dst,
+		.size = size,
+		.next = desc_iova,
+	};
+
+	/* Tell the device the address of the descriptor. */
+	writeq(desc_iova, registers + IOAT2_CHAINADDR_OFFSET);
+
+	/* (Re)Enable the channel interrupt and abort on any errors */
+	chanctrl = IOAT_CHANCTRL_INT_REARM | IOAT_CHANCTRL_ANY_ERR_ABORT_EN;
+	writew(chanctrl, registers + IOAT_CHANCTRL_OFFSET);
+
+	/* Kick off @count DMA copy operation(s). */
+	writew(count, registers + IOAT_CHAN_DMACOUNT_OFFSET);
+}
+
+static void ioat_memcpy_start(struct vfio_pci_device *device,
+			      iova_t src, iova_t dst, u64 size,
+			      u64 count)
+{
+	__ioat_memcpy_start(device, src, dst, size, count, false);
+}
+
+static void ioat_send_msi(struct vfio_pci_device *device)
+{
+	struct ioat_state *ioat = to_ioat_state(device);
+
+	__ioat_memcpy_start(device,
+			    to_iova(device, &ioat->send_msi_src),
+			    to_iova(device, &ioat->send_msi_dst),
+			    sizeof(ioat->send_msi_src), 1, true);
+
+	VFIO_ASSERT_EQ(ioat_memcpy_wait(device), 0);
+}
+
+const struct vfio_pci_driver_ops ioat_ops = {
+	.name = "ioat",
+	.probe = ioat_probe,
+	.init = ioat_init,
+	.remove = ioat_remove,
+	.memcpy_start = ioat_memcpy_start,
+	.memcpy_wait = ioat_memcpy_wait,
+	.send_msi = ioat_send_msi,
+};
diff --git a/tools/testing/selftests/vfio/lib/libvfio.mk b/tools/testing/selftests/vfio/lib/libvfio.mk
index 10a1a9316e4c..2ec24bb597fc 100644
--- a/tools/testing/selftests/vfio/lib/libvfio.mk
+++ b/tools/testing/selftests/vfio/lib/libvfio.mk
@@ -1,8 +1,15 @@
+include $(top_srcdir)/scripts/subarch.include
+ARCH ?= $(SUBARCH)
+
 VFIO_DIR := $(selfdir)/vfio
 
 LIBVFIO_C := lib/vfio_pci_device.c
 LIBVFIO_C += lib/vfio_pci_driver.c
 
+ifeq ($(ARCH:x86_64=x86),x86)
+LIBVFIO_C += lib/drivers/ioat.c
+endif
+
 LIBVFIO_O := $(patsubst %.c, $(OUTPUT)/%.o, $(LIBVFIO_C))
 
 LIBVFIO_O_DIRS := $(shell dirname $(LIBVFIO_O) | uniq)
diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_driver.c b/tools/testing/selftests/vfio/lib/vfio_pci_driver.c
index c98bd2d31d8a..aa47360e47a9 100644
--- a/tools/testing/selftests/vfio/lib/vfio_pci_driver.c
+++ b/tools/testing/selftests/vfio/lib/vfio_pci_driver.c
@@ -4,7 +4,15 @@
 #include "../../../kselftest.h"
 #include <vfio_util.h>
 
-static struct vfio_pci_driver_ops *driver_ops[] = {};
+#ifdef __x86_64__
+extern struct vfio_pci_driver_ops ioat_ops;
+#endif
+
+static struct vfio_pci_driver_ops *driver_ops[] = {
+#ifdef __x86_64__
+	&ioat_ops,
+#endif
+};
 
 void vfio_pci_driver_probe(struct vfio_pci_device *device)
 {
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 22/33] vfio: selftests: Add driver for Intel DSA

[David Matlack]

Add a driver to VFIO selftests for Intel DSA devices.

For now the driver only supports up to 32 batches and 1024 copies per
batch, which were the limits of the hardware this commit was tested
with. This is sufficient to generate 9+ minutes of DMA memcpys at a rate
of over 30 GB/s. This should be plenty to stress test VFIO and the IOMMU.

The driver does not yet support requesting interrupt handles, as this
commit was not tested against hardware that requires it.

Cc: Vinicius Costa Gomes <vinicius.gomes@intel.com>
Cc: Dave Jiang <dave.jiang@intel.com>
Signed-off-by: David Matlack <dmatlack@google.com>
---
 .../testing/selftests/vfio/lib/drivers/dsa.c  | 416 ++++++++++++++++++
 tools/testing/selftests/vfio/lib/libvfio.mk   |   1 +
 .../selftests/vfio/lib/vfio_pci_driver.c      |   2 +
 3 files changed, 419 insertions(+)
 create mode 100644 tools/testing/selftests/vfio/lib/drivers/dsa.c

diff --git a/tools/testing/selftests/vfio/lib/drivers/dsa.c b/tools/testing/selftests/vfio/lib/drivers/dsa.c
new file mode 100644
index 000000000000..504a4c7db945
--- /dev/null
+++ b/tools/testing/selftests/vfio/lib/drivers/dsa.c
@@ -0,0 +1,416 @@
+// SPDX-License-Identifier: GPL-2.0-only
+#include <stdint.h>
+#include <unistd.h>
+
+#include <linux/bits.h>
+#include <linux/errno.h>
+#include <linux/idxd.h>
+#include <linux/io.h>
+#include <linux/pci_ids.h>
+#include <linux/sizes.h>
+
+#include <vfio_util.h>
+
+#include <drivers/dma/idxd/registers.h>
+
+/* Vectors 1+ are available for work queue completion interrupts. */
+#define MSIX_VECTOR 1
+
+struct dsa_state {
+	/* Descriptors for copy and batch operations. */
+	struct dsa_hw_desc batch[32];
+	struct dsa_hw_desc copy[1024];
+
+	/* Completion records for copy and batch operations. */
+	struct dsa_completion_record copy_completion;
+	struct dsa_completion_record batch_completion;
+
+	/* Cached device registers (and derived data) for easy access */
+	union gen_cap_reg gen_cap;
+	union wq_cap_reg wq_cap;
+	union group_cap_reg group_cap;
+	union engine_cap_reg engine_cap;
+	union offsets_reg table_offsets;
+	void *wqcfg_table;
+	void *grpcfg_table;
+	u64 max_batches;
+	u64 max_copies_per_batch;
+
+	/* The number of ongoing memcpy operations. */
+	u64 memcpy_count;
+
+	/* Buffers used by dsa_send_msi() to generate an interrupt */
+	u64 send_msi_src;
+	u64 send_msi_dst;
+};
+
+static inline struct dsa_state *to_dsa_state(struct vfio_pci_device *device)
+{
+	return device->driver.region.vaddr;
+}
+
+static bool dsa_int_handle_request_required(struct vfio_pci_device *device)
+{
+	void *bar0 = device->bars[0].vaddr;
+	union gen_cap_reg gen_cap;
+	u32 cmd_cap;
+
+	gen_cap.bits = readq(bar0 + IDXD_GENCAP_OFFSET);
+	if (!gen_cap.cmd_cap)
+		return false;
+
+	cmd_cap = readl(bar0 + IDXD_CMDCAP_OFFSET);
+	return (cmd_cap >> IDXD_CMD_REQUEST_INT_HANDLE) & 1;
+}
+
+static int dsa_probe(struct vfio_pci_device *device)
+{
+	if (!vfio_pci_device_match(device, PCI_VENDOR_ID_INTEL,
+				   PCI_DEVICE_ID_INTEL_DSA_SPR0))
+		return -EINVAL;
+
+	if (dsa_int_handle_request_required(device)) {
+		printf("Device requires requesting interrupt handles\n");
+		return -EINVAL;
+	}
+
+	return 0;
+}
+
+static void dsa_check_sw_err(struct vfio_pci_device *device)
+{
+	void *reg = device->bars[0].vaddr + IDXD_SWERR_OFFSET;
+	union sw_err_reg err = {};
+	int i;
+
+	for (i = 0; i < ARRAY_SIZE(err.bits); i++) {
+		err.bits[i] = readq(reg + offsetof(union sw_err_reg, bits[i]));
+
+		/* No errors */
+		if (i == 0 && !err.valid)
+			return;
+	}
+
+	fprintf(stderr, "SWERR: 0x%016lx 0x%016lx 0x%016lx 0x%016lx\n",
+		err.bits[0], err.bits[1], err.bits[2], err.bits[3]);
+
+	fprintf(stderr, "  valid: 0x%x\n", err.valid);
+	fprintf(stderr, "  overflow: 0x%x\n", err.overflow);
+	fprintf(stderr, "  desc_valid: 0x%x\n", err.desc_valid);
+	fprintf(stderr, "  wq_idx_valid: 0x%x\n", err.wq_idx_valid);
+	fprintf(stderr, "  batch: 0x%x\n", err.batch);
+	fprintf(stderr, "  fault_rw: 0x%x\n", err.fault_rw);
+	fprintf(stderr, "  priv: 0x%x\n", err.priv);
+	fprintf(stderr, "  error: 0x%x\n", err.error);
+	fprintf(stderr, "  wq_idx: 0x%x\n", err.wq_idx);
+	fprintf(stderr, "  operation: 0x%x\n", err.operation);
+	fprintf(stderr, "  pasid: 0x%x\n", err.pasid);
+	fprintf(stderr, "  batch_idx: 0x%x\n", err.batch_idx);
+	fprintf(stderr, "  invalid_flags: 0x%x\n", err.invalid_flags);
+	fprintf(stderr, "  fault_addr: 0x%lx\n", err.fault_addr);
+
+	VFIO_FAIL("Software Error Detected!\n");
+}
+
+static void dsa_command(struct vfio_pci_device *device, u32 cmd)
+{
+	union idxd_command_reg cmd_reg = { .cmd = cmd };
+	u32 sleep_ms = 1, attempts = 5000 / sleep_ms;
+	void *bar0 = device->bars[0].vaddr;
+	u32 status;
+	u8 err;
+
+	writel(cmd_reg.bits, bar0 + IDXD_CMD_OFFSET);
+
+	for (;;) {
+		dsa_check_sw_err(device);
+
+		status = readl(bar0 + IDXD_CMDSTS_OFFSET);
+		if (!(status & IDXD_CMDSTS_ACTIVE))
+			break;
+
+		VFIO_ASSERT_GT(--attempts, 0);
+		usleep(sleep_ms * 1000);
+	}
+
+	err = status & IDXD_CMDSTS_ERR_MASK;
+	VFIO_ASSERT_EQ(err, 0, "Error issuing command 0x%x: 0x%x\n", cmd, err);
+}
+
+static void dsa_wq_init(struct vfio_pci_device *device)
+{
+	struct dsa_state *dsa = to_dsa_state(device);
+	union wq_cap_reg wq_cap = dsa->wq_cap;
+	union wqcfg wqcfg;
+	u64 wqcfg_size;
+	int i;
+
+	VFIO_ASSERT_GT((u32)wq_cap.num_wqs, 0);
+
+	wqcfg = (union wqcfg) {
+		.wq_size = wq_cap.total_wq_size,
+		.mode = 1,
+		.priority = 1,
+		/*
+		 * Disable Address Translation Service (if enabled) so that VFIO
+		 * selftests using this driver can generate I/O page faults.
+		 */
+		.wq_ats_disable = wq_cap.wq_ats_support,
+		.max_xfer_shift = dsa->gen_cap.max_xfer_shift,
+		.max_batch_shift = dsa->gen_cap.max_batch_shift,
+		.op_config[0] = BIT(DSA_OPCODE_MEMMOVE) | BIT(DSA_OPCODE_BATCH),
+	};
+
+	wqcfg_size = 1UL << (wq_cap.wqcfg_size + IDXD_WQCFG_MIN);
+
+	for (i = 0; i < wqcfg_size / sizeof(wqcfg.bits[0]); i++)
+		writel(wqcfg.bits[i], dsa->wqcfg_table + offsetof(union wqcfg, bits[i]));
+}
+
+static void dsa_group_init(struct vfio_pci_device *device)
+{
+	struct dsa_state *dsa = to_dsa_state(device);
+	union group_cap_reg group_cap = dsa->group_cap;
+	union engine_cap_reg engine_cap = dsa->engine_cap;
+
+	VFIO_ASSERT_GT((u32)group_cap.num_groups, 0);
+	VFIO_ASSERT_GT((u32)engine_cap.num_engines, 0);
+
+	/* Assign work queue 0 and engine 0 to group 0 */
+	writeq(1, dsa->grpcfg_table + offsetof(struct grpcfg, wqs[0]));
+	writeq(1, dsa->grpcfg_table + offsetof(struct grpcfg, engines));
+}
+
+static void dsa_register_cache_init(struct vfio_pci_device *device)
+{
+	struct dsa_state *dsa = to_dsa_state(device);
+	void *bar0 = device->bars[0].vaddr;
+
+	dsa->gen_cap.bits = readq(bar0 + IDXD_GENCAP_OFFSET);
+	dsa->wq_cap.bits = readq(bar0 + IDXD_WQCAP_OFFSET);
+	dsa->group_cap.bits = readq(bar0 + IDXD_GRPCAP_OFFSET);
+	dsa->engine_cap.bits = readq(bar0 + IDXD_ENGCAP_OFFSET);
+
+	dsa->table_offsets.bits[0] = readq(bar0 + IDXD_TABLE_OFFSET);
+	dsa->table_offsets.bits[1] = readq(bar0 + IDXD_TABLE_OFFSET + 8);
+
+	dsa->wqcfg_table = bar0 + dsa->table_offsets.wqcfg * IDXD_TABLE_MULT;
+	dsa->grpcfg_table = bar0 + dsa->table_offsets.grpcfg * IDXD_TABLE_MULT;
+
+	dsa->max_batches = 1U << (dsa->wq_cap.total_wq_size + IDXD_WQCFG_MIN);
+	dsa->max_batches = min(dsa->max_batches, ARRAY_SIZE(dsa->batch));
+
+	dsa->max_copies_per_batch = 1UL << dsa->gen_cap.max_batch_shift;
+	dsa->max_copies_per_batch = min(dsa->max_copies_per_batch, ARRAY_SIZE(dsa->copy));
+}
+
+static void dsa_init(struct vfio_pci_device *device)
+{
+	struct dsa_state *dsa = to_dsa_state(device);
+
+	VFIO_ASSERT_GE(device->driver.region.size, sizeof(*dsa));
+
+	vfio_pci_config_writew(device, PCI_COMMAND,
+			       PCI_COMMAND_MEMORY |
+			       PCI_COMMAND_MASTER |
+			       PCI_COMMAND_INTX_DISABLE);
+
+	dsa_command(device, IDXD_CMD_RESET_DEVICE);
+
+	dsa_register_cache_init(device);
+	dsa_wq_init(device);
+	dsa_group_init(device);
+
+	dsa_command(device, IDXD_CMD_ENABLE_DEVICE);
+	dsa_command(device, IDXD_CMD_ENABLE_WQ);
+
+	vfio_pci_msix_enable(device, MSIX_VECTOR, 1);
+
+	device->driver.max_memcpy_count =
+		dsa->max_batches * dsa->max_copies_per_batch;
+	device->driver.max_memcpy_size = 1UL << dsa->gen_cap.max_xfer_shift;
+	device->driver.msi = MSIX_VECTOR;
+}
+
+static void dsa_remove(struct vfio_pci_device *device)
+{
+	dsa_command(device, IDXD_CMD_RESET_DEVICE);
+	vfio_pci_msix_disable(device);
+}
+
+static int dsa_completion_wait(struct vfio_pci_device *device,
+			       struct dsa_completion_record *completion)
+{
+	u8 status;
+
+	for (;;) {
+		dsa_check_sw_err(device);
+
+		status = READ_ONCE(completion->status);
+		if (status)
+			break;
+
+		usleep(1000);
+	}
+
+	if (status == DSA_COMP_SUCCESS)
+		return 0;
+
+	printf("Error detected during memcpy operation: 0x%x\n", status);
+	return -1;
+}
+
+static void dsa_copy_desc_init(struct vfio_pci_device *device,
+			       struct dsa_hw_desc *desc,
+			       iova_t src, iova_t dst, u64 size,
+			       bool interrupt)
+{
+	struct dsa_state *dsa = to_dsa_state(device);
+	u16 flags;
+
+	flags = IDXD_OP_FLAG_CRAV | IDXD_OP_FLAG_RCR;
+
+	if (interrupt)
+		flags |= IDXD_OP_FLAG_RCI;
+
+	*desc = (struct dsa_hw_desc) {
+		.opcode = DSA_OPCODE_MEMMOVE,
+		.flags = flags,
+		.priv = 1,
+		.src_addr = src,
+		.dst_addr = dst,
+		.xfer_size = size,
+		.completion_addr = to_iova(device, &dsa->copy_completion),
+		.int_handle = interrupt ? MSIX_VECTOR : 0,
+	};
+}
+
+static void dsa_batch_desc_init(struct vfio_pci_device *device,
+				struct dsa_hw_desc *desc,
+				u64 count)
+{
+	struct dsa_state *dsa = to_dsa_state(device);
+
+	*desc = (struct dsa_hw_desc) {
+		.opcode = DSA_OPCODE_BATCH,
+		.flags = IDXD_OP_FLAG_CRAV,
+		.priv = 1,
+		.completion_addr = to_iova(device, &dsa->batch_completion),
+		.desc_list_addr = to_iova(device, &dsa->copy[0]),
+		.desc_count = count,
+	};
+}
+
+static void dsa_desc_write(struct vfio_pci_device *device, struct dsa_hw_desc *desc)
+{
+	/* Write the contents (not address) of the 64-byte descriptor to the device. */
+	iosubmit_cmds512(device->bars[2].vaddr, desc, 1);
+}
+
+static void dsa_memcpy_one(struct vfio_pci_device *device,
+			   iova_t src, iova_t dst, u64 size, bool interrupt)
+{
+	struct dsa_state *dsa = to_dsa_state(device);
+
+	memset(&dsa->copy_completion, 0, sizeof(dsa->copy_completion));
+
+	dsa_copy_desc_init(device, &dsa->copy[0], src, dst, size, interrupt);
+	dsa_desc_write(device, &dsa->copy[0]);
+}
+
+static void dsa_memcpy_batch(struct vfio_pci_device *device,
+			     iova_t src, iova_t dst, u64 size, u64 count)
+{
+	struct dsa_state *dsa = to_dsa_state(device);
+	int i;
+
+	memset(&dsa->batch_completion, 0, sizeof(dsa->batch_completion));
+
+	for (i = 0; i < ARRAY_SIZE(dsa->copy); i++) {
+		struct dsa_hw_desc *copy_desc = &dsa->copy[i];
+
+		dsa_copy_desc_init(device, copy_desc, src, dst, size, false);
+
+		/* Don't request completions for individual copies. */
+		copy_desc->flags &= ~IDXD_OP_FLAG_RCR;
+	}
+
+	for (i = 0; i < ARRAY_SIZE(dsa->batch) && count; i++) {
+		struct dsa_hw_desc *batch_desc = &dsa->batch[i];
+		int nr_copies;
+
+		nr_copies = min(count, dsa->max_copies_per_batch);
+		count -= nr_copies;
+
+		/*
+		 * Batches must have at least 2 copies, so handle the case where
+		 * there is exactly 1 copy left by doing one less copy in this
+		 * batch and then 2 in the next.
+		 */
+		if (count == 1) {
+			nr_copies--;
+			count++;
+		}
+
+		dsa_batch_desc_init(device, batch_desc, nr_copies);
+
+		/* Request a completion for the last batch. */
+		if (!count)
+			batch_desc->flags |= IDXD_OP_FLAG_RCR;
+
+		dsa_desc_write(device, batch_desc);
+	}
+
+	VFIO_ASSERT_EQ(count, 0, "Failed to start %lu copies.\n", count);
+}
+
+static void dsa_memcpy_start(struct vfio_pci_device *device,
+			     iova_t src, iova_t dst, u64 size, u64 count)
+{
+	struct dsa_state *dsa = to_dsa_state(device);
+
+	/* DSA devices require at least 2 copies per batch. */
+	if (count == 1)
+		dsa_memcpy_one(device, src, dst, size, false);
+	else
+		dsa_memcpy_batch(device, src, dst, size, count);
+
+	dsa->memcpy_count = count;
+}
+
+static int dsa_memcpy_wait(struct vfio_pci_device *device)
+{
+	struct dsa_state *dsa = to_dsa_state(device);
+	int r;
+
+	if (dsa->memcpy_count == 1)
+		r = dsa_completion_wait(device, &dsa->copy_completion);
+	else
+		r = dsa_completion_wait(device, &dsa->batch_completion);
+
+	dsa->memcpy_count = 0;
+
+	return r;
+}
+
+static void dsa_send_msi(struct vfio_pci_device *device)
+{
+	struct dsa_state *dsa = to_dsa_state(device);
+
+	dsa_memcpy_one(device,
+		       to_iova(device, &dsa->send_msi_src),
+		       to_iova(device, &dsa->send_msi_dst),
+		       sizeof(dsa->send_msi_src), true);
+
+	VFIO_ASSERT_EQ(dsa_completion_wait(device, &dsa->copy_completion), 0);
+}
+
+const struct vfio_pci_driver_ops dsa_ops = {
+	.name = "dsa",
+	.probe = dsa_probe,
+	.init = dsa_init,
+	.remove = dsa_remove,
+	.memcpy_start = dsa_memcpy_start,
+	.memcpy_wait = dsa_memcpy_wait,
+	.send_msi = dsa_send_msi,
+};
diff --git a/tools/testing/selftests/vfio/lib/libvfio.mk b/tools/testing/selftests/vfio/lib/libvfio.mk
index 2ec24bb597fc..51db107dc8fe 100644
--- a/tools/testing/selftests/vfio/lib/libvfio.mk
+++ b/tools/testing/selftests/vfio/lib/libvfio.mk
@@ -8,6 +8,7 @@ LIBVFIO_C += lib/vfio_pci_driver.c
 
 ifeq ($(ARCH:x86_64=x86),x86)
 LIBVFIO_C += lib/drivers/ioat.c
+LIBVFIO_C += lib/drivers/dsa.c
 endif
 
 LIBVFIO_O := $(patsubst %.c, $(OUTPUT)/%.o, $(LIBVFIO_C))
diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_driver.c b/tools/testing/selftests/vfio/lib/vfio_pci_driver.c
index aa47360e47a9..e5e8723ecb41 100644
--- a/tools/testing/selftests/vfio/lib/vfio_pci_driver.c
+++ b/tools/testing/selftests/vfio/lib/vfio_pci_driver.c
@@ -5,11 +5,13 @@
 #include <vfio_util.h>
 
 #ifdef __x86_64__
+extern struct vfio_pci_driver_ops dsa_ops;
 extern struct vfio_pci_driver_ops ioat_ops;
 #endif
 
 static struct vfio_pci_driver_ops *driver_ops[] = {
 #ifdef __x86_64__
+	&dsa_ops,
 	&ioat_ops,
 #endif
 };
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 23/33] vfio: selftests: Move helper to get cdev path to libvfio

[David Matlack]

Move the helper function to get the VFIO cdev path to libvfio so that it
can be used in libvfio in a subsequent commit.

No functional change intended.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 .../selftests/vfio/lib/include/vfio_util.h    |  2 +
 .../selftests/vfio/lib/vfio_pci_device.c      | 31 ++++++++++++++
 .../selftests/vfio/vfio_iommufd_setup_test.c  | 40 +++----------------
 3 files changed, 38 insertions(+), 35 deletions(-)

diff --git a/tools/testing/selftests/vfio/lib/include/vfio_util.h b/tools/testing/selftests/vfio/lib/include/vfio_util.h
index c824c993d82e..2b2e2a163e92 100644
--- a/tools/testing/selftests/vfio/lib/include/vfio_util.h
+++ b/tools/testing/selftests/vfio/lib/include/vfio_util.h
@@ -162,6 +162,8 @@ struct vfio_pci_device {
 	struct vfio_pci_driver driver;
 };
 
+const char *vfio_pci_get_cdev_path(const char *bdf);
+
 struct vfio_pci_device *vfio_pci_device_init(const char *bdf, int iommu_type);
 void vfio_pci_device_cleanup(struct vfio_pci_device *device);
 void vfio_pci_device_reset(struct vfio_pci_device *device);
diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_device.c b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
index 63f033047b4c..e8a9d46701c0 100644
--- a/tools/testing/selftests/vfio/lib/vfio_pci_device.c
+++ b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
@@ -1,4 +1,5 @@
 // SPDX-License-Identifier: GPL-2.0-only
+#include <dirent.h>
 #include <fcntl.h>
 #include <libgen.h>
 #include <stdlib.h>
@@ -328,6 +329,36 @@ static void vfio_pci_device_setup(struct vfio_pci_device *device, const char *bd
 		device->msi_eventfds[i] = -1;
 }
 
+const char *vfio_pci_get_cdev_path(const char *bdf)
+{
+	char dir_path[PATH_MAX];
+	struct dirent *entry;
+	char *cdev_path;
+	DIR *dir;
+
+	cdev_path = calloc(PATH_MAX, 1);
+	VFIO_ASSERT_NOT_NULL(cdev_path);
+
+	snprintf(dir_path, sizeof(dir_path), "/sys/bus/pci/devices/%s/vfio-dev/", bdf);
+
+	dir = opendir(dir_path);
+	VFIO_ASSERT_NOT_NULL(dir, "Failed to open directory %s\n", dir_path);
+
+	while ((entry = readdir(dir)) != NULL) {
+		/* Find the file that starts with "vfio" */
+		if (strncmp("vfio", entry->d_name, 4))
+			continue;
+
+		snprintf(cdev_path, PATH_MAX, "/dev/vfio/devices/%s", entry->d_name);
+		break;
+	}
+
+	VFIO_ASSERT_NE(cdev_path[0], 0, "Failed to find vfio cdev file.\n");
+	VFIO_ASSERT_EQ(closedir(dir), 0);
+
+	return cdev_path;
+}
+
 struct vfio_pci_device *vfio_pci_device_init(const char *bdf, int iommu_type)
 {
 	struct vfio_pci_device *device;
diff --git a/tools/testing/selftests/vfio/vfio_iommufd_setup_test.c b/tools/testing/selftests/vfio/vfio_iommufd_setup_test.c
index 851032f79a31..6965785cc32f 100644
--- a/tools/testing/selftests/vfio/vfio_iommufd_setup_test.c
+++ b/tools/testing/selftests/vfio/vfio_iommufd_setup_test.c
@@ -1,8 +1,4 @@
 // SPDX-License-Identifier: GPL-2.0
-#include <assert.h>
-#include <dirent.h>
-#include <fcntl.h>
-
 #include <uapi/linux/types.h>
 #include <linux/limits.h>
 #include <linux/sizes.h>
@@ -11,39 +7,15 @@
 
 #include <stdint.h>
 #include <stdio.h>
-#include <string.h>
 #include <sys/ioctl.h>
 #include <unistd.h>
 
 #include "../kselftest_harness.h"
 
-static const char iommu_dev_path[] = "/dev/iommu";
-char cdev_path[PATH_MAX] = { '\0' };
-
-static void set_cdev_path(const char *bdf)
-{
-	char dir_path[PATH_MAX];
-	DIR *dir;
-	struct dirent *entry;
-
-	snprintf(dir_path, sizeof(dir_path), "/sys/bus/pci/devices/%s/vfio-dev/", bdf);
-
-	dir = opendir(dir_path);
-	assert(dir);
-
-	/* Find the file named "vfio<number>" */
-	while ((entry = readdir(dir)) != NULL) {
-		if (!strncmp("vfio", entry->d_name, 4)) {
-			snprintf(cdev_path, sizeof(cdev_path), "/dev/vfio/devices/%s",
-				 entry->d_name);
-			break;
-		}
-	}
-
-	assert(strlen(cdev_path) > 0);
+#include <vfio_util.h>
 
-	closedir(dir);
-}
+static const char *iommu_dev_path = "/dev/iommu";
+static const char *cdev_path;
 
 static int vfio_device_bind_iommufd_ioctl(int cdev_fd, int iommufd)
 {
@@ -148,15 +120,13 @@ TEST_F(vfio_cdev, attach_invalid_pt_fails)
 
 int main(int argc, char *argv[])
 {
-	char *bdf;
-
 	if (argc != 2) {
 		printf("Usage: %s bus:device:function\n", argv[0]);
 		return 1;
 	}
 
-	bdf = argv[1];
-	set_cdev_path(bdf);
+	cdev_path = vfio_pci_get_cdev_path(argv[1]);
+
 	printf("Using cdev device %s\n", cdev_path);
 
 	return test_harness_run(1, argv);
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 24/33] vfio: selftests: Encapsulate IOMMU mode

[David Matlack]

Encapsulate the "IOMMU mode" a test should use behind a new struct.
In the future this will be used to support other types of IOMMUs besides
VFIO_TYPE1_IOMMU, and allow users to select the mode on the command
line.

No functional change intended.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 .../selftests/vfio/lib/include/vfio_util.h    | 12 +++++-
 .../selftests/vfio/lib/vfio_pci_device.c      | 42 ++++++++++++++++---
 .../selftests/vfio/vfio_dma_mapping_test.c    |  2 +-
 .../selftests/vfio/vfio_pci_device_test.c     |  4 +-
 .../selftests/vfio/vfio_pci_driver_test.c     |  4 +-
 5 files changed, 52 insertions(+), 12 deletions(-)

diff --git a/tools/testing/selftests/vfio/lib/include/vfio_util.h b/tools/testing/selftests/vfio/lib/include/vfio_util.h
index 2b2e2a163e92..bff8f550274b 100644
--- a/tools/testing/selftests/vfio/lib/include/vfio_util.h
+++ b/tools/testing/selftests/vfio/lib/include/vfio_util.h
@@ -47,6 +47,12 @@
 	VFIO_LOG_AND_EXIT(_fmt, ##__VA_ARGS__);			\
 } while (0)
 
+struct vfio_iommu_mode {
+	const char *name;
+	const char *container_path;
+	unsigned long group_iommu_type;
+};
+
 struct vfio_pci_bar {
 	struct vfio_region_info info;
 	void *vaddr;
@@ -144,6 +150,8 @@ struct vfio_pci_driver {
 
 struct vfio_pci_device {
 	int fd;
+
+	const struct vfio_iommu_mode *iommu_mode;
 	int group_fd;
 	int container_fd;
 
@@ -164,7 +172,9 @@ struct vfio_pci_device {
 
 const char *vfio_pci_get_cdev_path(const char *bdf);
 
-struct vfio_pci_device *vfio_pci_device_init(const char *bdf, int iommu_type);
+extern const char *default_iommu_mode;
+
+struct vfio_pci_device *vfio_pci_device_init(const char *bdf, const char *iommu_mode);
 void vfio_pci_device_cleanup(struct vfio_pci_device *device);
 void vfio_pci_device_reset(struct vfio_pci_device *device);
 
diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_device.c b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
index e8a9d46701c0..4a95cf0c1b1c 100644
--- a/tools/testing/selftests/vfio/lib/vfio_pci_device.c
+++ b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
@@ -18,7 +18,6 @@
 #include "../../../kselftest.h"
 #include <vfio_util.h>
 
-#define VFIO_DEV_PATH	"/dev/vfio/vfio"
 #define PCI_SYSFS_PATH	"/sys/bus/pci/devices"
 
 #define ioctl_assert(_fd, _op, _arg) do {					  \
@@ -257,10 +256,11 @@ static unsigned int vfio_pci_get_group_from_dev(const char *bdf)
 
 static void vfio_pci_container_setup(struct vfio_pci_device *device)
 {
+	const char *path = device->iommu_mode->container_path;
 	int version;
 
-	device->container_fd = open(VFIO_DEV_PATH, O_RDWR);
-	VFIO_ASSERT_GE(device->container_fd, 0, "open(%s) failed\n", VFIO_DEV_PATH);
+	device->container_fd = open(path, O_RDWR);
+	VFIO_ASSERT_GE(device->container_fd, 0, "open(%s) failed\n", path);
 
 	version = ioctl(device->container_fd, VFIO_GET_API_VERSION);
 	VFIO_ASSERT_EQ(version, VFIO_API_VERSION);
@@ -286,8 +286,9 @@ static void vfio_pci_group_setup(struct vfio_pci_device *device, const char *bdf
 	ioctl_assert(device->group_fd, VFIO_GROUP_SET_CONTAINER, &device->container_fd);
 }
 
-static void vfio_pci_iommu_setup(struct vfio_pci_device *device, unsigned long iommu_type)
+static void vfio_pci_iommu_setup(struct vfio_pci_device *device)
 {
+	unsigned long iommu_type = device->iommu_mode->group_iommu_type;
 	int ret;
 
 	INIT_LIST_HEAD(&device->dma_regions);
@@ -359,16 +360,45 @@ const char *vfio_pci_get_cdev_path(const char *bdf)
 	return cdev_path;
 }
 
-struct vfio_pci_device *vfio_pci_device_init(const char *bdf, int iommu_type)
+static const struct vfio_iommu_mode iommu_modes[] = {
+	{
+		.name = "vfio_type1_iommu",
+		.container_path = "/dev/vfio/vfio",
+		.group_iommu_type = VFIO_TYPE1_IOMMU,
+	},
+};
+
+const char *default_iommu_mode = "vfio_type1_iommu";
+
+static const struct vfio_iommu_mode *lookup_iommu_mode(const char *iommu_mode)
+{
+	int i;
+
+	if (!iommu_mode)
+		iommu_mode = default_iommu_mode;
+
+	for (i = 0; i < ARRAY_SIZE(iommu_modes); i++) {
+		if (strcmp(iommu_mode, iommu_modes[i].name))
+			continue;
+
+		return &iommu_modes[i];
+	}
+
+	VFIO_FAIL("Unrecognized IOMMU mode: %s\n", iommu_mode);
+}
+
+struct vfio_pci_device *vfio_pci_device_init(const char *bdf, const char *iommu_mode)
 {
 	struct vfio_pci_device *device;
 
 	device = calloc(1, sizeof(*device));
 	VFIO_ASSERT_NOT_NULL(device);
 
+	device->iommu_mode = lookup_iommu_mode(iommu_mode);
+
 	vfio_pci_container_setup(device);
 	vfio_pci_group_setup(device, bdf);
-	vfio_pci_iommu_setup(device, iommu_type);
+	vfio_pci_iommu_setup(device);
 	vfio_pci_device_setup(device, bdf);
 
 	vfio_pci_driver_probe(device);
diff --git a/tools/testing/selftests/vfio/vfio_dma_mapping_test.c b/tools/testing/selftests/vfio/vfio_dma_mapping_test.c
index d85621d165d9..2ecf2514f30c 100644
--- a/tools/testing/selftests/vfio/vfio_dma_mapping_test.c
+++ b/tools/testing/selftests/vfio/vfio_dma_mapping_test.c
@@ -104,7 +104,7 @@ FIXTURE(vfio_dma_mapping_test)
 
 FIXTURE_SETUP(vfio_dma_mapping_test)
 {
-	self->device = vfio_pci_device_init(test_config.bdf, VFIO_TYPE1_IOMMU);
+	self->device = vfio_pci_device_init(test_config.bdf, default_iommu_mode);
 }
 
 FIXTURE_TEARDOWN(vfio_dma_mapping_test)
diff --git a/tools/testing/selftests/vfio/vfio_pci_device_test.c b/tools/testing/selftests/vfio/vfio_pci_device_test.c
index 1a26df0f2280..45707c6d1c09 100644
--- a/tools/testing/selftests/vfio/vfio_pci_device_test.c
+++ b/tools/testing/selftests/vfio/vfio_pci_device_test.c
@@ -27,7 +27,7 @@ FIXTURE(vfio_pci_device_test) {
 };
 
 FIXTURE_SETUP(vfio_pci_device_test) {
-	self->device = vfio_pci_device_init(device_bdf, VFIO_TYPE1_IOMMU);
+	self->device = vfio_pci_device_init(device_bdf, default_iommu_mode);
 }
 
 FIXTURE_TEARDOWN(vfio_pci_device_test) {
@@ -113,7 +113,7 @@ FIXTURE_VARIANT_ADD(vfio_pci_irq_test, msix) {
 };
 
 FIXTURE_SETUP(vfio_pci_irq_test) {
-	self->device = vfio_pci_device_init(device_bdf, VFIO_TYPE1_IOMMU);
+	self->device = vfio_pci_device_init(device_bdf, default_iommu_mode);
 }
 
 FIXTURE_TEARDOWN(vfio_pci_irq_test) {
diff --git a/tools/testing/selftests/vfio/vfio_pci_driver_test.c b/tools/testing/selftests/vfio/vfio_pci_driver_test.c
index 5d14d892b796..0f05cc3774a6 100644
--- a/tools/testing/selftests/vfio/vfio_pci_driver_test.c
+++ b/tools/testing/selftests/vfio/vfio_pci_driver_test.c
@@ -62,7 +62,7 @@ FIXTURE(vfio_pci_driver_test) {
 FIXTURE_SETUP(vfio_pci_driver_test) {
 	struct vfio_pci_driver *driver;
 
-	self->device = vfio_pci_device_init(device_bdf, VFIO_TYPE1_IOMMU);
+	self->device = vfio_pci_device_init(device_bdf, default_iommu_mode);
 
 	driver = &self->device->driver;
 
@@ -228,7 +228,7 @@ int main(int argc, char *argv[])
 
 	device_bdf = argv[1];
 
-	device = vfio_pci_device_init(device_bdf, VFIO_TYPE1_IOMMU);
+	device = vfio_pci_device_init(device_bdf, default_iommu_mode);
 	if (!device->driver.ops) {
 		fprintf(stderr, "No driver found for device %s\n", device_bdf);
 		return KSFT_SKIP;
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 25/33] vfio: selftests: Add [-i iommu_mode] option to all tests

[David Matlack]

Add a new command line option to all tests that allow them to select
which iommu_mode to use. This makes it possible to replicate tests with
different modes and therefore get test coverage of different parts of
the kernel without having to change the tests themselves.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 .../selftests/vfio/lib/include/vfio_util.h    |  1 +
 .../selftests/vfio/lib/vfio_pci_device.c      | 15 ++++++++
 .../selftests/vfio/vfio_dma_mapping_test.c    | 14 +++++---
 .../selftests/vfio/vfio_pci_device_test.c     | 33 ++++++++++++-----
 .../selftests/vfio/vfio_pci_driver_test.c     | 35 ++++++++++++++-----
 5 files changed, 77 insertions(+), 21 deletions(-)

diff --git a/tools/testing/selftests/vfio/lib/include/vfio_util.h b/tools/testing/selftests/vfio/lib/include/vfio_util.h
index bff8f550274b..9989528c21b9 100644
--- a/tools/testing/selftests/vfio/lib/include/vfio_util.h
+++ b/tools/testing/selftests/vfio/lib/include/vfio_util.h
@@ -173,6 +173,7 @@ struct vfio_pci_device {
 const char *vfio_pci_get_cdev_path(const char *bdf);
 
 extern const char *default_iommu_mode;
+void iommu_mode_help(const char *flag);
 
 struct vfio_pci_device *vfio_pci_device_init(const char *bdf, const char *iommu_mode);
 void vfio_pci_device_cleanup(struct vfio_pci_device *device);
diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_device.c b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
index 4a95cf0c1b1c..950c83fee3ed 100644
--- a/tools/testing/selftests/vfio/lib/vfio_pci_device.c
+++ b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
@@ -370,6 +370,21 @@ static const struct vfio_iommu_mode iommu_modes[] = {
 
 const char *default_iommu_mode = "vfio_type1_iommu";
 
+void iommu_mode_help(const char *flag)
+{
+	int i;
+
+	printf("  %s: The iommu mode to use for the test (default: %s)\n"
+	       "\n"
+	       "      Available modes:\n",
+	       flag, default_iommu_mode);
+
+	for (i = 0; i < ARRAY_SIZE(iommu_modes); i++)
+		printf("        %s\n", iommu_modes[i].name);
+
+	printf("\n");
+}
+
 static const struct vfio_iommu_mode *lookup_iommu_mode(const char *iommu_mode)
 {
 	int i;
diff --git a/tools/testing/selftests/vfio/vfio_dma_mapping_test.c b/tools/testing/selftests/vfio/vfio_dma_mapping_test.c
index 2ecf2514f30c..e2f636838130 100644
--- a/tools/testing/selftests/vfio/vfio_dma_mapping_test.c
+++ b/tools/testing/selftests/vfio/vfio_dma_mapping_test.c
@@ -18,6 +18,7 @@ static struct {
 	u64 iova;
 	int mmap_flags;
 	const char *bdf;
+	const char *iommu_mode;
 } test_config;
 
 struct iommu_mapping {
@@ -104,7 +105,7 @@ FIXTURE(vfio_dma_mapping_test)
 
 FIXTURE_SETUP(vfio_dma_mapping_test)
 {
-	self->device = vfio_pci_device_init(test_config.bdf, default_iommu_mode);
+	self->device = vfio_pci_device_init(test_config.bdf, test_config.iommu_mode);
 }
 
 FIXTURE_TEARDOWN(vfio_dma_mapping_test)
@@ -172,13 +173,15 @@ TEST_F(vfio_dma_mapping_test, dma_map_unmap)
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-b backing_src] segment:bus:device.function\n"
+	printf("Usage: %s [-b backing_src] [-i iommu_mode] segment:bus:device.function\n"
 	       "  -b: Which backing memory to use (default: anonymous)\n"
 	       "\n"
 	       "      anonymous\n"
 	       "      anonymous_hugetlb_2mb\n"
-	       "      anonymous_hugetlb_1gb\n",
+	       "      anonymous_hugetlb_1gb\n"
+	       "\n",
 	       name);
+	iommu_mode_help("-i");
 	exit(1);
 }
 
@@ -211,11 +214,14 @@ int main(int argc, char *argv[])
 	const char *backing_src = "anonymous";
 	int c;
 
-	while ((c = getopt(argc, argv, "b:")) != -1) {
+	while ((c = getopt(argc, argv, "b:i:")) != -1) {
 		switch (c) {
 		case 'b':
 			backing_src = optarg;
 			break;
+		case 'i':
+			test_config.iommu_mode = optarg;
+			break;
 		default:
 			help(argv[0]);
 		}
diff --git a/tools/testing/selftests/vfio/vfio_pci_device_test.c b/tools/testing/selftests/vfio/vfio_pci_device_test.c
index 45707c6d1c09..842f0f2f3523 100644
--- a/tools/testing/selftests/vfio/vfio_pci_device_test.c
+++ b/tools/testing/selftests/vfio/vfio_pci_device_test.c
@@ -14,7 +14,8 @@
 
 #include "../kselftest_harness.h"
 
-const char *device_bdf;
+static const char *device_bdf;
+static const char *iommu_mode;
 
 /*
  * Limit the number of MSIs enabled/disabled by the test regardless of the
@@ -27,7 +28,7 @@ FIXTURE(vfio_pci_device_test) {
 };
 
 FIXTURE_SETUP(vfio_pci_device_test) {
-	self->device = vfio_pci_device_init(device_bdf, default_iommu_mode);
+	self->device = vfio_pci_device_init(device_bdf, iommu_mode);
 }
 
 FIXTURE_TEARDOWN(vfio_pci_device_test) {
@@ -113,7 +114,7 @@ FIXTURE_VARIANT_ADD(vfio_pci_irq_test, msix) {
 };
 
 FIXTURE_SETUP(vfio_pci_irq_test) {
-	self->device = vfio_pci_device_init(device_bdf, default_iommu_mode);
+	self->device = vfio_pci_device_init(device_bdf, iommu_mode);
 }
 
 FIXTURE_TEARDOWN(vfio_pci_irq_test) {
@@ -165,14 +166,30 @@ TEST_F(vfio_pci_device_test, reset)
 	vfio_pci_device_reset(self->device);
 }
 
+static void help(const char *name)
+{
+	printf("Usage: %s [-i iommu_mode] segment:bus:device.function\n", name);
+	iommu_mode_help("-i");
+	exit(1);
+}
+
 int main(int argc, char *argv[])
 {
-	if (argc != 2) {
-		fprintf(stderr, "usage: %s segment:bus:device.function\n", argv[0]);
-		return KSFT_FAIL;
+	int c;
+
+	while ((c = getopt(argc, argv, "i:")) != -1) {
+		switch (c) {
+		case 'i':
+			iommu_mode = optarg;
+			break;
+		default:
+			help(argv[0]);
+		}
 	}
 
-	device_bdf = argv[1];
+	if (optind >= argc)
+		help(argv[0]);
 
-	return test_harness_run(1, argv);
+	device_bdf = argv[optind];
+	return test_harness_run(0, NULL);
 }
diff --git a/tools/testing/selftests/vfio/vfio_pci_driver_test.c b/tools/testing/selftests/vfio/vfio_pci_driver_test.c
index 0f05cc3774a6..e94b77125e5e 100644
--- a/tools/testing/selftests/vfio/vfio_pci_driver_test.c
+++ b/tools/testing/selftests/vfio/vfio_pci_driver_test.c
@@ -11,7 +11,8 @@
 
 #include "../kselftest_harness.h"
 
-const char *device_bdf;
+static const char *device_bdf;
+static const char *iommu_mode;
 
 #define ASSERT_NO_MSI(_eventfd) do {			\
 	u64 __value;					\
@@ -62,7 +63,7 @@ FIXTURE(vfio_pci_driver_test) {
 FIXTURE_SETUP(vfio_pci_driver_test) {
 	struct vfio_pci_driver *driver;
 
-	self->device = vfio_pci_device_init(device_bdf, default_iommu_mode);
+	self->device = vfio_pci_device_init(device_bdf, iommu_mode);
 
 	driver = &self->device->driver;
 
@@ -217,23 +218,39 @@ TEST_F_TIMEOUT(vfio_pci_driver_test, memcpy_storm, 60)
 	ASSERT_NO_MSI(self->msi_fd);
 }
 
+static void help(const char *name)
+{
+	printf("Usage: %s [-i iommu_mode] segment:bus:device.function\n", name);
+	iommu_mode_help("-i");
+	exit(1);
+}
+
 int main(int argc, char *argv[])
 {
 	struct vfio_pci_device *device;
-
-	if (argc != 2) {
-		fprintf(stderr, "usage: %s segment:bus:device.function\n", argv[0]);
-		return KSFT_FAIL;
+	int c;
+
+	while ((c = getopt(argc, argv, "i:")) != -1) {
+		switch (c) {
+		case 'i':
+			iommu_mode = optarg;
+			break;
+		default:
+			help(argv[0]);
+		}
 	}
 
-	device_bdf = argv[1];
+	if (optind >= argc)
+		help(argv[0]);
+
+	device_bdf = argv[optind];
 
-	device = vfio_pci_device_init(device_bdf, default_iommu_mode);
+	device = vfio_pci_device_init(device_bdf, iommu_mode);
 	if (!device->driver.ops) {
 		fprintf(stderr, "No driver found for device %s\n", device_bdf);
 		return KSFT_SKIP;
 	}
 	vfio_pci_device_cleanup(device);
 
-	return test_harness_run(1, argv);
+	return test_harness_run(0, NULL);
 }
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 26/33] vfio: selftests: Add vfio_type1v2_mode

[David Matlack]

Add a new IOMMU mode for using VFIO_TYPE1v2_IOMMU.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 tools/testing/selftests/vfio/lib/vfio_pci_device.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_device.c b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
index 950c83fee3ed..b349d901cfdc 100644
--- a/tools/testing/selftests/vfio/lib/vfio_pci_device.c
+++ b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
@@ -366,6 +366,11 @@ static const struct vfio_iommu_mode iommu_modes[] = {
 		.container_path = "/dev/vfio/vfio",
 		.group_iommu_type = VFIO_TYPE1_IOMMU,
 	},
+	{
+		.name = "vfio_type1v2_iommu",
+		.container_path = "/dev/vfio/vfio",
+		.group_iommu_type = VFIO_TYPE1v2_IOMMU,
+	},
 };
 
 const char *default_iommu_mode = "vfio_type1_iommu";
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 27/33] vfio: selftests: Add iommufd_compat_type1{,v2} modes

[David Matlack]

Add new IOMMU modes for using iommufd in compatibility mode with
VFIO_TYPE1_IOMMU and VFIO_TYPE1v2_IOMMU.

In these modes, VFIO selftests will open /dev/iommu and treats it as a
container FD (as if it had opened /dev/vfio/vfio) and the kernel
translates the container ioctls to iommufd calls transparently.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 tools/testing/selftests/vfio/lib/vfio_pci_device.c | 10 ++++++++++
 1 file changed, 10 insertions(+)

diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_device.c b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
index b349d901cfdc..205bbfb3d54e 100644
--- a/tools/testing/selftests/vfio/lib/vfio_pci_device.c
+++ b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
@@ -371,6 +371,16 @@ static const struct vfio_iommu_mode iommu_modes[] = {
 		.container_path = "/dev/vfio/vfio",
 		.group_iommu_type = VFIO_TYPE1v2_IOMMU,
 	},
+	{
+		.name = "iommufd_compat_type1",
+		.container_path = "/dev/iommu",
+		.group_iommu_type = VFIO_TYPE1_IOMMU,
+	},
+	{
+		.name = "iommufd_compat_type1v2",
+		.container_path = "/dev/iommu",
+		.group_iommu_type = VFIO_TYPE1v2_IOMMU,
+	},
 };
 
 const char *default_iommu_mode = "vfio_type1_iommu";
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 28/33] vfio: selftests: Add iommufd mode

[David Matlack]

Add a new IOMMU mode for using iommufd directly. In this mode userspace
opens /dev/iommu and binds it to a device FD acquired through
/dev/vfio/devices/vfioX.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 .../selftests/vfio/lib/include/vfio_util.h    |   3 +
 .../selftests/vfio/lib/vfio_pci_device.c      | 175 ++++++++++++++----
 2 files changed, 144 insertions(+), 34 deletions(-)

diff --git a/tools/testing/selftests/vfio/lib/include/vfio_util.h b/tools/testing/selftests/vfio/lib/include/vfio_util.h
index 9989528c21b9..b03b3d8f5a52 100644
--- a/tools/testing/selftests/vfio/lib/include/vfio_util.h
+++ b/tools/testing/selftests/vfio/lib/include/vfio_util.h
@@ -155,6 +155,9 @@ struct vfio_pci_device {
 	int group_fd;
 	int container_fd;
 
+	int iommufd;
+	u32 ioas_id;
+
 	struct vfio_device_info info;
 	struct vfio_region_info config_space;
 	struct vfio_pci_bar bars[PCI_STD_NUM_BARS];
diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_device.c b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
index 205bbfb3d54e..d996e012a646 100644
--- a/tools/testing/selftests/vfio/lib/vfio_pci_device.c
+++ b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
@@ -10,10 +10,12 @@
 #include <sys/ioctl.h>
 #include <sys/mman.h>
 
+#include <uapi/linux/types.h>
 #include <linux/limits.h>
 #include <linux/mman.h>
 #include <linux/types.h>
 #include <linux/vfio.h>
+#include <linux/iommufd.h>
 
 #include "../../../kselftest.h"
 #include <vfio_util.h>
@@ -140,32 +142,80 @@ static void vfio_pci_irq_get(struct vfio_pci_device *device, u32 index,
 	ioctl_assert(device->fd, VFIO_DEVICE_GET_IRQ_INFO, irq_info);
 }
 
-void vfio_pci_dma_map(struct vfio_pci_device *device,
-		      struct vfio_dma_region *region)
+static void vfio_iommu_dma_map(struct vfio_pci_device *device,
+			       struct vfio_dma_region *region)
 {
-	struct vfio_iommu_type1_dma_map map = {
-		.argsz = sizeof(map),
+	struct vfio_iommu_type1_dma_map args = {
+		.argsz = sizeof(args),
 		.flags = VFIO_DMA_MAP_FLAG_READ | VFIO_DMA_MAP_FLAG_WRITE,
 		.vaddr = (u64)region->vaddr,
 		.iova = region->iova,
 		.size = region->size,
 	};
 
-	ioctl_assert(device->container_fd, VFIO_IOMMU_MAP_DMA, &map);
+	ioctl_assert(device->container_fd, VFIO_IOMMU_MAP_DMA, &args);
+}
+
+static void iommufd_dma_map(struct vfio_pci_device *device,
+			    struct vfio_dma_region *region)
+{
+	struct iommu_ioas_map args = {
+		.size = sizeof(args),
+		.flags = IOMMU_IOAS_MAP_READABLE |
+			 IOMMU_IOAS_MAP_WRITEABLE |
+			 IOMMU_IOAS_MAP_FIXED_IOVA,
+		.user_va = (u64)region->vaddr,
+		.iova = region->iova,
+		.length = region->size,
+		.ioas_id = device->ioas_id,
+	};
+
+	ioctl_assert(device->iommufd, IOMMU_IOAS_MAP, &args);
+}
+
+void vfio_pci_dma_map(struct vfio_pci_device *device,
+		      struct vfio_dma_region *region)
+{
+	if (device->iommufd)
+		iommufd_dma_map(device, region);
+	else
+		vfio_iommu_dma_map(device, region);
 
 	list_add(&region->link, &device->dma_regions);
 }
 
-void vfio_pci_dma_unmap(struct vfio_pci_device *device,
-			struct vfio_dma_region *region)
+static void vfio_iommu_dma_unmap(struct vfio_pci_device *device,
+				 struct vfio_dma_region *region)
 {
-	struct vfio_iommu_type1_dma_unmap unmap = {
-		.argsz = sizeof(unmap),
+	struct vfio_iommu_type1_dma_unmap args = {
+		.argsz = sizeof(args),
 		.iova = region->iova,
 		.size = region->size,
 	};
 
-	ioctl_assert(device->container_fd, VFIO_IOMMU_UNMAP_DMA, &unmap);
+	ioctl_assert(device->container_fd, VFIO_IOMMU_UNMAP_DMA, &args);
+}
+
+static void iommufd_dma_unmap(struct vfio_pci_device *device,
+			      struct vfio_dma_region *region)
+{
+	struct iommu_ioas_unmap args = {
+		.size = sizeof(args),
+		.iova = region->iova,
+		.length = region->size,
+		.ioas_id = device->ioas_id,
+	};
+
+	ioctl_assert(device->iommufd, IOMMU_IOAS_UNMAP, &args);
+}
+
+void vfio_pci_dma_unmap(struct vfio_pci_device *device,
+			struct vfio_dma_region *region)
+{
+	if (device->iommufd)
+		iommufd_dma_unmap(device, region);
+	else
+		vfio_iommu_dma_unmap(device, region);
 
 	list_del(&region->link);
 }
@@ -254,18 +304,6 @@ static unsigned int vfio_pci_get_group_from_dev(const char *bdf)
 	return group;
 }
 
-static void vfio_pci_container_setup(struct vfio_pci_device *device)
-{
-	const char *path = device->iommu_mode->container_path;
-	int version;
-
-	device->container_fd = open(path, O_RDWR);
-	VFIO_ASSERT_GE(device->container_fd, 0, "open(%s) failed\n", path);
-
-	version = ioctl(device->container_fd, VFIO_GET_API_VERSION);
-	VFIO_ASSERT_EQ(version, VFIO_API_VERSION);
-}
-
 static void vfio_pci_group_setup(struct vfio_pci_device *device, const char *bdf)
 {
 	struct vfio_group_status group_status = {
@@ -286,26 +324,34 @@ static void vfio_pci_group_setup(struct vfio_pci_device *device, const char *bdf
 	ioctl_assert(device->group_fd, VFIO_GROUP_SET_CONTAINER, &device->container_fd);
 }
 
-static void vfio_pci_iommu_setup(struct vfio_pci_device *device)
+static void vfio_pci_container_setup(struct vfio_pci_device *device, const char *bdf)
 {
 	unsigned long iommu_type = device->iommu_mode->group_iommu_type;
+	const char *path = device->iommu_mode->container_path;
+	int version;
 	int ret;
 
-	INIT_LIST_HEAD(&device->dma_regions);
+	device->container_fd = open(path, O_RDWR);
+	VFIO_ASSERT_GE(device->container_fd, 0, "open(%s) failed\n", path);
+
+	version = ioctl(device->container_fd, VFIO_GET_API_VERSION);
+	VFIO_ASSERT_EQ(version, VFIO_API_VERSION, "Unsupported version: %d\n", version);
+
+	vfio_pci_group_setup(device, bdf);
 
 	ret = ioctl(device->container_fd, VFIO_CHECK_EXTENSION, iommu_type);
 	VFIO_ASSERT_GT(ret, 0, "VFIO IOMMU type %lu not supported\n", iommu_type);
 
 	ioctl_assert(device->container_fd, VFIO_SET_IOMMU, (void *)iommu_type);
+
+	device->fd = ioctl(device->group_fd, VFIO_GROUP_GET_DEVICE_FD, bdf);
+	VFIO_ASSERT_GE(device->fd, 0);
 }
 
-static void vfio_pci_device_setup(struct vfio_pci_device *device, const char *bdf)
+static void vfio_pci_device_setup(struct vfio_pci_device *device)
 {
 	int i;
 
-	device->fd = ioctl(device->group_fd, VFIO_GROUP_GET_DEVICE_FD, bdf);
-	VFIO_ASSERT_GE(device->fd, 0);
-
 	device->info.argsz = sizeof(device->info);
 	ioctl_assert(device->fd, VFIO_DEVICE_GET_INFO, &device->info);
 
@@ -381,6 +427,9 @@ static const struct vfio_iommu_mode iommu_modes[] = {
 		.container_path = "/dev/iommu",
 		.group_iommu_type = VFIO_TYPE1v2_IOMMU,
 	},
+	{
+		.name = "iommufd",
+	},
 };
 
 const char *default_iommu_mode = "vfio_type1_iommu";
@@ -417,6 +466,57 @@ static const struct vfio_iommu_mode *lookup_iommu_mode(const char *iommu_mode)
 	VFIO_FAIL("Unrecognized IOMMU mode: %s\n", iommu_mode);
 }
 
+static void vfio_device_bind_iommufd(int device_fd, int iommufd)
+{
+	struct vfio_device_bind_iommufd args = {
+		.argsz = sizeof(args),
+		.iommufd = iommufd,
+	};
+
+	ioctl_assert(device_fd, VFIO_DEVICE_BIND_IOMMUFD, &args);
+}
+
+static u32 iommufd_ioas_alloc(int iommufd)
+{
+	struct iommu_ioas_alloc args = {
+		.size = sizeof(args),
+	};
+
+	ioctl_assert(iommufd, IOMMU_IOAS_ALLOC, &args);
+	return args.out_ioas_id;
+}
+
+static void vfio_device_attach_iommufd_pt(int device_fd, u32 pt_id)
+{
+	struct vfio_device_attach_iommufd_pt args = {
+		.argsz = sizeof(args),
+		.pt_id = pt_id,
+	};
+
+	ioctl_assert(device_fd, VFIO_DEVICE_ATTACH_IOMMUFD_PT, &args);
+}
+
+static void vfio_pci_iommufd_setup(struct vfio_pci_device *device, const char *bdf)
+{
+	const char *cdev_path = vfio_pci_get_cdev_path(bdf);
+
+	device->fd = open(cdev_path, O_RDWR);
+	VFIO_ASSERT_GE(device->fd, 0);
+	free((void *)cdev_path);
+
+	/*
+	 * Require device->iommufd to be >0 so that a simple non-0 check can be
+	 * used to check if iommufd is enabled. In practice open() will never
+	 * return 0 unless stdin is closed.
+	 */
+	device->iommufd = open("/dev/iommu", O_RDWR);
+	VFIO_ASSERT_GT(device->iommufd, 0);
+
+	vfio_device_bind_iommufd(device->fd, device->iommufd);
+	device->ioas_id = iommufd_ioas_alloc(device->iommufd);
+	vfio_device_attach_iommufd_pt(device->fd, device->ioas_id);
+}
+
 struct vfio_pci_device *vfio_pci_device_init(const char *bdf, const char *iommu_mode)
 {
 	struct vfio_pci_device *device;
@@ -424,13 +524,16 @@ struct vfio_pci_device *vfio_pci_device_init(const char *bdf, const char *iommu_
 	device = calloc(1, sizeof(*device));
 	VFIO_ASSERT_NOT_NULL(device);
 
+	INIT_LIST_HEAD(&device->dma_regions);
+
 	device->iommu_mode = lookup_iommu_mode(iommu_mode);
 
-	vfio_pci_container_setup(device);
-	vfio_pci_group_setup(device, bdf);
-	vfio_pci_iommu_setup(device);
-	vfio_pci_device_setup(device, bdf);
+	if (device->iommu_mode->container_path)
+		vfio_pci_container_setup(device, bdf);
+	else
+		vfio_pci_iommufd_setup(device, bdf);
 
+	vfio_pci_device_setup(device);
 	vfio_pci_driver_probe(device);
 
 	return device;
@@ -454,8 +557,12 @@ void vfio_pci_device_cleanup(struct vfio_pci_device *device)
 		VFIO_ASSERT_EQ(close(device->msi_eventfds[i]), 0);
 	}
 
-	VFIO_ASSERT_EQ(close(device->group_fd), 0);
-	VFIO_ASSERT_EQ(close(device->container_fd), 0);
+	if (device->iommufd) {
+		VFIO_ASSERT_EQ(close(device->iommufd), 0);
+	} else {
+		VFIO_ASSERT_EQ(close(device->group_fd), 0);
+		VFIO_ASSERT_EQ(close(device->container_fd), 0);
+	}
 
 	free(device);
 }
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 29/33] vfio: selftests: Make iommufd the default iommu_mode

[David Matlack]

Now that VFIO selftests support iommufd, make it the default mode.
IOMMUFD is the successor to VFIO_TYPE1{,v2}_IOMMU and all new features
are being added there, so it's a slightly better fit as the default
mode.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 tools/testing/selftests/vfio/lib/vfio_pci_device.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_device.c b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
index d996e012a646..d3c3958136cc 100644
--- a/tools/testing/selftests/vfio/lib/vfio_pci_device.c
+++ b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
@@ -432,7 +432,7 @@ static const struct vfio_iommu_mode iommu_modes[] = {
 	},
 };
 
-const char *default_iommu_mode = "vfio_type1_iommu";
+const char *default_iommu_mode = "iommufd";
 
 void iommu_mode_help(const char *flag)
 {
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 30/33] vfio: selftests: Add a script to help with running VFIO selftests

[David Matlack]

Introduce run.sh, a script to help with running VFIO selftests. The
script is intended to be used for both humans manually running VFIO
selftests, and to incorporate into test automation where VFIO selftests
may run alongside other tests. As such the script aims to be hermetic,
returning the system to the state it was before the test started.

The script takes as input the BDF of a device to use and a command to
run (typically the command would be a VFIO selftest). e.g.

  $ ./run.sh -d 0000:6a:01.0 ./vfio_pci_device_test

 or

  $ ./run.sh -d 0000:6a:01.0 -- ./vfio_pci_device_test

The script then handles unbinding device 0000:6a:01.0 from its current
driver, binding it to vfio-pci, running the test, unbinding from
vfio-pci, and binding back to the original driver.

When run.sh runs the provided test, it does so by appending the BDF as
the last parameter. For example:

  $ ./run.sh -d 0000:6a:01.0 -- echo hello

Results in the following being printed to stdout:

  hello 0000:6a:01.0

The script also supports a mode where it can break out into a shell so
that multiple tests can be run manually.

  $ ./run.sh -d 0000:6a:01.0 -s
  ... bind to vfio-pci and launch $SHELL ...
  $ echo $BDF
  0000:6a:01.0
  $ ./vfio_pci_device_test $BDF
  ...
  $ exit
  ... unbind from vfio-pci ...
  $

Choosing which device to use is up to the user.

In the future this script should be extensible to tests that want to use
multiple devices. The script can support accepting -d BDF multiple times
and parse them into an array, setup all the devices, pass the list of
BDFs to the test, and then cleanup all the devices.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 tools/testing/selftests/vfio/Makefile |   1 +
 tools/testing/selftests/vfio/run.sh   | 110 ++++++++++++++++++++++++++
 2 files changed, 111 insertions(+)
 create mode 100755 tools/testing/selftests/vfio/run.sh

diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftests/vfio/Makefile
index 21fb1809035e..2ab86bd930b0 100644
--- a/tools/testing/selftests/vfio/Makefile
+++ b/tools/testing/selftests/vfio/Makefile
@@ -3,6 +3,7 @@ TEST_GEN_PROGS_EXTENDED += vfio_dma_mapping_test
 TEST_GEN_PROGS_EXTENDED += vfio_iommufd_setup_test
 TEST_GEN_PROGS_EXTENDED += vfio_pci_device_test
 TEST_GEN_PROGS_EXTENDED += vfio_pci_driver_test
+TEST_PROGS_EXTENDED := run.sh
 include ../lib.mk
 include lib/libvfio.mk
 
diff --git a/tools/testing/selftests/vfio/run.sh b/tools/testing/selftests/vfio/run.sh
new file mode 100755
index 000000000000..b461cc1b2f11
--- /dev/null
+++ b/tools/testing/selftests/vfio/run.sh
@@ -0,0 +1,110 @@
+# SPDX-License-Identifier: GPL-2.0-or-later
+
+readonly VFIO_PCI_DRIVER=/sys/bus/pci/drivers/vfio-pci
+
+function bind() {
+	echo "Binding ${1} to ${2}"
+	echo "${1}" > "${2}/bind"
+}
+
+function unbind() {
+	echo "Unbinding ${1} from ${2}"
+	echo "${1}" > "${2}/unbind"
+}
+
+function set_sriov_numvfs() {
+	echo "Setting ${1} sriov_numvfs to ${2}"
+	echo ${2} > /sys/bus/pci/devices/${1}/sriov_numvfs
+}
+
+function add_id() {
+	if echo $(echo ${1} | tr : ' ') > ${2}/new_id 2> /dev/null; then
+		echo "Added ${1} to ${2}"
+		return 0
+	fi
+
+	return 1
+}
+
+function remove_id() {
+	echo "Removing ${1} from ${2}"
+	echo $(echo ${1} | tr : ' ') > ${2}/remove_id
+}
+
+function cleanup() {
+	if [ "${new_driver}" ]; then unbind ${bdf} ${new_driver} ; fi
+	if [ "${new_id}"     ]; then remove_id ${device_id} ${VFIO_PCI_DRIVER} ; fi
+	if [ "${old_driver}" ]; then bind ${bdf} ${old_driver} ; fi
+	if [ "${old_numvfs}" ]; then set_sriov_numvfs ${bdf} ${old_numvfs} ; fi
+}
+
+function usage() {
+	echo "usage: $0 [-d segment:bus:device.function] [-s] [-h] [cmd ...]" >&2
+	echo >&2
+	echo "  -d: The BDF of the device to use for the test (required)" >&2
+	echo "  -h: Show this help message" >&2
+	echo "  -s: Drop into a shell rather than running a command" >&2
+	echo >&2
+	echo "   cmd: The command to run and arguments to pass to it." >&2
+	echo "        Required when not using -s. The SBDF will be " >&2
+	echo "        appended to the argument list." >&2
+	exit 1
+}
+
+function main() {
+	while getopts "d:hs" opt; do
+		case $opt in
+			d) bdf="$OPTARG" ;;
+			s) shell=true ;;
+			*) usage ;;
+		esac
+	done
+
+	# Shift past all optional arguments.
+	shift $((OPTIND - 1))
+
+	# Check that the user passed in the command to run.
+	[ ! "${shell}" ] && [ $# = 0 ] && usage
+
+	# Check that the user passed in a BDF.
+	[ "${bdf}" ] || usage
+
+	trap cleanup EXIT
+	set -e
+
+	test -d /sys/bus/pci/devices/${bdf}
+
+	device_id=$(lspci -s ${bdf} -n | cut -d' ' -f3)
+
+	if [ -f /sys/bus/pci/devices/${bdf}/sriov_numvfs ]; then
+		old_numvfs=$(cat /sys/bus/pci/devices/${bdf}/sriov_numvfs)
+		set_sriov_numvfs ${bdf} 0
+	fi
+
+	if [ -L /sys/bus/pci/devices/${bdf}/driver ]; then
+		old_driver=$(readlink -m /sys/bus/pci/devices/${bdf}/driver)
+		unbind ${bdf} ${old_driver}
+	fi
+
+	# Add the device ID to vfio-pci. If it hasn't already been added, this will
+	# succeed and bind the device to vfio-pci. If it has already been added, this
+	# will fail and we have to manually bind the device.
+	if add_id ${device_id} ${VFIO_PCI_DRIVER}; then
+		new_id=true
+	else
+		bind ${bdf} ${VFIO_PCI_DRIVER}
+	fi
+
+	new_driver=${VFIO_PCI_DRIVER}
+
+	echo
+	if [ "${shell}" ]; then
+		echo "Dropping into ${SHELL} with BDF=${bdf}"
+		BDF=${bdf} ${SHELL}
+	else
+		"$@" ${bdf}
+	fi
+	echo
+}
+
+main "$@"
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 31/33] KVM: selftests: Build and link sefltests/vfio/lib into KVM selftests

[David Matlack]

Include libvfio.mk into the KVM selftests Makefile and link it into all
KVM selftests by adding it to LIBKVM_OBJS.

Note that KVM selftests build their own copy of sefltests/vfio/lib and
the resulting object files are placed in $(OUTPUT)/lib. This allows the
KVM and VFIO selftests to apply different CFLAGS when building without
conflicting with each other.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 tools/testing/selftests/kvm/Makefile.kvm | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 065029b35355..85f651743325 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -252,12 +252,15 @@ pgste-option = $(call try-run, echo 'int main(void) { return 0; }' | \
 LDLIBS += -ldl
 LDFLAGS += -pthread $(no-pie-option) $(pgste-option)
 
+include ../vfio/lib/libvfio.mk
+
 LIBKVM_C := $(filter %.c,$(LIBKVM))
 LIBKVM_S := $(filter %.S,$(LIBKVM))
 LIBKVM_C_OBJ := $(patsubst %.c, $(OUTPUT)/%.o, $(LIBKVM_C))
 LIBKVM_S_OBJ := $(patsubst %.S, $(OUTPUT)/%.o, $(LIBKVM_S))
 LIBKVM_STRING_OBJ := $(patsubst %.c, $(OUTPUT)/%.o, $(LIBKVM_STRING))
-LIBKVM_OBJS = $(LIBKVM_C_OBJ) $(LIBKVM_S_OBJ) $(LIBKVM_STRING_OBJ)
+
+LIBKVM_OBJS = $(LIBKVM_C_OBJ) $(LIBKVM_S_OBJ) $(LIBKVM_STRING_OBJ) $(LIBVFIO_O)
 SPLIT_TEST_GEN_PROGS := $(patsubst %, $(OUTPUT)/%, $(SPLIT_TESTS))
 SPLIT_TEST_GEN_OBJ := $(patsubst %, $(OUTPUT)/$(SRCARCH)/%.o, $(SPLIT_TESTS))
 
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 32/33] KVM: selftests: Test sending a vfio-pci device IRQ to a VM

[David Matlack]

Add a new selftest called vfio_pci_device_irq_test that routes and
delivers an MSI from a vfio-pci device into a guest.

Note that this test currently uses vfio_pci_irq_trigger(), in which VFIO
emulates a device sending the MSI. In the future we want to replace this
with a real MSI so we can test the hardware path (e.g. VT-d on Intel).

This test only supports x86_64 for now, but can be ported to other
architectures in the future.

Signed-off-by: David Matlack <dmatlack@google.com>
---
 tools/testing/selftests/kvm/Makefile.kvm      |   1 +
 .../testing/selftests/kvm/include/kvm_util.h  |   4 +
 tools/testing/selftests/kvm/lib/kvm_util.c    |  21 +++
 .../selftests/kvm/vfio_pci_device_irq_test.c  | 129 ++++++++++++++++++
 4 files changed, 155 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/vfio_pci_device_irq_test.c

diff --git a/tools/testing/selftests/kvm/Makefile.kvm b/tools/testing/selftests/kvm/Makefile.kvm
index 85f651743325..fefb6408b2e9 100644
--- a/tools/testing/selftests/kvm/Makefile.kvm
+++ b/tools/testing/selftests/kvm/Makefile.kvm
@@ -139,6 +139,7 @@ TEST_GEN_PROGS_x86 += rseq_test
 TEST_GEN_PROGS_x86 += steal_time
 TEST_GEN_PROGS_x86 += system_counter_offset_test
 TEST_GEN_PROGS_x86 += pre_fault_memory_test
+TEST_GEN_PROGS_x86 += vfio_pci_device_irq_test
 
 # Compiled outputs used by test targets
 TEST_GEN_PROGS_EXTENDED_x86 += x86/nx_huge_pages_test
diff --git a/tools/testing/selftests/kvm/include/kvm_util.h b/tools/testing/selftests/kvm/include/kvm_util.h
index 373912464fb4..860883ac53e1 100644
--- a/tools/testing/selftests/kvm/include/kvm_util.h
+++ b/tools/testing/selftests/kvm/include/kvm_util.h
@@ -866,6 +866,8 @@ void vcpu_args_set(struct kvm_vcpu *vcpu, unsigned int num, ...);
 void kvm_irq_line(struct kvm_vm *vm, uint32_t irq, int level);
 int _kvm_irq_line(struct kvm_vm *vm, uint32_t irq, int level);
 
+void kvm_add_irqfd(struct kvm_vm *vm, u32 gsi, int fd);
+
 #define KVM_MAX_IRQ_ROUTES		4096
 
 struct kvm_irq_routing *kvm_gsi_routing_create(void);
@@ -874,6 +876,8 @@ void kvm_gsi_routing_irqchip_add(struct kvm_irq_routing *routing,
 int _kvm_gsi_routing_write(struct kvm_vm *vm, struct kvm_irq_routing *routing);
 void kvm_gsi_routing_write(struct kvm_vm *vm, struct kvm_irq_routing *routing);
 
+void kvm_route_gsi(struct kvm_vm *vm, struct kvm_irq_routing_entry *entry);
+
 const char *exit_reason_str(unsigned int exit_reason);
 
 vm_paddr_t vm_phy_page_alloc(struct kvm_vm *vm, vm_paddr_t paddr_min,
diff --git a/tools/testing/selftests/kvm/lib/kvm_util.c b/tools/testing/selftests/kvm/lib/kvm_util.c
index 815bc45dd8dc..3ae752da5065 100644
--- a/tools/testing/selftests/kvm/lib/kvm_util.c
+++ b/tools/testing/selftests/kvm/lib/kvm_util.c
@@ -1871,6 +1871,16 @@ void kvm_irq_line(struct kvm_vm *vm, uint32_t irq, int level)
 	TEST_ASSERT(ret >= 0, KVM_IOCTL_ERROR(KVM_IRQ_LINE, ret));
 }
 
+void kvm_add_irqfd(struct kvm_vm *vm, u32 gsi, int fd)
+{
+	struct kvm_irqfd arg = {
+		.gsi = gsi,
+		.fd = fd,
+	};
+
+	vm_ioctl(vm, KVM_IRQFD, &arg);
+}
+
 struct kvm_irq_routing *kvm_gsi_routing_create(void)
 {
 	struct kvm_irq_routing *routing;
@@ -1921,6 +1931,17 @@ void kvm_gsi_routing_write(struct kvm_vm *vm, struct kvm_irq_routing *routing)
 	TEST_ASSERT(!ret, KVM_IOCTL_ERROR(KVM_SET_GSI_ROUTING, ret));
 }
 
+void kvm_route_gsi(struct kvm_vm *vm, struct kvm_irq_routing_entry *entry)
+{
+	u8 buf[sizeof(struct kvm_irq_routing) + sizeof(*entry)] = {};
+	struct kvm_irq_routing *routes = (void *)&buf;
+
+	routes->nr = 1;
+	routes->entries[0] = *entry;
+
+	vm_ioctl(vm, KVM_SET_GSI_ROUTING, routes);
+}
+
 /*
  * VM Dump
  *
diff --git a/tools/testing/selftests/kvm/vfio_pci_device_irq_test.c b/tools/testing/selftests/kvm/vfio_pci_device_irq_test.c
new file mode 100644
index 000000000000..c792fc169028
--- /dev/null
+++ b/tools/testing/selftests/kvm/vfio_pci_device_irq_test.c
@@ -0,0 +1,129 @@
+// SPDX-License-Identifier: GPL-2.0
+#include "kvm_util.h"
+#include "test_util.h"
+#include "apic.h"
+#include "processor.h"
+
+#include <pthread.h>
+#include <time.h>
+#include <linux/vfio.h>
+#include <vfio_util.h>
+
+static bool guest_ready_for_irq;
+static bool guest_received_irq;
+
+#define TIMEOUT_NS (10ULL * 1000 * 1000 * 1000)
+
+static void guest_enable_interrupts(void)
+{
+	x2apic_enable();
+	asm volatile ("sti");
+}
+
+void kvm_route_msi(struct kvm_vm *vm, u32 gsi, struct kvm_vcpu *vcpu, u8 vector)
+{
+	struct kvm_irq_routing_entry entry = {
+		.gsi = gsi,
+		.type = KVM_IRQ_ROUTING_MSI,
+		.u.msi.address_lo = 0xFEE00000 | (vcpu->id << 12),
+		.u.msi.data = vector,
+	};
+
+	kvm_route_gsi(vm, &entry);
+}
+
+static void guest_irq_handler(struct ex_regs *regs)
+{
+	WRITE_ONCE(guest_received_irq, true);
+	GUEST_DONE();
+}
+
+static void guest_code(void)
+{
+	guest_enable_interrupts();
+	WRITE_ONCE(guest_ready_for_irq, true);
+
+	for (;;)
+		continue;
+}
+
+void *vcpu_thread_main(void *arg)
+{
+	struct kvm_vcpu *vcpu = arg;
+	struct ucall uc;
+
+	vcpu_run(vcpu);
+	TEST_ASSERT_EQ(UCALL_DONE, get_ucall(vcpu, &uc));
+
+	return NULL;
+}
+
+static void help(const char *name)
+{
+	printf("Usage: %s [-i iommu_mode] segment:bus:device.function\n", name);
+	iommu_mode_help("-i");
+	exit(1);
+}
+
+int main(int argc, char **argv)
+{
+	/* Random non-reserved vector and GSI to use for the device IRQ */
+	const u8 vector = 0xe0;
+	const u32 gsi = 32;
+
+	struct timespec start, elapsed;
+	struct vfio_pci_device *device;
+	const char *iommu_mode = NULL;
+	struct kvm_vcpu *vcpu;
+	pthread_t vcpu_thread;
+	struct kvm_vm *vm;
+	int c;
+
+	while ((c = getopt(argc, argv, "i:")) != -1) {
+		switch (c) {
+		case 'i':
+			iommu_mode = optarg;
+			break;
+		default:
+			help(argv[0]);
+		}
+	}
+
+	if (optind >= argc)
+		help(argv[0]);
+
+	vm = vm_create_with_one_vcpu(&vcpu, guest_code);
+	vm_install_exception_handler(vm, vector, guest_irq_handler);
+
+	device = vfio_pci_device_init(argv[optind], iommu_mode);
+	TEST_REQUIRE(device->msix_info.count > 0);
+
+	vfio_pci_msix_enable(device, 0, 1);
+	kvm_add_irqfd(vm, gsi, device->msi_eventfds[0]);
+	kvm_route_msi(vm, gsi, vcpu, vector);
+
+	pthread_create(&vcpu_thread, NULL, vcpu_thread_main, vcpu);
+
+	while (!READ_ONCE(guest_ready_for_irq))
+		sync_global_from_guest(vm, guest_ready_for_irq);
+
+	/*
+	 * TODO: Get the device to send a physical MSI to exercise IRQ Bypass
+	 * (e.g. VT-d on Intel), rather than manually synthesizing a
+	 * notification from VFIO.
+	 */
+	vfio_pci_irq_trigger(device, VFIO_PCI_MSIX_IRQ_INDEX, 0);
+
+	clock_gettime(CLOCK_MONOTONIC, &start);
+
+	while (!READ_ONCE(guest_received_irq)) {
+		elapsed = timespec_elapsed(start);
+		TEST_ASSERT(timespec_to_ns(elapsed) < TIMEOUT_NS, "vCPU never received IRQ\n");
+		sync_global_from_guest(vm, guest_received_irq);
+	}
+
+	pthread_join(vcpu_thread, NULL);
+	vfio_pci_device_cleanup(device);
+
+	return 0;
+}
-- 
2.49.0.1151.ga128411c76-goog

[RFC PATCH 33/33] KVM: selftests: Use real device MSIs in vfio_pci_device_irq_test

[David Matlack]

Use real device MSIs to deliver interrupts into the guest in
vfio_pci_device_irq_test (when possible). If the device has a driver, we
can use the driver to make the device send a physical MSI, rather than
synthesizing an eventfd notification in software. This allows this test
to exercise IRQ Bypass (e.g. VT-d device-posted interrupts in Intel).

Signed-off-by: David Matlack <dmatlack@google.com>
---
 .../selftests/kvm/vfio_pci_device_irq_test.c  | 66 +++++++++++++++----
 1 file changed, 55 insertions(+), 11 deletions(-)

diff --git a/tools/testing/selftests/kvm/vfio_pci_device_irq_test.c b/tools/testing/selftests/kvm/vfio_pci_device_irq_test.c
index c792fc169028..8cbaf38357c4 100644
--- a/tools/testing/selftests/kvm/vfio_pci_device_irq_test.c
+++ b/tools/testing/selftests/kvm/vfio_pci_device_irq_test.c
@@ -7,6 +7,8 @@
 #include <pthread.h>
 #include <time.h>
 #include <linux/vfio.h>
+#include <linux/sizes.h>
+
 #include <vfio_util.h>
 
 static bool guest_ready_for_irq;
@@ -60,11 +62,54 @@ void *vcpu_thread_main(void *arg)
 
 static void help(const char *name)
 {
-	printf("Usage: %s [-i iommu_mode] segment:bus:device.function\n", name);
+	printf("Usage: %s [-i iommu_mode] [-d] segment:bus:device.function\n", name);
+	printf("  -d: Send a real MSI from the device, rather than synthesizing\n"
+	       "      an eventfd signal from VFIO. Note that this option requires\n"
+	       "      a VFIO selftests driver that supports the device.\n");
 	iommu_mode_help("-i");
 	exit(1);
 }
 
+static int setup_msi(struct vfio_pci_device *device, bool use_device_msi)
+{
+	const int flags = MAP_SHARED | MAP_ANONYMOUS;
+	const int prot = PROT_READ | PROT_WRITE;
+	struct vfio_dma_region *region;
+
+	if (use_device_msi) {
+		/* A driver is required to generate an MSI. */
+		TEST_REQUIRE(device->driver.ops);
+
+		/* Set up a DMA-able region for the driver to use. */
+		region = &device->driver.region;
+		region->iova = 0;
+		region->size = SZ_2M;
+		region->vaddr = mmap(NULL, region->size, prot, flags, -1, 0);
+		TEST_ASSERT(region->vaddr != MAP_FAILED, "mmap() failed\n");
+		vfio_pci_dma_map(device, region);
+
+		vfio_pci_driver_init(device);
+
+		return device->driver.msi;
+	}
+
+	TEST_REQUIRE(device->msix_info.count > 0);
+	vfio_pci_msix_enable(device, 0, 1);
+	return 0;
+}
+
+static void send_msi(struct vfio_pci_device *device, bool use_device_msi, int msi)
+{
+	if (use_device_msi) {
+		printf("Sending MSI %d from the device\n", msi);
+		TEST_ASSERT_EQ(msi, device->driver.msi);
+		vfio_pci_driver_send_msi(device);
+	} else {
+		printf("Notifying the eventfd for MSI %d from VFIO\n", msi);
+		vfio_pci_irq_trigger(device, VFIO_PCI_MSIX_IRQ_INDEX, msi);
+	}
+}
+
 int main(int argc, char **argv)
 {
 	/* Random non-reserved vector and GSI to use for the device IRQ */
@@ -74,16 +119,21 @@ int main(int argc, char **argv)
 	struct timespec start, elapsed;
 	struct vfio_pci_device *device;
 	const char *iommu_mode = NULL;
+	bool use_device_msi = false;
 	struct kvm_vcpu *vcpu;
 	pthread_t vcpu_thread;
 	struct kvm_vm *vm;
+	int msi;
 	int c;
 
-	while ((c = getopt(argc, argv, "i:")) != -1) {
+	while ((c = getopt(argc, argv, "i:d")) != -1) {
 		switch (c) {
 		case 'i':
 			iommu_mode = optarg;
 			break;
+		case 'd':
+			use_device_msi = true;
+			break;
 		default:
 			help(argv[0]);
 		}
@@ -96,10 +146,9 @@ int main(int argc, char **argv)
 	vm_install_exception_handler(vm, vector, guest_irq_handler);
 
 	device = vfio_pci_device_init(argv[optind], iommu_mode);
-	TEST_REQUIRE(device->msix_info.count > 0);
+	msi = setup_msi(device, use_device_msi);
 
-	vfio_pci_msix_enable(device, 0, 1);
-	kvm_add_irqfd(vm, gsi, device->msi_eventfds[0]);
+	kvm_add_irqfd(vm, gsi, device->msi_eventfds[msi]);
 	kvm_route_msi(vm, gsi, vcpu, vector);
 
 	pthread_create(&vcpu_thread, NULL, vcpu_thread_main, vcpu);
@@ -107,12 +156,7 @@ int main(int argc, char **argv)
 	while (!READ_ONCE(guest_ready_for_irq))
 		sync_global_from_guest(vm, guest_ready_for_irq);
 
-	/*
-	 * TODO: Get the device to send a physical MSI to exercise IRQ Bypass
-	 * (e.g. VT-d on Intel), rather than manually synthesizing a
-	 * notification from VFIO.
-	 */
-	vfio_pci_irq_trigger(device, VFIO_PCI_MSIX_IRQ_INDEX, 0);
+	send_msi(device, use_device_msi, msi);
 
 	clock_gettime(CLOCK_MONOTONIC, &start);
 
-- 
2.49.0.1151.ga128411c76-goog

Re: [RFC PATCH 00/33] vfio: Introduce selftests for VFIO

[Yi Liu]

On 2025/5/24 07:29, David Matlack wrote:
> This series introduces VFIO selftests, located in
> tools/testing/selftests/vfio/.
> 
> VFIO selftests aim to enable kernel developers to write and run tests
> that take the form of userspace programs that interact with VFIO and
> IOMMUFD uAPIs. VFIO selftests can be used to write functional tests for
> new features, regression tests for bugs, and performance tests for
> optimizations.
> 
> These tests are designed to interact with real PCI devices, i.e. they do
> not rely on mocking out or faking any behavior in the kernel. This
> allows the tests to exercise not only VFIO but also IOMMUFD, the IOMMU
> driver, interrupt remapping, IRQ handling, etc.
> 
> We chose selftests to host these tests primarily to enable integration
> with the existing KVM selftests. As explained in the next section,
> enabling KVM developers to test the interaction between VFIO and KVM is
> one of the motivators of this series.

interesting. Two quick questions:
1) does this selftest support all the vfio iommu drivers (typ1 and 
spapr_tce)? Maybe also the iommufd vfio_compat as well.
2) I know Alex has a test suit as the below. Has this series referred
    it?

https://github.com/awilliam/tests/commits/for-clg/

> Motivation
> -----------------------------------------------------------------------
> 
> The main motivation for this series is upcoming development in the
> kernel to support Hypervisor Live Updates [1][2]. Live Update is a
> specialized reboot process where selected devices are kept operational
> and their kernel state is preserved and recreated across a kexec. For
> devices, DMA and interrupts may continue during the reboot. VFIO-bound
> devices are the main target, since the first usecase of Live Updates is
> to enable host kernel upgrades in a Cloud Computing environment without
> disrupting running customer VMs.
> 
> To prepare for upcoming support for Live Updates in VFIO, IOMMUFD, IOMMU
> drivers, the PCI layer, etc., we'd like to first lay the ground work for
> exercising and testing VFIO from kernel selftests. This way when we
> eventually upstream support for Live Updates, we can also upstream tests
> for those changes, rather than purely relying on Live Update integration
> tests which would be hard to share and reproduce upstream.
> 
> But even without Live Updates, VFIO and IOMMUFD are becoming an
> increasingly critical component of running KVM-based VMs in cloud
> environments. Virtualized networking and storage are increasingly being
> offloaded to smart NICs/cards, and demand for high performance
> networking, storage, and AI are also leading to NICs, SSDs, and GPUs
> being directly attached to VMs via VFIO.
> 
> VFIO selftests increases our ability to test in several ways.
> 
>   - It enables developers sending VFIO, IOMMUFD, etc. commits upstream to
>     test their changes against all existing VFIO selftests, reducing the
>     probability of regressions.
> 
>   - It enables developers sending VFIO, IOMMUFD, etc. commits upstream to
>     include tests alongside their changes, increasing the quality of the
>     code that is merged.
> 
>   - It enables testing the interaction between VFIO and KVM. There are
>     some paths in KVM that are only exercised through VFIO, such as IRQ
>     bypass. VFIO selftests provides a helper library to enable KVM
>     developers to write KVM selftests to test those interactions [3].
> 
> Design
> -----------------------------------------------------------------------
> 
> VFIO selftests are designed around interacting with with VFIO-managed PCI
> devices. As such, the core data struture is struct vfio_pci_device, which
> represents a single PCI device.
> 
>    struct vfio_pci_device *device;
> 
>    device = vfio_pci_device_init("0000:6a:01.0", iommu_mode);
> 
>    ...
> 
>    vfio_pci_device_cleanup(device);
> 
> vfio_pci_device_init() sets up a container or iommufd, depending on the
> iommu_mode argument, to manage DMA mappings, fetches information about
> the device and what interrupts it supports from VFIO and caches it, and
> mmap()s all mappable BARs for the test to use.
> 
> There are helper methods that operate on struct vfio_pci_device to do
> things like read and write to PCI config space, enable/disable IRQs, and
> map memory for DMA,
> 
> struct vfio_pci_device and its methods do not care about what device
> they are actually interacting with. It can be a GPU, a NIC, an SSD, etc.
> 
> To keep things simple initially, VFIO selftests only support a single
> device per group and per container/iommufd. But it should be possible to
> relax those restrictions in the future, e.g. to enable testing with
> multiple devices in the same container/iommufd.
> 
> Driver Framework
> -----------------------------------------------------------------------
> 
> In order to support VFIO selftests where a device is generating DMA and
> interrupts on command, the VFIO selftests supports a driver framework.
> 
> This framework abstracts away device-specific details allowing VFIO
> selftests to be written in a generic way, and then run against different
> devices depending on what hardware developers have access to.
> 
> The framework also aims to support carrying drivers out-of-tree, e.g.
> so that companies can run VFIO selftests with custom/test hardware.
> 
> Drivers must implement the following methods:
> 
>   - probe():        Check if the driver supports a given device.
>   - init():         Initialize the driver.
>   - remove():       Deinitialize the driver and reset the device.
>   - memcpy_start(): Kick off a series of repeated memcpys (DMA reads and
>                     DMA writes).
>   - memcpy_wait():  Wait for a memcpy operation to complete.
>   - send_msi():     Make the device send an MSI interrupt.
> 
> memcpy_start/wait() are for generating DMA. We separate the operation
> into 2 steps so that tests can trigger a long-running DMA operation. We
> expect to use this to stress test Live Updates by kicking off a
> long-running mempcy operation and then performing a Live Update. These
> methods are required to not generate any interrupts.
> 
> send_msi() is used for testing MSI and MSI-x interrupts. The driver
> tells the test which MSI it will be using via device->driver.msi.
> 
> It's the responsibility of the test to set up a region of memory
> and map it into the device for use by the driver, e.g. for in-memory
> descriptors, before calling init().
> 
> A demo of the driver framework can be found in
> tools/testing/selftests/vfio/vfio_pci_driver_test.c.
> 
> In addition, this series introduces a new KVM selftest to demonstrate
> delivering a device MSI directly into a guest, which can be found in
> tools/testing/selftests/kvm/vfio_pci_device_irq_test.c.
> 
> Tests
> -----------------------------------------------------------------------
> 
> There are 5 tests in this series, mostly to demonstrate as a
> proof-of-concept:
> 
>   - tools/testing/selftests/vfio/vfio_pci_device_test.c
>   - tools/testing/selftests/vfio/vfio_pci_driver_test.c
>   - tools/testing/selftests/vfio/vfio_iommufd_setup_test.c
>   - tools/testing/selftests/vfio/vfio_dma_mapping_test.c
>   - tools/testing/selftests/kvm/vfio_pci_device_irq_test.c
> 
> Integrating with KVM selftests
> -----------------------------------------------------------------------
> 
> To support testing the interactions between VFIO and KVM, the VFIO
> selftests support sharing its library with the KVM selftest. The patches
> at the end of this series demonstrate how that works.
> 
> Essentially, we allow the KVM selftests to build their own copy of
> tools/testing/selftests/vfio/lib/ and link it into KVM selftests
> binaries. This requires minimal changes to the KVM selftests Makefile.
> 
> Future Areas of Development
> -----------------------------------------------------------------------
> 
> Library:
> 
>   - Driver support for devices that can be used on AMD, ARM, and other
>     platforms.
>   - Driver support for a device available in QEMU VMs.
>   - Support for tests that use multiple devices.
>   - Support for IOMMU groups with multiple devices.
>   - Support for multiple devices sharing the same container/iommufd.
>   - Sharing TEST_ASSERT() macros and other common code between KVM
>     and VFIO selftests.
> 
> Tests:
> 
>   - DMA mapping performance tests for BARs/HugeTLB/etc.
>   - Live Update selftests.
>   - Porting Sean's KVM selftest for posted interrupts to use the VFIO
>     selftests library [3]
> 
> This series can also be found on GitHub:
> 
>    https://github.com/dmatlack/linux/tree/vfio/selftests/rfc
> 
> Cc: Alex Williamson <alex.williamson@redhat.com>
> Cc: Jason Gunthorpe <jgg@nvidia.com>
> Cc: Kevin Tian <kevin.tian@intel.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Sean Christopherson <seanjc@google.com>
> Cc: Vipin Sharma <vipinsh@google.com>
> Cc: Josh Hilke <jrhilke@google.com>
> Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
> Cc: Saeed Mahameed <saeedm@nvidia.com>
> Cc: Saeed Mahameed <saeedm@nvidia.com>
> Cc: Adithya Jayachandran <ajayachandra@nvidia.com>
> Cc: Parav Pandit <parav@nvidia.com>
> Cc: Leon Romanovsky <leonro@nvidia.com>
> Cc: Vinicius Costa Gomes <vinicius.gomes@intel.com>
> Cc: Dave Jiang <dave.jiang@intel.com>
> Cc: Dan Williams <dan.j.williams@intel.com>
> 
> [1] https://lore.kernel.org/all/f35359d5-63e1-8390-619f-67961443bfe1@google.com/
> [2] https://lore.kernel.org/all/20250515182322.117840-1-pasha.tatashin@soleen.com/
> [3] https://lore.kernel.org/kvm/20250404193923.1413163-68-seanjc@google.com/
> 
> David Matlack (28):
>    selftests: Create tools/testing/selftests/vfio
>    vfio: selftests: Add a helper library for VFIO selftests
>    vfio: selftests: Introduce vfio_pci_device_test
>    tools headers: Add stub definition for __iomem
>    tools headers: Import asm-generic MMIO helpers
>    tools headers: Import x86 MMIO helper overrides
>    tools headers: Import iosubmit_cmds512()
>    tools headers: Import drivers/dma/ioat/{hw.h,registers.h}
>    tools headers: Import drivers/dma/idxd/registers.h
>    tools headers: Import linux/pci_ids.h
>    vfio: selftests: Keep track of DMA regions mapped into the device
>    vfio: selftests: Enable asserting MSI eventfds not firing
>    vfio: selftests: Add a helper for matching vendor+device IDs
>    vfio: selftests: Add driver framework
>    vfio: sefltests: Add vfio_pci_driver_test
>    vfio: selftests: Add driver for Intel CBDMA
>    vfio: selftests: Add driver for Intel DSA
>    vfio: selftests: Move helper to get cdev path to libvfio
>    vfio: selftests: Encapsulate IOMMU mode
>    vfio: selftests: Add [-i iommu_mode] option to all tests
>    vfio: selftests: Add vfio_type1v2_mode
>    vfio: selftests: Add iommufd_compat_type1{,v2} modes
>    vfio: selftests: Add iommufd mode
>    vfio: selftests: Make iommufd the default iommu_mode
>    vfio: selftests: Add a script to help with running VFIO selftests
>    KVM: selftests: Build and link sefltests/vfio/lib into KVM selftests
>    KVM: selftests: Test sending a vfio-pci device IRQ to a VM
>    KVM: selftests: Use real device MSIs in vfio_pci_device_irq_test
> 
> Josh Hilke (5):
>    vfio: selftests: Test basic VFIO and IOMMUFD integration
>    vfio: selftests: Move vfio dma mapping test to their own file
>    vfio: selftests: Add test to reset vfio device.
>    vfio: selftests: Use command line to set hugepage size for DMA mapping
>      test
>    vfio: selftests: Validate 2M/1G HugeTLB are mapped as 2M/1G in IOMMU
> 
>   MAINTAINERS                                   |    7 +
>   tools/arch/x86/include/asm/io.h               |  101 +
>   tools/arch/x86/include/asm/special_insns.h    |   27 +
>   tools/include/asm-generic/io.h                |  482 +++
>   tools/include/asm/io.h                        |   11 +
>   tools/include/drivers/dma/idxd/registers.h    |  601 +++
>   tools/include/drivers/dma/ioat/hw.h           |  270 ++
>   tools/include/drivers/dma/ioat/registers.h    |  251 ++
>   tools/include/linux/compiler.h                |    4 +
>   tools/include/linux/io.h                      |    4 +-
>   tools/include/linux/pci_ids.h                 | 3212 +++++++++++++++++
>   tools/testing/selftests/Makefile              |    1 +
>   tools/testing/selftests/kvm/Makefile.kvm      |    6 +-
>   .../testing/selftests/kvm/include/kvm_util.h  |    4 +
>   tools/testing/selftests/kvm/lib/kvm_util.c    |   21 +
>   .../selftests/kvm/vfio_pci_device_irq_test.c  |  173 +
>   tools/testing/selftests/vfio/.gitignore       |    7 +
>   tools/testing/selftests/vfio/Makefile         |   20 +
>   .../testing/selftests/vfio/lib/drivers/dsa.c  |  416 +++
>   .../testing/selftests/vfio/lib/drivers/ioat.c |  235 ++
>   .../selftests/vfio/lib/include/vfio_util.h    |  271 ++
>   tools/testing/selftests/vfio/lib/libvfio.mk   |   26 +
>   .../selftests/vfio/lib/vfio_pci_device.c      |  573 +++
>   .../selftests/vfio/lib/vfio_pci_driver.c      |  126 +
>   tools/testing/selftests/vfio/run.sh           |  110 +
>   .../selftests/vfio/vfio_dma_mapping_test.c    |  239 ++
>   .../selftests/vfio/vfio_iommufd_setup_test.c  |  133 +
>   .../selftests/vfio/vfio_pci_device_test.c     |  195 +
>   .../selftests/vfio/vfio_pci_driver_test.c     |  256 ++
>   29 files changed, 7780 insertions(+), 2 deletions(-)
>   create mode 100644 tools/arch/x86/include/asm/io.h
>   create mode 100644 tools/arch/x86/include/asm/special_insns.h
>   create mode 100644 tools/include/asm-generic/io.h
>   create mode 100644 tools/include/asm/io.h
>   create mode 100644 tools/include/drivers/dma/idxd/registers.h
>   create mode 100644 tools/include/drivers/dma/ioat/hw.h
>   create mode 100644 tools/include/drivers/dma/ioat/registers.h
>   create mode 100644 tools/include/linux/pci_ids.h
>   create mode 100644 tools/testing/selftests/kvm/vfio_pci_device_irq_test.c
>   create mode 100644 tools/testing/selftests/vfio/.gitignore
>   create mode 100644 tools/testing/selftests/vfio/Makefile
>   create mode 100644 tools/testing/selftests/vfio/lib/drivers/dsa.c
>   create mode 100644 tools/testing/selftests/vfio/lib/drivers/ioat.c
>   create mode 100644 tools/testing/selftests/vfio/lib/include/vfio_util.h
>   create mode 100644 tools/testing/selftests/vfio/lib/libvfio.mk
>   create mode 100644 tools/testing/selftests/vfio/lib/vfio_pci_device.c
>   create mode 100644 tools/testing/selftests/vfio/lib/vfio_pci_driver.c
>   create mode 100755 tools/testing/selftests/vfio/run.sh
>   create mode 100644 tools/testing/selftests/vfio/vfio_dma_mapping_test.c
>   create mode 100644 tools/testing/selftests/vfio/vfio_iommufd_setup_test.c
>   create mode 100644 tools/testing/selftests/vfio/vfio_pci_device_test.c
>   create mode 100644 tools/testing/selftests/vfio/vfio_pci_driver_test.c
> 
> 
> base-commit: a11a72229881d8ac1d52ea727101bc9c744189c1
> prerequisite-patch-id: 3bae97c9e1093148763235f47a84fa040b512d04

-- 
Regards,
Yi Liu

Re: [RFC PATCH 00/33] vfio: Introduce selftests for VFIO

[Jason Gunthorpe]

On Fri, May 23, 2025 at 11:29:45PM +0000, David Matlack wrote:
> Drivers must implement the following methods:
> 
>  - probe():        Check if the driver supports a given device.
>  - init():         Initialize the driver.
>  - remove():       Deinitialize the driver and reset the device.
>  - memcpy_start(): Kick off a series of repeated memcpys (DMA reads and
>                    DMA writes).
>  - memcpy_wait():  Wait for a memcpy operation to complete.
>  - send_msi():     Make the device send an MSI interrupt.
> 
> memcpy_start/wait() are for generating DMA. We separate the operation
> into 2 steps so that tests can trigger a long-running DMA operation. We
> expect to use this to stress test Live Updates by kicking off a
> long-running mempcy operation and then performing a Live Update. These
> methods are required to not generate any interrupts.

I like this, it is a smart way to go about building a testing
framework.

Joel had sent something that looks related:

https://lore.kernel.org/r/5zoh5r6eovbpijic22htkqik6mvyfbma5w7kjzcpz7kgbjufd2@yw6ymwy2a54s

IMHO it would be very nice to lean into what Joel was thinking of a
'libvfn' where the above device stuff is inside a library and it can
be re-used for other purposes.

We keep running into scenarios where it would be really nice to be
able to do DMA testing and we just don't have easy ways to build SW to
do that.

A reusable mini-driver framework that can trigger DMA is a huge leap
forward.

> Library:
> 
>  - Driver support for devices that can be used on AMD, ARM, and other
>    platforms.

I would be very happy to see a mlx5 driver option. mlx5 class HW is
widely available, cheap on ebay, and it would make this usable by
pretty much anyone who wants.

Jason

Re: [RFC PATCH 07/33] vfio: selftests: Use command line to set hugepage size for DMA mapping test

[Jason Gunthorpe]

On Fri, May 23, 2025 at 11:29:52PM +0000, David Matlack wrote:
> From: Josh Hilke <jrhilke@google.com>
> 
> Add a command line arg to vfio_dma_mapping_test to choose the size of
> page which is mapped in VFIO.

This doesn't seem right..

Tests should run automously, test all possible sizes using a fixture.

Jason

Re: [RFC PATCH 08/33] vfio: selftests: Validate 2M/1G HugeTLB are mapped as 2M/1G in IOMMU

[Jason Gunthorpe]

On Fri, May 23, 2025 at 11:29:53PM +0000, David Matlack wrote:
> From: Josh Hilke <jrhilke@google.com>
> 
> Update vfio dma mapping test to verify that the IOMMU uses 2M and 1G
> mappings when 2M and 1G HugeTLB pages are mapped into a device
> respectively.
> 
> This validation is done by inspecting the contents of the I/O page
> tables via /sys/kernel/debug/iommu/intel/. This validation is skipped if
> that directory is not available (i.e. non-Intel IOMMUs).
> 
> Signed-off-by: Josh Hilke <jrhilke@google.com>
> [reword commit message, refactor code]
> Signed-off-by: David Matlack <dmatlack@google.com>
> ---
>  .../selftests/vfio/vfio_dma_mapping_test.c    | 126 +++++++++++++++++-
>  1 file changed, 119 insertions(+), 7 deletions(-)

FWIW, I'm thinking to add an iommufd ioctl to report back on the # of
PTEs of each page size within a range. This would be after we get the
new page table stuff merged.

Jason

Re: [RFC PATCH 13/33] tools headers: Import drivers/dma/ioat/{hw.h,registers.h}

[Jason Gunthorpe]

On Fri, May 23, 2025 at 11:29:58PM +0000, David Matlack wrote:
> Import drivers/dma/ioat/{hw.h,registers.h} into tools/include/ so that
> they can be used in VFIO selftests to interact with Intel CBDMA devices.
> 
> Changes made when importing:
>  - Drop system_has_dca_enabled() prototype from hw.h
> 
> Cc: Dave Jiang <dave.jiang@intel.com>
> Cc: Dan Williams <dan.j.williams@intel.com>
> Signed-off-by: David Matlack <dmatlack@google.com>
> ---
>  tools/include/drivers/dma/ioat/hw.h        | 270 +++++++++++++++++++++
>  tools/include/drivers/dma/ioat/registers.h | 251 +++++++++++++++++++
>  2 files changed, 521 insertions(+)
>  create mode 100644 tools/include/drivers/dma/ioat/hw.h
>  create mode 100644 tools/include/drivers/dma/ioat/registers.h

I think you'd be better to not duplicate the code but just #include a
shared header from  drivers/dma/ioat/ and so on for the other patches
too

Jason

Re: [RFC PATCH 29/33] vfio: selftests: Make iommufd the default iommu_mode

[Jason Gunthorpe]

On Fri, May 23, 2025 at 11:30:14PM +0000, David Matlack wrote:
> Now that VFIO selftests support iommufd, make it the default mode.
> IOMMUFD is the successor to VFIO_TYPE1{,v2}_IOMMU and all new features
> are being added there, so it's a slightly better fit as the default
> mode.

As before, the tests should run through all combinations on their own,
use a fixture.

It would be much better to fix the kselftests so you could regex
select the tests to run and use that as the commandline way to choose
what test combination to use.

Jason

Re: [RFC PATCH 00/33] vfio: Introduce selftests for VFIO

[David Matlack]

On Mon, May 26, 2025 at 2:07 AM Yi Liu <yi.l.liu@intel.com> wrote:
>
> On 2025/5/24 07:29, David Matlack wrote:
> > This series introduces VFIO selftests, located in
> > tools/testing/selftests/vfio/.
> >
> > VFIO selftests aim to enable kernel developers to write and run tests
> > that take the form of userspace programs that interact with VFIO and
> > IOMMUFD uAPIs. VFIO selftests can be used to write functional tests for
> > new features, regression tests for bugs, and performance tests for
> > optimizations.
> >
> > These tests are designed to interact with real PCI devices, i.e. they do
> > not rely on mocking out or faking any behavior in the kernel. This
> > allows the tests to exercise not only VFIO but also IOMMUFD, the IOMMU
> > driver, interrupt remapping, IRQ handling, etc.
> >
> > We chose selftests to host these tests primarily to enable integration
> > with the existing KVM selftests. As explained in the next section,
> > enabling KVM developers to test the interaction between VFIO and KVM is
> > one of the motivators of this series.
>
> interesting. Two quick questions:
> 1) does this selftest support all the vfio iommu drivers (typ1 and
> spapr_tce)? Maybe also the iommufd vfio_compat as well.

This series supports the following IOMMU drivers [1]:
  - VFIO Type1 IOMMU
  - VFIO Type1v2 IOMMU
  - IOMMUFD compat-mode with Type1
  - IOMMUFD compat-mode with Type1v2
  - IOMMUFD

I have not lookied at spapr_tce or any other modes, but I implemented
the code in such a way that other modes could be added in the future.

[1] https://github.com/dmatlack/linux/blob/9832935d7a44aab725ff627c6acf22b8a17d407f/tools/testing/selftests/vfio/lib/vfio_pci_device.c#L409

> 2) I know Alex has a test suit as the below. Has this series referred
>     it?
>
> https://github.com/awilliam/tests/commits/for-clg/

I was not aware of these tests, thanks for sharing the link.

If we think the approach taken in this series makes sense, we could
presumably port those tests to tools/testing/selftests/vfio/. I would
be happy to help with that.

Re: [RFC PATCH 00/33] vfio: Introduce selftests for VFIO

[David Matlack]

On Mon, May 26, 2025 at 10:09 AM Jason Gunthorpe <jgg@nvidia.com> wrote:
>
> On Fri, May 23, 2025 at 11:29:45PM +0000, David Matlack wrote:
> > Drivers must implement the following methods:
> >
> >  - probe():        Check if the driver supports a given device.
> >  - init():         Initialize the driver.
> >  - remove():       Deinitialize the driver and reset the device.
> >  - memcpy_start(): Kick off a series of repeated memcpys (DMA reads and
> >                    DMA writes).
> >  - memcpy_wait():  Wait for a memcpy operation to complete.
> >  - send_msi():     Make the device send an MSI interrupt.
> >
> > memcpy_start/wait() are for generating DMA. We separate the operation
> > into 2 steps so that tests can trigger a long-running DMA operation. We
> > expect to use this to stress test Live Updates by kicking off a
> > long-running mempcy operation and then performing a Live Update. These
> > methods are required to not generate any interrupts.
>
> I like this, it is a smart way to go about building a testing
> framework.
>
> Joel had sent something that looks related:
>
> https://lore.kernel.org/r/5zoh5r6eovbpijic22htkqik6mvyfbma5w7kjzcpz7kgbjufd2@yw6ymwy2a54s

Thanks for sharing, I've started to take a look. Joel, please take a
look at this series too and let me know your thoughts.

>
> IMHO it would be very nice to lean into what Joel was thinking of a
> 'libvfn' where the above device stuff is inside a library and it can
> be re-used for other purposes.
>
> We keep running into scenarios where it would be really nice to be
> able to do DMA testing and we just don't have easy ways to build SW to
> do that.
>
> A reusable mini-driver framework that can trigger DMA is a huge leap
> forward.

How broad do you think the reusability should go?

I structured the library (which includes the driver framework and
drivers) so that it is reusable across other selftests (i.e. not just
in tools/testing/selftests/vfio). The last 3 patches in this series
show it being used in KVM selftests for example. IOMMU-focused tests
in tools/testing/selftests/iommu could also use it.

But it's not reusable outside of selftests, or outside of the kernel
source tree. My intuition is the former wouldn't be too hard to
support, but the latter would be challenging.

>
> > Library:
> >
> >  - Driver support for devices that can be used on AMD, ARM, and other
> >    platforms.
>
> I would be very happy to see a mlx5 driver option. mlx5 class HW is
> widely available, cheap on ebay, and it would make this usable by
> pretty much anyone who wants.

I was also thinking of using NVMe for this (cheap, broadly available),
but I'm a little worried someone might accidentally corrupt their boot
disk if they accidentally pass in the wrong BDF :)

Do you think mlx5 HW could support the current driver API?

Re: [RFC PATCH 08/33] vfio: selftests: Validate 2M/1G HugeTLB are mapped as 2M/1G in IOMMU

[David Matlack]

On Mon, May 26, 2025 at 10:16 AM Jason Gunthorpe <jgg@nvidia.com> wrote:
>
> On Fri, May 23, 2025 at 11:29:53PM +0000, David Matlack wrote:
> > From: Josh Hilke <jrhilke@google.com>
> >
> > Update vfio dma mapping test to verify that the IOMMU uses 2M and 1G
> > mappings when 2M and 1G HugeTLB pages are mapped into a device
> > respectively.
> >
> > This validation is done by inspecting the contents of the I/O page
> > tables via /sys/kernel/debug/iommu/intel/. This validation is skipped if
> > that directory is not available (i.e. non-Intel IOMMUs).
> >
> > Signed-off-by: Josh Hilke <jrhilke@google.com>
> > [reword commit message, refactor code]
> > Signed-off-by: David Matlack <dmatlack@google.com>
> > ---
> >  .../selftests/vfio/vfio_dma_mapping_test.c    | 126 +++++++++++++++++-
> >  1 file changed, 119 insertions(+), 7 deletions(-)
>
> FWIW, I'm thinking to add an iommufd ioctl to report back on the # of
> PTEs of each page size within a range. This would be after we get the
> new page table stuff merged.

Thanks for the heads up. We are using Intel DebugFS because it's
available and better than nothing, but a generic way to validate
mappings sizes would be way better.

Re: [RFC PATCH 13/33] tools headers: Import drivers/dma/ioat/{hw.h,registers.h}

[David Matlack]

On Mon, May 26, 2025 at 10:18 AM Jason Gunthorpe <jgg@nvidia.com> wrote:
>
> On Fri, May 23, 2025 at 11:29:58PM +0000, David Matlack wrote:
> > Import drivers/dma/ioat/{hw.h,registers.h} into tools/include/ so that
> > they can be used in VFIO selftests to interact with Intel CBDMA devices.
> >
> > Changes made when importing:
> >  - Drop system_has_dca_enabled() prototype from hw.h
> >
> > Cc: Dave Jiang <dave.jiang@intel.com>
> > Cc: Dan Williams <dan.j.williams@intel.com>
> > Signed-off-by: David Matlack <dmatlack@google.com>
> > ---
> >  tools/include/drivers/dma/ioat/hw.h        | 270 +++++++++++++++++++++
> >  tools/include/drivers/dma/ioat/registers.h | 251 +++++++++++++++++++
> >  2 files changed, 521 insertions(+)
> >  create mode 100644 tools/include/drivers/dma/ioat/hw.h
> >  create mode 100644 tools/include/drivers/dma/ioat/registers.h
>
> I think you'd be better to not duplicate the code but just #include a
> shared header from  drivers/dma/ioat/ and so on for the other patches
> too

I think that makes sense for drivers/ headers and maybe
linux/pci_ids.h if the relevant maintainers are ok with it. The
changes needed to the headers in the kernel would be pretty minimal.

But it starts to get complicated when kernel headers include other
kernel headers, and/or contain a lot of code that can't be compiled in
tools/, e.g. asm-generic/io.h [1]. For those headers, duplicating
seemed like the cleanest option.

[1] https://lore.kernel.org/kvm/20250523233018.1702151-11-dmatlack@google.com/

Re: [RFC PATCH 00/33] vfio: Introduce selftests for VFIO

[Jason Gunthorpe]

On Tue, May 27, 2025 at 04:01:52PM -0700, David Matlack wrote:
> > A reusable mini-driver framework that can trigger DMA is a huge leap
> > forward.
> 
> How broad do you think the reusability should go?
> 
> I structured the library (which includes the driver framework and
> drivers) so that it is reusable across other selftests (i.e. not just
> in tools/testing/selftests/vfio). The last 3 patches in this series
> show it being used in KVM selftests for example. IOMMU-focused tests
> in tools/testing/selftests/iommu could also use it.

I think having it as a usable library within selftests is a good place
to start at least. It shows it has a clean API boundary at least.

> But it's not reusable outside of selftests, or outside of the kernel
> source tree. My intuition is the former wouldn't be too hard to
> support, but the latter would be challenging.

And then we can see if there is interest to move it outside.

> I was also thinking of using NVMe for this (cheap, broadly available),
> but I'm a little worried someone might accidentally corrupt their boot
> disk if they accidentally pass in the wrong BDF :)

Yeah, you can't do memcpy on NVMe without being destructive.

You'd want an alternative stimulus API that was more like 'DMA write
something random to X", then you could DMA READ from the media and use
that as a non-destructive test.

> Do you think mlx5 HW could support the current driver API?

I think it can do memcpy. It would require copying a lot of code but
it is "straightforward" to setup a loopback QP and then issue RDMA
WRITE operations to memcpy data. It would act almost the same as IDXD.

There are examples doing this in the kernel, and we have examples in
rdma-core how to boot the device under VFIO.

Jason

Re: [RFC PATCH 00/33] vfio: Introduce selftests for VFIO

[David Matlack]

On Tue, May 27, 2025 at 5:11 PM Jason Gunthorpe <jgg@nvidia.com> wrote:
>
> On Tue, May 27, 2025 at 04:01:52PM -0700, David Matlack wrote:
> > > A reusable mini-driver framework that can trigger DMA is a huge leap
> > > forward.
> >
> > How broad do you think the reusability should go?
> >
> > I structured the library (which includes the driver framework and
> > drivers) so that it is reusable across other selftests (i.e. not just
> > in tools/testing/selftests/vfio). The last 3 patches in this series
> > show it being used in KVM selftests for example. IOMMU-focused tests
> > in tools/testing/selftests/iommu could also use it.
>
> I think having it as a usable library within selftests is a good place
> to start at least. It shows it has a clean API boundary at least.
>
> > But it's not reusable outside of selftests, or outside of the kernel
> > source tree. My intuition is the former wouldn't be too hard to
> > support, but the latter would be challenging.
>
> And then we can see if there is interest to move it outside.

Sounds good to me.

> > I was also thinking of using NVMe for this (cheap, broadly available),
> > but I'm a little worried someone might accidentally corrupt their boot
> > disk if they accidentally pass in the wrong BDF :)
>
> Yeah, you can't do memcpy on NVMe without being destructive.
>
> You'd want an alternative stimulus API that was more like 'DMA write
> something random to X", then you could DMA READ from the media and use
> that as a non-destructive test.

Yeah we would need a different driver API. Intel DSA supports a Memory
Fill operation, which would be similar. But the nice thing about
memcpy is you can validate that the contents of memory are "correct"
after the DMA. With NVMe we wouldn't be able to have any guarantees
about what exactly would get written.

If mlx5 HW is truly cheap and broadly available then maybe we just
align on that for baremetal tests and not worry about NVMe. That way
we can keep the memcpy API and be able to validate the contents of
DMAs.

For running these tests in VMs, Joel's pci-ats-testdev [1] looks like
a good option.

[1] https://github.com/Joelgranados/qemu/blob/pcie-testdev/hw/misc/pcie-ats-testdev.c

>
> > Do you think mlx5 HW could support the current driver API?
>
> I think it can do memcpy. It would require copying a lot of code but
> it is "straightforward" to setup a loopback QP and then issue RDMA
> WRITE operations to memcpy data. It would act almost the same as IDXD.
>
> There are examples doing this in the kernel, and we have examples in
> rdma-core how to boot the device under VFIO.

Good to know. I'd need some help from someone from Nvidia to write the
driver though, or it might take a while.

Re: [RFC PATCH 07/33] vfio: selftests: Use command line to set hugepage size for DMA mapping test

[David Matlack]

On Mon, May 26, 2025 at 10:15 AM Jason Gunthorpe <jgg@nvidia.com> wrote:
>
> On Fri, May 23, 2025 at 11:29:52PM +0000, David Matlack wrote:
> > From: Josh Hilke <jrhilke@google.com>
> >
> > Add a command line arg to vfio_dma_mapping_test to choose the size of
> > page which is mapped in VFIO.
>
> This doesn't seem right..
>
> Tests should run automously, test all possible sizes using a fixture.

This test uses a fixture already. I assume you're referring to
FIXTURE_VARIANT()?

I'll explore doing this. For a single dimension this looks possible.
But for multiple dimensions (e.g. cross product of iommu_mode and
backing_src) I don't see a clear way to do it. But that's just after a
cursory look.

For context, the pattern of passing in test configuration via flags
rather than automatically testing all combinations is something
inherited from KVM selftests. That's the common pattern there. There's
some work happening there to encode configurations at a higher level
using testcase files and a runner [1].

There are also some challenges with making VFIO selftests (or any
selftest that uses tools/testing/selftests/vfio/lib) truly autonomous:

 - The library needs to know which device to use. In this RFC that
works by the user passing in BDF as a positional argument to each
test.
 - For tests that use HugeTLB (like this one), the test requires the
user to have already allocated HugeTLB memory for it to use.

One idea would be to have tests test all possible iommu_modes by
default (with the ability to override and pick a specific mode) and
then let everything else be driven by flags.

[1] https://lore.kernel.org/kvm/20250222005943.3348627-1-vipinsh@google.com/

Re: [RFC PATCH 07/33] vfio: selftests: Use command line to set hugepage size for DMA mapping test

[Jason Gunthorpe]

On Fri, May 30, 2025 at 09:50:22AM -0700, David Matlack wrote:
> On Mon, May 26, 2025 at 10:15 AM Jason Gunthorpe <jgg@nvidia.com> wrote:
> >
> > On Fri, May 23, 2025 at 11:29:52PM +0000, David Matlack wrote:
> > > From: Josh Hilke <jrhilke@google.com>
> > >
> > > Add a command line arg to vfio_dma_mapping_test to choose the size of
> > > page which is mapped in VFIO.
> >
> > This doesn't seem right..
> >
> > Tests should run automously, test all possible sizes using a fixture.
> 
> This test uses a fixture already. I assume you're referring to
> FIXTURE_VARIANT()?

Yes


> I'll explore doing this. For a single dimension this looks possible.
> But for multiple dimensions (e.g. cross product of iommu_mode and
> backing_src) I don't see a clear way to do it. But that's just after a
> cursory look.

Explicitly list all the combinations with macros?

Enhance the userspace tests allow code to generate the
variants? Kernel tests can do this:

/**
 * KUNIT_CASE_PARAM - A helper for creation a parameterized &struct kunit_case
 *
 * @test_name: a reference to a test case function.
 * @gen_params: a reference to a parameter generator function.
 *
 * The generator function::
 *
 *	const void* gen_params(const void *prev, char *desc)
 *
 * is used to lazily generate a series of arbitrarily typed values that fit into
 * a void*. The argument @prev is the previously returned value, which should be
 * used to derive the next value; @prev is set to NULL on the initial generator
 * call. When no more values are available, the generator must return NULL.
 * Optionally write a string into @desc (size of KUNIT_PARAM_DESC_SIZE)
 * describing the parameter.
 */
#define KUNIT_CASE_PARAM(test_name, gen_params)			\
		{ .run_case = test_name, .name = #test_name,	\
		  .generate_params = gen_params, .module_name = KBUILD_MODNAME}

> For context, the pattern of passing in test configuration via flags
> rather than automatically testing all combinations is something
> inherited from KVM selftests. That's the common pattern there. There's
> some work happening there to encode configurations at a higher level
> using testcase files and a runner [1].

IMHO it is not good, it should be done with fixtures and variants
slicing the test by matching the test fixture/name/etc.

It doesn't really make sense to have a C test runner that you have to
invoke from bash just because the C stuff is not very good..

>  - The library needs to know which device to use. In this RFC that
> works by the user passing in BDF as a positional argument to each
> test.

I'd probably say it should scan the available vfio devices and pick
the first that it understands how to use. Command line would be an
option.

>  - For tests that use HugeTLB (like this one), the test requires the
> user to have already allocated HugeTLB memory for it to use.

Other tests do this already, like the iommufd test assumes it can get
hugetlb mmaps. Skip the tests on allocation failure is the best I
think you can do.

It would be nice to have test metadata so a runner frame work could
know to provide this stuff in the environment.

Jason

Re: [RFC PATCH 08/33] vfio: selftests: Validate 2M/1G HugeTLB are mapped as 2M/1G in IOMMU

[David Matlack]

On Fri, May 23, 2025 at 4:30 PM David Matlack <dmatlack@google.com> wrote:
> +static int iommu_mapping_get(const char *bdf, u64 iova,
> +                            struct iommu_mapping *mapping)
> +{
> +       if (access("/sys/kernel/debug/iommu/intel", F_OK))
> +               return intel_iommu_mapping_get(bdf, iova, mapping);
> +

Oops, this should be !access(...).

> +       return -EOPNOTSUPP;
> +}

Re: [RFC PATCH 08/33] vfio: selftests: Validate 2M/1G HugeTLB are mapped as 2M/1G in IOMMU

[Andy Shevchenko]

On Fri, May 30, 2025 at 02:12:36PM -0700, David Matlack wrote:
> On Fri, May 23, 2025 at 4:30 PM David Matlack <dmatlack@google.com> wrote:

...

> > +       if (access("/sys/kernel/debug/iommu/intel", F_OK))
> > +               return intel_iommu_mapping_get(bdf, iova, mapping);
> > +
> 
> Oops, this should be !access(...).

Hmm... Generally speaking that code is a hack. Is there any guarantee that
debugfs mount point is fixed to /sys/kernel/debug ?

> > +       return -EOPNOTSUPP;

-- 
With Best Regards,
Andy Shevchenko

Re: [RFC PATCH 08/33] vfio: selftests: Validate 2M/1G HugeTLB are mapped as 2M/1G in IOMMU

[Greg Kroah-Hartman]

On Sat, May 31, 2025 at 10:09:34PM +0300, Andy Shevchenko wrote:
> On Fri, May 30, 2025 at 02:12:36PM -0700, David Matlack wrote:
> > On Fri, May 23, 2025 at 4:30 PM David Matlack <dmatlack@google.com> wrote:
> 
> ...
> 
> > > +       if (access("/sys/kernel/debug/iommu/intel", F_OK))
> > > +               return intel_iommu_mapping_get(bdf, iova, mapping);
> > > +
> > 
> > Oops, this should be !access(...).
> 
> Hmm... Generally speaking that code is a hack. Is there any guarantee that
> debugfs mount point is fixed to /sys/kernel/debug ?

No, userspace can mount it anywhere :)

But the kernel does create the mount point at /sys/kernel/debug/ by
default, so hopefully it does end up there if it is present in the
system, so you should be able to rely on it for testing.

thanks,

greg k-h

Re: [RFC PATCH 00/33] vfio: Introduce selftests for VFIO

[Cédric Le Goater]

On 5/24/25 01:29, David Matlack wrote:
> This series introduces VFIO selftests, located in
> tools/testing/selftests/vfio/.
> 
> VFIO selftests aim to enable kernel developers to write and run tests
> that take the form of userspace programs that interact with VFIO and
> IOMMUFD uAPIs. VFIO selftests can be used to write functional tests for
> new features, regression tests for bugs, and performance tests for
> optimizations.
> 
> These tests are designed to interact with real PCI devices, i.e. they do
> not rely on mocking out or faking any behavior in the kernel. This
> allows the tests to exercise not only VFIO but also IOMMUFD, the IOMMU
> driver, interrupt remapping, IRQ handling, etc.
> 
> We chose selftests to host these tests primarily to enable integration
> with the existing KVM selftests. As explained in the next section,
> enabling KVM developers to test the interaction between VFIO and KVM is
> one of the motivators of this series.
> 
> Motivation
> -----------------------------------------------------------------------
> 
> The main motivation for this series is upcoming development in the
> kernel to support Hypervisor Live Updates [1][2]. Live Update is a
> specialized reboot process where selected devices are kept operational
> and their kernel state is preserved and recreated across a kexec. For
> devices, DMA and interrupts may continue during the reboot. VFIO-bound
> devices are the main target, since the first usecase of Live Updates is
> to enable host kernel upgrades in a Cloud Computing environment without
> disrupting running customer VMs.
> 
> To prepare for upcoming support for Live Updates in VFIO, IOMMUFD, IOMMU
> drivers, the PCI layer, etc., we'd like to first lay the ground work for
> exercising and testing VFIO from kernel selftests. This way when we
> eventually upstream support for Live Updates, we can also upstream tests
> for those changes, rather than purely relying on Live Update integration
> tests which would be hard to share and reproduce upstream.
> 
> But even without Live Updates, VFIO and IOMMUFD are becoming an
> increasingly critical component of running KVM-based VMs in cloud
> environments. Virtualized networking and storage are increasingly being
> offloaded to smart NICs/cards, and demand for high performance
> networking, storage, and AI are also leading to NICs, SSDs, and GPUs
> being directly attached to VMs via VFIO.
> 
> VFIO selftests increases our ability to test in several ways.
> 
>   - It enables developers sending VFIO, IOMMUFD, etc. commits upstream to
>     test their changes against all existing VFIO selftests, reducing the
>     probability of regressions.
> 
>   - It enables developers sending VFIO, IOMMUFD, etc. commits upstream to
>     include tests alongside their changes, increasing the quality of the
>     code that is merged.
> 
>   - It enables testing the interaction between VFIO and KVM. There are
>     some paths in KVM that are only exercised through VFIO, such as IRQ
>     bypass. VFIO selftests provides a helper library to enable KVM
>     developers to write KVM selftests to test those interactions [3].
> 
> Design
> -----------------------------------------------------------------------
> 
> VFIO selftests are designed around interacting with with VFIO-managed PCI
> devices. As such, the core data struture is struct vfio_pci_device, which
> represents a single PCI device.
> 
>    struct vfio_pci_device *device;
> 
>    device = vfio_pci_device_init("0000:6a:01.0", iommu_mode);
> 
>    ...
> 
>    vfio_pci_device_cleanup(device);
> 
> vfio_pci_device_init() sets up a container or iommufd, depending on the
> iommu_mode argument, to manage DMA mappings, fetches information about
> the device and what interrupts it supports from VFIO and caches it, and
> mmap()s all mappable BARs for the test to use.
> 
> There are helper methods that operate on struct vfio_pci_device to do
> things like read and write to PCI config space, enable/disable IRQs, and
> map memory for DMA,
> 
> struct vfio_pci_device and its methods do not care about what device
> they are actually interacting with. It can be a GPU, a NIC, an SSD, etc.
> 
> To keep things simple initially, VFIO selftests only support a single
> device per group and per container/iommufd. But it should be possible to
> relax those restrictions in the future, e.g. to enable testing with
> multiple devices in the same container/iommufd.
> 
> Driver Framework
> -----------------------------------------------------------------------
> 
> In order to support VFIO selftests where a device is generating DMA and
> interrupts on command, the VFIO selftests supports a driver framework.
> 
> This framework abstracts away device-specific details allowing VFIO
> selftests to be written in a generic way, and then run against different
> devices depending on what hardware developers have access to.
> 
> The framework also aims to support carrying drivers out-of-tree, e.g.
> so that companies can run VFIO selftests with custom/test hardware.
> > Drivers must implement the following methods:
> 
>   - probe():        Check if the driver supports a given device.
>   - init():         Initialize the driver.
>   - remove():       Deinitialize the driver and reset the device.
>   - memcpy_start(): Kick off a series of repeated memcpys (DMA reads and
>                     DMA writes).
>   - memcpy_wait():  Wait for a memcpy operation to complete.
>   - send_msi():     Make the device send an MSI interrupt.
> 
> memcpy_start/wait() are for generating DMA. We separate the operation
> into 2 steps so that tests can trigger a long-running DMA operation. We
> expect to use this to stress test Live Updates by kicking off a
> long-running mempcy operation and then performing a Live Update. These
> methods are required to not generate any interrupts.
> 
> send_msi() is used for testing MSI and MSI-x interrupts. The driver
> tells the test which MSI it will be using via device->driver.msi.
> 
> It's the responsibility of the test to set up a region of memory
> and map it into the device for use by the driver, e.g. for in-memory
> descriptors, before calling init().
> 
> A demo of the driver framework can be found in
> tools/testing/selftests/vfio/vfio_pci_driver_test.c.
> 
> In addition, this series introduces a new KVM selftest to demonstrate
> delivering a device MSI directly into a guest, which can be found in
> tools/testing/selftests/kvm/vfio_pci_device_irq_test.c.
> 
> Tests
> -----------------------------------------------------------------------
> 
> There are 5 tests in this series, mostly to demonstrate as a
> proof-of-concept:
> 
>   - tools/testing/selftests/vfio/vfio_pci_device_test.c
>   - tools/testing/selftests/vfio/vfio_pci_driver_test.c
>   - tools/testing/selftests/vfio/vfio_iommufd_setup_test.c
>   - tools/testing/selftests/vfio/vfio_dma_mapping_test.c
>   - tools/testing/selftests/kvm/vfio_pci_device_irq_test.c
> 
> Integrating with KVM selftests
> -----------------------------------------------------------------------
> 
> To support testing the interactions between VFIO and KVM, the VFIO
> selftests support sharing its library with the KVM selftest. The patches
> at the end of this series demonstrate how that works.
> 
> Essentially, we allow the KVM selftests to build their own copy of
> tools/testing/selftests/vfio/lib/ and link it into KVM selftests
> binaries. This requires minimal changes to the KVM selftests Makefile.
> 
> Future Areas of Development
> -----------------------------------------------------------------------
> 
> Library:
> 
>   - Driver support for devices that can be used on AMD, ARM, and other
>     platforms.
>   - Driver support for a device available in QEMU VMs.

The QEMU igb and igbvf devices are good candidates.

I have been using these in QEMU VMs to test a few scenarios, IOMMUFD
and also VFIO migration with a custom VFIO PCI igbvf variant driver,
with dummy get/set handlers.

We also have the mdev mtty device driver but I think it is less useful.
Having a PCI device is more interesting for test coverage.

Anyhow, this is an excellent proposal. I plan to give it a try as soon
as possible, once the recent QEMU proposals for VFIO have made progress.

>   - Support for tests that use multiple devices.
>   - Support for IOMMU groups with multiple devices.
>   - Support for multiple devices sharing the same container/iommufd.
>   - Sharing TEST_ASSERT() macros and other common code between KVM
>     and VFIO selftests.
> 
> Tests:
> 
>   - DMA mapping performance tests for BARs/HugeTLB/etc.


Thanks,

C.




>   - Live Update selftests.
>   - Porting Sean's KVM selftest for posted interrupts to use the VFIO
>     selftests library [3]
> 
> This series can also be found on GitHub:
> 
>    https://github.com/dmatlack/linux/tree/vfio/selftests/rfc
> 
> Cc: Alex Williamson <alex.williamson@redhat.com>
> Cc: Jason Gunthorpe <jgg@nvidia.com>
> Cc: Kevin Tian <kevin.tian@intel.com>
> Cc: Paolo Bonzini <pbonzini@redhat.com>
> Cc: Sean Christopherson <seanjc@google.com>
> Cc: Vipin Sharma <vipinsh@google.com>
> Cc: Josh Hilke <jrhilke@google.com>
> Cc: Pasha Tatashin <pasha.tatashin@soleen.com>
> Cc: Saeed Mahameed <saeedm@nvidia.com>
> Cc: Saeed Mahameed <saeedm@nvidia.com>
> Cc: Adithya Jayachandran <ajayachandra@nvidia.com>
> Cc: Parav Pandit <parav@nvidia.com>
> Cc: Leon Romanovsky <leonro@nvidia.com>
> Cc: Vinicius Costa Gomes <vinicius.gomes@intel.com>
> Cc: Dave Jiang <dave.jiang@intel.com>
> Cc: Dan Williams <dan.j.williams@intel.com>
> 
> [1] https://lore.kernel.org/all/f35359d5-63e1-8390-619f-67961443bfe1@google.com/
> [2] https://lore.kernel.org/all/20250515182322.117840-1-pasha.tatashin@soleen.com/
> [3] https://lore.kernel.org/kvm/20250404193923.1413163-68-seanjc@google.com/
> 
> David Matlack (28):
>    selftests: Create tools/testing/selftests/vfio
>    vfio: selftests: Add a helper library for VFIO selftests
>    vfio: selftests: Introduce vfio_pci_device_test
>    tools headers: Add stub definition for __iomem
>    tools headers: Import asm-generic MMIO helpers
>    tools headers: Import x86 MMIO helper overrides
>    tools headers: Import iosubmit_cmds512()
>    tools headers: Import drivers/dma/ioat/{hw.h,registers.h}
>    tools headers: Import drivers/dma/idxd/registers.h
>    tools headers: Import linux/pci_ids.h
>    vfio: selftests: Keep track of DMA regions mapped into the device
>    vfio: selftests: Enable asserting MSI eventfds not firing
>    vfio: selftests: Add a helper for matching vendor+device IDs
>    vfio: selftests: Add driver framework
>    vfio: sefltests: Add vfio_pci_driver_test
>    vfio: selftests: Add driver for Intel CBDMA
>    vfio: selftests: Add driver for Intel DSA
>    vfio: selftests: Move helper to get cdev path to libvfio
>    vfio: selftests: Encapsulate IOMMU mode
>    vfio: selftests: Add [-i iommu_mode] option to all tests
>    vfio: selftests: Add vfio_type1v2_mode
>    vfio: selftests: Add iommufd_compat_type1{,v2} modes
>    vfio: selftests: Add iommufd mode
>    vfio: selftests: Make iommufd the default iommu_mode
>    vfio: selftests: Add a script to help with running VFIO selftests
>    KVM: selftests: Build and link sefltests/vfio/lib into KVM selftests
>    KVM: selftests: Test sending a vfio-pci device IRQ to a VM
>    KVM: selftests: Use real device MSIs in vfio_pci_device_irq_test
> 
> Josh Hilke (5):
>    vfio: selftests: Test basic VFIO and IOMMUFD integration
>    vfio: selftests: Move vfio dma mapping test to their own file
>    vfio: selftests: Add test to reset vfio device.
>    vfio: selftests: Use command line to set hugepage size for DMA mapping
>      test
>    vfio: selftests: Validate 2M/1G HugeTLB are mapped as 2M/1G in IOMMU
> 
>   MAINTAINERS                                   |    7 +
>   tools/arch/x86/include/asm/io.h               |  101 +
>   tools/arch/x86/include/asm/special_insns.h    |   27 +
>   tools/include/asm-generic/io.h                |  482 +++
>   tools/include/asm/io.h                        |   11 +
>   tools/include/drivers/dma/idxd/registers.h    |  601 +++
>   tools/include/drivers/dma/ioat/hw.h           |  270 ++
>   tools/include/drivers/dma/ioat/registers.h    |  251 ++
>   tools/include/linux/compiler.h                |    4 +
>   tools/include/linux/io.h                      |    4 +-
>   tools/include/linux/pci_ids.h                 | 3212 +++++++++++++++++
>   tools/testing/selftests/Makefile              |    1 +
>   tools/testing/selftests/kvm/Makefile.kvm      |    6 +-
>   .../testing/selftests/kvm/include/kvm_util.h  |    4 +
>   tools/testing/selftests/kvm/lib/kvm_util.c    |   21 +
>   .../selftests/kvm/vfio_pci_device_irq_test.c  |  173 +
>   tools/testing/selftests/vfio/.gitignore       |    7 +
>   tools/testing/selftests/vfio/Makefile         |   20 +
>   .../testing/selftests/vfio/lib/drivers/dsa.c  |  416 +++
>   .../testing/selftests/vfio/lib/drivers/ioat.c |  235 ++
>   .../selftests/vfio/lib/include/vfio_util.h    |  271 ++
>   tools/testing/selftests/vfio/lib/libvfio.mk   |   26 +
>   .../selftests/vfio/lib/vfio_pci_device.c      |  573 +++
>   .../selftests/vfio/lib/vfio_pci_driver.c      |  126 +
>   tools/testing/selftests/vfio/run.sh           |  110 +
>   .../selftests/vfio/vfio_dma_mapping_test.c    |  239 ++
>   .../selftests/vfio/vfio_iommufd_setup_test.c  |  133 +
>   .../selftests/vfio/vfio_pci_device_test.c     |  195 +
>   .../selftests/vfio/vfio_pci_driver_test.c     |  256 ++
>   29 files changed, 7780 insertions(+), 2 deletions(-)
>   create mode 100644 tools/arch/x86/include/asm/io.h
>   create mode 100644 tools/arch/x86/include/asm/special_insns.h
>   create mode 100644 tools/include/asm-generic/io.h
>   create mode 100644 tools/include/asm/io.h
>   create mode 100644 tools/include/drivers/dma/idxd/registers.h
>   create mode 100644 tools/include/drivers/dma/ioat/hw.h
>   create mode 100644 tools/include/drivers/dma/ioat/registers.h
>   create mode 100644 tools/include/linux/pci_ids.h
>   create mode 100644 tools/testing/selftests/kvm/vfio_pci_device_irq_test.c
>   create mode 100644 tools/testing/selftests/vfio/.gitignore
>   create mode 100644 tools/testing/selftests/vfio/Makefile
>   create mode 100644 tools/testing/selftests/vfio/lib/drivers/dsa.c
>   create mode 100644 tools/testing/selftests/vfio/lib/drivers/ioat.c
>   create mode 100644 tools/testing/selftests/vfio/lib/include/vfio_util.h
>   create mode 100644 tools/testing/selftests/vfio/lib/libvfio.mk
>   create mode 100644 tools/testing/selftests/vfio/lib/vfio_pci_device.c
>   create mode 100644 tools/testing/selftests/vfio/lib/vfio_pci_driver.c
>   create mode 100755 tools/testing/selftests/vfio/run.sh
>   create mode 100644 tools/testing/selftests/vfio/vfio_dma_mapping_test.c
>   create mode 100644 tools/testing/selftests/vfio/vfio_iommufd_setup_test.c
>   create mode 100644 tools/testing/selftests/vfio/vfio_pci_device_test.c
>   create mode 100644 tools/testing/selftests/vfio/vfio_pci_driver_test.c
> 
> 
> base-commit: a11a72229881d8ac1d52ea727101bc9c744189c1
> prerequisite-patch-id: 3bae97c9e1093148763235f47a84fa040b512d04

Re: [RFC PATCH 30/33] vfio: selftests: Add a script to help with running VFIO selftests

[Alex Williamson]

On Fri, 23 May 2025 23:30:15 +0000
David Matlack <dmatlack@google.com> wrote:

> Introduce run.sh, a script to help with running VFIO selftests. The
> script is intended to be used for both humans manually running VFIO
> selftests, and to incorporate into test automation where VFIO selftests
> may run alongside other tests. As such the script aims to be hermetic,
> returning the system to the state it was before the test started.
> 
> The script takes as input the BDF of a device to use and a command to
> run (typically the command would be a VFIO selftest). e.g.
> 
>   $ ./run.sh -d 0000:6a:01.0 ./vfio_pci_device_test
> 
>  or
> 
>   $ ./run.sh -d 0000:6a:01.0 -- ./vfio_pci_device_test
> 
> The script then handles unbinding device 0000:6a:01.0 from its current
> driver, binding it to vfio-pci, running the test, unbinding from
> vfio-pci, and binding back to the original driver.
> 
> When run.sh runs the provided test, it does so by appending the BDF as
> the last parameter. For example:
> 
>   $ ./run.sh -d 0000:6a:01.0 -- echo hello
> 
> Results in the following being printed to stdout:
> 
>   hello 0000:6a:01.0
> 
> The script also supports a mode where it can break out into a shell so
> that multiple tests can be run manually.
> 
>   $ ./run.sh -d 0000:6a:01.0 -s
>   ... bind to vfio-pci and launch $SHELL ...
>   $ echo $BDF
>   0000:6a:01.0
>   $ ./vfio_pci_device_test $BDF
>   ...
>   $ exit
>   ... unbind from vfio-pci ...
>   $
> 
> Choosing which device to use is up to the user.
> 
> In the future this script should be extensible to tests that want to use
> multiple devices. The script can support accepting -d BDF multiple times
> and parse them into an array, setup all the devices, pass the list of
> BDFs to the test, and then cleanup all the devices.
> 
> Signed-off-by: David Matlack <dmatlack@google.com>
> ---
>  tools/testing/selftests/vfio/Makefile |   1 +
>  tools/testing/selftests/vfio/run.sh   | 110 ++++++++++++++++++++++++++
>  2 files changed, 111 insertions(+)
>  create mode 100755 tools/testing/selftests/vfio/run.sh
> 
> diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftests/vfio/Makefile
> index 21fb1809035e..2ab86bd930b0 100644
> --- a/tools/testing/selftests/vfio/Makefile
> +++ b/tools/testing/selftests/vfio/Makefile
> @@ -3,6 +3,7 @@ TEST_GEN_PROGS_EXTENDED += vfio_dma_mapping_test
>  TEST_GEN_PROGS_EXTENDED += vfio_iommufd_setup_test
>  TEST_GEN_PROGS_EXTENDED += vfio_pci_device_test
>  TEST_GEN_PROGS_EXTENDED += vfio_pci_driver_test
> +TEST_PROGS_EXTENDED := run.sh
>  include ../lib.mk
>  include lib/libvfio.mk
>  
> diff --git a/tools/testing/selftests/vfio/run.sh b/tools/testing/selftests/vfio/run.sh
> new file mode 100755
> index 000000000000..b461cc1b2f11
> --- /dev/null
> +++ b/tools/testing/selftests/vfio/run.sh
> @@ -0,0 +1,110 @@
> +# SPDX-License-Identifier: GPL-2.0-or-later
> +
> +readonly VFIO_PCI_DRIVER=/sys/bus/pci/drivers/vfio-pci
> +
> +function bind() {
> +	echo "Binding ${1} to ${2}"
> +	echo "${1}" > "${2}/bind"
> +}
> +
> +function unbind() {
> +	echo "Unbinding ${1} from ${2}"
> +	echo "${1}" > "${2}/unbind"
> +}
> +
> +function set_sriov_numvfs() {
> +	echo "Setting ${1} sriov_numvfs to ${2}"
> +	echo ${2} > /sys/bus/pci/devices/${1}/sriov_numvfs
> +}
> +
> +function add_id() {
> +	if echo $(echo ${1} | tr : ' ') > ${2}/new_id 2> /dev/null; then
> +		echo "Added ${1} to ${2}"
> +		return 0
> +	fi
> +
> +	return 1
> +}

I'd suggest using the newer driver_override mechanism rather than
new_id/remove_id.

I appreciate the work here, I think this could be really useful.  As
Jason notes a few times, I'd also like to see this automatically
iterate through all the combinations of options.  Maybe we could also
think about whether we can define some build environment options to
specify devices to use and this could all be run automatically from
'make kselftest'.  Thanks,

Alex

> +
> +function remove_id() {
> +	echo "Removing ${1} from ${2}"
> +	echo $(echo ${1} | tr : ' ') > ${2}/remove_id
> +}
> +
> +function cleanup() {
> +	if [ "${new_driver}" ]; then unbind ${bdf} ${new_driver} ; fi
> +	if [ "${new_id}"     ]; then remove_id ${device_id} ${VFIO_PCI_DRIVER} ; fi
> +	if [ "${old_driver}" ]; then bind ${bdf} ${old_driver} ; fi
> +	if [ "${old_numvfs}" ]; then set_sriov_numvfs ${bdf} ${old_numvfs} ; fi
> +}
> +
> +function usage() {
> +	echo "usage: $0 [-d segment:bus:device.function] [-s] [-h] [cmd ...]" >&2
> +	echo >&2
> +	echo "  -d: The BDF of the device to use for the test (required)" >&2
> +	echo "  -h: Show this help message" >&2
> +	echo "  -s: Drop into a shell rather than running a command" >&2
> +	echo >&2
> +	echo "   cmd: The command to run and arguments to pass to it." >&2
> +	echo "        Required when not using -s. The SBDF will be " >&2
> +	echo "        appended to the argument list." >&2
> +	exit 1
> +}
> +
> +function main() {
> +	while getopts "d:hs" opt; do
> +		case $opt in
> +			d) bdf="$OPTARG" ;;
> +			s) shell=true ;;
> +			*) usage ;;
> +		esac
> +	done
> +
> +	# Shift past all optional arguments.
> +	shift $((OPTIND - 1))
> +
> +	# Check that the user passed in the command to run.
> +	[ ! "${shell}" ] && [ $# = 0 ] && usage
> +
> +	# Check that the user passed in a BDF.
> +	[ "${bdf}" ] || usage
> +
> +	trap cleanup EXIT
> +	set -e
> +
> +	test -d /sys/bus/pci/devices/${bdf}
> +
> +	device_id=$(lspci -s ${bdf} -n | cut -d' ' -f3)
> +
> +	if [ -f /sys/bus/pci/devices/${bdf}/sriov_numvfs ]; then
> +		old_numvfs=$(cat /sys/bus/pci/devices/${bdf}/sriov_numvfs)
> +		set_sriov_numvfs ${bdf} 0
> +	fi
> +
> +	if [ -L /sys/bus/pci/devices/${bdf}/driver ]; then
> +		old_driver=$(readlink -m /sys/bus/pci/devices/${bdf}/driver)
> +		unbind ${bdf} ${old_driver}
> +	fi
> +
> +	# Add the device ID to vfio-pci. If it hasn't already been added, this will
> +	# succeed and bind the device to vfio-pci. If it has already been added, this
> +	# will fail and we have to manually bind the device.
> +	if add_id ${device_id} ${VFIO_PCI_DRIVER}; then
> +		new_id=true
> +	else
> +		bind ${bdf} ${VFIO_PCI_DRIVER}
> +	fi
> +
> +	new_driver=${VFIO_PCI_DRIVER}
> +
> +	echo
> +	if [ "${shell}" ]; then
> +		echo "Dropping into ${SHELL} with BDF=${bdf}"
> +		BDF=${bdf} ${SHELL}
> +	else
> +		"$@" ${bdf}
> +	fi
> +	echo
> +}
> +
> +main "$@"

Re: [RFC PATCH 04/33] vfio: selftests: Test basic VFIO and IOMMUFD integration

[Alex Williamson]

On Fri, 23 May 2025 23:29:49 +0000
David Matlack <dmatlack@google.com> wrote:
> +int main(int argc, char *argv[])
> +{
> +	char *bdf;
> +
> +	if (argc != 2) {
> +		printf("Usage: %s bus:device:function\n", argv[0]);

segment:bus:device.function?

> +		return 1;

1 or KSFT_FAIL?  Thanks,

Alex

> +	}
> +
> +	bdf = argv[1];
> +	set_cdev_path(bdf);
> +	printf("Using cdev device %s\n", cdev_path);
> +
> +	return test_harness_run(1, argv);
> +}

Re: [RFC PATCH 00/33] vfio: Introduce selftests for VFIO

[Joel Granados]

[-- Attachment #1: Type: text/plain, Size: 1573 bytes --]

On Tue, May 27, 2025 at 04:01:52PM -0700, David Matlack wrote:
> On Mon, May 26, 2025 at 10:09 AM Jason Gunthorpe <jgg@nvidia.com> wrote:
> >
> > On Fri, May 23, 2025 at 11:29:45PM +0000, David Matlack wrote:
> > > Drivers must implement the following methods:
> > >
> > >  - probe():        Check if the driver supports a given device.
> > >  - init():         Initialize the driver.
> > >  - remove():       Deinitialize the driver and reset the device.
> > >  - memcpy_start(): Kick off a series of repeated memcpys (DMA reads and
> > >                    DMA writes).
> > >  - memcpy_wait():  Wait for a memcpy operation to complete.
> > >  - send_msi():     Make the device send an MSI interrupt.
> > >
> > > memcpy_start/wait() are for generating DMA. We separate the operation
> > > into 2 steps so that tests can trigger a long-running DMA operation. We
> > > expect to use this to stress test Live Updates by kicking off a
> > > long-running mempcy operation and then performing a Live Update. These
> > > methods are required to not generate any interrupts.
> >
> > I like this, it is a smart way to go about building a testing
> > framework.
> >
> > Joel had sent something that looks related:
> >
> > https://lore.kernel.org/r/5zoh5r6eovbpijic22htkqik6mvyfbma5w7kjzcpz7kgbjufd2@yw6ymwy2a54s
> 
> Thanks for sharing, I've started to take a look. Joel, please take a
> look at this series too and let me know your thoughts.
Added to my back log. Looking forward to see if this fits into iommutests.

Best

-- 

Joel Granados

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]

Re: [RFC PATCH 02/33] vfio: selftests: Add a helper library for VFIO selftests

[liulongfang]

On 2025/5/24 7:29, David Matlack wrote:
> Add a basic helper library to be used by VFIO selftests.
> 
> The basic unit of the library is struct vfio_pci_device, which
> represents a single PCI device that is bound to the vfio-pci driver. The
> library currently only supports a single device per group and container,
> and VFIO IOMMU types.
> 
> The code in this library was heavily based on prior work done by
> Raghavendra Rao Ananta <rananta@google.com>, and the VFIO_ASSERT*()
> macros were written by Vipin Sharma <vipinsh@google.com>.
> 
> Separate that Makefile rules for building the library into a separate
> script so that the library can be built by and linked into KVM selftests
> in a subsequent commit.
> 
> Signed-off-by: David Matlack <dmatlack@google.com>
> ---
>  tools/testing/selftests/vfio/Makefile         |  13 +
>  .../selftests/vfio/lib/include/vfio_util.h    | 126 +++++++
>  tools/testing/selftests/vfio/lib/libvfio.mk   |  17 +
>  .../selftests/vfio/lib/vfio_pci_device.c      | 330 ++++++++++++++++++
>  4 files changed, 486 insertions(+)
>  create mode 100644 tools/testing/selftests/vfio/lib/include/vfio_util.h
>  create mode 100644 tools/testing/selftests/vfio/lib/libvfio.mk
>  create mode 100644 tools/testing/selftests/vfio/lib/vfio_pci_device.c
> 
> diff --git a/tools/testing/selftests/vfio/Makefile b/tools/testing/selftests/vfio/Makefile
> index 2bba39aff5d9..f8933dcdc8e1 100644
> --- a/tools/testing/selftests/vfio/Makefile
> +++ b/tools/testing/selftests/vfio/Makefile

You should provide a configuration file that allows users to select which features
require the self-testing function to be enabled.

Thanks.

> @@ -1,2 +1,15 @@
>  CFLAGS = $(KHDR_INCLUDES)
>  include ../lib.mk
> +include lib/libvfio.mk
> +
> +CFLAGS += -I$(top_srcdir)/tools/include
> +CFLAGS += -MD
> +
> +$(TEST_GEN_PROGS_EXTENDED): %: %.o $(LIBVFIO_O)
> +	$(CC) $(CFLAGS) $(CPPFLAGS) $(LDFLAGS) $< $(LIBVFIO_O) $(LDLIBS) -o $@
> +
> +TEST_GEN_PROGS_EXTENDED_O = $(patsubst %, %.o, $(TEST_GEN_PROGS_EXTENDED))
> +TEST_DEP_FILES = $(patsubst %.o, %.d, $(TEST_GEN_PROGS_EXTENDED_O) $(LIBVFIO_O))
> +-include $(TEST_DEP_FILES)
> +
> +EXTRA_CLEAN += $(TEST_GEN_PROGS_EXTENDED_O) $(TEST_DEP_FILES)
> diff --git a/tools/testing/selftests/vfio/lib/include/vfio_util.h b/tools/testing/selftests/vfio/lib/include/vfio_util.h
> new file mode 100644
> index 000000000000..ea54754a8b16
> --- /dev/null
> +++ b/tools/testing/selftests/vfio/lib/include/vfio_util.h
> @@ -0,0 +1,126 @@
> +/* SPDX-License-Identifier: GPL-2.0-only */
> +#ifndef SELFTESTS_VFIO_LIB_INCLUDE_VFIO_UTIL_H
> +#define SELFTESTS_VFIO_LIB_INCLUDE_VFIO_UTIL_H
> +
> +#include <string.h>
> +#include <linux/vfio.h>
> +#include <linux/list.h>
> +#include <linux/pci_regs.h>
> +
> +#include "../../../kselftest.h"
> +
> +#define VFIO_LOG_AND_EXIT(...) do {		\
> +	fprintf(stderr, "  " __VA_ARGS__);	\
> +	fprintf(stderr, "\n");			\
> +	exit(KSFT_FAIL);			\
> +} while (0)
> +
> +#define VFIO_ASSERT_OP(_lhs, _rhs, _op, ...) do {				\
> +	typeof(_lhs) __lhs = (_lhs);						\
> +	typeof(_rhs) __rhs = (_rhs);						\
> +										\
> +	if (__lhs _op __rhs)							\
> +		break;								\
> +										\
> +	fprintf(stderr, "%s:%u: Assertion Failure\n\n", __FILE__, __LINE__);	\
> +	fprintf(stderr, "  Expression: " #_lhs " " #_op " " #_rhs "\n");	\
> +	fprintf(stderr, "  Observed: %#lx %s %#lx\n",				\
> +			(u64)__lhs, #_op, (u64)__rhs);				\
> +	fprintf(stderr, "  [errno: %d - %s\n]", errno, strerror(errno));	\
> +	VFIO_LOG_AND_EXIT(__VA_ARGS__);						\
> +} while (0)
> +
> +#define VFIO_ASSERT_EQ(_a, _b, ...) VFIO_ASSERT_OP(_a, _b, ==, ##__VA_ARGS__)
> +#define VFIO_ASSERT_NE(_a, _b, ...) VFIO_ASSERT_OP(_a, _b, !=, ##__VA_ARGS__)
> +#define VFIO_ASSERT_LT(_a, _b, ...) VFIO_ASSERT_OP(_a, _b, <, ##__VA_ARGS__)
> +#define VFIO_ASSERT_LE(_a, _b, ...) VFIO_ASSERT_OP(_a, _b, <=, ##__VA_ARGS__)
> +#define VFIO_ASSERT_GT(_a, _b, ...) VFIO_ASSERT_OP(_a, _b, >, ##__VA_ARGS__)
> +#define VFIO_ASSERT_GE(_a, _b, ...) VFIO_ASSERT_OP(_a, _b, >=, ##__VA_ARGS__)
> +#define VFIO_ASSERT_TRUE(_a, ...) VFIO_ASSERT_NE(false, (_a), ##__VA_ARGS__)
> +#define VFIO_ASSERT_FALSE(_a, ...) VFIO_ASSERT_EQ(false, (_a), ##__VA_ARGS__)
> +#define VFIO_ASSERT_NULL(_a, ...) VFIO_ASSERT_EQ(NULL, _a, ##__VA_ARGS__)
> +#define VFIO_ASSERT_NOT_NULL(_a, ...) VFIO_ASSERT_NE(NULL, _a, ##__VA_ARGS__)
> +
> +#define VFIO_FAIL(_fmt, ...) do {				\
> +	fprintf(stderr, "%s:%u: FAIL\n\n", __FILE__, __LINE__);	\
> +	VFIO_LOG_AND_EXIT(_fmt, ##__VA_ARGS__);			\
> +} while (0)
> +
> +struct vfio_pci_bar {
> +	struct vfio_region_info info;
> +	void *vaddr;
> +};
> +
> +struct vfio_pci_device {
> +	int fd;
> +	int group_fd;
> +	int container_fd;
> +
> +	struct vfio_device_info info;
> +	struct vfio_region_info config_space;
> +	struct vfio_pci_bar bars[PCI_STD_NUM_BARS];
> +
> +	struct vfio_irq_info msi_info;
> +	struct vfio_irq_info msix_info;
> +
> +	/* eventfds for MSI and MSI-x interrupts */
> +	int msi_eventfds[PCI_MSIX_FLAGS_QSIZE + 1];
> +};
> +
> +struct vfio_pci_device *vfio_pci_device_init(const char *bdf, int iommu_type);
> +void vfio_pci_device_cleanup(struct vfio_pci_device *device);
> +
> +void vfio_pci_dma_map(struct vfio_pci_device *device, u64 iova, u64 size,
> +		      void *vaddr);
> +void vfio_pci_dma_unmap(struct vfio_pci_device *device, u64 iova, u64 size);
> +
> +void vfio_pci_config_access(struct vfio_pci_device *device, bool write,
> +			    size_t config, size_t size, void *data);
> +
> +#define vfio_pci_config_read(_device, _offset, _type) ({			    \
> +	_type __data;								    \
> +	vfio_pci_config_access((_device), false, _offset, sizeof(__data), &__data); \
> +	__data;									    \
> +})
> +
> +#define vfio_pci_config_readb(_d, _o) vfio_pci_config_read(_d, _o, u8)
> +#define vfio_pci_config_readw(_d, _o) vfio_pci_config_read(_d, _o, u16)
> +#define vfio_pci_config_readl(_d, _o) vfio_pci_config_read(_d, _o, u32)
> +
> +#define vfio_pci_config_write(_device, _offset, _value, _type) do {		  \
> +	_type __data = (_value);						  \
> +	vfio_pci_config_access((_device), true, _offset, sizeof(_type), &__data); \
> +} while (0)
> +
> +#define vfio_pci_config_writeb(_d, _o, _v) vfio_pci_config_write(_d, _o, _v, u8)
> +#define vfio_pci_config_writew(_d, _o, _v) vfio_pci_config_write(_d, _o, _v, u16)
> +#define vfio_pci_config_writel(_d, _o, _v) vfio_pci_config_write(_d, _o, _v, u32)
> +
> +void vfio_pci_irq_enable(struct vfio_pci_device *device, u32 index,
> +			 u32 vector, int count);
> +void vfio_pci_irq_disable(struct vfio_pci_device *device, u32 index);
> +void vfio_pci_irq_trigger(struct vfio_pci_device *device, u32 index, u32 vector);
> +
> +static inline void vfio_pci_msi_enable(struct vfio_pci_device *device,
> +				       u32 vector, int count)
> +{
> +	vfio_pci_irq_enable(device, VFIO_PCI_MSI_IRQ_INDEX, vector, count);
> +}
> +
> +static inline void vfio_pci_msi_disable(struct vfio_pci_device *device)
> +{
> +	vfio_pci_irq_disable(device, VFIO_PCI_MSI_IRQ_INDEX);
> +}
> +
> +static inline void vfio_pci_msix_enable(struct vfio_pci_device *device,
> +					u32 vector, int count)
> +{
> +	vfio_pci_irq_enable(device, VFIO_PCI_MSIX_IRQ_INDEX, vector, count);
> +}
> +
> +static inline void vfio_pci_msix_disable(struct vfio_pci_device *device)
> +{
> +	vfio_pci_irq_disable(device, VFIO_PCI_MSIX_IRQ_INDEX);
> +}
> +
> +#endif /* SELFTESTS_VFIO_LIB_INCLUDE_VFIO_UTIL_H */
> diff --git a/tools/testing/selftests/vfio/lib/libvfio.mk b/tools/testing/selftests/vfio/lib/libvfio.mk
> new file mode 100644
> index 000000000000..388533f8d407
> --- /dev/null
> +++ b/tools/testing/selftests/vfio/lib/libvfio.mk
> @@ -0,0 +1,17 @@
> +VFIO_DIR := $(selfdir)/vfio
> +
> +LIBVFIO_C := lib/vfio_pci_device.c
> +
> +LIBVFIO_O := $(patsubst %.c, $(OUTPUT)/%.o, $(LIBVFIO_C))
> +
> +LIBVFIO_O_DIRS := $(shell dirname $(LIBVFIO_O) | uniq)
> +
> +CFLAGS += -I$(VFIO_DIR)/lib/include
> +
> +$(LIBVFIO_O): $(OUTPUT)/%.o : $(VFIO_DIR)/%.c $(LIBVFIO_O_DIRS)
> +	$(CC) $(CFLAGS) $(CPPFLAGS) $(TARGET_ARCH) -c $< -o $@
> +
> +$(LIBVFIO_O_DIRS):
> +	mkdir -p $@
> +
> +EXTRA_CLEAN += $(LIBVFIO_O)
> diff --git a/tools/testing/selftests/vfio/lib/vfio_pci_device.c b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
> new file mode 100644
> index 000000000000..922f21138532
> --- /dev/null
> +++ b/tools/testing/selftests/vfio/lib/vfio_pci_device.c
> @@ -0,0 +1,330 @@
> +// SPDX-License-Identifier: GPL-2.0-only
> +#include <fcntl.h>
> +#include <libgen.h>
> +#include <stdlib.h>
> +#include <string.h>
> +#include <unistd.h>
> +
> +#include <sys/eventfd.h>
> +#include <sys/ioctl.h>
> +#include <sys/mman.h>
> +
> +#include <linux/limits.h>
> +#include <linux/mman.h>
> +#include <linux/types.h>
> +#include <linux/vfio.h>
> +
> +#include "../../../kselftest.h"
> +#include <vfio_util.h>
> +
> +#define VFIO_DEV_PATH	"/dev/vfio/vfio"
> +#define PCI_SYSFS_PATH	"/sys/bus/pci/devices"
> +
> +#define ioctl_assert(_fd, _op, _arg) do {					  \
> +	void *__arg = (_arg);							  \
> +	int __ret = ioctl((_fd), (_op), (__arg));				  \
> +	VFIO_ASSERT_EQ(__ret, 0,						  \
> +		    "ioctl(%s, %s, %s) returned %d\n", #_fd, #_op, #_arg, __ret); \
> +} while (0)
> +
> +static void vfio_pci_irq_set(struct vfio_pci_device *device,
> +			     u32 index, u32 vector, u32 count, int *fds)
> +{
> +	u8 buf[sizeof(struct vfio_irq_set) + sizeof(int) * count] = {};
> +	struct vfio_irq_set *irq = (void *)&buf;
> +	int *irq_fds = (void *)&irq->data;
> +
> +	irq->argsz = sizeof(buf);
> +	irq->flags = VFIO_IRQ_SET_ACTION_TRIGGER;
> +	irq->index = index;
> +	irq->start = vector;
> +	irq->count = count;
> +
> +	if (count) {
> +		irq->flags |= VFIO_IRQ_SET_DATA_EVENTFD;
> +		memcpy(irq_fds, fds, sizeof(int) * count);
> +	} else {
> +		irq->flags |= VFIO_IRQ_SET_DATA_NONE;
> +	}
> +
> +	ioctl_assert(device->fd, VFIO_DEVICE_SET_IRQS, irq);
> +}
> +
> +void vfio_pci_irq_trigger(struct vfio_pci_device *device, u32 index, u32 vector)
> +{
> +	struct vfio_irq_set irq = {
> +		.argsz = sizeof(irq),
> +		.flags = VFIO_IRQ_SET_ACTION_TRIGGER | VFIO_IRQ_SET_DATA_NONE,
> +		.index = index,
> +		.start = vector,
> +		.count = 1,
> +	};
> +
> +	ioctl_assert(device->fd, VFIO_DEVICE_SET_IRQS, &irq);
> +}
> +
> +static void check_supported_irq_index(u32 index)
> +{
> +	/* VFIO selftests only supports MSI and MSI-x for now. */
> +	VFIO_ASSERT_TRUE(index == VFIO_PCI_MSI_IRQ_INDEX ||
> +			 index == VFIO_PCI_MSIX_IRQ_INDEX,
> +			 "Unsupported IRQ index: %u\n", index);
> +}
> +
> +void vfio_pci_irq_enable(struct vfio_pci_device *device, u32 index, u32 vector,
> +			 int count)
> +{
> +	int i;
> +
> +	check_supported_irq_index(index);
> +
> +	for (i = vector; i < vector + count; i++) {
> +		VFIO_ASSERT_LT(device->msi_eventfds[i], 0);
> +		device->msi_eventfds[i] = eventfd(0, 0);
> +		VFIO_ASSERT_GE(device->msi_eventfds[i], 0);
> +	}
> +
> +	vfio_pci_irq_set(device, index, vector, count, device->msi_eventfds + vector);
> +}
> +
> +void vfio_pci_irq_disable(struct vfio_pci_device *device, u32 index)
> +{
> +	int i;
> +
> +	check_supported_irq_index(index);
> +
> +	for (i = 0; i < ARRAY_SIZE(device->msi_eventfds); i++) {
> +		if (device->msi_eventfds[i] < 0)
> +			continue;
> +
> +		VFIO_ASSERT_EQ(close(device->msi_eventfds[i]), 0);
> +		device->msi_eventfds[i] = -1;
> +	}
> +
> +	vfio_pci_irq_set(device, index, 0, 0, NULL);
> +}
> +
> +static void vfio_pci_irq_get(struct vfio_pci_device *device, u32 index,
> +			     struct vfio_irq_info *irq_info)
> +{
> +	irq_info->argsz = sizeof(*irq_info);
> +	irq_info->index = index;
> +
> +	ioctl_assert(device->fd, VFIO_DEVICE_GET_IRQ_INFO, irq_info);
> +}
> +
> +void vfio_pci_dma_map(struct vfio_pci_device *device, u64 iova, u64 size, void *vaddr)
> +{
> +	struct vfio_iommu_type1_dma_map map = {
> +		.argsz = sizeof(map),
> +		.flags = VFIO_DMA_MAP_FLAG_READ | VFIO_DMA_MAP_FLAG_WRITE,
> +		.vaddr = (u64)vaddr,
> +		.iova = iova,
> +		.size = size,
> +	};
> +
> +	ioctl_assert(device->container_fd, VFIO_IOMMU_MAP_DMA, &map);
> +}
> +
> +void vfio_pci_dma_unmap(struct vfio_pci_device *device, u64 iova, u64 size)
> +{
> +	struct vfio_iommu_type1_dma_unmap unmap = {
> +		.argsz = sizeof(unmap),
> +		.iova = iova,
> +		.size = size,
> +	};
> +
> +	ioctl_assert(device->container_fd, VFIO_IOMMU_UNMAP_DMA, &unmap);
> +}
> +
> +static void vfio_pci_region_get(struct vfio_pci_device *device, int index,
> +				struct vfio_region_info *info)
> +{
> +	memset(info, 0, sizeof(*info));
> +
> +	info->argsz = sizeof(*info);
> +	info->index = index;
> +
> +	ioctl_assert(device->fd, VFIO_DEVICE_GET_REGION_INFO, info);
> +}
> +
> +static void vfio_pci_bar_map(struct vfio_pci_device *device, int index)
> +{
> +	struct vfio_pci_bar *bar = &device->bars[index];
> +	int prot = 0;
> +
> +	VFIO_ASSERT_LT(index, PCI_STD_NUM_BARS);
> +	VFIO_ASSERT_NULL(bar->vaddr);
> +	VFIO_ASSERT_TRUE(bar->info.flags & VFIO_REGION_INFO_FLAG_MMAP);
> +
> +	if (bar->info.flags & VFIO_REGION_INFO_FLAG_READ)
> +		prot |= PROT_READ;
> +	if (bar->info.flags & VFIO_REGION_INFO_FLAG_WRITE)
> +		prot |= PROT_WRITE;
> +
> +	bar->vaddr = mmap(NULL, bar->info.size, prot, MAP_FILE | MAP_SHARED,
> +			  device->fd, bar->info.offset);
> +	VFIO_ASSERT_NE(bar->vaddr, MAP_FAILED);
> +}
> +
> +static void vfio_pci_bar_unmap(struct vfio_pci_device *device, int index)
> +{
> +	struct vfio_pci_bar *bar = &device->bars[index];
> +
> +	VFIO_ASSERT_LT(index, PCI_STD_NUM_BARS);
> +	VFIO_ASSERT_NOT_NULL(bar->vaddr);
> +
> +	VFIO_ASSERT_EQ(munmap(bar->vaddr, bar->info.size), 0);
> +	bar->vaddr = NULL;
> +}
> +
> +static void vfio_pci_bar_unmap_all(struct vfio_pci_device *device)
> +{
> +	int i;
> +
> +	for (i = 0; i < PCI_STD_NUM_BARS; i++) {
> +		if (device->bars[i].vaddr)
> +			vfio_pci_bar_unmap(device, i);
> +	}
> +}
> +
> +void vfio_pci_config_access(struct vfio_pci_device *device, bool write,
> +			    size_t config, size_t size, void *data)
> +{
> +	struct vfio_region_info *config_space = &device->config_space;
> +	int ret;
> +
> +	if (write)
> +		ret = pwrite(device->fd, data, size, config_space->offset + config);
> +	else
> +		ret = pread(device->fd, data, size, config_space->offset + config);
> +
> +	VFIO_ASSERT_EQ(ret, size, "Failed to %s PCI config space: 0x%lx\n",
> +		       write ? "write to" : "read from", config);
> +}
> +
> +static unsigned int vfio_pci_get_group_from_dev(const char *bdf)
> +{
> +	char dev_iommu_group_path[PATH_MAX] = {0};
> +	char sysfs_path[PATH_MAX] = {0};
> +	unsigned int group;
> +	int ret;
> +
> +	snprintf(sysfs_path, PATH_MAX, "%s/%s/iommu_group", PCI_SYSFS_PATH, bdf);
> +
> +	ret = readlink(sysfs_path, dev_iommu_group_path, sizeof(dev_iommu_group_path));
> +	VFIO_ASSERT_NE(ret, -1, "Failed to get the IOMMU group for device: %s\n", bdf);
> +
> +	ret = sscanf(basename(dev_iommu_group_path), "%u", &group);
> +	VFIO_ASSERT_EQ(ret, 1, "Failed to get the IOMMU group for device: %s\n", bdf);
> +
> +	return group;
> +}
> +
> +static void vfio_pci_container_setup(struct vfio_pci_device *device)
> +{
> +	int version;
> +
> +	device->container_fd = open(VFIO_DEV_PATH, O_RDWR);
> +	VFIO_ASSERT_GE(device->container_fd, 0, "open(%s) failed\n", VFIO_DEV_PATH);
> +
> +	version = ioctl(device->container_fd, VFIO_GET_API_VERSION);
> +	VFIO_ASSERT_EQ(version, VFIO_API_VERSION);
> +}
> +
> +static void vfio_pci_group_setup(struct vfio_pci_device *device, const char *bdf)
> +{
> +	struct vfio_group_status group_status = {
> +		.argsz = sizeof(group_status),
> +	};
> +	char group_path[32];
> +	int group;
> +
> +	group = vfio_pci_get_group_from_dev(bdf);
> +	snprintf(group_path, sizeof(group_path), "/dev/vfio/%d", group);
> +
> +	device->group_fd = open(group_path, O_RDWR);
> +	VFIO_ASSERT_GE(device->group_fd, 0, "open(%s) failed\n", group_path);
> +
> +	ioctl_assert(device->group_fd, VFIO_GROUP_GET_STATUS, &group_status);
> +	VFIO_ASSERT_TRUE(group_status.flags & VFIO_GROUP_FLAGS_VIABLE);
> +
> +	ioctl_assert(device->group_fd, VFIO_GROUP_SET_CONTAINER, &device->container_fd);
> +}
> +
> +static void vfio_pci_iommu_setup(struct vfio_pci_device *device, unsigned long iommu_type)
> +{
> +	int ret;
> +
> +	ret = ioctl(device->container_fd, VFIO_CHECK_EXTENSION, iommu_type);
> +	VFIO_ASSERT_GT(ret, 0, "VFIO IOMMU type %lu not supported\n", iommu_type);
> +
> +	ioctl_assert(device->container_fd, VFIO_SET_IOMMU, (void *)iommu_type);
> +}
> +
> +static void vfio_pci_device_setup(struct vfio_pci_device *device, const char *bdf)
> +{
> +	int i;
> +
> +	device->fd = ioctl(device->group_fd, VFIO_GROUP_GET_DEVICE_FD, bdf);
> +	VFIO_ASSERT_GE(device->fd, 0);
> +
> +	device->info.argsz = sizeof(device->info);
> +	ioctl_assert(device->fd, VFIO_DEVICE_GET_INFO, &device->info);
> +
> +	vfio_pci_region_get(device, VFIO_PCI_CONFIG_REGION_INDEX, &device->config_space);
> +
> +	/* Sanity check VFIO does not advertise mmap for config space */
> +	VFIO_ASSERT_TRUE(!(device->config_space.flags & VFIO_REGION_INFO_FLAG_MMAP),
> +			 "PCI config space should not support mmap()\n");
> +
> +	for (i = 0; i < PCI_STD_NUM_BARS; i++) {
> +		struct vfio_pci_bar *bar = device->bars + i;
> +
> +		vfio_pci_region_get(device, i, &bar->info);
> +		if (bar->info.flags & VFIO_REGION_INFO_FLAG_MMAP)
> +			vfio_pci_bar_map(device, i);
> +	}
> +
> +	vfio_pci_irq_get(device, VFIO_PCI_MSI_IRQ_INDEX, &device->msi_info);
> +	vfio_pci_irq_get(device, VFIO_PCI_MSIX_IRQ_INDEX, &device->msix_info);
> +
> +	for (i = 0; i < ARRAY_SIZE(device->msi_eventfds); i++)
> +		device->msi_eventfds[i] = -1;
> +}
> +
> +struct vfio_pci_device *vfio_pci_device_init(const char *bdf, int iommu_type)
> +{
> +	struct vfio_pci_device *device;
> +
> +	device = calloc(1, sizeof(*device));
> +	VFIO_ASSERT_NOT_NULL(device);
> +
> +	vfio_pci_container_setup(device);
> +	vfio_pci_group_setup(device, bdf);
> +	vfio_pci_iommu_setup(device, iommu_type);
> +	vfio_pci_device_setup(device, bdf);
> +
> +	return device;
> +}
> +
> +void vfio_pci_device_cleanup(struct vfio_pci_device *device)
> +{
> +	int i;
> +
> +	vfio_pci_bar_unmap_all(device);
> +
> +	VFIO_ASSERT_EQ(close(device->fd), 0);
> +
> +	for (i = 0; i < ARRAY_SIZE(device->msi_eventfds); i++) {
> +		if (device->msi_eventfds[i] < 0)
> +			continue;
> +
> +		VFIO_ASSERT_EQ(close(device->msi_eventfds[i]), 0);
> +	}
> +
> +	VFIO_ASSERT_EQ(close(device->group_fd), 0);
> +	VFIO_ASSERT_EQ(close(device->container_fd), 0);
> +
> +	free(device);
> +}
> 

Re: [RFC PATCH 30/33] vfio: selftests: Add a script to help with running VFIO selftests

[David Matlack]

On Mon, Jun 2, 2025 at 4:08 PM Alex Williamson
<alex.williamson@redhat.com> wrote:
> On Fri, 23 May 2025 23:30:15 +0000
> > +function add_id() {
> > +     if echo $(echo ${1} | tr : ' ') > ${2}/new_id 2> /dev/null; then
> > +             echo "Added ${1} to ${2}"
> > +             return 0
> > +     fi
> > +
> > +     return 1
> > +}
>
> I'd suggest using the newer driver_override mechanism rather than
> new_id/remove_id.

Thanks for the tip, I'll include that in the next version.

> I appreciate the work here, I think this could be really useful.  As
> Jason notes a few times, I'd also like to see this automatically
> iterate through all the combinations of options.

Will do.

> Maybe we could also
> think about whether we can define some build environment options to
> specify devices to use and this could all be run automatically from
> 'make kselftest'.

That's an interesting idea. Maybe an environment variable that tests
will look for and use if they aren't provided with an explicit BDF
string on the command line?

    export VFIO_SELFTESTS_DEFAULT_BDF=XXXX:YY:ZZ.AA

Thanks for taking a look!

Re: [RFC PATCH 04/33] vfio: selftests: Test basic VFIO and IOMMUFD integration

[David Matlack]

On Mon, Jun 2, 2025 at 4:08 PM Alex Williamson
<alex.williamson@redhat.com> wrote:
>
> On Fri, 23 May 2025 23:29:49 +0000
> David Matlack <dmatlack@google.com> wrote:
> > +int main(int argc, char *argv[])
> > +{
> > +     char *bdf;
> > +
> > +     if (argc != 2) {
> > +             printf("Usage: %s bus:device:function\n", argv[0]);
>
> segment:bus:device.function?

Oops, yes. Will fix in the next version.

>
> > +             return 1;
>
> 1 or KSFT_FAIL?

Yeah KSFT_FAIL is probably the better option.

Thanks.

Re: [RFC PATCH 29/33] vfio: selftests: Make iommufd the default iommu_mode

[David Matlack]

On Mon, May 26, 2025 at 10:20 AM Jason Gunthorpe <jgg@nvidia.com> wrote:
>
> On Fri, May 23, 2025 at 11:30:14PM +0000, David Matlack wrote:
> > Now that VFIO selftests support iommufd, make it the default mode.
> > IOMMUFD is the successor to VFIO_TYPE1{,v2}_IOMMU and all new features
> > are being added there, so it's a slightly better fit as the default
> > mode.
>
> As before, the tests should run through all combinations on their own,
> use a fixture.

Ack, I'll rework the series so that tests automatically run
through all IOMMU modes.

> It would be much better to fix the kselftests so you could regex
> select the tests to run and use that as the commandline way to choose
> what test combination to use.

Re: [RFC PATCH 08/33] vfio: selftests: Validate 2M/1G HugeTLB are mapped as 2M/1G in IOMMU

[David Matlack]

On Sun, Jun 1, 2025 at 12:45 AM Greg Kroah-Hartman
<gregkh@linuxfoundation.org> wrote:
>
> On Sat, May 31, 2025 at 10:09:34PM +0300, Andy Shevchenko wrote:
> > On Fri, May 30, 2025 at 02:12:36PM -0700, David Matlack wrote:
> > > On Fri, May 23, 2025 at 4:30 PM David Matlack <dmatlack@google.com> wrote:
> >
> > ...
> >
> > > > +       if (access("/sys/kernel/debug/iommu/intel", F_OK))
> > > > +               return intel_iommu_mapping_get(bdf, iova, mapping);
> > > > +
> > >
> > > Oops, this should be !access(...).
> >
> > Hmm... Generally speaking that code is a hack. Is there any guarantee that
> > debugfs mount point is fixed to /sys/kernel/debug ?
>
> No, userspace can mount it anywhere :)
>
> But the kernel does create the mount point at /sys/kernel/debug/ by
> default, so hopefully it does end up there if it is present in the
> system, so you should be able to rely on it for testing.

I'll leave the hard-coded path in there for now then. If/when someone
wants to run these tests on a system where debugfs is mounted
somewhere else, we can change the code to find the debugfs mount point
instead. In the meantime, the test will still work on such a system,
it will just skip the mapping level assertions.

Re: [RFC PATCH 07/33] vfio: selftests: Use command line to set hugepage size for DMA mapping test

[David Matlack]

On 2025-05-30 02:25 PM, Jason Gunthorpe wrote:
> On Fri, May 30, 2025 at 09:50:22AM -0700, David Matlack wrote:
> > I'll explore doing this. For a single dimension this looks possible.
> > But for multiple dimensions (e.g. cross product of iommu_mode and
> > backing_src) I don't see a clear way to do it. But that's just after a
> > cursory look.
> 
> Explicitly list all the combinations with macros?
> 
> Enhance the userspace tests allow code to generate the
> variants? Kernel tests can do this:

I got a chance to play around with generating fixture variants today and
eneded up with this, which I think is pretty clean.

tools/testing/selftests/vfio/lib/include/vfio_util.h:

  #define ALL_IOMMU_MODES_VARIANT_ADD(...) \
  __IOMMU_MODE_VARIANT_ADD(vfio_type1_iommu, ##__VA_ARGS__); \
  __IOMMU_MODE_VARIANT_ADD(vfio_type1v2_iommu, ##__VA_ARGS__); \
  __IOMMU_MODE_VARIANT_ADD(iommufd_compat_type1, ##__VA_ARGS__); \
  __IOMMU_MODE_VARIANT_ADD(iommufd_compat_type1v2, ##__VA_ARGS__); \
  __IOMMU_MODE_VARIANT_ADD(iommufd, ##__VA_ARGS__)

tools/testing/selftests/vfio/vfio_dma_mapping_test.c:

  #define __IOMMU_MODE_VARIANT_ADD(_iommu_mode, _name, _size, _mmap_flags)	\
  FIXTURE_VARIANT_ADD(vfio_dma_mapping_test, _iommu_mode ## _name)		\
  {										\
  	.iommu_mode = #_iommu_mode,						\
  	.size = (_size),							\
  	.mmap_flags = MAP_ANONYMOUS | MAP_PRIVATE | (_mmap_flags),		\
  }

  ALL_IOMMU_MODES_VARIANT_ADD(anonymous, 0, 0);
  ALL_IOMMU_MODES_VARIANT_ADD(anonymous_hugetlb_2mb, SZ_2M, MAP_HUGETLB | MAP_HUGE_2MB);
  ALL_IOMMU_MODES_VARIANT_ADD(anonymous_hugetlb_1gb, SZ_1G, MAP_HUGETLB | MAP_HUGE_1GB);

  #undef __IOMMU_MODE_VARIANT_ADD

Let me know if you think this looks reasonable.

Re: [RFC PATCH 07/33] vfio: selftests: Use command line to set hugepage size for DMA mapping test

[Jason Gunthorpe]

On Fri, Jun 06, 2025 at 11:05:46PM +0000, David Matlack wrote:
> On 2025-05-30 02:25 PM, Jason Gunthorpe wrote:
> > On Fri, May 30, 2025 at 09:50:22AM -0700, David Matlack wrote:
> > > I'll explore doing this. For a single dimension this looks possible.
> > > But for multiple dimensions (e.g. cross product of iommu_mode and
> > > backing_src) I don't see a clear way to do it. But that's just after a
> > > cursory look.
> > 
> > Explicitly list all the combinations with macros?
> > 
> > Enhance the userspace tests allow code to generate the
> > variants? Kernel tests can do this:
> 
> I got a chance to play around with generating fixture variants today and
> eneded up with this, which I think is pretty clean.
> 
> tools/testing/selftests/vfio/lib/include/vfio_util.h:
> 
>   #define ALL_IOMMU_MODES_VARIANT_ADD(...) \
>   __IOMMU_MODE_VARIANT_ADD(vfio_type1_iommu, ##__VA_ARGS__); \
>   __IOMMU_MODE_VARIANT_ADD(vfio_type1v2_iommu, ##__VA_ARGS__); \
>   __IOMMU_MODE_VARIANT_ADD(iommufd_compat_type1, ##__VA_ARGS__); \
>   __IOMMU_MODE_VARIANT_ADD(iommufd_compat_type1v2, ##__VA_ARGS__); \
>   __IOMMU_MODE_VARIANT_ADD(iommufd, ##__VA_ARGS__)
> 
> tools/testing/selftests/vfio/vfio_dma_mapping_test.c:
> 
>   #define __IOMMU_MODE_VARIANT_ADD(_iommu_mode, _name, _size, _mmap_flags)	\
>   FIXTURE_VARIANT_ADD(vfio_dma_mapping_test, _iommu_mode ## _name)		\
>   {										\
>   	.iommu_mode = #_iommu_mode,						\
>   	.size = (_size),							\
>   	.mmap_flags = MAP_ANONYMOUS | MAP_PRIVATE | (_mmap_flags),		\
>   }
> 
>   ALL_IOMMU_MODES_VARIANT_ADD(anonymous, 0, 0);
>   ALL_IOMMU_MODES_VARIANT_ADD(anonymous_hugetlb_2mb, SZ_2M, MAP_HUGETLB | MAP_HUGE_2MB);
>   ALL_IOMMU_MODES_VARIANT_ADD(anonymous_hugetlb_1gb, SZ_1G, MAP_HUGETLB | MAP_HUGE_1GB);
> 
>   #undef __IOMMU_MODE_VARIANT_ADD
> 
> Let me know if you think this looks reasonable.

Seems reasonable enough to me, not worth inventing a programmatic way to
generate them..

Jason

Re: [PATCH RFC 17/33] vfio: selftests: Enable asserting MSI eventfds not firing

[Sairaj Kodilkar]

On 5/24/2025 5:00 AM, David Matlack wrote:
> Make it possible to assert that a given MSI eventfd did _not_ fire by
> adding a helper to mark an eventfd non-blocking. Demonstrate this in
> vfio_pci_device_test by asserting the MSI eventfd did not fire before
> vfio_pci_irq_trigger().
> 
> Signed-off-by: David Matlack <dmatlack@google.com>
> ---
>   tools/testing/selftests/vfio/lib/include/vfio_util.h | 12 ++++++++++++
>   tools/testing/selftests/vfio/vfio_pci_device_test.c  | 10 +++++++++-
>   2 files changed, 21 insertions(+), 1 deletion(-)
> 
> diff --git a/tools/testing/selftests/vfio/lib/include/vfio_util.h b/tools/testing/selftests/vfio/lib/include/vfio_util.h
> index ab96a6628f0e..2b96be07f182 100644
> --- a/tools/testing/selftests/vfio/lib/include/vfio_util.h
> +++ b/tools/testing/selftests/vfio/lib/include/vfio_util.h
> @@ -2,6 +2,7 @@
>   #ifndef SELFTESTS_VFIO_LIB_INCLUDE_VFIO_UTIL_H
>   #define SELFTESTS_VFIO_LIB_INCLUDE_VFIO_UTIL_H
>   
> +#include <fcntl.h>
>   #include <string.h>
>   #include <linux/vfio.h>
>   #include <linux/list.h>
> @@ -116,6 +117,17 @@ void vfio_pci_irq_enable(struct vfio_pci_device *device, u32 index,
>   void vfio_pci_irq_disable(struct vfio_pci_device *device, u32 index);
>   void vfio_pci_irq_trigger(struct vfio_pci_device *device, u32 index, u32 vector);
>   
> +static inline void fcntl_set_nonblock(int fd)
> +{
> +	int r;
> +
> +	r = fcntl(fd, F_GETFL, 0);

fcntl F_GETFL does not expect argument

> +	VFIO_ASSERT_NE(r, -1, "F_GETFL failed for fd %d\n", fd);

May be print errno as well  ?

> +
> +	r = fcntl(fd, F_SETFL, r | O_NONBLOCK);
> +	VFIO_ASSERT_NE(r, -1, "F_SETFL O_NONBLOCK failed for fd %d\n", fd);
> +}
> +

Thanks
Sairaj

Re: [PATCH RFC 17/33] vfio: selftests: Enable asserting MSI eventfds not firing

[David Matlack]

On Mon, Jun 30, 2025 at 5:24 AM Sairaj Kodilkar <sarunkod@amd.com> wrote:
>
> On 5/24/2025 5:00 AM, David Matlack wrote:
> > Make it possible to assert that a given MSI eventfd did _not_ fire by
> > adding a helper to mark an eventfd non-blocking. Demonstrate this in
> > vfio_pci_device_test by asserting the MSI eventfd did not fire before
> > vfio_pci_irq_trigger().
> >
> > Signed-off-by: David Matlack <dmatlack@google.com>
> > ---
> >   tools/testing/selftests/vfio/lib/include/vfio_util.h | 12 ++++++++++++
> >   tools/testing/selftests/vfio/vfio_pci_device_test.c  | 10 +++++++++-
> >   2 files changed, 21 insertions(+), 1 deletion(-)
> >
> > diff --git a/tools/testing/selftests/vfio/lib/include/vfio_util.h b/tools/testing/selftests/vfio/lib/include/vfio_util.h
> > index ab96a6628f0e..2b96be07f182 100644
> > --- a/tools/testing/selftests/vfio/lib/include/vfio_util.h
> > +++ b/tools/testing/selftests/vfio/lib/include/vfio_util.h
> > @@ -2,6 +2,7 @@
> >   #ifndef SELFTESTS_VFIO_LIB_INCLUDE_VFIO_UTIL_H
> >   #define SELFTESTS_VFIO_LIB_INCLUDE_VFIO_UTIL_H
> >
> > +#include <fcntl.h>
> >   #include <string.h>
> >   #include <linux/vfio.h>
> >   #include <linux/list.h>
> > @@ -116,6 +117,17 @@ void vfio_pci_irq_enable(struct vfio_pci_device *device, u32 index,
> >   void vfio_pci_irq_disable(struct vfio_pci_device *device, u32 index);
> >   void vfio_pci_irq_trigger(struct vfio_pci_device *device, u32 index, u32 vector);
> >
> > +static inline void fcntl_set_nonblock(int fd)
> > +{
> > +     int r;
> > +
> > +     r = fcntl(fd, F_GETFL, 0);
>
> fcntl F_GETFL does not expect argument

Good point, this could simply be:

  r = fcntl(fd, F_GETFL);

>
> > +     VFIO_ASSERT_NE(r, -1, "F_GETFL failed for fd %d\n", fd);
>
> May be print errno as well  ?

All of the VFIO_ASSERT*() macros print errno by default [1], since
it's relevant often enough.

The idea to print errno by default came from the KVM selftests [2]. In
a future series I'd like to find a way to share the assert code
between the KVM and VFIO selftests since there's a lot of overlap and
the KVM selftests also support useful things like backtraces on
assertion failures that I'd love to have in VFIO selftests.

[1] https://github.com/dmatlack/linux/blob/44c8e1e805698286e43cf2a471f540eee75e94a1/tools/testing/selftests/vfio/lib/include/vfio_util.h#L30
[2] https://github.com/dmatlack/linux/blob/44c8e1e805698286e43cf2a471f540eee75e94a1/tools/testing/selftests/kvm/lib/assert.c#L79