* [PATCH 0/2] qemu vfio-pci: VGA passthrough support
@ 2013-02-18 20:53 Alex Williamson
2013-02-18 20:53 ` [PATCH 1/2] qemu vfio-pci: Add support for VGA MMIO and I/O port access Alex Williamson
2013-02-18 20:53 ` [PATCH 2/2] qemu vfio-pci: Graphics device quirks Alex Williamson
0 siblings, 2 replies; 3+ messages in thread
From: Alex Williamson @ 2013-02-18 20:53 UTC (permalink / raw)
To: alex.williamson, qemu-devel; +Cc: kvm
I've pushed kernel side vfio vga support to my next branch and intend
to try to get those in for kernel v3.9. Those changes can be found
here: git://github.com/awilliam/linux-vfio.git (next)
This is the matching QEMU updates. Patch 1/2 adds basic VGA range
support. This can be used with the option "-vga none" to disable
emulated VGA and also requires vfio-pci device option "x-vga=on". As
noted by the x- prefix, this support is entirely experimental.
Expectations should range anywhere from working perfectly to crashing
the host.
Patch 2/2 contains numerous quirks for devices that I have onhand to
test, but hopefully covers a faily broad spectrum (at least for
ATI/AMD and Nvidia). These quirks trap various device specific
addresses and insert device virtual addresses in place of device
physical addresses. If we could identity map devices we could avoid
these, but that also puts numerous constraints on our configuration.
The vfio kernel VGA support makes use of the VGA arbiter to switch
between host VGA devices and I have been successful in running
multiple devices simultaneously. At this point only secondary
devices have been tested. The primary graphics likely needs extra
work to completely turn off host kernel access. Intel integrated
graphics have not been tested and are most certainly broken as
they require access to devices other than the VGA device alone.
Nvidia devices will work with nouveau in Linux guests, but do not
work with Nvidia drivers in Windows (probably Linux too). I think
we need to put these behind emulated PCIe root ports before they'll
work, but Q35 doesn't work well enough for that yet. ATI/AMD cards
seem to have the best hope of working with their Catalyst drivers.
For best results, use Q35, -vga none, x-vga=on, and x-bustype=2,
for example:
-M q35 -vga none -device vfio-pci,host=x:xx.x,x-vga=on,x-bustype=2
All the qemu patches necessary for this can be found in this branch:
git://github.com/awilliam/qemu-vfio.git (vfio-vga-v3)
This series will depend on a kernel header update which I'll send
patches for once the kernel patches are upstream. Thanks,
Alex
---
Alex Williamson (2):
qemu vfio-pci: Add support for VGA MMIO and I/O port access
qemu vfio-pci: Graphics device quirks
hw/vfio_pci.c | 651 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 649 insertions(+), 2 deletions(-)
^ permalink raw reply [flat|nested] 3+ messages in thread
* [PATCH 1/2] qemu vfio-pci: Add support for VGA MMIO and I/O port access
2013-02-18 20:53 [PATCH 0/2] qemu vfio-pci: VGA passthrough support Alex Williamson
@ 2013-02-18 20:53 ` Alex Williamson
2013-02-18 20:53 ` [PATCH 2/2] qemu vfio-pci: Graphics device quirks Alex Williamson
1 sibling, 0 replies; 3+ messages in thread
From: Alex Williamson @ 2013-02-18 20:53 UTC (permalink / raw)
To: alex.williamson, qemu-devel; +Cc: kvm
Most VGA cards need some kind of quirk to fully operate since they
hide backdoors to get to other registers outside of PCI config space
within the registers, but this provides the base infrastructure. If
we could identity map PCI resources for assigned devices we would need
a lot fewer quirks.
To enable this, use a kernel side vfio-pci driver that incorporates
VGA support (patches posted), and use the -vga none option and add the
x-vga=on option for the vfio-pci device. The "x-" denotes this as an
experimental feature. You may also need to use a cached copy of the
VGA BIOS for your device, passing it to vfio-pci using the romfile=
option.
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
hw/vfio_pci.c | 173 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 173 insertions(+)
diff --git a/hw/vfio_pci.c b/hw/vfio_pci.c
index 330a687..307cbc1 100644
--- a/hw/vfio_pci.c
+++ b/hw/vfio_pci.c
@@ -59,6 +59,18 @@ typedef struct VFIOBAR {
uint8_t nr; /* cache the BAR number for debug */
} VFIOBAR;
+typedef struct VFIOVGARegion {
+ MemoryRegion mem;
+ off_t offset;
+ int nr;
+} VFIOVGARegion;
+
+typedef struct VFIOVGA {
+ off_t fd_offset;
+ int fd;
+ VFIOVGARegion region[3];
+} VFIOVGA;
+
typedef struct VFIOINTx {
bool pending; /* interrupt pending */
bool kvm_accel; /* set when QEMU bypass through KVM enabled */
@@ -127,11 +139,16 @@ typedef struct VFIODevice {
int nr_vectors; /* Number of MSI/MSIX vectors currently in use */
int interrupt; /* Current interrupt type */
VFIOBAR bars[PCI_NUM_REGIONS - 1]; /* No ROM */
+ VFIOVGA vga; /* 0xa0000, 0x3b0, 0x3c0 */
PCIHostDeviceAddress host;
QLIST_ENTRY(VFIODevice) next;
struct VFIOGroup *group;
+ uint32_t features;
+#define VFIO_FEATURE_ENABLE_VGA_BIT 0
+#define VFIO_FEATURE_ENABLE_VGA (1 << VFIO_FEATURE_ENABLE_VGA_BIT)
uint8_t bustype;
bool reset_works;
+ bool has_vga;
} VFIODevice;
typedef struct VFIOGroup {
@@ -959,6 +976,89 @@ static const MemoryRegionOps vfio_bar_ops = {
.endianness = DEVICE_LITTLE_ENDIAN,
};
+static void vfio_vga_write(void *opaque, hwaddr addr,
+ uint64_t data, unsigned size)
+{
+ VFIOVGARegion *region = opaque;
+ VFIOVGA *vga = container_of(region, VFIOVGA, region[region->nr]);
+ union {
+ uint8_t byte;
+ uint16_t word;
+ uint32_t dword;
+ uint64_t qword;
+ } buf;
+ off_t offset = vga->fd_offset + region->offset + addr;
+
+ switch (size) {
+ case 1:
+ buf.byte = data;
+ break;
+ case 2:
+ buf.word = cpu_to_le16(data);
+ break;
+ case 4:
+ buf.dword = cpu_to_le32(data);
+ break;
+ default:
+ hw_error("vfio: unsupported write size, %d bytes\n", size);
+ break;
+ }
+
+ if (pwrite(vga->fd, &buf, size, offset) != size) {
+ error_report("%s(,0x%"HWADDR_PRIx", 0x%"PRIx64", %d) failed: %m",
+ __func__, region->offset + addr, data, size);
+ }
+
+ DPRINTF("%s(0x%"HWADDR_PRIx", 0x%"PRIx64", %d)\n",
+ __func__, region->offset + addr, data, size);
+}
+
+static uint64_t vfio_vga_read(void *opaque, hwaddr addr, unsigned size)
+{
+ VFIOVGARegion *region = opaque;
+ VFIOVGA *vga = container_of(region, VFIOVGA, region[region->nr]);
+ union {
+ uint8_t byte;
+ uint16_t word;
+ uint32_t dword;
+ uint64_t qword;
+ } buf;
+ uint64_t data = 0;
+ off_t offset = vga->fd_offset + region->offset + addr;
+
+ if (pread(vga->fd, &buf, size, offset) != size) {
+ error_report("%s(,0x%"HWADDR_PRIx", %d) failed: %m",
+ __func__, region->offset + addr, size);
+ return (uint64_t)-1;
+ }
+
+ switch (size) {
+ case 1:
+ data = buf.byte;
+ break;
+ case 2:
+ data = le16_to_cpu(buf.word);
+ break;
+ case 4:
+ data = le32_to_cpu(buf.dword);
+ break;
+ default:
+ hw_error("vfio: unsupported read size, %d bytes\n", size);
+ break;
+ }
+
+ DPRINTF("%s(0x%"HWADDR_PRIx", %d) = 0x%"PRIx64"\n",
+ __func__, region->offset + addr, size, data);
+
+ return data;
+}
+
+static const MemoryRegionOps vfio_vga_ops = {
+ .read = vfio_vga_read,
+ .write = vfio_vga_write,
+ .endianness = DEVICE_LITTLE_ENDIAN,
+};
+
/*
* PCI config space
*/
@@ -1479,6 +1579,27 @@ static void vfio_map_bars(VFIODevice *vdev)
for (i = 0; i < PCI_ROM_SLOT; i++) {
vfio_map_bar(vdev, i);
}
+
+ if (vdev->has_vga) {
+ memory_region_init_io(&vdev->vga.region[0].mem, &vfio_vga_ops,
+ &vdev->vga.region[0], "vfio-vga-mmio@0xa0000",
+ 0xbffff - 0xa0000 + 1);
+ memory_region_add_subregion_overlap(pci_address_space(&vdev->pdev),
+ 0xa0000, &vdev->vga.region[0].mem,
+ 1);
+
+ memory_region_init_io(&vdev->vga.region[1].mem, &vfio_vga_ops,
+ &vdev->vga.region[1], "vfio-vga-io@0x3b0",
+ 0x3bb - 0x3b0 + 1);
+ memory_region_add_subregion_overlap(pci_address_space_io(&vdev->pdev),
+ 0x3b0, &vdev->vga.region[1].mem, 1);
+
+ memory_region_init_io(&vdev->vga.region[2].mem, &vfio_vga_ops,
+ &vdev->vga.region[2], "vfio-vga-io@0x3c0",
+ 0x3df - 0x3c0 + 1);
+ memory_region_add_subregion_overlap(pci_address_space_io(&vdev->pdev),
+ 0x3c0, &vdev->vga.region[2].mem, 1);
+ }
}
static void vfio_unmap_bars(VFIODevice *vdev)
@@ -1488,6 +1609,20 @@ static void vfio_unmap_bars(VFIODevice *vdev)
for (i = 0; i < PCI_ROM_SLOT; i++) {
vfio_unmap_bar(vdev, i);
}
+
+ if (vdev->has_vga) {
+ memory_region_del_subregion(pci_address_space(&vdev->pdev),
+ &vdev->vga.region[0].mem);
+ memory_region_destroy(&vdev->vga.region[0].mem);
+
+ memory_region_del_subregion(pci_address_space_io(&vdev->pdev),
+ &vdev->vga.region[1].mem);
+ memory_region_destroy(&vdev->vga.region[1].mem);
+
+ memory_region_del_subregion(pci_address_space_io(&vdev->pdev),
+ &vdev->vga.region[2].mem);
+ memory_region_destroy(&vdev->vga.region[2].mem);
+ }
}
/*
@@ -2011,6 +2146,42 @@ static int vfio_get_device(VFIOGroup *group, const char *name, VFIODevice *vdev)
}
vdev->config_offset = reg_info.offset;
+ if ((vdev->features & VFIO_FEATURE_ENABLE_VGA) &&
+ dev_info.num_regions > VFIO_PCI_VGA_REGION_INDEX) {
+ struct vfio_region_info vga_info = {
+ .argsz = sizeof(vga_info),
+ .index = VFIO_PCI_VGA_REGION_INDEX,
+ };
+
+ ret = ioctl(vdev->fd, VFIO_DEVICE_GET_REGION_INFO, &vga_info);
+ if (ret) {
+ error_report(
+ "vfio: Device does not support requested feature x-vga");
+ goto error;
+ }
+
+ if (!(vga_info.flags & VFIO_REGION_INFO_FLAG_READ) ||
+ !(vga_info.flags & VFIO_REGION_INFO_FLAG_WRITE) ||
+ vga_info.size < 0xbffff + 1) {
+ error_report("vfio: Unexpected VGA info, flags 0x%lx, size 0x%lx",
+ (unsigned long)vga_info.flags,
+ (unsigned long)vga_info.size);
+ goto error;
+ }
+
+ vdev->vga.fd_offset = vga_info.offset;
+ vdev->vga.fd = vdev->fd;
+
+ vdev->vga.region[0].offset = 0xa0000;
+ vdev->vga.region[0].nr = 0;
+ vdev->vga.region[1].offset = 0x3b0;
+ vdev->vga.region[1].nr = 1;
+ vdev->vga.region[2].offset = 0x3c0;
+ vdev->vga.region[2].nr = 2;
+
+ vdev->has_vga = true;
+ }
+
error:
if (ret) {
QLIST_REMOVE(vdev, next);
@@ -2229,6 +2400,8 @@ static Property vfio_pci_dev_properties[] = {
DEFINE_PROP_UINT32("x-intx-mmap-timeout-ms", VFIODevice,
intx.mmap_timeout, 1100),
DEFINE_PROP_UINT8("x-bustype", VFIODevice, bustype, 0),
+ DEFINE_PROP_BIT("x-vga", VFIODevice, features,
+ VFIO_FEATURE_ENABLE_VGA_BIT, false),
/*
* TODO - support passed fds... is this necessary?
* DEFINE_PROP_STRING("vfiofd", VFIODevice, vfiofd_name),
^ permalink raw reply related [flat|nested] 3+ messages in thread
* [PATCH 2/2] qemu vfio-pci: Graphics device quirks
2013-02-18 20:53 [PATCH 0/2] qemu vfio-pci: VGA passthrough support Alex Williamson
2013-02-18 20:53 ` [PATCH 1/2] qemu vfio-pci: Add support for VGA MMIO and I/O port access Alex Williamson
@ 2013-02-18 20:53 ` Alex Williamson
1 sibling, 0 replies; 3+ messages in thread
From: Alex Williamson @ 2013-02-18 20:53 UTC (permalink / raw)
To: alex.williamson, qemu-devel; +Cc: kvm
Apparently graphics vendors need to come up with new ways to retrieve
PCI BAR addresses on every revision of their chip. These are the ones
that I've found on the following assortment of cards:
Advanced Micro Devices [AMD] nee ATI Cedar PRO [Radeon HD 5450/6350]
Advanced Micro Devices [AMD] nee ATI RV370 [Radeon X550]
NVIDIA Corporation G98 [GeForce 8400 GS]
NVIDIA Corporation G86 [Quadro NVS 290]
NVIDIA Corporation G72 [GeForce 7300 LE]
With these quirks, most ATI/AMD and Nvidia cards will post and work
with standard VGA drivers. ATI/AMD devices may also work with the
Catalyst driver (try x-bustype=2). I suspect Nvidia accelerated
drivers will require working root ports, they currently report error
code 43 under Windows.
It's relatively easy to figure out many of these quirks. First enable
DEBUG_UNASSIGNED in exec.c, then enable DEBUG_VFIO in hw/vfio_pci.c.
Log the output and can kill Qemu when Unassigned access errors start
to spew (ignore the ones at very low offsets). If the unassigned
access matches a range covered by the device (consult lspci or
/proc/iomem), then look back in the vfio-pci debug output for read
from the device that returned an address or partial address matching
the unassigned access. Then follow these examples for creating quirks
to trap that access and return the emulated BAR address.
None of these would be necessary if we could identity map devices.
Signed-off-by: Alex Williamson <alex.williamson@redhat.com>
---
hw/vfio_pci.c | 478 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 476 insertions(+), 2 deletions(-)
diff --git a/hw/vfio_pci.c b/hw/vfio_pci.c
index 307cbc1..769ab05 100644
--- a/hw/vfio_pci.c
+++ b/hw/vfio_pci.c
@@ -48,6 +48,16 @@
do { } while (0)
#endif
+struct VFIODevice;
+
+typedef struct VFIOQuirk {
+ MemoryRegion mem;
+ struct VFIODevice *vdev;
+ QLIST_ENTRY(VFIOQuirk) next;
+ uint32_t data;
+ uint32_t data2;
+} VFIOQuirk;
+
typedef struct VFIOBAR {
off_t fd_offset; /* offset of BAR within device fd */
int fd; /* device fd, allows us to pass VFIOBAR as opaque data */
@@ -57,12 +67,14 @@ typedef struct VFIOBAR {
size_t size;
uint32_t flags; /* VFIO region flags (rd/wr/mmap) */
uint8_t nr; /* cache the BAR number for debug */
+ QLIST_HEAD(, VFIOQuirk) quirks;
} VFIOBAR;
typedef struct VFIOVGARegion {
MemoryRegion mem;
off_t offset;
int nr;
+ QLIST_HEAD(, VFIOQuirk) quirks;
} VFIOVGARegion;
typedef struct VFIOVGA {
@@ -82,8 +94,6 @@ typedef struct VFIOINTx {
QEMUTimer *mmap_timer; /* enable mmaps after periods w/o interrupts */
} VFIOINTx;
-struct VFIODevice;
-
typedef struct VFIOMSIVector {
EventNotifier interrupt; /* eventfd triggered on interrupt */
struct VFIODevice *vdev; /* back pointer to device */
@@ -1060,6 +1070,460 @@ static const MemoryRegionOps vfio_vga_ops = {
};
/*
+ * Device specific quirks
+ */
+
+/*
+ * Device 1002:68f9 (Advanced Micro Devices [AMD] nee ATI Cedar PRO [Radeon
+ * HD 5450/6350]) reports the upper byte of the physical address of the
+ * I/O port BAR4 through VGA register 0x3c3. The BAR is 256 bytes, so the
+ * lower byte is known to be zero. Test for this quirk on all ATI/AMD
+ * devices.
+ */
+static uint64_t vfio_ati_3c3_quirk_read(void *opaque,
+ hwaddr addr, unsigned size)
+{
+ VFIOQuirk *quirk = opaque;
+ VFIODevice *vdev = quirk->vdev;
+ PCIDevice *pdev = &vdev->pdev;
+ uint64_t data = vfio_vga_read(&vdev->vga.region[2], addr + 0x3, size);
+
+ if (data == quirk->data) {
+ data = pci_get_byte(pdev->config + PCI_BASE_ADDRESS_4 + 1);
+ DPRINTF("%s(0x3c3, 1) = 0x%"PRIx64"\n", __func__, data);
+ }
+
+ return data;
+}
+
+static const MemoryRegionOps vfio_ati_3c3_quirk = {
+ .read = vfio_ati_3c3_quirk_read,
+ .endianness = DEVICE_LITTLE_ENDIAN,
+};
+
+/*
+ * Some devices seem to read 0xff from 0x3c3 during init so probing
+ * is unreliable. Save the physical address and return the virtual
+ * address any time the read matches the physical address.
+ */
+static void vfio_vga_probe_ati_3c3_quirk(VFIODevice *vdev)
+{
+ PCIDevice *pdev = &vdev->pdev;
+ off_t physoffset = vdev->config_offset + PCI_BASE_ADDRESS_4;
+ uint32_t physbar;
+ VFIOQuirk *quirk;
+
+ if (pci_get_word(pdev->config + PCI_VENDOR_ID) != 0x1002 ||
+ vdev->bars[4].size < 256) {
+ return;
+ }
+
+ /* Get I/O port BAR physical address */
+ if (pread(vdev->fd, &physbar, 4, physoffset) != 4) {
+ error_report("vfio: probe failed for ATI/AMD 0x3c3 quirk on device "
+ "%04x:%02x:%02x.%x", vdev->host.domain,
+ vdev->host.bus, vdev->host.slot, vdev->host.function);
+ return;
+ }
+
+ quirk = g_malloc0(sizeof(*quirk));
+ quirk->vdev = vdev;
+ quirk->data = (physbar >> 8) & 0xff;
+
+ memory_region_init_io(&quirk->mem, &vfio_ati_3c3_quirk, quirk,
+ "vfio-ati-3c3-quirk", 1);
+ memory_region_add_subregion(&vdev->vga.region[2].mem, 3, &quirk->mem);
+
+ QLIST_INSERT_HEAD(&vdev->vga.region[2].quirks, quirk, next);
+
+ fprintf(stderr, "vfio: Enabled ATI/AMD quirk 0x3c3 for device "
+ "%04x:%02x:%02x.%x\n", vdev->host.domain,
+ vdev->host.bus, vdev->host.slot, vdev->host.function);
+}
+
+/*
+ * Device 10de:01d1 (NVIDIA Corporation G72 [GeForce 7300 LE]) reads the
+ * BAR1 physical address using a read from VGA register 0x3d0. A specific
+ * series of writes to 0x3d0 & 0x3d4 precede this access. When the conditions
+ * are met, insert the virtual BAR1 address in place of the physical address.
+ * Unfortunately we can't seem to probe for this at init time as we don't
+ * know exactly how to generate this return value. The writes we track alone
+ * generate 0xffffffff at init time.
+ */
+static uint64_t vfio_nvidia_3d0_quirk_read(void *opaque,
+ hwaddr addr, unsigned size)
+{
+ VFIOQuirk *quirk = opaque;
+ VFIODevice *vdev = quirk->vdev;
+ PCIDevice *pdev = &vdev->pdev;
+ uint64_t data = vfio_vga_read(&vdev->vga.region[2], addr + 0x10, size);
+
+ if (quirk->data == 3 && size == 4 && addr == 0 && data == quirk->data2) {
+ data = pci_get_long(pdev->config + PCI_BASE_ADDRESS_1);
+ DPRINTF("%s(0x3d0, 4) = 0x%"PRIx64"\n", __func__, data);
+ }
+
+ quirk->data = 0;
+
+ return data;
+}
+
+static void vfio_nvidia_3d0_quirk_write(void *opaque, hwaddr addr,
+ uint64_t data, unsigned size)
+{
+ VFIOQuirk *quirk = opaque;
+ VFIODevice *vdev = quirk->vdev;
+
+ vfio_vga_write(&vdev->vga.region[2], addr + 0x10, data, size);
+
+ switch (quirk->data) {
+ case 0:
+ quirk->data = (addr == 4 && data == 0x338 && size == 2) ? 1 : 0;
+ break;
+ case 1:
+ quirk->data = (addr == 0 && data == 0x1814 && size == 4) ? 2 : 0;
+ break;
+ case 2:
+ quirk->data = (addr == 4 && data == 0x538 && size == 2) ? 3 : 0;
+ break;
+ default:
+ quirk->data = 0;
+ }
+}
+
+static const MemoryRegionOps vfio_nvidia_3d0_quirk = {
+ .read = vfio_nvidia_3d0_quirk_read,
+ .write = vfio_nvidia_3d0_quirk_write,
+ .endianness = DEVICE_LITTLE_ENDIAN,
+};
+
+static void vfio_vga_probe_nvidia_3d0_quirk(VFIODevice *vdev)
+{
+ PCIDevice *pdev = &vdev->pdev;
+ off_t physoffset = vdev->config_offset + PCI_BASE_ADDRESS_1;
+ uint32_t physbar;
+ VFIOQuirk *quirk;
+
+ if (pci_get_word(pdev->config + PCI_VENDOR_ID) != 0x10de ||
+ !vdev->bars[1].size) {
+ return;
+ }
+
+ if (pread(vdev->fd, &physbar, 4, physoffset) != 4) {
+ error_report("vfio: probe failed for NVIDIA 0x3d0 quirk on device "
+ "%04x:%02x:%02x.%x", vdev->host.domain,
+ vdev->host.bus, vdev->host.slot, vdev->host.function);
+ return;
+ }
+
+ quirk = g_malloc0(sizeof(*quirk));
+ quirk->vdev = vdev;
+ quirk->data2 = physbar;
+
+ memory_region_init_io(&quirk->mem, &vfio_nvidia_3d0_quirk, quirk,
+ "vfio-nvidia-3d0-quirk", 6);
+ memory_region_add_subregion(&vdev->vga.region[2].mem, 0x10, &quirk->mem);
+
+ QLIST_INSERT_HEAD(&vdev->vga.region[2].quirks, quirk, next);
+
+ fprintf(stderr, "vfio: Enabled NVIDIA quirk 0x3d0 for device "
+ "%04x:%02x:%02x.%x\n", vdev->host.domain,
+ vdev->host.bus, vdev->host.slot, vdev->host.function);
+}
+
+/*
+ * Device 1002:68f9 (Advanced Micro Devices [AMD] nee ATI Cedar PRO [Radeon
+ * HD 5450/6350]) reports the physical address of MMIO BAR0 through a
+ * write/read operation on I/O port BAR4. When uint32_t 0x4010 is written
+ * to offset 0x0, the subsequent read from offset 0x4 returns the contents
+ * of BAR0. Test for this quirk on all ATI/AMD devices.
+ */
+static uint64_t vfio_ati_4010_quirk_read(void *opaque,
+ hwaddr addr, unsigned size)
+{
+ VFIOQuirk *quirk = opaque;
+ VFIODevice *vdev = quirk->vdev;
+ PCIDevice *pdev = &vdev->pdev;
+ uint64_t data = vfio_bar_read(&vdev->bars[4], addr, size);
+
+ if (addr == 4 && size == 4 && quirk->data) {
+ data = pci_get_long(pdev->config + PCI_BASE_ADDRESS_0);
+ DPRINTF("%s(BAR4+0x4) = 0x%"PRIx64"\n", __func__, data);
+ }
+
+ quirk->data = 0;
+
+ return data;
+}
+
+static void vfio_ati_4010_quirk_write(void *opaque, hwaddr addr,
+ uint64_t data, unsigned size)
+{
+ VFIOQuirk *quirk = opaque;
+ VFIODevice *vdev = quirk->vdev;
+
+ vfio_bar_write(&vdev->bars[4], addr, data, size);
+
+ quirk->data = (addr == 0 && size == 4 && data == 0x4010) ? 1 : 0;
+}
+
+static const MemoryRegionOps vfio_ati_4010_quirk = {
+ .read = vfio_ati_4010_quirk_read,
+ .write = vfio_ati_4010_quirk_write,
+ .endianness = DEVICE_LITTLE_ENDIAN,
+};
+
+static void vfio_probe_ati_4010_quirk(VFIODevice *vdev, int nr)
+{
+ PCIDevice *pdev = &vdev->pdev;
+ off_t physoffset = vdev->config_offset + PCI_BASE_ADDRESS_0;
+ uint32_t physbar0;
+ uint64_t data;
+ VFIOQuirk *quirk;
+
+ if (!vdev->has_vga ||
+ pci_get_word(pdev->config + PCI_VENDOR_ID) != 0x1002 ||
+ nr != 4 || !vdev->bars[0].size) {
+ return;
+ }
+
+ /* Get I/O port BAR physical address */
+ if (pread(vdev->fd, &physbar0, 4, physoffset) != 4) {
+ error_report("vfio: probe failed for ATI/AMD 0x4010 quirk on device "
+ "%04x:%02x:%02x.%x", vdev->host.domain,
+ vdev->host.bus, vdev->host.slot, vdev->host.function);
+ return;
+ }
+
+ /* Write 0x4010 to I/O port BAR offset 0 */
+ vfio_bar_write(&vdev->bars[4], 0, 0x4010, 4);
+ /* Read back result */
+ data = vfio_bar_read(&vdev->bars[4], 4, 4);
+
+ /* If the register matches the physical address of BAR0, we need a quirk */
+ if (data != physbar0) {
+ return;
+ }
+
+ quirk = g_malloc0(sizeof(*quirk));
+ quirk->vdev = vdev;
+
+ memory_region_init_io(&quirk->mem, &vfio_ati_4010_quirk, quirk,
+ "vfio-ati-4010-quirk", 8);
+ memory_region_add_subregion_overlap(&vdev->bars[nr].mem, 0, &quirk->mem, 1);
+
+ QLIST_INSERT_HEAD(&vdev->bars[nr].quirks, quirk, next);
+
+ fprintf(stderr, "vfio: Enabled ATI/AMD quirk 0x4010 for device "
+ "%04x:%02x:%02x.%x\n", vdev->host.domain,
+ vdev->host.bus, vdev->host.slot, vdev->host.function);
+}
+
+/*
+ * Device 1002:5b63 (Advanced Micro Devices [AMD] nee ATI RV370 [Radeon X550])
+ * retrieves the upper half of the MMIO BAR0 physical address by writing
+ * 0xf10 to I/O port BAR1 offset 0 and reading the result from offset 6.
+ */
+static uint64_t vfio_ati_f10_quirk_read(void *opaque,
+ hwaddr addr, unsigned size)
+{
+ VFIOQuirk *quirk = opaque;
+ VFIODevice *vdev = quirk->vdev;
+ PCIDevice *pdev = &vdev->pdev;
+ uint64_t data = vfio_bar_read(&vdev->bars[1], addr, size);
+
+ if (addr == 6 && size == 2 && quirk->data) {
+ data = pci_get_word(pdev->config + PCI_BASE_ADDRESS_0 + 2);
+ DPRINTF("%s(BAR1+0x6) = 0x%"PRIx64"\n", __func__, data);
+ }
+
+ quirk->data = 0;
+
+ return data;
+}
+
+static void vfio_ati_f10_quirk_write(void *opaque, hwaddr addr,
+ uint64_t data, unsigned size)
+{
+ VFIOQuirk *quirk = opaque;
+ VFIODevice *vdev = quirk->vdev;
+
+ vfio_bar_write(&vdev->bars[1], addr, data, size);
+
+ quirk->data = (addr == 0 && size == 4 && data == 0xf10) ? 1 : 0;
+}
+
+static const MemoryRegionOps vfio_ati_f10_quirk = {
+ .read = vfio_ati_f10_quirk_read,
+ .write = vfio_ati_f10_quirk_write,
+ .endianness = DEVICE_LITTLE_ENDIAN,
+};
+
+static void vfio_probe_ati_f10_quirk(VFIODevice *vdev, int nr)
+{
+ PCIDevice *pdev = &vdev->pdev;
+ off_t physoffset = vdev->config_offset + PCI_BASE_ADDRESS_0;
+ uint32_t physbar0;
+ uint64_t data;
+ VFIOQuirk *quirk;
+
+ if (!vdev->has_vga ||
+ pci_get_word(pdev->config + PCI_VENDOR_ID) != 0x1002 ||
+ nr != 1 || !vdev->bars[0].size) {
+ return;
+ }
+
+ /* Get I/O port BAR physical address */
+ if (pread(vdev->fd, &physbar0, 4, physoffset) != 4) {
+ error_report("vfio: probe failed for ATI/AMD 0xf10 quirk on device "
+ "%04x:%02x:%02x.%x", vdev->host.domain,
+ vdev->host.bus, vdev->host.slot, vdev->host.function);
+ return;
+ }
+
+ vfio_bar_write(&vdev->bars[1], 0, 0xf10, 4);
+ data = vfio_bar_read(&vdev->bars[1], 0x6, 2);
+
+ /* If the register matches the physical address of BAR0, we need a quirk */
+ if (data != (le32_to_cpu(physbar0) >> 16)) {
+ return;
+ }
+
+ quirk = g_malloc0(sizeof(*quirk));
+ quirk->vdev = vdev;
+
+ memory_region_init_io(&quirk->mem, &vfio_ati_f10_quirk, quirk,
+ "vfio-ati-f10-quirk", 8);
+ memory_region_add_subregion_overlap(&vdev->bars[nr].mem, 0, &quirk->mem, 1);
+
+ QLIST_INSERT_HEAD(&vdev->bars[nr].quirks, quirk, next);
+
+ fprintf(stderr, "vfio: Enabled ATI/AMD quirk 0xf10 for device "
+ "%04x:%02x:%02x.%x\n", vdev->host.domain,
+ vdev->host.bus, vdev->host.slot, vdev->host.function);
+}
+
+/*
+ * Device 10de:06e4 (NVIDIA Corporation G98 [GeForce 8400 GS]) writes 0x8801c
+ * to 0ffset 0x8 of it's I/O port BAR5 and reads back the MMIO BAR3 base
+ * address for offset 0xc. Trap this and return the virtual BAR3 address.
+ * This quirk doesn't probe reliably, so compare runtime.
+ */
+static uint64_t vfio_nvidia_8801c_quirk_read(void *opaque,
+ hwaddr addr, unsigned size)
+{
+ VFIOQuirk *quirk = opaque;
+ VFIODevice *vdev = quirk->vdev;
+ PCIDevice *pdev = &vdev->pdev;
+ uint64_t data = vfio_bar_read(&vdev->bars[5], addr + 0x8, size);
+
+ if (addr == 4 && size == 4 && quirk->data && data == quirk->data2) {
+ data = pci_get_long(pdev->config + PCI_BASE_ADDRESS_3);
+ DPRINTF("%s(BAR5+0xc) = 0x%"PRIx64"\n", __func__, data);
+ }
+
+ quirk->data = 0;
+
+ return data;
+}
+
+static void vfio_nvidia_8801c_quirk_write(void *opaque, hwaddr addr,
+ uint64_t data, unsigned size)
+{
+ VFIOQuirk *quirk = opaque;
+ VFIODevice *vdev = quirk->vdev;
+
+ vfio_bar_write(&vdev->bars[5], addr + 0x8, data, size);
+
+ quirk->data = (addr == 0 && size == 4 && data == 0x8801c) ? 1 : 0;
+}
+
+static const MemoryRegionOps vfio_nvidia_8801c_quirk = {
+ .read = vfio_nvidia_8801c_quirk_read,
+ .write = vfio_nvidia_8801c_quirk_write,
+ .endianness = DEVICE_LITTLE_ENDIAN,
+};
+
+static void vfio_probe_nvidia_8801c_quirk(VFIODevice *vdev, int nr)
+{
+ PCIDevice *pdev = &vdev->pdev;
+ off_t physoffset = vdev->config_offset + PCI_BASE_ADDRESS_3;
+ uint32_t physbar3;
+ VFIOQuirk *quirk;
+
+ if (!vdev->has_vga ||
+ pci_get_word(pdev->config + PCI_VENDOR_ID) != 0x10de ||
+ nr != 5 || !vdev->bars[3].size) {
+ return;
+ }
+
+ /* Get I/O port BAR physical address */
+ if (pread(vdev->fd, &physbar3, 4, physoffset) != 4) {
+ error_report("vfio: probe failed for NVIDIA 0x8801c quirk on device "
+ "%04x:%02x:%02x.%x", vdev->host.domain,
+ vdev->host.bus, vdev->host.slot, vdev->host.function);
+ return;
+ }
+
+ quirk = g_malloc0(sizeof(*quirk));
+ quirk->vdev = vdev;
+ quirk->data2 = physbar3;
+
+ memory_region_init_io(&quirk->mem, &vfio_nvidia_8801c_quirk, quirk,
+ "vfio-nvidia-8801c-quirk", 8);
+ memory_region_add_subregion_overlap(&vdev->bars[nr].mem, 0x8,
+ &quirk->mem, 1);
+
+ QLIST_INSERT_HEAD(&vdev->bars[nr].quirks, quirk, next);
+
+ fprintf(stderr, "vfio: Enabled NVIDIA quirk 0x8801c for device "
+ "%04x:%02x:%02x.%x\n", vdev->host.domain,
+ vdev->host.bus, vdev->host.slot, vdev->host.function);
+}
+
+/*
+ * Common quirk probe entry points.
+ */
+static void vfio_vga_quirk_setup(VFIODevice *vdev)
+{
+ vfio_vga_probe_ati_3c3_quirk(vdev);
+ vfio_vga_probe_nvidia_3d0_quirk(vdev);
+}
+
+static void vfio_vga_quirk_teardown(VFIODevice *vdev)
+{
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(vdev->vga.region); i++) {
+ while (!QLIST_EMPTY(&vdev->vga.region[i].quirks)) {
+ VFIOQuirk *quirk = QLIST_FIRST(&vdev->vga.region[i].quirks);
+ memory_region_del_subregion(&vdev->vga.region[i].mem, &quirk->mem);
+ QLIST_REMOVE(quirk, next);
+ g_free(quirk);
+ }
+ }
+}
+
+static void vfio_bar_quirk_setup(VFIODevice *vdev, int nr)
+{
+ vfio_probe_ati_4010_quirk(vdev, nr);
+ vfio_probe_nvidia_8801c_quirk(vdev, nr);
+ vfio_probe_ati_f10_quirk(vdev, nr);
+}
+
+static void vfio_bar_quirk_teardown(VFIODevice *vdev, int nr)
+{
+ VFIOBAR *bar = &vdev->bars[nr];
+
+ while (!QLIST_EMPTY(&bar->quirks)) {
+ VFIOQuirk *quirk = QLIST_FIRST(&bar->quirks);
+ memory_region_del_subregion(&bar->mem, &quirk->mem);
+ QLIST_REMOVE(quirk, next);
+ g_free(quirk);
+ }
+}
+
+/*
* PCI config space
*/
static uint32_t vfio_pci_read_config(PCIDevice *pdev, uint32_t addr, int len)
@@ -1460,6 +1924,8 @@ static void vfio_unmap_bar(VFIODevice *vdev, int nr)
return;
}
+ vfio_bar_quirk_teardown(vdev, nr);
+
memory_region_del_subregion(&bar->mem, &bar->mmap_mem);
munmap(bar->mmap, memory_region_size(&bar->mmap_mem));
@@ -1570,6 +2036,8 @@ static void vfio_map_bar(VFIODevice *vdev, int nr)
error_report("%s unsupported. Performance may be slow", name);
}
}
+
+ vfio_bar_quirk_setup(vdev, nr);
}
static void vfio_map_bars(VFIODevice *vdev)
@@ -1599,6 +2067,7 @@ static void vfio_map_bars(VFIODevice *vdev)
0x3df - 0x3c0 + 1);
memory_region_add_subregion_overlap(pci_address_space_io(&vdev->pdev),
0x3c0, &vdev->vga.region[2].mem, 1);
+ vfio_vga_quirk_setup(vdev);
}
}
@@ -1611,6 +2080,7 @@ static void vfio_unmap_bars(VFIODevice *vdev)
}
if (vdev->has_vga) {
+ vfio_vga_quirk_teardown(vdev);
memory_region_del_subregion(pci_address_space(&vdev->pdev),
&vdev->vga.region[0].mem);
memory_region_destroy(&vdev->vga.region[0].mem);
@@ -2109,6 +2579,7 @@ static int vfio_get_device(VFIOGroup *group, const char *name, VFIODevice *vdev)
vdev->bars[i].fd_offset = reg_info.offset;
vdev->bars[i].fd = vdev->fd;
vdev->bars[i].nr = i;
+ QLIST_INIT(&vdev->bars[i].quirks);
}
reg_info.index = VFIO_PCI_ROM_REGION_INDEX;
@@ -2174,10 +2645,13 @@ static int vfio_get_device(VFIOGroup *group, const char *name, VFIODevice *vdev)
vdev->vga.region[0].offset = 0xa0000;
vdev->vga.region[0].nr = 0;
+ QLIST_INIT(&vdev->vga.region[0].quirks);
vdev->vga.region[1].offset = 0x3b0;
vdev->vga.region[1].nr = 1;
+ QLIST_INIT(&vdev->vga.region[1].quirks);
vdev->vga.region[2].offset = 0x3c0;
vdev->vga.region[2].nr = 2;
+ QLIST_INIT(&vdev->vga.region[2].quirks);
vdev->has_vga = true;
}
^ permalink raw reply related [flat|nested] 3+ messages in thread
end of thread, other threads:[~2013-02-18 20:53 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2013-02-18 20:53 [PATCH 0/2] qemu vfio-pci: VGA passthrough support Alex Williamson
2013-02-18 20:53 ` [PATCH 1/2] qemu vfio-pci: Add support for VGA MMIO and I/O port access Alex Williamson
2013-02-18 20:53 ` [PATCH 2/2] qemu vfio-pci: Graphics device quirks Alex Williamson
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox