Re: [Qemu-devel] VFIO-VGA Issue

[Alex Williamson <alex.williamson@redhat.com>]

On Thu, 2013-04-11 at 13:59 -0400, deniv@lavabit.com wrote:
> > On Wed, 2013-04-10 at 16:32 -0400, deniv@lavabit.com wrote:
> >> >> However, turning gfx_passthru off did
> >> >> the trick. Win7 started loading with cirrus and switched to HD7750
> >> >> halfway
> >> >> through boot proccess. I didn't do any testing just let Windows
> >> >> calculate
> >> >> its score. The result was 7.4 and Aero was working.
> >> >
> >> > You should be able to do this with vfio too, use -vga cirrus and don't
> >> > use the x-vga option on pci-assign.  The x-vga enables legacy VGA
> >> > support for boot and primary console, as a secondary head normal PCI
> >> > device assignment should be sufficient.
> >> >
> >> Oh, how I wish it was true! Trying to load with cirrus and vfio-pci
> >> results in BSOD:
> >> Attemp to reset the display driver and recover from timeout failed.
> >
> > Is this a fresh windows install?  Windows doesn't like change and will
> > BSOD pretty easily when attaching graphics to an existing image.
> >
> >> Trying the old pci-assign with kvm results in non-working GFX. Windows
> >> shows code 10 and sometimes code 43 for the card.
> >
> > What happens if you don't use a q35 machine?  Windows is very particular
> > about the PCIe type of a device and will often show Code 10 if it
> > doesn't have a type compatible with a root complex.  Alternatively you
> > can use the q35 config in the docs directory with the -readconfig option
> > and the bus= option on the pci-assign device to place the graphics
> > behind a root port.
> >
> 
> After many attempts I have VGA passthrough working. Each test has been
> performed in a clean Win7 install with kvm enabled. Installation was done
> with i440fx machine and changed to q35 after virtio drivers were
> installed. I did that because there's no AHCI driver for q35 yet and win7
> doesn't have drivers for lsi scsi.
> 
> 1. q35/pc, vga=none, vfio-pci, x-vga=on. Nothing on the VM's screen, debug
> output can be seen in the previous mails.
> 2. q35/pc, vga=cirrus, vfio-pci, x-vga=on. Screen corruption on host
> (correction to the previous mails: corruption also happens even in kms
> console). Nothing on the VM's screen.
> 3. q35/pc, vga=cirrus, vfio-pci, no x-vga. BSOD: Attempt to reset the
> display driver and recover from timeout failed.
> 4. q35, vga=cirrus, pci-assign. Windows boots, the graphic card doesn't
> start: Code 10.
> 5. pc, vga=cirrus, pci-assign. IT'S ALIVE! Ironically, this is the oldest
> way to assign pci devices in kvm. Why I could make it work previously?
> Silly me!

What does your /sys/kernel/iommu_groups look like for the group
containing your VGA device?  I'm curious if it includes the PCIe root
port and whether you're attaching those to vfio-pci or leaving them
bound to pcieport.  If the former, that may contribute to why you're
having problems with vfio-pci.

> After some testing I can say that the card can tolerate guest restarts,
> but it also can freeze the host in process. I got such a freeze (after a
> clean shutdown) once in about 10 restarts. On the other hand, killing the
> guest during a heavy 3d benchmark and starting the guest again was fine.
> Also, the card fails to clock up after a reboot, which causes poor
> performance in heavy loads. Thanks to Xen wiki I found that they "fix"
> this by ejecting the graphic card from Windows' panel. Oh, and the last
> thing, I once had code 43 after a guest reboot, but rebooting the guest
> again fixed it.
> 
> All in all, I'm quite happy now. Just wish that host freeze I had won't
> happen again. Is it a bug in qemu, the catalyst driver, or VT-d?
> Hopefully, q35 and vfio-pci will work some day. I wonder why q35 +
> pci-assign doesn't work,

This is likely because pci-assign doesn't mangle the PCI express type to
something Windows finds acceptable on a root complex.

>  and what's the problem with q35/pc + vfio-pci (no
> x-vga). If vfio-pci is the direct replacement for pci-assign it must be a
> bug in vfio, huh?

It's either a bug in vfio or the above configuration issue with the
pcieport.  I'd appreciate if you could advise how you're configuring the
group for use.  Thanks,

Alex