Re: [Qemu-devel] VFIO-VGA Issue

[deniv <deniv@lavabit.com>]

Sorry for the long delay.

Alex Williamson:
> On Thu, 2013-04-11 at 13:59 -0400, deniv@lavabit.com wrote:
>>> On Wed, 2013-04-10 at 16:32 -0400, deniv@lavabit.com wrote:
>>>>>> However, turning gfx_passthru off did
>>>>>> the trick. Win7 started loading with cirrus and switched to HD7750
>>>>>> halfway
>>>>>> through boot proccess. I didn't do any testing just let Windows
>>>>>> calculate
>>>>>> its score. The result was 7.4 and Aero was working.
>>>>>
>>>>> You should be able to do this with vfio too, use -vga cirrus and don't
>>>>> use the x-vga option on pci-assign.  The x-vga enables legacy VGA
>>>>> support for boot and primary console, as a secondary head normal PCI
>>>>> device assignment should be sufficient.
>>>>>
>>>> Oh, how I wish it was true! Trying to load with cirrus and vfio-pci
>>>> results in BSOD:
>>>> Attemp to reset the display driver and recover from timeout failed.
>>>
>>> Is this a fresh windows install?  Windows doesn't like change and will
>>> BSOD pretty easily when attaching graphics to an existing image.
>>>
>>>> Trying the old pci-assign with kvm results in non-working GFX. Windows
>>>> shows code 10 and sometimes code 43 for the card.
>>>
>>> What happens if you don't use a q35 machine?  Windows is very particular
>>> about the PCIe type of a device and will often show Code 10 if it
>>> doesn't have a type compatible with a root complex.  Alternatively you
>>> can use the q35 config in the docs directory with the -readconfig option
>>> and the bus= option on the pci-assign device to place the graphics
>>> behind a root port.
>>>
>>
>> After many attempts I have VGA passthrough working. Each test has been
>> performed in a clean Win7 install with kvm enabled. Installation was done
>> with i440fx machine and changed to q35 after virtio drivers were
>> installed. I did that because there's no AHCI driver for q35 yet and win7
>> doesn't have drivers for lsi scsi.
>>
>> 1. q35/pc, vga=none, vfio-pci, x-vga=on. Nothing on the VM's screen, debug
>> output can be seen in the previous mails.
>> 2. q35/pc, vga=cirrus, vfio-pci, x-vga=on. Screen corruption on host
>> (correction to the previous mails: corruption also happens even in kms
>> console). Nothing on the VM's screen.
>> 3. q35/pc, vga=cirrus, vfio-pci, no x-vga. BSOD: Attempt to reset the
>> display driver and recover from timeout failed.
>> 4. q35, vga=cirrus, pci-assign. Windows boots, the graphic card doesn't
>> start: Code 10.
>> 5. pc, vga=cirrus, pci-assign. IT'S ALIVE! Ironically, this is the oldest
>> way to assign pci devices in kvm. Why I could make it work previously?
>> Silly me!
> 
> What does your /sys/kernel/iommu_groups look like for the group
> containing your VGA device?  I'm curious if it includes the PCIe root
> port and whether you're attaching those to vfio-pci or leaving them
> bound to pcieport.  If the former, that may contribute to why you're
> having problems with vfio-pci.
Yes, the group containing the VGA device includes PCIe root port. No, I
did not attach it to vfio-pci. I tried this right now, it didn't make
any difference.

For what it's worth, I also noticed errors in dmesg output when Windows
BSODs (q35, vga=cirrus, vfio-pci, no x-vga). There are about ten
thousand lines of
---
dmar: DRHD: handling fault status reg 3
dmar: DMAR:[DMA Read] Request device [01:00.0] fault addr 1200b6000
DMAR:[fault reason 06] PTE Read access is not set
---
Fault addresses start at 11fff6000 (always the same) and go to about
1201b3000 (varies on each start).

Those read faults are followed a bunch of
---
dmar: DRHD: handling fault status reg 3
dmar: DMAR:[DMA Write] Request device [01:00.0] fault addr 11ffed000
DMAR:[fault reason 05] PTE Write access is not set
---
Fault addresses 11fef3000-11fff0000 (the last address varies).