Re: [PATCH 0/3] VFIO-based PCI device assignment for QEMU 1.2

[Andreas Hartmann <andihartmann@01019freenet.de>]

Alex Williamson schrieb:
> On Mon, 2012-08-13 at 17:48 +0200, Andreas Hartmann wrote:
>> Alex Williamson wrote:
>>> On Mon, 2012-08-13 at 08:27 -0500, Anthony Liguori wrote:
>>>> Alex Williamson <alex.williamson@redhat.com> writes:
>>>>
>>>>> VFIO kernel support was just merged into Linux, so I'd like to
>>>>> formally propose inclusion of the QEMU vfio-pci driver for
>>>>> QEMU 1.2.  Included here is support for x86 PCI device assignment.
>>>>> PCI INTx is not yet enabled, but devices making use of either MSI
>>>>> or MSI-X work.  The level irqfd and eoifd support I've proposed
>>>>> for KVM enable an accelerated patch for this through KVM.  I'd
>>>>> like to get this base driver in first and enable the remaining
>>>>> support in-tree.
>>>>>
>>>>> I've split this version up a little from the RFC to make it a bit
>>>>> easier to review.  Review comments from Blue Swirl and Avi are
>>>>> already incorporated, including Avi's requests to simplify both
>>>>> the PCI BAR mapping and unmapping paths.
>>>>
>>>> Hi Alex,
>>>>
>>>> Thanks for pushing this forward!  Hopefully this will finally kill
>>>> off qemu-kvm.git for good.
>>>>
>>>> I think this series is going to have to wait for 1.3 to open up.  We
>>>> have a very short release window for this release and I'd feel a lot
>>>> more comfortable having such a significant feature spend some time in
>>>> the development cycle getting testing/review.
>>>>
>>>> I'd like to see a few Reviewed-by's too for this series before it
>>>> goes in.  I expect they won't be hard to get but I also expect it
>>>> will take a few more revisions of this series to get there.
>>>
>>> That's disappointing, but I can understand your reluctance.  Blue
>>> Swirl reviewed the RFC and could perhaps add a Reviewed-by.  Alexey
>>> has been working on the POWER port and I'm sure could provide a
>>> Reviewed-by.  We also have a few early adopters that are already
>>> making use of this code.
>>
>> I'm running qemu with vfio patch since Jun 05, 2012
>> (awilliam-qemu-vfio-v0.14.0-rc0-6402-g323cf9f.tar.gz). I didn't
>> encounter any problem so far.
>>
>> If you like, I could compile a more actual version, too (if there have
>> been any changes).
> 
> The only change in the version proposed for qemu is that legacy
> interrupt support has been removed until we can agree on interfaces in
> kvm and plumb an EOI path through qemu.  IIRC, the devices you're using
> require legacy interrupt support.
> 
>> To see more about my use case:
>> http://permalink.gmane.org/gmane.linux.drivers.rt2x00.user/1051
>>
>> You may add a Tested-by Andreas Hartmann <andihartmann@01019freenet.de>
>> if you like.
>>
>> Unfortunately, I'm only running the vfio VM (kvm) with this version of
>> qemu, but I'm running parallel 4 other VM's with the unchanged version
>> of qemu (kvm-0.15.0-123.2.x86_64), too.
>> One of these 4 VM's uses PCIe passthrough.
>>
>> I now tried to run all VMs with the new version of qemu. At this point,
>> I unfortunately run into a problem with the VM which passes through a
>> PCIe device. The error message is (during start of VM):
>>
>> virsh start VM
>> error: Failed to start domain VM
>> error: internal error process exited while connecting to monitor: qemu-system-x86_64: -device pci-assign,host=04:00.0,id=hostdev0,bus=pci.0,addr=0x6: Parameter 'driver' expects device type
>>
>> The xml file for libvirt looks like this: 
> 
> libvirt doesn't yet support vfio-pci and current qemu.git doesn't yet
> support pci-assign.  To use libvirt for device assignment, your only
> option right now is to use a qemu-kvm.git based version of qemu.

If I'm using your qemu instead of qemu from kvm-0.15 (opensuse package),
this error comes up when passing through a PCIe device, which works
absolutely fine with kvm 0.15. I would have expected, that your qemu
works with the legacy way of handling pcie passthrough, too (with
pci-stub module).

This would mean, that all users get errors if they use the traditional
way. IOW: there are changes needed (which?) to move from kvm 0.15 to
your qemu version.

> 
>> <domain type='kvm'>
>>   <name>VM</name>
>>   <uuid>44444444-4444-2222-7777-111111111111</uuid>
>>   <memory>1048576</memory>
>>   <currentMemory>262144</currentMemory>
>>   <vcpu>1</vcpu>
>>   <os>
>>     <type arch='x86_64' machine='pc-0.14'>hvm</type>
>>     <boot dev='hd'/>
>>   </os>
>>   <features>
>>     <acpi/>
>>     <pae/>
>>   </features>
>>   <clock offset='utc'/>
>>   <on_poweroff>destroy</on_poweroff>
>>   <on_reboot>restart</on_reboot>
>>   <on_crash>destroy</on_crash>
>>   <devices>
>>     <emulator>/usr/local/bin/qemu-system-x86_64</emulator>
>>
>>     <hostdev mode='subsystem' type='pci' managed='yes'>
>>       <source>
>>         <address domain='0x0000' bus='0x04' slot='0x00' function='0x0'/>
>>       </source>
>>     </hostdev>
>>
>>     <disk type='file' device='disk'>
>>       <source file='/hds/vm.qed' type='aio'/>
>>       <target dev='vda' bus='virtio'/>
>>       <driver name='qemu' type='qed' cache='none'/>
>>     </disk>
>>
>>     <input type='mouse' bus='ps2'/>
>>     <graphics type='vnc' autoport='yes'/>
>>     <video>
>>       <model type='cirrus' vram='9216' heads='1'/>
>>     </video>
>>   </devices>
>> </domain>
>>
>>
>> Maybe, this is fixed in a newer version of qemu for vfio?

Kind regards,
Andreas