[Qemu-devel] Nested PCI passthrough

[Qemu-devel] Nested PCI passthrough

[fassl]

Hello,

i am trying to pass through a graphic card to a guest within a guest.
So far i can see a text console within the target vm which says
"radeon: ring 0 test failed", so it times out in radeon_vce_ring_test
function. I am using qemu 2.8.50 at revision
0f6bcf68a99efdc531b209551f2b760b0bdcc554.

The relevant lowermost host arguments are:
-machine pc-q35-2.8,accel=kvm,kernel-irqchip=split
-device intel-iommu,intremap=on,eim=on

If i dont set the x-vga flag for the passed through device in the
lowermost host the VM within the VM does not reset(?) the device during
shutdown and the screen freezes. If i do, the screen goes black and no
signal is going to the screen anymore and i can restart the target VM
without the whole machine freezing. (one has to set
CONFIG_VFIO_PCI_VGA=y in the kernel of the first VM, or call
pci_register_vga to get this)

Also with irqchip=split the first vm cannot shutdown gracefully and
crashes during shutdown.

I also can see some IRTE vector and trigger mode inconsistencies, can
they cause this?

Or are there some quirks to be done to get this going?

If anybody could lead me into the right direction or if there is work
going on in this field i would be more than glad to help testing.

Thank you in advance

Re: [Qemu-devel] Nested PCI passthrough

[Alex Williamson]

On Sat, 21 Jan 2017 04:16:25 +0100
fassl <superfassl@gmail.com> wrote:

> Hello,
> 
> i am trying to pass through a graphic card to a guest within a guest.
> So far i can see a text console within the target vm which says
> "radeon: ring 0 test failed", so it times out in radeon_vce_ring_test
> function. I am using qemu 2.8.50 at revision
> 0f6bcf68a99efdc531b209551f2b760b0bdcc554.
> 
> The relevant lowermost host arguments are:
> -machine pc-q35-2.8,accel=kvm,kernel-irqchip=split
> -device intel-iommu,intremap=on,eim=on
> 
> If i dont set the x-vga flag for the passed through device in the
> lowermost host the VM within the VM does not reset(?) the device during
> shutdown and the screen freezes. If i do, the screen goes black and no
> signal is going to the screen anymore and i can restart the target VM
> without the whole machine freezing. (one has to set
> CONFIG_VFIO_PCI_VGA=y in the kernel of the first VM, or call
> pci_register_vga to get this)
> 
> Also with irqchip=split the first vm cannot shutdown gracefully and
> crashes during shutdown.
> 
> I also can see some IRTE vector and trigger mode inconsistencies, can
> they cause this?
> 
> Or are there some quirks to be done to get this going?
> 
> If anybody could lead me into the right direction or if there is work
> going on in this field i would be more than glad to help testing.
> 
> Thank you in advance

This shouldn't even be possible right now, vfio-pci should generate an
abort when used with intel-iommu.  There is work in progress to make
some form of vfio-pci work with intel-iommu, but devices that don't
support function level reset (such as graphics cards) are going to have
a more difficult time than most being supported in this configuration.
What is your use case that you're even attempting this (or consider it
a sane thing to do)?  Thanks,

Alex

Re: [Qemu-devel] Nested PCI passthrough

[Peter Xu]

On Sat, Jan 21, 2017 at 04:16:25AM +0100, fassl wrote:
> Hello,
> 
> i am trying to pass through a graphic card to a guest within a guest.
> So far i can see a text console within the target vm which says
> "radeon: ring 0 test failed", so it times out in radeon_vce_ring_test
> function. I am using qemu 2.8.50 at revision
> 0f6bcf68a99efdc531b209551f2b760b0bdcc554.
> 
> The relevant lowermost host arguments are:
> -machine pc-q35-2.8,accel=kvm,kernel-irqchip=split
> -device intel-iommu,intremap=on,eim=on
> 
> If i dont set the x-vga flag for the passed through device in the
> lowermost host the VM within the VM does not reset(?) the device during
> shutdown and the screen freezes. If i do, the screen goes black and no
> signal is going to the screen anymore and i can restart the target VM
> without the whole machine freezing. (one has to set
> CONFIG_VFIO_PCI_VGA=y in the kernel of the first VM, or call
> pci_register_vga to get this)
> 
> Also with irqchip=split the first vm cannot shutdown gracefully and
> crashes during shutdown.
> 
> I also can see some IRTE vector and trigger mode inconsistencies, can
> they cause this?

This should not be related. Even you don't passthrough devices, they
should be possibly there as well as long as you are running Linux
inside guest (of course, you should have enabled IOMMU_DEBUG).

-- peterx