Re: [Xen-devel] Reset pass-thru devices in a VM

[Chao Gao <chao.gao@intel.com>]

On Fri, Aug 09, 2019 at 02:42:09PM +0200, Roger Pau Monné wrote:
>On Fri, Aug 09, 2019 at 04:38:33PM +0800, Chao Gao wrote:
>> Hi everyone,
>> 
>> I have a device which only supports secondary bus reset. After being
>> assigned to a VM, it would be placed under host bridge. For devices
>> under host bridge, secondary bus reset is not applicable. Thus, a VM
>> has no way to reset this device.
>
>I think in general we don't allow guests to perform any kind of reset
>of PCI devices, that's always in control of the hardware domain.

But reset is trapped and performed by the hardware domain. I don't think
in this way, guest could access registers or gain more permissions over
some registers than it should be.

>
>How are for example BARs going to be placed after such reset?

I don't know BARs are relocated after reset. I will figure it out.
Considering KVM/Qemu do support device reset, I think there
should be some means.

>
>> This device's usage would be limited without PCI reset (for example, its
>> driver cannot re-initialize the device properly without PCI reset, which
>> means in VM device won't be usable after unloading the driver), it would
>> be much better if there is a way available to VMs to reset the device.
>
>Is this something common (ie: requiring device reset functionality)
>for drivers to work correctly?

I don't think it is common. I am not sure whether it is required for GPU
pass-thru in some cases. But I believe that I saw some online materials
about GPU pass-thru mentioned how to enable FLR for a VM.

>
>So far we seem to have managed to get away without it.
>
>> In my mind, a straightfoward solution is to create a virtual bridge
>> for a VM and place the pass-thru device under a virtual bridge. But it
>> isn't supported in Xen (KVM/QEMU supports) and enabling it looks need
>> a lot of efforts. Alternatively, emulating FLR (Function Level Reset)
>> capability for this device might be a feasible way and only needs
>> relatively few changes. I am planning to enable an opt-in feature
>> (like 'permissive') to allow qemu to expose FLR capability to guest for
>> pass-thru devices as long as this device is resetable on dom0 (i.e. the
>> device has 'reset' attribute under its sysfs). And when guest initiates
>> an FLR, qemu just echo 1 to the 'reset' attribute on dom0.
>
>So you would expose the device as FLR capable and just implement it as
>a secondary bus reset on the device model?

For the device I mentioned, yes. Actually, for other devices, it can be
any supported reset method. There are several ways to reset a function;
secondary bus reset is the last choice.

>
>That seems feasible, but as noted above I would be worried about the
>resources owned by the device, and how they are going to be placed
>after such reset. Note you would also have to notify Xen somehow of
>such reset, so it tears down all the state related to the device.

I will figure out what should be done in Xen side.

Thanks
Chao

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel