Re: KVM vs. PCI-E Function Level Reset (FLR) ...

[Sheng Yang <sheng@linux.intel.com>]

On Wednesday 14 July 2010 04:41:01 Casey Leedom wrote:
>   I've ground my way through most of the Linux kernel code supporting KVM
> and "Device Assignment" but I haven't been able to untangle the path of
> when/where PCI-E Function Level Resets (FLRs) are applied and what happens
> if the Virtual Machine OS/Driver attempts to perform an FLR on the PCI
> Pass Through Device.
> 
>   It looks like the Linux KVM kernel support code issues an FLR against an
> Assigned Device when the device is assigned and when it's freed but it's
> not clear when those actions are taken.  For instance, if a device is
> assigned to a VM and then the VM reboots itself, is that counted as
> another assignment point? I.e. will KVM issue a new pci_reset_function()
> on the device so it shows up reset in the newly rebooted VM?

The assignment/deassignment happened when guest was created/destroyed. Currently 
it wouldn't issue a FLR when guest reset. 
 
>   And what happens if the VM OS/Driver attempts to write the PCI Pass
> Through Device's PCI-E FLR bit?  I assume that that write (and the
> following polling reads) are trapped by the KVM code but I can't find the
> code which implements the PCI Configuration Space emulation to see if the
> FLR is implemented there.  For instance, if I run Linux 2.6.30 in the VM
> and my Device Driver calls pci_reset_function() in its "probe()" function
> will that result in a Device FLR? it doesn't appear to be the case ...

The PCI configuration space emulated is in QEmu rather than KVM. You can check 
qemu-kvm/hw/device-assignment.c. We didn't emulate FLR capability now. (OK, some 
other device specific reset method may involved, you can check pci_dev_reset())

>       Note that it's impossible for a Device Driver to call
> pci_reset_function() under Linux 2.6.31 and later because a call to
> device_lock() was added to pci_dev_reset() in chageset 8e9394ce on Feb 17,
> 2010 by Greg Kroah-Hartman.  This means that a call to
> pci_reset_function() in a device driver's "probe()" routine will result in
> an immediate deadlock.

What I saw the code is like this:

static int pci_dev_reset(struct pci_dev *dev, int probe)  
{                                                         
        int rc;                                           
                                                          
        might_sleep();                                    
                                                          
        if (!probe) {                                     
                pci_block_user_cfg_access(dev);           
                /* block PM suspend, driver probe, etc. */
                device_lock(&dev->dev);                   
        }                                                
[...]

So seems it's fine with _probe_ set, to use with probe() routine.

--
regards
Yang, Sheng


> 
>   Thanks for any help and/or insight into these questions!
> 
> Casey
> --
> To unsubscribe from this list: send the line "unsubscribe kvm" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html