Re: xl/SR-IOV: disposition of VFs when PF disappears?

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Mon, Oct 27, 2014 at 05:53:01PM +0000, Anirban Chakraborty wrote:
> 
> 
> On 10/27/14, 6:35 AM, "Konrad Rzeszutek Wilk" <konrad.wilk@oracle.com>
> wrote:
> 
> >On Mon, Oct 27, 2014 at 12:57:46PM +0000, Ian Campbell wrote:
> >> On Mon, 2014-10-27 at 12:36 +0000, Jan Beulich wrote:
> >> > All,
> >> > 
> >> > Intel reports that the sequence
> >> > 
> >> > - xl pci-assignable-add <VF>
> >> > - briefly run guest using that device [not sure whether that's really
> >>a
> >> >   necessary step]
> >> > - xl pci-assignable-add <PF of VF>
> >> > 
> >> > results in both VF and PF being listed as assignable (the fact that
> >>as a
> >> > result the PF handed to a guest doesn't work is secondary here, as I
> >> > think this is a driver issue). Is that really how it should be?
> >>Shouldn't
> >> > instead all VFs get removed when the PF device (e.g. due to the
> >> > PF driver getting unloaded, which is a necessary part of making it
> >> > assignable) goes away? Or is it required for the admin to manually
> >> > remove the assignable VFs prior to making the PF go away?
> >
> >I am not sure I see the problem. If the user wishes to give the PF and
> >VF to a guest they should be able to do so?
> 
> Theoretically, yes a guest can have a PF and all its VFs. However, from
> security perspective PF having the privilege of resetting the device etc.,
> should stay in a privileged domain. Most of the NICs have some sort of
> PF-VF communication where the PF driver would ensure that VF drivers are
> notified of imminent PF removal so that the VF drivers can prepare for a
> graceful halt of IO. Ideally, a PF removal should do a hot unplug of the
> VFs from the guests and admin should not have to manually remove them.

We seem to be talking about two different things.

1) Assigning a PF and VF to a guest. While it is stupid it should be
   be possible. We could add an warning to the 'xl pci-assign' command
   if somebody does that, but it should be possible.

2). PF removal. Currently if you try to unload the PF and the VFs
   are in use (pciback owns them), the unloading will not happen. Until
   all of the VFs have been de-assigned.
   Is the "bug" here that the reporter (Intel?) wants the VFs to be
   automatically yanked out of a guest when the system admin wants to
   unload the PF?

> 
> Anirban
>  
> >
> >> 
> >> xl is just controlling/exposing the set of devices which are bound to
> >> pciback here. (pci-assignable-list is literally a readdir loop over the
> >> relevant sysfs dir).
> >> 
> >> I'm not sure if it should be up to (lib)xl, pciback or the core Linux
> >> pci stuff to handle the creation/destruction of VF devices when the PF
> >> driver is unbound/assigned. In fact I'm not even sure if VF lifetime is
> >> in any way tied to the PF driver state.
> >
> >It is. When we detect that the device is a VF we set some flag so that the
> >PF won't try to de-allocate the VFs.
> >
> >> 
> >> I've added Konrad for a kernel-size pciback perspective.
> >> 
> >> Ian.
> >> 
> >
> >_______________________________________________
> >Xen-devel mailing list
> >Xen-devel@lists.xen.org
> >http://lists.xen.org/xen-devel
>