From mboxrd@z Thu Jan 1 00:00:00 1970 From: Anirban Chakraborty Subject: Re: xl/SR-IOV: disposition of VFs when PF disappears? Date: Mon, 27 Oct 2014 19:53:09 +0000 Message-ID: References: <544E4A4E020000780004267D@mail.emea.novell.com> <1414414666.23883.13.camel@eu.citrix.com> <20141027133534.GC4050@laptop.dumpdata.com> <20141027180723.GD12989@laptop.dumpdata.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: Received: from mail6.bemta5.messagelabs.com ([195.245.231.135]) by lists.xen.org with esmtp (Exim 4.72) (envelope-from ) id 1XiqLo-00084M-PH for xen-devel@lists.xenproject.org; Mon, 27 Oct 2014 19:53:12 +0000 In-Reply-To: <20141027180723.GD12989@laptop.dumpdata.com> Content-Language: en-US Content-ID: <5E246B7B4E633F4DB1D80742E20AA48B@namprd05.prod.outlook.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Konrad Rzeszutek Wilk Cc: Wei Liu , Stefano Stabellini , "ian.jackson@eu.citrix.com" , Ian Campbell , Jan Beulich , xen-devel List-Id: xen-devel@lists.xenproject.org On 10/27/14, 11:07 AM, "Konrad Rzeszutek Wilk" wrote: >On Mon, Oct 27, 2014 at 05:53:01PM +0000, Anirban Chakraborty wrote: >> >> >> On 10/27/14, 6:35 AM, "Konrad Rzeszutek Wilk" >> wrote: >> >> >On Mon, Oct 27, 2014 at 12:57:46PM +0000, Ian Campbell wrote: >> >> On Mon, 2014-10-27 at 12:36 +0000, Jan Beulich wrote: >> >> > All, >> >> > >> >> > Intel reports that the sequence >> >> > >> >> > - xl pci-assignable-add >> >> > - briefly run guest using that device [not sure whether that's >>really >> >>a >> >> > necessary step] >> >> > - xl pci-assignable-add >> >> > >> >> > results in both VF and PF being listed as assignable (the fact that >> >>as a >> >> > result the PF handed to a guest doesn't work is secondary here, as >>I >> >> > think this is a driver issue). Is that really how it should be? >> >>Shouldn't >> >> > instead all VFs get removed when the PF device (e.g. due to the >> >> > PF driver getting unloaded, which is a necessary part of making it >> >> > assignable) goes away? Or is it required for the admin to manually >> >> > remove the assignable VFs prior to making the PF go away? >> > >> >I am not sure I see the problem. If the user wishes to give the PF and >> >VF to a guest they should be able to do so? >> >> Theoretically, yes a guest can have a PF and all its VFs. However, from >> security perspective PF having the privilege of resetting the device >>etc., >> should stay in a privileged domain. Most of the NICs have some sort of >> PF-VF communication where the PF driver would ensure that VF drivers are >> notified of imminent PF removal so that the VF drivers can prepare for a >> graceful halt of IO. Ideally, a PF removal should do a hot unplug of the >> VFs from the guests and admin should not have to manually remove them. > >We seem to be talking about two different things. > >1) Assigning a PF and VF to a guest. While it is stupid it should be > be possible. We could add an warning to the 'xl pci-assign' command > if somebody does that, but it should be possible. Agreed. > >2). PF removal. Currently if you try to unload the PF and the VFs > are in use (pciback owns them), the unloading will not happen. Until > all of the VFs have been de-assigned. > Is the "bug" here that the reporter (Intel?) wants the VFs to be > automatically yanked out of a guest when the system admin wants to > unload the PF? > >> >> Anirban >> >> > >> >> >> >> xl is just controlling/exposing the set of devices which are bound to >> >> pciback here. (pci-assignable-list is literally a readdir loop over >>the >> >> relevant sysfs dir). >> >> >> >> I'm not sure if it should be up to (lib)xl, pciback or the core Linux >> >> pci stuff to handle the creation/destruction of VF devices when the >>PF >> >> driver is unbound/assigned. In fact I'm not even sure if VF lifetime >>is >> >> in any way tied to the PF driver state. >> > >> >It is. When we detect that the device is a VF we set some flag so that >>the >> >PF won't try to de-allocate the VFs. >> > >> >> >> >> I've added Konrad for a kernel-size pciback perspective. >> >> >> >> Ian. >> >> >> > >> >_______________________________________________ >> >Xen-devel mailing list >> >Xen-devel@lists.xen.org >> >http://lists.xen.org/xen-devel >>