From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrew Cooper Subject: Re: PV passthrough of sibling igbvf's Date: Tue, 16 Oct 2012 15:42:08 +0100 Message-ID: <507D7240.2010305@citrix.com> References: <507D6D84.8050602@redhat.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <507D6D84.8050602@redhat.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: Laszlo Ersek Cc: Igor Mammedov , Drew Jones , "xen-devel@lists.xen.org" , Paolo Bonzini , Konrad Rzeszutek Wilk List-Id: xen-devel@lists.xenproject.org On 16/10/12 15:21, Laszlo Ersek wrote: > Hi, > > this is with reference to > -- RHEL-5.9 Beta > host & guest. Nonetheless I think my question applies to current > upstream Linux -- if not, I'd greatly appreciate commit hashes. > > Consider several igbvf's that belong to the same PF (port): they are > functions that share a (bus, device) pair (aka "slot") in dom0. The VPCI > implementation of pciback_add_pci_dev() [drivers/xen/pciback/vpci.c] > will assign these sibling functions to the same virtual slot. In other > words, VFs that are siblings in dom0 end up as siblings in the PV domU. > > (Upstream path and function: "drivers/xen/xen-pciback/vpci.c", > __xen_pcibk_add_pci_dev().) > > This logic appears to date back to > . > > > The RHEL-5.9 Beta PV domU does something like this: > > pci_scan_slot > /* for each function: */ > pci_scan_single_device > pci_scan_device > pci_bus_read_config_dword(bus, devfn, PCI_VENDOR_ID, &l) > pci_setup_device > pci_read_config_byte(dev, PCI_HEADER_TYPE, &hdr_type) > dev->multifunction = !!(hdr_type & 0x80); > if (dev_scanned && !dev->multifunc && func == 0): break > > Current upstream Linux has gone through several changes here, but the > gist is the same: if function 0 is successfully scanned and it > explicitly reports itself non-multi (--> no_next_fn()), then the rest of > the functions on the slot is skipped. > > Problem is, function 0 of the igbvf I'm looking at reports itself as > non-multifunction, and thus the domU doesn't find the rest of the > passed-through functions. Which is correct, because the virtual function itself is only a single function. I would hazard a guess that the real bug is trying to fake up 8 individual virtual functions as an 8-fuction device, which seems like a toolstack bug to me. While it would certainly be possible to trap and emulate reads like this, I would think it would be decidedly hacky, thus preferably avoided. ~Andrew > > > pciback seems to have no overlay for PCI_HEADER_TYPE in array > "header_common" [upstream: drivers/xen/xen-pciback/conf_space_header.c], > thus pciback_config_read() / xen_pcibk_config_read() pass through the > header type transparently when the domU reads it in pci_setup_device(). > > > I wonder if > > - a new dom0 overlay should be introduced for PCI_HEADER_TYPE, to the > tune of upstream Linux commit fd5b221b (ie. vendor-id/device-id), that > would perhaps check the # of sibling functions in the given vpci slot, > and fake the MSB in "hdr_type", > > - or the domU's slot scanning logic should be changed, > > - or the igbvf I'm looking at reports an incorrect "hdr_type" (in which > case we'd still have to fake something). The legacy > "/etc/xen/xend-pci-quirks.sxp" interface is only suitable for giving > extra write access to the domU, thus not good enough here. > > Thanks a lot! > Laszlo > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel -- Andrew Cooper - Dom0 Kernel Engineer, Citrix XenServer T: +44 (0)1223 225 900, http://www.citrix.com