PV passthrough of sibling igbvf's

[Laszlo Ersek <lersek@redhat.com>]

Hi,

this is with reference to
<https://bugzilla.redhat.com/show_bug.cgi?id=865736> -- RHEL-5.9 Beta
host & guest. Nonetheless I think my question applies to current
upstream Linux -- if not, I'd greatly appreciate commit hashes.

Consider several igbvf's that belong to the same PF (port): they are
functions that share a (bus, device) pair (aka "slot") in dom0. The VPCI
implementation of pciback_add_pci_dev() [drivers/xen/pciback/vpci.c]
will assign these sibling functions to the same virtual slot. In other
words, VFs that are siblings in dom0 end up as siblings in the PV domU.

(Upstream path and function: "drivers/xen/xen-pciback/vpci.c",
__xen_pcibk_add_pci_dev().)

This logic appears to date back to
<http://xenbits.xensource.com/xen-unstable.hg/rev/5b433b4fca34>.


The RHEL-5.9 Beta PV domU does something like this:

  pci_scan_slot
    /* for each function: */
      pci_scan_single_device
         pci_scan_device
           pci_bus_read_config_dword(bus, devfn, PCI_VENDOR_ID, &l)
           pci_setup_device
             pci_read_config_byte(dev, PCI_HEADER_TYPE, &hdr_type)
             dev->multifunction = !!(hdr_type & 0x80);
      if (dev_scanned && !dev->multifunc && func == 0): break

Current upstream Linux has gone through several changes here, but the
gist is the same: if function 0 is successfully scanned and it
explicitly reports itself non-multi (--> no_next_fn()), then the rest of
the functions on the slot is skipped.

Problem is, function 0 of the igbvf I'm looking at reports itself as
non-multifunction, and thus the domU doesn't find the rest of the
passed-through functions.


pciback seems to have no overlay for PCI_HEADER_TYPE in array
"header_common" [upstream: drivers/xen/xen-pciback/conf_space_header.c],
thus pciback_config_read() / xen_pcibk_config_read() pass through the
header type transparently when the domU reads it in pci_setup_device().


I wonder if

- a new dom0 overlay should be introduced for PCI_HEADER_TYPE, to the
tune of upstream Linux commit fd5b221b (ie. vendor-id/device-id), that
would perhaps check the # of sibling functions in the given vpci slot,
and fake the MSB in "hdr_type",

- or the domU's slot scanning logic should be changed,

- or the igbvf I'm looking at reports an incorrect "hdr_type" (in which
case we'd still have to fake something). The legacy
"/etc/xen/xend-pci-quirks.sxp" interface is only suitable for giving
extra write access to the domU, thus not good enough here.

Thanks a lot!
Laszlo