Re: Intel S3420GPLX VT-d VF question

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Mon, Dec 20, 2010 at 09:24:17PM -0500, Michael A. Collins wrote:
> On Thu, 09 Dec 2010 22:18:45 -0500, "Michael A. Collins"
> <mike@ark-net.org> wrote:
> >On Wed, 8 Dec 2010 12:50:35 -0500, Konrad Rzeszutek Wilk
> ><konrad.wilk@oracle.com> wrote:
> >>Take your time.
> >
> >Seems as the igb module was loaded in the initramfs and was not
> >reloaded using the options specified in my igb.conf file under
> >/etc/modprobe.d.  I rmmod igb and did a modprobe igb max_vfs=7 and
> >suddenly all the Virtual Functions showed up.  So that was the
> >problem
> >all along.  Until I figure out how to edit the initramfs to load igb
> >with max_vfs option, I'm putting the rmmod and modprobe in the
> >rc.local file, that seems to work well and it gets executed before
> >xend.  I am build xen-4.0-testing to replace the rpms I used.  Once
> >all that is done and I test passing through one of the Virtual
> >Functions to a vm, I'll report back.
> >Mike
> 
> 
> Ok, so I can pass one of the virtual functions through to a HVM.  I
> think I'm still not out of the water though.  It's acting very
> strangely and I have not successfully been able to very layer-2

And if you tried to use the device from dom0, or even baremetal, you
had not trouble with the VIFs, right?

> connectivity with it.  I am seeing the following in my HVM's qemu
> log:
> register_real_device: Assigning real physical device 04:10.4 ...
> pt_dev_is_virtfn: 0000:04:10.4 is a SR-IOV Virtual Function
> pt_iomul_init: Error: pt_iomul_init can't open file
> /dev/xen/pci_iomul: No such file or directory: 0x4:0x10.0x4
> pt_register_regions: IO region registered (size=0x00004000
> base_addr=0xb2448004)
> pt_register_regions: IO region registered (size=0x00004000
> base_addr=0xb2468004)
> pt_msix_init: get MSI-X table bar base b2468000
> pt_msix_init: table_off = 0, total_entries = 3
> pt_msix_init: errno = 2
                ^^^^^^^
That does not sound good, but it might be just b/c it could
not find /dev/xen/pci_iomul.

> pt_msix_init: mapping physical MSI-X table to 7fe3c4359000
> register_real_device: Real physical device 04:10.4 registered
> successfuly!
> IRQ type = INTx
> generate a sci for PHP.
> deassert due to disable GPE bit.
> pt_pci_write_config: Warning: Guest attempt to set address to unused
> Base Address Register. [00:06.0][Offset:30h][Length:4]
> pt_iomem_map: e_phys=20000000 maddr=b2448000 type=0 len=16384
> index=0 first_map=1
> pt_iomem_map: e_phys=20004000 maddr=b2468000 type=0 len=16384
> index=3 first_map=1
> 
> 
> Now, I never see a pt_msi_setup: msi mapped with pirq XX line, which
> leads me to believe that the pass through isn't actually working.  I
> am still using the mayoung built xen-4.0.1-6 rpms from koji, since I
> have not been able to successfully compile xen stubdom.  I guess gcc
> 4.5 is just too new and untested for me to troubleshoot and get
> through.  Hopefully, someone on here can point me in the right
> direction.

Hmm. I am bit out of ideas.. Do you know which line in the QEMU code
would trigger that? You could instrument QEMU a bit?