Re: Intel S3420GPLX VT-d VF question

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

> The kernel I am using is myoung's pre-built rpm from http://myoung.fedorapeople.org/dom0/x86_64/kernel-2.6.32.26-174.2.xendom0.fc12.x86_64.rpm.

Sweet. That is quite up-to-date.

> I am also using the 4.0.1 rpms from
> http://pasik.reaktio.net/fedora/xen-4.0.1-6.fc14.src.rpm.

Uhh, try a more modern version. The xen-unstable.hg - you will have
to compile it yourself. Pasi's awesome PVOPS Wiki mentions the appropiate
steps and I think Boris's blog has details as well.

> I am following the instructions from
> http://wiki.xen.org/xenwiki/Fedora13Xen4Tutorial.
> 
> The pci_pt_e820_access=on setting was taken from the following blog: http://blogs.oracle.com/wim/2010/10/playing_with_sr_iov_in_oracle.html.

Oh gosh. That is with 2.6.18 and Xen 3.4.

> I was under the impression that that option had to be set for the
> VFs to be seen by the kernel.

Well, they were workarounds for that verison of kernel. It might be
that they are required in 2.6.32.. but lets assume that those
are actually fixed.

> I have not tried it without, but will do that tonight.  The
> blacklist settings was referenced in the above article, so I stuck
> it in.  I will try to load the igbvf tonight as well.

Yeah, try it. If it works you should see suddenly seven extra NICs
(ifconfig -a) in your box. If that works then we know the kernel
driver is working. 
> 
> I do not see the 7 Virtual Functions in lspci.

Ah! Those "01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
01:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
01:10.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
01:10.2 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
01:10.4 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
01:10.6 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
01:11.0 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
01:11.2 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)
01:11.4 Ethernet controller: Intel Corporation 82576 Virtual Function (rev 01)"

must show up. That is the proof that your max_ig.. parameter worked and the card
is now able to do SR-IOV. You might have need to upgrade your lspci - I think I had
it as "Unknown device" at some point.

Anyhow, if you see those extra devices, _then_ you can blacklist the igvf module.

> 
> Thanks for the reply, I will try without the e820 option and loading
> the igbvf module tonight if that works, I'll let you know.  If not,
> I'll respond with the full xen dmesg output.

Take your time.