Re: [PATCH] x86/msi: Validate the guest-identified PCI devices in pci_prepare_msix()

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

On Thu, Jan 23, 2014 at 08:24:11AM +0000, Jan Beulich wrote:
> >>> On 22.01.14 at 22:40, Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:
> > On Wed, Jan 22, 2014 at 12:08:42PM +0000, Jan Beulich wrote:
> >> "Fixing the wrong thing" presumably, after taking a closer look at
> >> Konrad's second crash: The device in question really appears to
> >> be MSI-X capable, yet alloc_pdev() didn't recognize it as such. I
> >> wonder whether the capability gets displayed/hidden dynamically
> >> based on some other enabling the driver may be doing on the
> >> device. In which case we'd need to allocate the structure on
> >> demand.
> > 
> > The device in question (02:00.1) is an SR-IOV 82576:
> > 
> > 02:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network 
> > Connection (rev 01)
> > 02:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network 
> > Connection (rev 01)
> > 
> > -bash-4.1# lspci -s 02:00.1 -v | more
> > 02:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network 
> > Connection (rev 01)
> >         Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
> >         Flags: fast devsel, IRQ 18
> >         Memory at f1400000 (32-bit, non-prefetchable) [disabled] [size=128K]
> >         Memory at f0800000 (32-bit, non-prefetchable) [disabled] [size=4M]
> >         I/O ports at d000 [disabled] [size=32]
> >         Memory at f1440000 (32-bit, non-prefetchable) [disabled] [size=16K]
> >         Expansion ROM at f0400000 [disabled] [size=4M]
> >         Capabilities: [40] Power Management version 3
> >         Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
> >         Capabilities: [70] MSI-X: Enable- Count=10 Masked-
> >         Capabilities: [a0] Express Endpoint, MSI 00
> >         Capabilities: [100] Advanced Error Reporting
> >         Capabilities: [140] Device Serial Number 00-1b-21-ff-ff-45-d9-ac
> >         Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
> >         Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
> >         Kernel driver in use: pciback
> >         Kernel modules: igb
> 
> So is this state with igb never having been bound to the device,
> or was it unbound before the device got handed to igb. I'm asking
> because I'm trying to understand why alloc_pdev() didn't find the
> MSI-X capability structure, and I continue to suspect that the
> driver may have done something to the device to make it visible.

I built the kernel without the igb driver just to eliminate it being
the culprit. Now I can boot without issues and this is what lspci
reports:

-bash-4.1# lspci -s 02:00.0 -v
02:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
        Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
        Flags: bus master, fast devsel, latency 0, IRQ 10
        Memory at f1420000 (32-bit, non-prefetchable) [size=128K]
        Memory at f1000000 (32-bit, non-prefetchable) [size=4M]
        I/O ports at e020 [size=32]
        Memory at f1444000 (32-bit, non-prefetchable) [size=16K]
        Expansion ROM at f0c00000 [disabled] [size=4M]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
        Capabilities: [70] MSI-X: Enable- Count=10 Masked-
        Capabilities: [a0] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] Device Serial Number 00-1b-21-ff-ff-45-d9-ac
        Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
        Capabilities: [160] Single Root I/O Virtualization (SR-IOV)

-bash-4.1# lspci -s 02:00.1 -v
02:00.1 Ethernet controller: Intel Corporation 82576 Gigabit Network Connection (rev 01)
        Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
        Flags: bus master, fast devsel, latency 0, IRQ 5
        Memory at f1400000 (32-bit, non-prefetchable) [size=128K]
        Memory at f0800000 (32-bit, non-prefetchable) [size=4M]
        I/O ports at e000 [size=32]
        Memory at f1440000 (32-bit, non-prefetchable) [size=16K]
        Expansion ROM at f0400000 [disabled] [size=4M]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
        Capabilities: [70] MSI-X: Enable- Count=10 Masked-
        Capabilities: [a0] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] Device Serial Number 00-1b-21-ff-ff-45-d9-ac
        Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
        Capabilities: [160] Single Root I/O Virtualization (SR-IOV)

> 
> Jan
>