Re: AMD_IOV: IO_PAGE_FALT trying to pass through Mellanox ConnectX HCA (debian testing)

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

> The mlx4_core driver in the domU is not happy:
> 
> [    0.411867] mlx4_core: Mellanox ConnectX core driver v0.01 (May 1, 2007)
> [    0.411879] mlx4_core: Initializing 0000:00:00.0
> [    0.412027] mlx4_core 0000:00:00.0: enabling device (0000 -> 0002)
> [    0.412027] mlx4_core 0000:00:00.0: Xen PCI enabling IRQ: 19
> [    1.417477] mlx4_core 0000:00:00.0: Installed FW has unsupported command
> interface revision 0.
> [    1.417509] mlx4_core 0000:00:00.0: (Installed FW version is 0.0.000)
> [    1.417527] mlx4_core 0000:00:00.0: This driver version supports only
> revisions 2 to 3.
> [    1.417549] mlx4_core 0000:00:00.0: QUERY_FW command failed, aborting.
> 
> When trying to boot a PV domU with kernel options iommu=soft and
> swiotlb=force, the output is slightly different. The full bootlog is attached

Don't use swiotlb=force unless neccessary.

> (testsqueeze.bootlog.txt). Here's the relevant excerpt:

That is b/c you are missing iommu=pv on Xen hypervisor line, and
you might need to make sure your driver is using the VM_IO flag.

There was some discussion on LKML about this and they proposed
a patch that wasn't neccessary. Don't remember the details but I can
look that up next week.
> 
> [    0.441684] mlx4_core: Mellanox ConnectX core driver v1.0-ofed1.5.2
> (August 4, 2010)
> [    0.441696] mlx4_core: Initializing 0000:00:00.0
> [    0.442044] mlx4_core 0000:00:00.0: enabling device (0000 -> 0002)
> [    0.442741] mlx4_core 0000:00:00.0: Xen PCI enabling IRQ: 19
> [    2.752125] mlx4_core 0000:00:00.0: NOP command failed to generate MSI-X
> interrupt IRQ 54).
> [    2.752158] mlx4_core 0000:00:00.0: Trying again without MSI-X.
> [    2.884105] mlx4_core 0000:00:00.0: NOP command failed to generate
> interrupt (IRQ 54), aborting.
> [    2.884138] mlx4_core 0000:00:00.0: BIOS or ACPI interrupt routing