qemu-xen-dir + PCI passthrough = BOOM

[Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>]

Hey,

I just started noticing it today - with qemu-xen (tip is
commit b97307ecaad98360f41ea36cd9674ef810c4f8cf
    xen_disk: mark ioreq as mapped before unmapping in error case)

when I try to pass in a PCI device at bootup it blows up with:

char device redirected to /dev/pts/2 (label serial0)
qemu: hardware error: xen: failed to populate ram at 40050000
CPU #0:
EAX=00000000 EBX=00000000 ECX=00000000 EDX=00000633
ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000
EIP=0000fff0 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0
ES =0000 00000000 0000ffff 00009300
CS =f000 ffff0000 0000ffff 00009b00
SS =0000 00000000 0000ffff 00009300
DS =0000 00000000 0000ffff 00009300
FS =0000 00000000 0000ffff 00009300
GS =0000 00000000 0000ffff 00009300
LDT=0000 00000000 0000ffff 00008200
TR =0000 00000000 0000ffff 00008b00
GDT=     00000000 0000ffff
IDT=     00000000 0000ffff
CR0=60000010 CR2=00000000 CR3=00000000 CR4=00000000
DR0=00000000 DR1=00000000 DR2=00000000 DR3=00000000 
DR6=ffff0ff0 DR7=00000400
EFER=0000000000000000
FCW=037f FSW=0000 [ST=0] FTW=00 MXCSR=00001f80
FPR0=0000000000000000 0000 FPR1=0000000000000000 0000
FPR2=0000000000000000 0000 FPR3=0000000000000000 0000
FPR4=0000000000000000 0000 FPR5=0000000000000000 0000
FPR6=0000000000000000 0000 FPR7=0000000000000000 0000
XMM00=00000000000000000000000000000000
XMM01=00000000000000000000000000000000
XMM02=00000000000000000000000000000000
XMM03=00000000000000000000000000000000
XMM04=00000000000000000000000000000000
XMM05=00000000000000000000000000000000
XMM06=00000000000000000000000000000000
XMM07=00000000000000000000000000000000
CPU #1:
EAX=00000000 EBX=00000000 ECX=00000000 EDX=00000633
ESI=00000000 EDI=00000000 EBP=00000000 ESP=00000000
EIP=0000fff0 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=1
ES =0000 00000000 0000ffff 00009300
CS =f000 ffff0000 0000ffff 00009b00


...
-bash-4.1# xl dmesg | tail
[ 2788.038463] xen_pciback: vpci(d3) [2013-12-04 19:43:40] System
requested SeaBIOS
: 0000:01:00.1: (d3) [2013-12-04 19:43:40] CPU speed is 3093 MHz
assign to virtua(d3) [2013-12-04 19:43:40] Relocating guest memory for
lowmem MMIO space disabled
l slot 0
[ 2788.076396] device vif3.0 entered promiscuous mode

If I don't have 'pci=' stanze in my guest config it boots just fine.

-bash-4.1# more /vm.cfg 
builder='hvm'
disk = [ 'file:/mnt/lab/latest/root_image.iso,hdc:cdrom,r']
memory=1024
boot="d"
vcpus=2
serial="pty"
vnclisten="0.0.0.0"
name="latest"
vif = [ 'mac=00:0F:4B:00:00:68, bridge=switch' ]
on_crash="preserve"
pci=["01:00.0"]

Any ideas?

This is with today's Xen and 3.13-rc2. The device in question is
-bash-4.1# lspci -s 01:00.0 -v
01:00.0 Ethernet controller: Intel Corporation 82576 Gigabit Network
Connection (rev 01)
        Subsystem: Intel Corporation Gigabit ET Dual Port Server Adapter
        Flags: fast devsel, IRQ 16
        Memory at fbc20000 (32-bit, non-prefetchable) [disabled]
[size=128K]
        Memory at fb800000 (32-bit, non-prefetchable) [disabled]
[size=4M]
        I/O ports at e020 [disabled] [size=32]
        Memory at fbc44000 (32-bit, non-prefetchable) [disabled]
[size=16K]
        Expansion ROM at fb400000 [disabled] [size=4M]
        Capabilities: [40] Power Management version 3
        Capabilities: [50] MSI: Enable- Count=1/1 Maskable+ 64bit+
        Capabilities: [70] MSI-X: Enable- Count=10 Masked-
        Capabilities: [a0] Express Endpoint, MSI 00
        Capabilities: [100] Advanced Error Reporting
        Capabilities: [140] Device Serial Number 00-1b-21-ff-ff-45-d9-ac
        Capabilities: [150] Alternative Routing-ID Interpretation (ARI)
        Capabilities: [160] Single Root I/O Virtualization (SR-IOV)
        Kernel driver in use: pciback
        Kernel modules: igb


Oh and of course it boots with the traditional QEMU.