[BUG] PCI passtrough causes driver crash in DomU (Xen 4.3, Linux kernel 3.12)

[Jonas Meurer <jonas@freesources.org>]

Hello,

as already written to xen-users a few days ago, I've troubles using pci
passthrough for an Intel 82574L ethernet device (kernel module e1000e).
The device is passed through by xen-pciback, and can be seen by 'lspci'
inside the DomU. Still, the corresponding network interface doesn't
appear in /proc/net/dev and even more striking, the DomU kernel driver
crashes with a traceback.

All this happens on a Debian/Jessie Dom0 with Xen 4.3 and Linux kernel
3.12. The DomU in question has a similar Debian/Jessie setup.

I searched the archives for similar issues and found the following
thread from October 2013 that describes a similar issue starting with
Linux kernel 3.8:
http://thread.gmane.org/gmane.comp.emulators.xen.user/80672

Below in my original post to xen-users you find details about the kernel
drvier crash and relevant logs.

I'll gladly provide more details if you give me advice on how to further
debug this issue :)

Kind regards,
 jonas


Am 03.03.2014 17:59, schrieb Jonas Meurer:
> Hello,
> 
> I fail to configure Xen the way that it passes a network device through
> to a VM (on Debian/Jessie, Xen 4.3, Linux kernel 3.12)
> 
> This is the network device in question:
> 
> # lspci -v -s 0000:03:00.0
> 03:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network
> Connection
> [...]
> Kernel driver in use: e1000e
> 
> Dynamic assignment seems to work, but passthrough works only halfways:
> 
> # echo 0000:03:00.0 > /sys/bus/pci/drivers/e1000e/unbind
> # echo 0000:03:00.0 > /sys/bus/pci/drivers/pciback/new_slot
> # echo 0000:03:00.0 > /sys/bus/pci/drivers/pciback/bind
> # lspci -v -s 0000:03:00.0
> 03:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network
> Connection
> [...]
> Kernel driver in use: pciback
> # xl pci-assignable-list
> 0000:03:00.0
> 
> Afterwards I do see the pci device inside the DomU (Debian/Jessie, Linux
> Kernel 3.12):
> 
> # lspci
> 00:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network
> Connection
> # lspci -v -s 0000:00:00.0
> 00:00.0 Ethernet controller: Intel Corporation 82574L Gigabit Network
> Connection
> [...]
> Kernel driver in use: e1000e
> 
> But the device is not recognized as network device, /proc/net/dev lists
> only the vif configured in DomU-Config.
> 
> I see the following traceback in dmesg inside DomU:
> 
> [    1.148546] xenbus_probe_frontend: Device with no driver:
> device/vbd/51712
> [    1.148547] xenbus_probe_frontend: Device with no driver: device/vif/0
> [    1.148548] xenbus_probe_frontend: Device with no driver: device/pci/0
> [...]
> [    1.206122] pcifront pci-0: Installing PCI frontend
> [...]
> [    1.209670] pcifront pci-0: Creating PCI Frontend Bus 0000:00
> [    1.209709] pcifront pci-0: PCI host bridge to bus 0000:00
> [    1.209712] pci_bus 0000:00: root bus resource [io  0x0000-0xffff]
> [    1.209713] pci_bus 0000:00: root bus resource [mem
> 0x00000000-0x3fffffffffff]
> [    1.209716] pci_bus 0000:00: root bus resource [bus 00-ff]
> [    1.209888] pci 0000:00:00.0: [8086:10d3] type 00 class 0x020000
> [    1.210001] pci 0000:00:00.0: reg 0x10: [mem 0xdfbc0000-0xdfbdffff]
> [    1.210066] pci 0000:00:00.0: reg 0x14: [mem 0xdfb00000-0xdfb7ffff]
> [    1.210134] pci 0000:00:00.0: reg 0x18: [io  0x7000-0x701f]
> [    1.210195] pci 0000:00:00.0: reg 0x1c: [mem 0xdfbe0000-0xdfbe3fff]
> [    1.223406] xen_netfront: Initialising Xen virtual ethernet driver
> [    1.227416] pcifront pci-0: claiming resource 0000:00:00.0/0
> [    1.227421] pcifront pci-0: claiming resource 0000:00:00.0/1
> [    1.227422] pcifront pci-0: claiming resource 0000:00:00.0/2
> [    1.227424] pcifront pci-0: claiming resource 0000:00:00.0/3
> [...]
> [    1.308278] e1000e: Intel(R) PRO/1000 Network Driver - 2.3.2-k
> [    1.308280] e1000e: Copyright(c) 1999 - 2013 Intel Corporation.
> [    1.308363] e1000e 0000:00:00.0: enabling device (0000 -> 0002)
> [    1.308536] e1000e 0000:00:00.0: Xen PCI mapped GSI40 to IRQ28
> [    1.309023] e1000e 0000:00:00.0: Interrupt Throttling Rate (ints/sec)
> set to dynamic conservative mode
> [    1.309462] BUG: unable to handle kernel paging request at
> ffffc900000a600c
> [    1.309472] IP: [<ffffffff812ab4be>] pci_enable_msix+0x30e/0x3e0
> [    1.309480] PGD ec5f067 PUD ec60067 PMD ec61067 PTE 80100000dfbe0465
> [    1.309489] Oops: 0003 [#1] SMP
> [    1.309494] Modules linked in: e1000e(+) ptp pps_core xen_netfront(+)
> xen_blkfront(+) xen_pcifront
> [    1.309503] CPU: 0 PID: 72 Comm: modprobe Not tainted 3.12-1-amd64 #1
> Debian 3.12.9-1
> [    1.309508] task: ffff88000c2cf840 ti: ffff880003806000 task.ti:
> ffff880003806000
> [    1.309512] RIP: e030:[<ffffffff812ab4be>]  [<ffffffff812ab4be>]
> pci_enable_msix+0x30e/0x3e0
> [    1.309516] RSP: e02b:ffff880003807b78  EFLAGS: 00010286
> [    1.309519] RAX: 000000000000000c RBX: ffff88000c1a7140 RCX:
> 0000000000000005
> [    1.309523] RDX: ffffc900000a600c RSI: 0000000000000001 RDI:
> ffff88000c200200
> [    1.309526] RBP: ffff880003ff5a00 R08: ffff88000c209c00 R09:
> ffff88000e800050
> [    1.309529] R10: 000000000000001f R11: 0000000000000044 R12:
> ffff88000c11f000
> [    1.309532] R13: 0000000000000000 R14: 0000000000000000 R15:
> 000000000000000c
> [    1.309539] FS:  00007f6dcf53b700(0000) GS:ffff88000fc00000(0000)
> knlGS:0000000000000000
> [    1.309543] CS:  e033 DS: 0000 ES: 0000 CR0: 0000000080050033
> [    1.309545] CR2: ffff8000006fdf00 CR3: 0000000003ff3000 CR4:
> 0000000000042660
> [    1.309549] Stack:
> [    1.309552]  ffff88000c11f868 00000000c0040800 ffff880003880800
> ffffffffa005a100
> [    1.309557]  ffff88000c11f098 0000000000000001 ffff880003880000
> ffff880003880800
> [    1.309562]  ffffffffa00519bf ffff88000c11f000 ffffffffa0055fc5
> ffff880003880800
> [    1.309568] Call Trace:
> [    1.309578]  [<ffffffffa00519bf>] ?
> e1000e_set_interrupt_capability+0xef/0x120 [e1000e]
> [    1.309586]  [<ffffffffa0055fc5>] ? e1000_probe+0x3a5/0xe30 [e1000e]
> [    1.309594]  [<ffffffff81293264>] ? local_pci_probe+0x34/0x60
> [    1.309599]  [<ffffffff81294582>] ? pci_device_probe+0x112/0x120
> [    1.309605]  [<ffffffff81345bf8>] ? driver_probe_device+0x68/0x220
> [    1.309609]  [<ffffffff81345e6b>] ? __driver_attach+0x7b/0x80
> [    1.309613]  [<ffffffff81345df0>] ? __device_attach+0x40/0x40
> [    1.309618]  [<ffffffff81343e73>] ? bus_for_each_dev+0x53/0x90
> [    1.309623]  [<ffffffff81345368>] ? bus_add_driver+0x1e8/0x290
> [    1.309628]  [<ffffffff81346406>] ? driver_register+0x56/0xd0
> [    1.309631]  [<ffffffffa0066000>] ? 0xffffffffa0065fff
> [    1.309637]  [<ffffffff8100210a>] ? do_one_initcall+0x10a/0x160
> [    1.309642]  [<ffffffff810c01f1>] ? load_module+0x1c11/0x24c0
> [    1.309647]  [<ffffffff810bcf40>] ? symbol_put_addr+0x30/0x30
> [    1.309652]  [<ffffffff810c0bcd>] ? SyS_finit_module+0x6d/0x70
> [    1.309658]  [<ffffffff81498fb9>] ? system_call_fastpath+0x16/0x1b
> [    1.309661] Code: 0c 89 02 8b 7b 0c e8 02 c9 df ff 49 63 c7 48 03 43
> 28 8b 30 0f b7 43 02 89 73 08 83 ce 01 c1 e0 04 83 c0 0c 48 63 d0 48 03
> 53 28 <89> 32 48 8b 43 18 41 83 c6 01 48 39 04 24 89 73 08 48 8d 58 e8
> [    1.309697] RIP  [<ffffffff812ab4be>] pci_enable_msix+0x30e/0x3e0
> [    1.309702]  RSP <ffff880003807b78>
> [    1.309704] CR2: ffffc900000a600c
> [    1.309708] ---[ end trace 14e095cf15d42207 ]---
> 
> 
> Any idea what I'm missing? Or is this a known bug?
> 
> Kind regards,
>  jonas
> 
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xen.org
> http://lists.xen.org/xen-users
>