From: Konrad Rzeszutek Wilk <konrad.wilk@oracle.com>
To: Samuel Monderer <SMonderer@advaoptical.com>
Cc: "xen-devel@lists.xen.org" <xen-devel@lists.xen.org>
Subject: Re: HVM crashes when passing second igb interface
Date: Wed, 7 May 2014 13:30:15 -0400 [thread overview]
Message-ID: <20140507173015.GC14576@phenom.dumpdata.com> (raw)
In-Reply-To: <67957e630ea04d1195dcf092280f5dce@MUC-SRV-MBX2.advaoptical.com>
On Wed, May 07, 2014 at 04:31:02PM +0000, Samuel Monderer wrote:
> Hi,
>
> I've upgraded my HVM kernel to 3.14.1 and know when I attach an igb PF interface I get the following error and crash
Could you give a bit more details on the guest configuration?
And perhaps the 'lspci' with the previous guest?
>
> After attaching the first interface as following
>
> [ 260.042238] ACPI: Error installing CMOS-RTC region handler
> [ 260.044658] pci 0000:00:00.0: no hotplug settings from platform
> [ 260.044661] pci 0000:00:00.0: using default PCI settings
> [ 260.044753] pci 0000:00:01.0: no hotplug settings from platform
> [ 260.044754] pci 0000:00:01.0: using default PCI settings
> [ 260.044881] ata_piix 0000:00:01.1: no hotplug settings from platform
> [ 260.044883] ata_piix 0000:00:01.1: using default PCI settings
> [ 260.044975] pci 0000:00:01.3: no hotplug settings from platform
> [ 260.044976] pci 0000:00:01.3: using default PCI settings
> [ 260.045100] pci 0000:00:02.0: no hotplug settings from platform
> [ 260.045102] pci 0000:00:02.0: using default PCI settings
> [ 260.045197] xen-platform-pci 0000:00:03.0: no hotplug settings from platform
> [ 260.045199] xen-platform-pci 0000:00:03.0: using default PCI settings
> [ 260.045286] pci 0000:00:04.0: no hotplug settings from platform
> [ 260.045341] pci 0000:00:00.0: no hotplug settings from platform
> [ 260.045342] pci 0000:00:00.0: using default PCI settings
> [ 260.045462] pci 0000:00:01.0: no hotplug settings from platform
> [ 260.045464] pci 0000:00:01.0: using default PCI settings
> [ 260.045556] ata_piix 0000:00:01.1: no hotplug settings from platform
> [ 260.045558] ata_piix 0000:00:01.1: using default PCI settings
> [ 260.045669] pci 0000:00:01.3: no hotplug settings from platform
> [ 260.045671] pci 0000:00:01.3: using default PCI settings
> [ 260.045833] pci 0000:00:02.0: no hotplug settings from platform
> [ 260.045835] pci 0000:00:02.0: using default PCI settings
> [ 260.045993] xen-platform-pci 0000:00:03.0: no hotplug settings from platform
> [ 260.045995] xen-platform-pci 0000:00:03.0: using default PCI settings
> [ 260.046129] pci 0000:00:04.0: no hotplug settings from platform
> [ 260.046448] dca service started, version 1.12.1
> [ 260.046451] pci 0000:00:00.0: no hotplug settings from platform
> [ 260.046454] pci 0000:00:00.0: using default PCI settings
> [ 260.046597] pci 0000:00:01.0: no hotplug settings from platform
> [ 260.046599] pci 0000:00:01.0: using default PCI settings
> [ 260.046689] ata_piix 0000:00:01.1: no hotplug settings from platform
> [ 260.046690] ata_piix 0000:00:01.1: using default PCI settings
> [ 260.046777] pci 0000:00:01.3: no hotplug settings from platform
> [ 260.046778] pci 0000:00:01.3: using default PCI settings
> [ 260.046886] pci 0000:00:02.0: no hotplug settings from platform
> [ 260.046887] pci 0000:00:02.0: using default PCI settings
> [ 260.046978] xen-platform-pci 0000:00:03.0: no hotplug settings from platform
> [ 260.046980] xen-platform-pci 0000:00:03.0: using default PCI settings
> [ 260.047128] pci 0000:00:04.0: no hotplug settings from platform
> [ 260.047174] pps_core: LinuxPPS API ver. 1 registered
> [ 260.047175] pps_core: Software ver. 5.3.6 - Copyright 2005-2007 Rodolfo Giometti <giometti@linux.it>
> [ 260.047859] PTP clock support registered
> [ 260.050174] igb: Intel(R) Gigabit Ethernet Network Driver - version 5.0.5-k
> [ 260.050177] igb: Copyright (c) 2007-2013 Intel Corporation.
> [ 260.050297] igb 0000:00:04.0: enabling device (0000 -> 0002)
> [ 260.139424] xen: --> pirq=17 -> irq=33 (gsi=33)
> [ 260.265105] igb 0000:00:04.0: added PHC on eth1
> [ 260.265107] igb 0000:00:04.0: Intel(R) Gigabit Ethernet Network Connection
> [ 260.265109] igb 0000:00:04.0: eth1: (PCIe:5.0Gb/s:Width x4) a0:36:9f:00:29:dd
> [ 260.265707] igb 0000:00:04.0: eth1: PBA No: G13159-006
> [ 260.265708] igb 0000:00:04.0: Using MSI-X interrupts. 4 rx queue(s), 4 tx queue(s)
> [ 260.280260] systemd-udevd[2260]: renamed network interface eth1 to eth6
>
> And After I attach the second igb PF interface the kernel crashes
>
> [ 546.765926] igb 0000:00:05.0: Xen PCI frontend has not registered MSI/MSI-X support!
.. and this means it cannot get interrupts.
What does your /var/log/xen/qemu-<guest> look like? Or 'xl dmesg'?
Do you see anything about not enough GSI?
> [ 546.769763] BUG: unable to handle kernel NULL pointer dereference at 0000000000000038
> [ 546.773715] IP: [<ffffffffa038e31b>] igb_reset_q_vector+0xb/0x50 [igb]
Ha! Looks like the igb driver is not able to deal with errors
when it cannot allocate all of the interrupts.
One workaround might be to use (on Xen command line):
'extra_guest_irqs=256'
There are some fixes (if this is the problem I think it is) that I
can suggest.
But before I pooint you that way please provide what I had asked
above - thanks!
> [ 546.773720] PGD 0
> [ 546.773720] Oops: 0000 [#1] SMP
> [ 546.773720] Modules linked in: igb ptp pps_core dca i2c_algo_bit nfsd auth_rpcgss oid_registry nfs_acl nfs lockd fscache sunrpc loop fuse crct10dif_pclmul crct10dif_common crc32_pclmul crc32c_intel ghash_clmulni_intel ppdev aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper parport_pc cryptd joydev parport ttm drm_kms_helper drm psmouse processor i2c_piix4 i2c_core pcspkr serio_raw thermal_sys evdev button ext4 crc16 mbcache jbd2 sr_mod cdrom ata_generic xen_netfront xen_blkfront ata_piix libata scsi_mod floppy
> [ 546.773720] CPU: 0 PID: 53 Comm: kworker/u256:1 Not tainted 3.14-1-amd64 #1 Debian 3.14.2-1
> [ 546.773720] Hardware name: Xen HVM domU, BIOS 4.3.0 12/04/2013
> [ 546.773720] Workqueue: kacpi_hotplug acpi_hotplug_work_fn
> [ 546.773720] task: ffff880036ed55b0 ti: ffff880000054000 task.ti: ffff880000054000
> [ 546.773720] RIP: 0010:[<ffffffffa038e31b>] [<ffffffffa038e31b>] igb_reset_q_vector+0xb/0x50 [igb]
> [ 546.773720] RSP: 0018:ffff880000055a90 EFLAGS: 00010206
> [ 546.773720] RAX: 0000000000000000 RBX: 0000000000000006 RCX: ffff880119e41200
> [ 546.773720] RDX: ffff88001712b858 RSI: 0000000000000007 RDI: ffff88009a48a880
> [ 546.773720] RBP: ffff88009a48a880 R08: 0000000000000004 R09: 0000000000000008
> [ 546.773720] R10: ffffffff818a6d40 R11: 0000000000000fd4 R12: 0000000000000008
> [ 546.773720] R13: 0000000000000000 R14: ffff88001712b000 R15: ffff88001712b000
> [ 546.773720] FS: 0000000000000000(0000) GS:ffff88009b400000(0000) knlGS:0000000000000000
> [ 546.773720] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 546.773720] CR2: 0000000000000038 CR3: 0000000098af2000 CR4: 00000000001406f0
> [ 546.773720] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 546.773720] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [ 546.773720] Stack:
> [ 546.773720] ffffffffa038e40d ffff88009a48a880 ffff88009a48a880 ffff88001712b000
> [ 546.773720] ffffffffa0391588 ffff88009a48a000 ffff88009a48a880 0000000000000000
> [ 546.773720] ffff88001712b000 ffff88009a48a000 ffff88009a48a880 ffff88001712b000
> [ 546.773720] Call Trace:
> [ 546.773720] [<ffffffffa038e40d>] ? igb_reset_interrupt_capability+0x3d/0x70 [igb]
> [ 546.773720] [<ffffffffa0391588>] ? igb_init_interrupt_scheme+0x3d8/0x3f0 [igb]
> [ 546.773720] [<ffffffffa0396b23>] ? igb_sriov_reinit+0x53/0xb0 [igb]
> [ 546.773720] [<ffffffffa0396c96>] ? igb_pci_enable_sriov+0x116/0x1e0 [igb]
> [ 546.773720] [<ffffffffa0397257>] ? igb_probe+0x4c7/0x11e0 [igb]
> [ 546.773720] [<ffffffff812b02da>] ? local_pci_probe+0x3a/0xa0
> [ 546.773720] [<ffffffff812b15ea>] ? pci_device_probe+0xca/0x120
> [ 546.773720] [<ffffffff81365d26>] ? driver_probe_device+0x76/0x220
> [ 546.773720] [<ffffffff81365ed0>] ? driver_probe_device+0x220/0x220
> [ 546.773720] [<ffffffff81364163>] ? bus_for_each_drv+0x53/0x90
> [ 546.773720] [<ffffffff81365c70>] ? device_attach+0x90/0xb0
> [ 546.773720] [<ffffffff812a8474>] ? pci_bus_add_device+0x34/0x60
> [ 546.773720] [<ffffffff812a8859>] ? pci_bus_add_devices+0x39/0xa0
> [ 546.773720] [<ffffffff814b0898>] ? enable_slot+0x278/0x2f0
> [ 546.773720] [<ffffffff812c6588>] ? acpiphp_check_bridge.part.6+0xf8/0x110
> [ 546.773720] [<ffffffff812c6d62>] ? hotplug_event+0x162/0x190
> [ 546.773720] [<ffffffff812c6db0>] ? hotplug_event_work+0x20/0x60
> [ 546.773720] [<ffffffff812e75d1>] ? acpi_hotplug_work_fn+0x13/0x1c
> [ 546.773720] [<ffffffff81078b9d>] ? process_one_work+0x16d/0x420
> [ 546.773720] [<ffffffff81079756>] ? worker_thread+0x116/0x3b0
> [ 546.773720] [<ffffffff81079640>] ? rescuer_thread+0x330/0x330
> [ 546.773720] [<ffffffff8107f911>] ? kthread+0xc1/0xe0
> [ 546.773720] [<ffffffff8107f850>] ? kthread_create_on_node+0x180/0x180
> [ 546.773720] [<ffffffff814c718c>] ? ret_from_fork+0x7c/0xb0
> [ 546.773720] [<ffffffff8107f850>] ? kthread_create_on_node+0x180/0x180
> [ 546.773720] Code: 44 09 c8 eb cf 0f 1f 00 b8 a1 ff ff ff c3 66 90 b8 ea ff ff ff c3 66 2e 0f 1f 84 00 00 00 00 00 48 63 f6 48 8b 84 f7 48 0f 00 00 <48> 8b 50 38 48 85 d2 74 10 0f b6 52 4e 48 c7 84 d7 80 02 00 00
> [ 546.773720] RIP [<ffffffffa038e31b>] igb_reset_q_vector+0xb/0x50 [igb]
> [ 546.773720] RSP <ffff880000055a90>
> [ 546.773720] CR2: 0000000000000038
> [ 546.773720] ---[ end trace 8b1d98daee2f26b8 ]---
> [ 546.926235] BUG: unable to handle kernel paging request at ffffffffffffffd8
> [ 546.928690] IP: [<ffffffff8107fe77>] kthread_data+0x7/0x10
> [ 546.930199] PGD 180f067 PUD 1811067 PMD 0
> [ 546.930199] Oops: 0000 [#2] SMP
> [ 546.930199] Modules linked in: igb ptp pps_core dca i2c_algo_bit nfsd auth_rpcgss oid_registry nfs_acl nfs lockd fscache sunrpc loop fuse crct10dif_pclmul crct10dif_common crc32_pclmul crc32c_intel ghash_clmulni_intel ppdev aesni_intel aes_x86_64 lrw gf128mul glue_helper ablk_helper parport_pc cryptd joydev parport ttm drm_kms_helper drm psmouse processor i2c_piix4 i2c_core pcspkr serio_raw thermal_sys evdev button ext4 crc16 mbcache jbd2 sr_mod cdrom ata_generic xen_netfront xen_blkfront ata_piix libata scsi_mod floppy
> [ 546.930199] CPU: 0 PID: 53 Comm: kworker/u256:1 Tainted: G D 3.14-1-amd64 #1 Debian 3.14.2-1
> [ 546.930199] Hardware name: Xen HVM domU, BIOS 4.3.0 12/04/2013
> [ 546.930199] task: ffff880036ed55b0 ti: ffff880000054000 task.ti: ffff880000054000
> [ 546.930199] RIP: 0010:[<ffffffff8107fe77>] [<ffffffff8107fe77>] kthread_data+0x7/0x10
> [ 546.930199] RSP: 0018:ffff8800000557b0 EFLAGS: 00010002
> [ 546.930199] RAX: 0000000000000000 RBX: 0000000000000000 RCX: 000000000000000f
> [ 546.930199] RDX: 0000000000000000 RSI: 0000000000000000 RDI: ffff880036ed55b0
> [ 546.930199] RBP: ffff880036ed55b0 R08: 0000000000000001 R09: 000000000000ba9a
> [ 546.930199] R10: ffffffff81a90a60 R11: 0000000000000000 R12: ffff88009b414380
> [ 546.930199] R13: 0000000000000000 R14: ffff880036ed55a0 R15: ffff880036ed55b0
> [ 546.930199] FS: 0000000000000000(0000) GS:ffff88009b400000(0000) knlGS:0000000000000000
> [ 546.930199] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
> [ 546.930199] CR2: 0000000000000028 CR3: 0000000098af2000 CR4: 00000000001406f0
> [ 546.930199] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> [ 546.930199] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
> [ 546.930199] Stack:
> [ 546.930199] ffffffff81079d68 ffff880036ed59d0 ffffffff814bbe1d 0000000000014380
> [ 546.930199] 0000000000014380 ffff880000055fd8 ffff880036ed55b0 ffff880036ed5bf0
> [ 546.930199] ffff880036ed58c8 ffff880036ed55a0 ffff88009eaf51b0 ffff880036ed55a0
> [ 546.930199] Call Trace:
> [ 546.930199] [<ffffffff81079d68>] ? wq_worker_sleeping+0x8/0x80
> [ 546.930199] [<ffffffff814bbe1d>] ? __schedule+0x45d/0x6e0
> [ 546.930199] [<ffffffff810617e7>] ? do_exit+0x6e7/0xa60
> [ 546.930199] [<ffffffff814c0282>] ? oops_end+0x92/0xd0
> [ 546.930199] [<ffffffff814b67a5>] ? no_context+0x272/0x27e
> [ 546.930199] [<ffffffff814c26d0>] ? __do_page_fault+0x80/0x4f0
> [ 546.930199] [<ffffffff8101257b>] ? __switch_to+0x11b/0x4b0
> [ 546.930199] [<ffffffff8113e45c>] ? pcpu_free_area+0x12c/0x1a0
> [ 546.930199] [<ffffffff8113e74a>] ? free_percpu+0x7a/0x140
> [ 546.930199] [<ffffffff814bf788>] ? page_fault+0x28/0x30
> [ 546.930199] [<ffffffffa038e31b>] ? igb_reset_q_vector+0xb/0x50 [igb]
> [ 546.930199] [<ffffffffa038e40d>] ? igb_reset_interrupt_capability+0x3d/0x70 [igb]
> [ 546.930199] [<ffffffffa0391588>] ? igb_init_interrupt_scheme+0x3d8/0x3f0 [igb]
> [ 546.930199] [<ffffffffa0396b23>] ? igb_sriov_reinit+0x53/0xb0 [igb]
> [ 546.930199] [<ffffffffa0396c96>] ? igb_pci_enable_sriov+0x116/0x1e0 [igb]
> [ 546.930199] [<ffffffffa0397257>] ? igb_probe+0x4c7/0x11e0 [igb]
> [ 546.930199] [<ffffffff812b02da>] ? local_pci_probe+0x3a/0xa0
> [ 546.930199] [<ffffffff812b15ea>] ? pci_device_probe+0xca/0x120
> [ 546.930199] [<ffffffff81365d26>] ? driver_probe_device+0x76/0x220
> [ 546.930199] [<ffffffff81365ed0>] ? driver_probe_device+0x220/0x220
> [ 546.930199] [<ffffffff81364163>] ? bus_for_each_drv+0x53/0x90
> [ 546.930199] [<ffffffff81365c70>] ? device_attach+0x90/0xb0
> [ 546.930199] [<ffffffff812a8474>] ? pci_bus_add_device+0x34/0x60
> [ 546.930199] [<ffffffff812a8859>] ? pci_bus_add_devices+0x39/0xa0
> [ 546.930199] [<ffffffff814b0898>] ? enable_slot+0x278/0x2f0
> [ 546.930199] [<ffffffff812c6588>] ? acpiphp_check_bridge.part.6+0xf8/0x110
> [ 546.930199] [<ffffffff812c6d62>] ? hotplug_event+0x162/0x190
> [ 546.930199] [<ffffffff812c6db0>] ? hotplug_event_work+0x20/0x60
> [ 546.930199] [<ffffffff812e75d1>] ? acpi_hotplug_work_fn+0x13/0x1c
> [ 546.930199] [<ffffffff81078b9d>] ? process_one_work+0x16d/0x420
> [ 546.930199] [<ffffffff81079756>] ? worker_thread+0x116/0x3b0
> [ 546.930199] [<ffffffff81079640>] ? rescuer_thread+0x330/0x330
> [ 546.930199] [<ffffffff8107f911>] ? kthread+0xc1/0xe0
> [ 546.930199] [<ffffffff8107f850>] ? kthread_create_on_node+0x180/0x180
> [ 546.930199] [<ffffffff814c718c>] ? ret_from_fork+0x7c/0xb0
> [ 546.930199] [<ffffffff8107f850>] ? kthread_create_on_node+0x180/0x180
> [ 546.930199] Code: 00 00 00 00 65 48 8b 04 25 80 c8 00 00 48 8b 80 c8 03 00 00 48 8b 40 c8 48 c1 e8 02 83 e0 01 c3 0f 1f 40 00 48 8b 87 c8 03 00 00 <48> 8b 40 d8 c3 0f 1f 40 00 48 83 ec 08 48 8b b7 c8 03 00 00 ba
> [ 546.930199] RIP [<ffffffff8107fe77>] kthread_data+0x7/0x10
> [ 546.930199] RSP <ffff8800000557b0>
> [ 546.930199] CR2: ffffffffffffffd8
> [ 546.930199] ---[ end trace 8b1d98daee2f26b9 ]---
> [ 546.930199] Fixing recursive fault but reboot is needed!
>
>
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xen.org
> http://lists.xen.org/xen-devel
next prev parent reply other threads:[~2014-05-07 17:30 UTC|newest]
Thread overview: 3+ messages / expand[flat|nested] mbox.gz Atom feed top
2014-05-07 16:31 HVM crashes when passing second igb interface Samuel Monderer
2014-05-07 17:30 ` Konrad Rzeszutek Wilk [this message]
2014-05-08 9:06 ` Samuel Monderer
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20140507173015.GC14576@phenom.dumpdata.com \
--to=konrad.wilk@oracle.com \
--cc=SMonderer@advaoptical.com \
--cc=xen-devel@lists.xen.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.