From mboxrd@z Thu Jan 1 00:00:00 1970 From: Konrad Rzeszutek Wilk Subject: Re: Kernel bug from 3.0 (was phy disks and vifs timing out in DomU) Date: Wed, 3 Aug 2011 11:28:41 -0400 Message-ID: <20110803152841.GA2860@dumpdata.com> References: <29902981.10.1311837224851.JavaMail.root@zimbra.overnetdata.com> <24093349.14.1311837878822.JavaMail.root@zimbra.overnetdata.com> <4E31820C.5030200@overnetdata.com> <1311870512.24408.153.camel@cthulhu.hellion.org.uk> <4E3266DE.9000606@overnetdata.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Return-path: Content-Disposition: inline In-Reply-To: <4E3266DE.9000606@overnetdata.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xensource.com Errors-To: xen-devel-bounces@lists.xensource.com To: Anthony Wright , Ian Campbell Cc: Todd Deshane , "xen-devel@lists.xensource.com" List-Id: xen-devel@lists.xenproject.org On Fri, Jul 29, 2011 at 08:53:02AM +0100, Anthony Wright wrote: > I've just upgraded to xen 4.1.1 with a stock 3.0 kernel on dom0 (with > the vga-support patch backported). I can't get my DomU's to work due to > the phy disks and vifs timing out in DomU and looking through my logs > this morning I'm getting a consistent kernel bug report with xen > mentioned at the top of the stack trace and vifdisconnect mentioned on Yikes! Ian any ideas what to try? Anthony, can you compile the kernel with debug=y and when this happens see what 'xl dmesg' gives? Also there is also the 'xl debug-keys g' which should dump the grants in use.. that might help a bit. > 2011 Jul 29 07:18:50 kernel: [ 33.213500] vif vif-1-0: vif1.0: failed to map tx ring. err=-12 status=-1 > 2011 Jul 29 07:18:50 kernel: [ 33.213516] vif vif-1-0: 1 mapping shared-frames 768/769 port 12 > 2011 Jul 29 07:19:01 /usr/sbin/cron[3719]: (root) CMD (/usr/monitor/monitor) > 2011 Jul 29 07:19:23 kernel: [ 66.043164] vif vif-1-0: 2 reading script > 2011 Jul 29 07:19:23 kernel: [ 66.045984] br-internal: port 1(vif1.0) entering disabled state > 2011 Jul 29 07:19:23 kernel: [ 66.046044] libfcoe_device_notification: NETDEV_UNREGISTER vif1.0 > 2011 Jul 29 07:19:23 kernel: [ 66.046082] br-internal: port 1(vif1.0) entering disabled state > 2011 Jul 29 07:19:23 kernel: [ 66.046279] br-internal: mixed no checksumming and other settings. > 2011 Jul 29 07:19:23 kernel: [ 66.050077] ------------[ cut here ]------------ > 2011 Jul 29 07:19:23 kernel: [ 66.050858] kernel BUG at mm/vmalloc.c:2164! > 2011 Jul 29 07:19:23 kernel: [ 66.051034] invalid opcode: 0000 [#1] SMP > 2011 Jul 29 07:19:23 kernel: [ 66.051034] Modules linked in: > 2011 Jul 29 07:19:23 kernel: [ 66.051034] > 2011 Jul 29 07:19:23 kernel: [ 66.051034] Pid: 20, comm: xenwatch Not tainted 3.0.0 #1 MSI MS-7309/MS-7309 > 2011 Jul 29 07:19:23 kernel: [ 66.051034] EIP: 0061:[] EFLAGS: 00010207 CPU: 1 > 2011 Jul 29 07:19:23 kernel: [ 66.051034] EIP is at free_vm_area+0xf/0x19 > 2011 Jul 29 07:19:23 kernel: [ 66.051034] EAX: 00000000 EBX: d0799700 ECX: 00000018 EDX: 00000000 > 2011 Jul 29 07:19:23 kernel: [ 66.051034] ESI: cf9e5800 EDI: d051a600 EBP: cf9e5c00 ESP: d0ce7eb4 > 2011 Jul 29 07:19:23 kernel: [ 66.051034] DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069 > 2011 Jul 29 07:19:23 kernel: [ 66.051034] Process xenwatch (pid: 20, ti=d0ce6000 task=d0c55140 task.ti=d0ce6000) > 2011 Jul 29 07:19:23 kernel: [ 66.051034] Stack: > 2011 Jul 29 07:19:23 kernel: [ 66.051034] cf9e5c00 c09e87aa fffc6e23 c0c4bd65 d0ce7ecc cf9e5844 d0ce7ecc d0ce7ecc > 2011 Jul 29 07:19:23 kernel: [ 66.051034] cf9e5c00 cf9e5800 d051a600 cf9e5c94 c09eace0 cffdbfe0 00000000 fffffffe > 2011 Jul 29 07:19:23 kernel: [ 66.051034] d0ce7f9c c061fe74 cffdbe60 d051a620 d051a600 d0ce7f9c c09e9f8c d051a600 > 2011 Jul 29 07:19:23 kernel: [ 66.051034] Call Trace: > 2011 Jul 29 07:19:23 kernel: [ 66.051034] [] ? xen_netbk_unmap_frontend_rings+0xbf/0xd3 > 2011 Jul 29 07:19:23 kernel: [ 66.051034] [] ? netdev_run_todo+0x1b7/0x1cc > 2011 Jul 29 07:19:23 kernel: [ 66.051034] [] ? xenvif_disconnect+0xd0/0xe4 > 2011 Jul 29 07:19:23 kernel: [ 66.051034] [] ? xenbus_rm+0x37/0x3e > 2011 Jul 29 07:19:23 kernel: [ 66.051034] [] ? netback_remove+0x40/0x5d > 2011 Jul 29 07:19:23 kernel: [ 66.051034] [] ? xenbus_dev_remove+0x2c/0x3d > 2011 Jul 29 07:19:23 kernel: [ 66.051034] [] ? __device_release_driver+0x42/0x79 > 2011 Jul 29 07:19:23 kernel: [ 66.051034] [] ? device_release_driver+0xf/0x17 > 2011 Jul 29 07:19:23 kernel: [ 66.051034] [] ? bus_remove_device+0x75/0x84 > 2011 Jul 29 07:19:23 kernel: [ 66.051034] [] ? device_del+0xe6/0x125 > 2011 Jul 29 07:19:23 kernel: [ 66.051034] [] ? device_unregister+0x8/0x10 > 2011 Jul 29 07:19:23 kernel: [ 66.051034] [] ? xenbus_dev_changed+0x71/0x129 > 2011 Jul 29 07:19:23 kernel: [ 66.051034] [] ? check_events+0x8/0xc > 2011 Jul 29 07:19:23 kernel: [ 66.051034] [] ? xenwatch_thread+0xeb/0x113 > 2011 Jul 29 07:19:23 kernel: [ 66.051034] [] ? wake_up_bit+0x53/0x53 > 2011 Jul 29 07:19:23 kernel: [ 66.051034] [] ? xenbus_thread+0x1cc/0x1cc > 2011 Jul 29 07:19:23 kernel: [ 66.051034] [] ? kthread+0x63/0x68 > 2011 Jul 29 07:19:23 kernel: [ 66.051034] [] ? kthread_worker_fn+0x122/0x122 > 2011 Jul 29 07:19:23 kernel: [ 66.051034] [] ? kernel_thread_helper+0x6/0x10 > 2011 Jul 29 07:19:23 kernel: [ 66.051034] Code: c1 00 00 00 01 89 f0 e8 a1 ff ff ff 81 6b 08 00 10 00 00 eb 02 31 db 89 d8 5b 5e c3 53 89 c3 8b 40 04 e8 9b ff ff ff 39 d8 74 04 <0f> 0b eb fe 5b e9 73 95 00 00 57 89 d7 56 31 f6 53 89 c3 eb 09 > 2011 Jul 29 07:19:23 kernel: [ 66.051034] EIP: [] free_vm_area+0xf/0x19 SS:ESP 0069:d0ce7eb4 > 2011 Jul 29 07:19:23 kernel: [ 66.051034] ---[ end trace b47a8d30fa29735c ]--- > 2011 Jul 29 07:19:23 logger: /etc/xen/scripts/xen-hotplug-cleanup: XENBUS_PATH=backend/qdisk/1/51714 > 2011 Jul 29 07:19:23 logger: /etc/xen/scripts/xen-hotplug-cleanup: XENBUS_PATH=backend/qdisk/1/51713