From mboxrd@z Thu Jan  1 00:00:00 1970
From: Arnd Hannemann <hannemann@nets.rwth-aachen.de>
Subject: xen dom0 2.6.32.15 kernel BUG at
	drivers/xen/grant-table.c:583
Date: Mon, 14 Jun 2010 09:53:36 +0200
Message-ID: <4C15E000.7060509@nets.rwth-aachen.de>
Mime-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-15
Content-Transfer-Encoding: 7BIT
Return-path: <xen-devel-bounces@lists.xensource.com>
List-Unsubscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xensource.com?subject=unsubscribe>
List-Post: <mailto:xen-devel@lists.xensource.com>
List-Help: <mailto:xen-devel-request@lists.xensource.com?subject=help>
List-Subscribe: <http://lists.xensource.com/mailman/listinfo/xen-devel>,
	<mailto:xen-devel-request@lists.xensource.com?subject=subscribe>
Sender: xen-devel-bounces@lists.xensource.com
Errors-To: xen-devel-bounces@lists.xensource.com
To: "xen-devel@lists.xensource.com" <xen-devel@lists.xensource.com>
List-Id: xen-devel@lists.xenproject.org

Hi,

we have regular but hard to reproduce (wait for a day or two starting domUs) kernel panics (see below) with latest
"xen/stable-2.6.32.x" git tree.

Any idea, anyone?

Best regards,
Arnd

Jun 11 20:03:16 vmhost1 logger: /etc/xen/scripts/vif-bridge: Successful vif-bridge online for vif861.1, bridge br0.
Jun 11 20:03:16 vmhost1 logger: /etc/xen/scripts/vif-bridge: Writing backend/vif/861/1/hotplug-status connected to xenstore.
Jun 11 20:03:16 vmhost1 [21717.809194] ------------[ cut here ]------------
Jun 11 20:03:16 vmhost1 [21717.809227] kernel BUG at drivers/xen/grant-table.c:583!
Jun 11 20:03:16 vmhost1 [21717.809251] invalid opcode: 0000 [#1]
Jun 11 20:03:16 vmhost1 SMP
Jun 11 20:03:16 vmhost1
Jun 11 20:03:16 vmhost1 [21717.809295] last sysfs file: /sys/devices/virtual/net/br0/bridge/topology_change_detected
Jun 11 20:03:16 vmhost1 [21717.809324] Modules linked in:
Jun 11 20:03:16 vmhost1 netconsole
Jun 11 20:03:16 vmhost1 raid0
Jun 11 20:03:16 vmhost1 md_mod
Jun 11 20:03:16 vmhost1 rtc_cmos
Jun 11 20:03:16 vmhost1 rtc_core
Jun 11 20:03:16 vmhost1 rtc_lib
Jun 11 20:03:16 vmhost1 thermal
Jun 11 20:03:16 vmhost1 processor
Jun 11 20:03:16 vmhost1 pl2303
Jun 11 20:03:16 vmhost1 thermal_sys
Jun 11 20:03:16 vmhost1 button
Jun 11 20:03:16 vmhost1 usbserial
Jun 11 20:03:16 vmhost1 acpi_processor
Jun 11 20:03:16 vmhost1 hwmon
Jun 11 20:03:16 vmhost1 sr_mod
Jun 11 20:03:16 vmhost1 cdrom
Jun 11 20:03:16 vmhost1 ipv6
Jun 11 20:03:16 vmhost1 evdev
Jun 11 20:03:16 vmhost1
Jun 11 20:03:16 vmhost1 [21717.809575]
Jun 11 20:03:16 vmhost1 [21717.809593] Pid: 0, comm: swapper Not tainted (2.6.32.15-xen4.0.0-dom0 #1) System Product Name
Jun 11 20:03:16 vmhost1 [21717.809624] EIP: 0061:[<c120f170>] EFLAGS: 00010282 CPU: 0
Jun 11 20:03:16 vmhost1 [21717.809652] EIP is at gnttab_copy_grant_page+0x1f0/0x260
Jun 11 20:03:16 vmhost1 [21717.809675] EAX: ffffffea EBX: c153be84 ECX: 00000001 EDX: 00000000
Jun 11 20:03:16 vmhost1 [21717.809700] ESI: 00007ff0 EDI: 0000003e EBP: c290dde0 ESP: c153be50
Jun 11 20:03:16 vmhost1 [21717.809726]  DS: 007b ES: 007b FS: 00d8 GS: 0000 SS: 0069
Jun 11 20:03:16 vmhost1 [21717.809750] Process swapper (pid: 0, ti=c153a000 task=c1543760 task.ti=c153a000)
Jun 11 20:03:16 vmhost1 [21717.809777] Stack:
Jun 11 20:03:16 vmhost1 [21717.809793]  00000000
Jun 11 20:03:16 vmhost1 0021412d
Jun 11 20:03:16 vmhost1 c28f02e0
Jun 11 20:03:16 vmhost1 0002c1ef
Jun 11 20:03:16 vmhost1 ec1ef000
Jun 11 20:03:16 vmhost1 ecf85adc
Jun 11 20:03:16 vmhost1 0000003e
Jun 11 20:03:16 vmhost1 ec1ef000
Jun 11 20:03:16 vmhost1
Jun 11 20:03:16 vmhost1 [21717.809917] <0>
Jun 11 20:03:16 vmhost1 00000000
Jun 11 20:03:16 vmhost1 eb317000
Jun 11 20:03:16 vmhost1 00000000
Jun 11 20:03:16 vmhost1 0000003e
Jun 11 20:03:16 vmhost1 eb2a0000
Jun 11 20:03:16 vmhost1 1412d001
Jun 11 20:03:16 vmhost1 00000000
Jun 11 20:03:16 vmhost1 0002c1ef
Jun 11 20:03:16 vmhost1
Jun 11 20:03:16 vmhost1 [21717.810059] <0>
Jun 11 20:03:16 vmhost1 00000000
Jun 11 20:03:16 vmhost1 c1627f54
Jun 11 20:03:16 vmhost1 c155b97
Jun 11 20:03:16 vmhost1 8
Jun 11 20:03:16 vmhost1 c1627f54
Jun 11 20:03:16 vmhost1 01c5b765
Jun 11 20:03:16 vmhost1 c121fff1
Jun 11 20:03:16 vmhost1 c1006a98
Jun 11 20:03:16 vmhost1 c3b90240
Jun 11 20:03:16 vmhost1
Jun 11 20:03:16 vmhost1 [21717.810146] Call Trace:
Jun 11 20:03:16 vmhost1 [21717.810168]  [<c121fff1>] ? net_tx_action+0x1d1/0x9b0
Jun 11 20:03:16 vmhost1 [21717.810197]  [<c1006a98>] ? check_events+0x8/0xc
Jun 11 20:03:16 vmhost1 [21717.810224]  [<c1040c1d>] ? run_timer_softirq+0x15d/0x1e0
Jun 11 20:03:16 vmhost1 [21717.810252]  [<c103bc2e>] ? tasklet_action+0x9e/0xb0
Jun 11 20:03:16 vmhost1 [21717.810278]  [<c103c378>] ? __do_softirq+0x88/0x110
Jun 11 20:03:16 vmhost1 [21717.810304]  [<c1210057>] ? __xen_evtchn_do_upcall+0xd7/0x160
Jun 11 20:03:16 vmhost1 [21717.810332]  [<c103c43d>] ? do_softirq+0x3d/0x40
Jun 11 20:03:16 vmhost1 [21717.810357]  [<c121063a>] ? xen_evtchn_do_upcall+0x2a/0x40
Jun 11 20:03:16 vmhost1 [21717.810385]  [<c1009da7>] ? xen_do_upcall+0x7/0xc
Jun 11 20:03:16 vmhost1 [21717.810411]  [<c10013a7>] ? hypercall_page+0x3a7/0x1010
Jun 11 20:03:16 vmhost1 [21717.810442]  [<c10061ef>] ? xen_safe_halt+0xf/0x20
Jun 11 20:03:16 vmhost1 [21717.810474]  [<c100382c>] ? xen_idle+0x1c/0x30
Jun 11 20:03:16 vmhost1 [21717.810501]  [<c10081fa>] ? cpu_idle+0x3a/0x60
Jun 11 20:03:16 vmhost1 [21717.810527]  [<c15787ef>] ? start_kernel+0x2c6/0x2cb
Jun 11 20:03:16 vmhost1 [21717.810559]  [<c1578367>] ? unknown_bootoption+0x0/0x190
Jun 11 20:03:16 vmhost1 [21717.810594]  [<c157b0e6>] ? xen_start_kernel+0x624/0x62c
Jun 11 20:03:16 vmhost1 [21717.810620] Code:
Jun 11 20:03:16 vmhost1 8d
Jun 11 20:03:16 vmhost1 5c
Jun 11 20:03:16 vmhost1 24
Jun 11 20:03:16 vmhost1 34
Jun 11 20:03:16 vmhost1 c1
Jun 11 20:03:16 vmhost1 e0
Jun 11 20:03:16 vmhost1 0c
Jun 11 20:03:16 vmhost1 83
Jun 11 20:03:16 vmhost1 c8
Jun 11 20:03:16 vmhost1 01
Jun 11 20:03:16 vmhost1 89
Jun 11 20:03:16 vmhost1 44
Jun 11 20:03:16 vmhost1 24
Jun 11 20:03:16 vmhost1 34
Jun 11 20:03:16 vmhost1 8b
Jun 11 20:03:16 vmhost1 44
Jun 11 20:03:16 vmhost1 24
Jun 11 20:03:16 vmhost1 0c
Jun 11 20:03:16 vmhost1 c7
Jun 11 20:03:16 vmhost1 44
Jun 11 20:03:16 vmhost1 24
Jun 11 20:03:16 vmhost1 40
Jun 11 20:03:16 vmhost1 00
Jun 11 20:03:16 vmhost1 00
Jun 11 20:03:16 vmhost1 00
Jun 11 20:03:16 vmhost1 00
Jun 11 20:03:16 vmhost1 89
Jun 11 20:03:16 vmhost1 44
Jun 11 20:03:16 vmhost1 24
Jun 11 20:03:16 vmhost1 3c
Jun 11 20:03:16 vmhost1 e8
Jun 11 20:03:16 vmhost1 b8
Jun 11 20:03:16 vmhost1 1e
Jun 11 20:03:16 vmhost1 df
Jun 11 20:03:16 vmhost1 ff
Jun 11 20:03:16 vmhost1 85
Jun 11 20:03:16 vmhost1 c0
Jun 11 20:03:16 vmhost1 0f
Jun 11 20:03:16 vmhost1 84
Jun 11 20:03:16 vmhost1 2c
Jun 11 20:03:16 vmhost1 ff
Jun 11 20:03:16 vmhost1 ff
Jun 11 20:03:16 vmhost1 ff
Jun 11 18:03:15 vmhost1 unparseable log message: "<0f> "
Jun 11 20:03:16 vmhost1 0b
Jun 11 20:03:16 vmhost1 eb
Jun 11 20:03:16 vmhost1 fe
Jun 11 20:03:16 vmhost1 0f
Jun 11 20:03:16 vmhost1 0b
Jun 11 20:03:16 vmhost1 eb
Jun 11 20:03:16 vmhost1 fe
Jun 11 20:03:16 vmhost1 0f
Jun 11 20:03:16 vmhost1 0b
Jun 11 20:03:16 vmhost1 eb
Jun 11 20:03:16 vmhost1 fe
Jun 11 20:03:16 vmhost1 8b
Jun 11 20:03:16 vmhost1 54
Jun 11 20:03:16 vmhost1 24
Jun 11 20:03:16 vmhost1 04
Jun 11 20:03:16 vmhost1 8b
Jun 11 20:03:16 vmhost1 44
Jun 11 20:03:16 vmhost1 24
Jun 11 20:03:16 vmhost1 0c
Jun 11 20:03:16 vmhost1 e8
Jun 11 20:03:16 vmhost1
Jun 11 20:03:16 vmhost1 [21717.811364] EIP: [<c120f170>]
Jun 11 20:03:16 vmhost1 gnttab_copy_grant_page+0x1f0/0x260
Jun 11 20:03:16 vmhost1 SS:ESP 0069:c153be50
Jun 11 20:03:16 vmhost1 [21717.811821] ---[ end trace 763de4792c00b5c0 ]---
Jun 11 20:03:16 vmhost1 [21717.811962] Kernel panic - not syncing: Fatal exception in interrupt
Jun 11 20:03:16 vmhost1 [21717.812043] Pid: 0, comm: swapper Tainted: G      D    2.6.32.15-xen4.0.0-dom0 #1
Jun 11 20:03:16 vmhost1 [21717.812192] Call Trace:
Jun 11 20:03:16 vmhost1 [21717.812276]  [<c141d2e5>] ? panic+0x42/0xe1
Jun 11 20:03:16 vmhost1 [21717.812350]  [<c100cc56>] ? oops_end+0x96/0xa0
Jun 11 20:03:16 vmhost1 [21717.812422]  [<c100a73f>] ? do_invalid_op+0x7f/0x90
Jun 11 20:03:16 vmhost1 [21717.812496]  [<c120f170>] ? gnttab_copy_grant_page+0x1f0/0x260
Jun 11 20:03:16 vmhost1 [21717.812574]  [<c10741e4>] ? __alloc_pages_nodemask+0xe4/0x5b0
Jun 11 20:03:16 vmhost1 [21717.812650]  [<c1006197>] ? xen_force_evtchn_callback+0x17/0x30
Jun 11 20:03:16 vmhost1 [21717.812726]  [<c1006a98>] ? check_events+0x8/0xc
Jun 11 20:03:16 vmhost1 [21717.812799]  [<c141f9d6>] ? error_code+0x66/0x6c
Jun 11 20:03:16 vmhost1 [21717.812872]  [<c100a6c0>] ? do_invalid_op+0x0/0x90
Jun 11 20:03:16 vmhost1 [21717.812945]  [<c120f170>] ? gnttab_copy_grant_page+0x1f0/0x260
Jun 11 20:03:16 vmhost1 [21717.813022]  [<c121fff1>] ? net_tx_action+0x1d1/0x9b0
Jun 11 20:03:16 vmhost1 [21717.813108]  [<c1006a98>] ? check_events+0x8/0xc
Jun 11 20:03:16 vmhost1 [21717.813185]  [<c1040c1d>] ? run_timer_softirq+0x15d/0x1e0
Jun 11 20:03:16 vmhost1 [21717.813260]  [<c103bc2e>] ? tasklet_action+0x9e/0xb0
Jun 11 20:03:16 vmhost1 [21717.813333]  [<c103c378>] ? __do_softirq+0x88/0x110
Jun 11 20:03:16 vmhost1 [21717.813407]  [<c1210057>] ? __xen_evtchn_do_upcall+0xd7/0x160
Jun 11 20:03:16 vmhost1 [21717.813483]  [<c103c43d>] ? do_softirq+0x3d/0x40
Jun 11 20:03:16 vmhost1 [21717.813556]  [<c121063a>] ? xen_evtchn_do_upcall+0x2a/0x40
Jun 11 20:03:16 vmhost1 [21717.813631]  [<c1009da7>] ? xen_do_upcall+0x7/0xc
Jun 11 20:03:16 vmhost1 [21717.813704]  [<c10013a7>] ? hypercall_page+0x3a7/0x1010
Jun 11 20:03:16 vmhost1 [21717.813779]  [<c10061ef>] ? xen_safe_halt+0xf/0x20
Jun 11 20:03:16 vmhost1 [21717.813852]  [<c100382c>] ? xen_idle+0x1c/0x30
Jun 11 20:03:16 vmhost1 [21717.813939]  [<c10081fa>] ? cpu_idle+0x3a/0x60
Jun 11 20:03:16 vmhost1 [21717.814024]  [<c15787ef>] ? start_kernel+0x2c6/0x2cb
Jun 11 20:03:16 vmhost1 [21717.814101]  [<c1578367>] ? unknown_bootoption+0x0/0x190
Jun 11 20:03:16 vmhost1 [21717.814181]  [<c157b0e6>] ? xen_start_kernel+0x624/0x62c