All of lore.kernel.org
 help / color / mirror / Atom feed
* RE: xm mem-set causes kernel panic
@ 2006-05-03 20:58 Krysan, Susan
  2006-05-03 21:29 ` Andrew Theurer
  0 siblings, 1 reply; 7+ messages in thread
From: Krysan, Susan @ 2006-05-03 20:58 UTC (permalink / raw)
  To: Ian Pratt, xen-devel
  Cc: Subrahmanian, Raj, Vessey, Bruce A, Puthiyaparambil, Aravindh,
	Carb, Brian A

We did run using unstable. We installed sles10 beta11 but we pulled and built xen-unstable changeset 9903 from xensource repository.  We have not tried with PAE hypervisor.  It is not our focus.  How desperately would you need us to try PAE?

Thanks,

> Sue Krysan
> Linux Systems Group
> Unisys Corporation
> 


-----Original Message-----
From: Ian Pratt [mailto:m+Ian.Pratt@cl.cam.ac.uk]
Sent: Wednesday, May 03, 2006 4:19 PM
To: Krysan, Susan; xen-devel@lists.xensource.com
Cc: Subrahmanian, Raj; Vessey, Bruce A; Puthiyaparambil, Aravindh; Carb,
Brian A; ian.pratt@cl.cam.ac.uk
Subject: RE: [Xen-devel] xm mem-set causes kernel panic


> When I boot the ES7000 with 32 GB of RAM, dom0 is allocated 
> almost all of it.  I issued an xm mem-set 0 512 command to 
> decrease dom0's memory to 512 MB.  Using xentop, I see the 
> amount of memory allocated to dom0 slowly decrease.  The 
> kernel panic occurs when dom0's memory is around 1 GB.  The 
> serial console output is attached.  Using SLES10 Beta 11 
> upgraded to xen changeset 9903.

Can you repro this on -unstable? What about with a PAE hypervisor?

Thanks,
Ian

^ permalink raw reply	[flat|nested] 7+ messages in thread
* RE: xm mem-set causes kernel panic
@ 2006-05-04 15:49 Puthiyaparambil, Aravindh
  0 siblings, 0 replies; 7+ messages in thread
From: Puthiyaparambil, Aravindh @ 2006-05-04 15:49 UTC (permalink / raw)
  To: Keir Fraser, Ian Pratt
  Cc: Carb, Brian A, xen-devel, Krysan, Susan, Subrahmanian, Raj,
	Vessey, Bruce A

> I don't think this behaviour is surprising. The kernel is probably
> taking about 1% of memory to track memory usage. So trying to balloon
> down to 1-2% of your initial memory reservation is rather harsh.

I agree.

> The 'fix' here is to stop users being able to shoot their feet off, by
> setting a lower bound on ballooning that is related to the initial
> memory size of the guest, and/or quiesce the balloon driver when
> low-memory and out-of-memory code paths are executed in the guest.

We will look into this and send in a patch.

Cheers,
Aravindh

^ permalink raw reply	[flat|nested] 7+ messages in thread
* RE: xm mem-set causes kernel panic
@ 2006-05-03 21:15 Ian Pratt
  2006-05-04 10:03 ` Keir Fraser
  0 siblings, 1 reply; 7+ messages in thread
From: Ian Pratt @ 2006-05-03 21:15 UTC (permalink / raw)
  To: Krysan, Susan, xen-devel
  Cc: Subrahmanian, Raj, Vessey, Bruce A, Puthiyaparambil, Aravindh,
	Carb, Brian A

> We did run using unstable. We installed sles10 beta11 but we 
> pulled and built xen-unstable changeset 9903 from xensource 
> repository.  We have not tried with PAE hypervisor.  It is 
> not our focus.  How desperately would you need us to try PAE?

Interesting. If you boot with dom0_mem=512 and then start a 15GB guest,
do you see problems shrinking the guest's memory?

I'd certainly be interested in hearing whether the problem exists on
PAE.

Ian

> -----Original Message-----
> From: Ian Pratt [mailto:m+Ian.Pratt@cl.cam.ac.uk]
> Sent: Wednesday, May 03, 2006 4:19 PM
> To: Krysan, Susan; xen-devel@lists.xensource.com
> Cc: Subrahmanian, Raj; Vessey, Bruce A; Puthiyaparambil, 
> Aravindh; Carb, Brian A; ian.pratt@cl.cam.ac.uk
> Subject: RE: [Xen-devel] xm mem-set causes kernel panic
> 
> 
> > When I boot the ES7000 with 32 GB of RAM, dom0 is allocated 
> almost all 
> > of it.  I issued an xm mem-set 0 512 command to decrease 
> dom0's memory 
> > to 512 MB.  Using xentop, I see the amount of memory 
> allocated to dom0 
> > slowly decrease.  The kernel panic occurs when dom0's 
> memory is around 
> > 1 GB.  The serial console output is attached.  Using SLES10 Beta 11 
> > upgraded to xen changeset 9903.
> 
> Can you repro this on -unstable? What about with a PAE hypervisor?
> 
> Thanks,
> Ian
> 

^ permalink raw reply	[flat|nested] 7+ messages in thread
* RE: xm mem-set causes kernel panic
@ 2006-05-03 20:18 Ian Pratt
  0 siblings, 0 replies; 7+ messages in thread
From: Ian Pratt @ 2006-05-03 20:18 UTC (permalink / raw)
  To: Krysan, Susan, xen-devel
  Cc: Subrahmanian, Raj, Vessey, Bruce A, Puthiyaparambil, Aravindh,
	Carb, Brian A

> When I boot the ES7000 with 32 GB of RAM, dom0 is allocated 
> almost all of it.  I issued an xm mem-set 0 512 command to 
> decrease dom0's memory to 512 MB.  Using xentop, I see the 
> amount of memory allocated to dom0 slowly decrease.  The 
> kernel panic occurs when dom0's memory is around 1 GB.  The 
> serial console output is attached.  Using SLES10 Beta 11 
> upgraded to xen changeset 9903.

Can you repro this on -unstable? What about with a PAE hypervisor?

Thanks,
Ian

^ permalink raw reply	[flat|nested] 7+ messages in thread
* xm mem-set causes kernel panic
@ 2006-05-03 19:01 Krysan, Susan
  0 siblings, 0 replies; 7+ messages in thread
From: Krysan, Susan @ 2006-05-03 19:01 UTC (permalink / raw)
  To: xen-devel
  Cc: Subrahmanian, Raj, Vessey, Bruce A, Puthiyaparambil, Aravindh,
	Carb, Brian A

[-- Attachment #1: Type: text/plain, Size: 483 bytes --]

When I boot the ES7000 with 32 GB of RAM, dom0 is allocated almost all of it.  I issued an xm mem-set 0 512 command to decrease dom0's memory to 512 MB.  Using xentop, I see the amount of memory allocated to dom0 slowly decrease.  The kernel panic occurs when dom0's memory is around 1 GB.  The serial console output is attached.  Using SLES10 Beta 11 upgraded to xen changeset 9903.

 <<serport2.txt>> 
Thanks,
> Sue Krysan
> Linux Systems Group
> Unisys Corporation
> 
> 

[-- Attachment #2: serport2.txt --]
[-- Type: text/plain, Size: 59886 bytes --]

Kernel panic - not syncing: Out of memory and no killable processes...
 (XEN) Domain 0 crashed: rebooting machine in 5 seconds.
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
f801cd57c>{blk_congestion_wait+132} <ffffffff8013fa4d>{autoremove_wake_function+0}
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
       <ffffffff80159715>{try_to_free_pages+342} <ffffffff80154cdf>{__alloc_pages+422}
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
       <ffffffff8023c4c2>{balloon_process+904} <ffffffff8023c13a>{balloon_process+0}
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
       <ffffffff8013bf43>{run_workqueue+153} <ffffffff8013c685>{worker_thread+0}
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
       <ffffffff8013c78e>{worker_thread+265} <ffffffff801236c3>{__wake_up_common+62}
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
       <ffffffff80124910>{default_wake_function+0} <ffffffff8013f913>{kthread+212}
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
       <ffffffff8013c685>{worker_thread+0} <ffffffff8010b8c6>{child_rip+8}
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
       <ffffffff8013c685>{worker_thread+0} <ffffffff8013f83f>{kthread+0}
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
       <ffffffff8010b8be>{child_rip+0}
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
 <0>general protection fault: 0000 [12] SMP 
CPU 15 
Modules linked in: ipv6 bridge button battery ac dm_mod e1000 ext3 jbd reiserfs fan thermal processor sg mptspi mptscsih mptbase piix sd_mod scsi_mod
Pid: 4687, comm: python Tainted: GF     2.6.16-xen #1
RIP: e030:[<ffffffff8011326b>] <ffffffff8011326b>{smp_really_stop_cpu+9}
RSP: e02b:ffff88000491bea8  EFLAGS: 00010002
RAX: ffffffffff5783c0 RBX: 0000000000000000 RCX: 0000000000000001
RDX: ffffffffff578000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 0000000000000000 R08: 0000000000000000 R09: 0000000000000000
R10: 0000000000000020 R11: 0000000000000000 R12: ffffffff80113262
R13: 0000000000000000 R14: ffff8807d087b7d8 R15: ffff8807d087b7d8
FS:  00002aae5150ffc0(0000) GS:ffffffff803fb780(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000
Process python (pid: 4687, threadinfo ffff8807d087a000, task ffff8807d12eb040)
Stack: 0000000000000000 ffffffff801134c8 ffff8807d6a56140 0000000000000000 
       000000000000012e ffffffff8014ee2d ffffffff80405b80 0000000000009700 
       000000000000012e ffff8807d6a56140 
Call Trace: <IRQ> <ffffffff801134c8>{smp_call_function_interrupt+79}
       <ffffffff8014ee2d>{handle_IRQ_event+78} <ffffffff8014ef10>{__do_IRQ+153}
       <ffffffff8010d661>{do_IRQ+60} <ffffffff8023834a>{evtchn_do_upcall+134}
       <ffffffff80113262>{smp_really_stop_cpu+0} <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff80113262>{smp_really_stop_cpu+0} <ffffffff801132d2>{__smp_call_function+100}
       <ffffffff801132d0>{__smp_call_function+98} <ffffffff80113262>{smp_really_stop_cpu+0}
       <ffffffff8011331b>{smp_send_stop+38} <ffffffff8012b885>{panic+141}
       <ffffffff8015909f>{shrink_slab+317} <ffffffff8013038d>{getnstimeofday+16}
       <ffffffff8014b7ec>{cpuset_excl_nodes_overlap+30} <ffffffff80153300>{out_of_memory+276}
       <ffffffff80154d56>{__alloc_pages+541} <ffffffff80109fc0>{__switch_to+740}
       <ffffffff80167143>{read_swap_cache_async+66} <ffffffff8015b333>{swapin_readahead+78}
       <ffffffff802ceb52>{.text.lock.spinlock+103} <ffffffff8015d670>{__handle_mm_fault+2324}
       <ffffffff8016d3cb>{kmem_cache_alloc+20} <ffffffff801180d0>{do_page_fault+1938}
       <ffffffff801248ff>{try_to_wake_up+852} <ffffffff8010b5c7>{error_exit+0}
       <ffffffff801db6e0>{__put_user_4+32} <ffffffff80128e40>{mm_release+102}
       <ffffffff8012d3d2>{exit_mm+22} <ffffffff8012eaa8>{do_exit+520}
       <ffffffff8012f1f3>{sys_exit_group+0} <ffffffff80138379>{get_signal_to_deliver+1652}
       <ffffffff8010a1f7>{do_signal+109} <ffffffff80124910>{default_wake_function+0}
       <ffffffff8010af9d>{sysret_signal+56} <ffffffff8010b271>{ptregscall_common+61}

Code: f4 eb fd 41 55 49 89 fd 48 c7 c7 c8 c9 3c 80 41 54 55 48 89 
RIP <ffffffff8011326b>{smp_really_stop_cpu+9} RSP <ffff88000491bea8>
Badness in do_unblank_screen at drivers/char/vt.c:2830

Call Trace: <IRQ> <ffffffff80228114>{do_unblank_screen+76}
       <ffffffff8011775b>{bust_spinlocks+28} <ffffffff8010bbbd>{oops_end+21}
       <ffffffff8010c25b>{die+58} <ffffffff8010d04a>{do_general_protection+291}
       <ffffffff80113262>{smp_really_stop_cpu+0} <ffffffff8010b5c7>{error_exit+0}
       <ffffffff80113262>{smp_really_stop_cpu+0} <ffffffff8011326b>{smp_really_stop_cpu+9}
       <ffffffff8011326b>{smp_really_stop_cpu+9} <ffffffff801134c8>{smp_call_function_interrupt+79}
       <ffffffff8014ee2d>{handle_IRQ_event+78} <ffffffff8014ef10>{__do_IRQ+153}
       <ffffffff8010d661>{do_IRQ+60} <ffffffff8023834a>{evtchn_do_upcall+134}
       <ffffffff80113262>{smp_really_stop_cpu+0} <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff80113262>{smp_really_stop_cpu+0} <ffffffff801132d2>{__smp_call_function+100}
       <ffffffff801132d0>{__smp_call_function+98} <ffffffff80113262>{smp_really_stop_cpu+0}
       <ffffffff8011331b>{smp_send_stop+38} <ffffffff8012b885>{panic+141}
       <ffffffff8015909f>{shrink_slab+317} <ffffffff8013038d>{getnstimeofday+16}
       <ffffffff8014b7ec>{cpuset_excl_nodes_overlap+30} <ffffffff80153300>{out_of_memory+276}
       <ffffffff80154d56>{__alloc_pages+541} <ffffffff80109fc0>{__switch_to+740}
       <ffffffff80167143>{read_swap_cache_async+66} <ffffffff8015b333>{swapin_readahead+78}
       <ffffffff802ceb52>{.text.lock.spinlock+103} <ffffffff8015d670>{__handle_mm_fault+2324}
       <ffffffff8016d3cb>{kmem_cache_alloc+20} <ffffffff801180d0>{do_page_fault+1938}
       <ffffffff801248ff>{try_to_wake_up+852} <ffffffff8010b5c7>{error_exit+0}
       <ffffffff801db6e0>{__put_user_4+32} <ffffffff80128e40>{mm_release+102}
       <ffffffff8012d3d2>{exit_mm+22} <ffffffff8012eaa8>{do_exit+520}
       <ffffffff8012f1f3>{sys_exit_group+0} <ffffffff80138379>{get_signal_to_deliver+1652}
       <ffffffff8010a1f7>{do_signal+109} <ffffffff80124910>{default_wake_function+0}
       <ffffffff8010af9d>{sysret_signal+56} <ffffffff8010b271>{ptregscall_common+61}
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
 <0>general protection fault: 0000 [13] SMP 
CPU 7 
Modules linked in: ipv6 bridge button battery ac dm_mod e1000 ext3 jbd reiserfs fan thermal processor sg mptspi mptscsih mptbase piix sd_mod scsi_mod
Pid: 5967, comm: gnome-panel Tainted: GF     2.6.16-xen #1
RIP: e030:[<ffffffff8011326b>] <ffffffff8011326b>{smp_really_stop_cpu+9}
RSP: e02b:ffff8807d6a7bea8  EFLAGS: 00010002
RAX: ffffffffff5781c0 RBX: 0000000000000000 RCX: 0000000000000001
RDX: ffffffffff578000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 0000000000000000 R08: ffff880024041368 R09: ffff8807c57e7b48
R10: 0000000000000007 R11: 0000000000000246 R12: ffffffff80113262
R13: 0000000000000000 R14: ffff8807c57e78f8 R15: ffff8807c57e78f8
FS:  00002ae36fe73e30(0000) GS:ffffffff803fb380(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000
Process gnome-panel (pid: 5967, threadinfo ffff8807c57e6000, task ffff8807d4817180)
Stack: 0000000000000000 ffffffff801134c8 ffff880000044380 0000000000000000 
       0000000000000116 ffffffff8014ee2d ffffffff80404f80 0000000000008b00 
       0000000000000116 ffff880000044380 
Call Trace: <IRQ> <ffffffff801134c8>{smp_call_function_interrupt+79}
       <ffffffff8014ee2d>{handle_IRQ_event+78} <ffffffff8014ef10>{__do_IRQ+153}
       <ffffffff8010d661>{do_IRQ+60} <ffffffff8023834a>{evtchn_do_upcall+134}
       <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff8010722a>{hypercall_page+554} <ffffffff8010722a>{hypercall_page+554}
       <ffffffff80237bfd>{force_evtchn_callback+10} <ffffffff801585f6>{shrink_zone+1774}
       <ffffffff802cd0ab>{thread_return+0} <ffffffff80134fb6>{lock_timer_base+27}
       <ffffffff80135028>{try_to_del_timer_sync+81} <ffffffff802cd47c>{io_schedule_timeout+50}
       <ffffffff801cd57c>{blk_congestion_wait+132} <ffffffff8013fa4d>{autoremove_wake_function+0}
       <ffffffff80159715>{try_to_free_pages+342} <ffffffff8015335c>{out_of_memory+368}
       <ffffffff80154cdf>{__alloc_pages+422} <ffffffff8015667c>{__do_page_cache_readahead+245}
       <ffffffff8015016c>{sync_page+0} <ffffffff802cd4b9>{io_schedule+50}
       <ffffffff802cdcfd>{__wait_on_bit_lock+91} <ffffffff80152893>{filemap_nopage+323}
       <ffffffff8015d2e3>{__handle_mm_fault+1415} <ffffffff801812b2>{__user_walk_fd+65}
       <ffffffff801180d0>{do_page_fault+1938} <ffffffff8018aa7b>{mntput_no_expire+25}
       <ffffffff8014dd09>{audit_syscall_exit+747} <ffffffff8010b5c7>{error_exit+0}

Code: f4 eb fd 41 55 49 89 fd 48 c7 c7 c8 c9 3c 80 41 54 55 48 89 
RIP <ffffffff8011326b>{smp_really_stop_cpu+9} RSP <ffff8807d6a7bea8>
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
general protection fault: 0000 [14] SMP  
CPU 11 
Modules linked in: ipv6 bridge button battery ac dm_mod e1000 ext3 jbd reiserfs fan thermal processor sg mptspi mptscsih mptbase piix sd_mod scsi_mod
Pid: 0, comm: swapper Tainted: GF     2.6.16-xen #1
RIP: e030:[<ffffffff8011326b>] <ffffffff8011326b>{smp_really_stop_cpu+9}
RSP: e02b:ffff88002c0cbea8  EFLAGS: 00010002
RAX: ffffffffff5782c0 RBX: 0000000000000000 RCX: 0000000000000001
RDX: ffffffffff578000 RSI: 0000000000000002 RDI: 0000000000000000
RBP: 0000000000000000 R08: 0000000100073353 R09: 000000000000000b
R10: 00000001000733b4 R11: 0000000000000246 R12: ffffffff80113262
R13: 0000000000000000 R14: ffff88000499fe58 R15: ffff88000499fe58
FS:  00002b8d6e4e6ca0(0000) GS:ffffffff803fb580(0000) knlGS:0000000000000000
CS:  e033 DS: 002b ES: 002b
Process swapper (pid: 0, threadinfo ffff88000499e000, task ffff8807d6abb180)
Stack: ffffffff80113262 ffffffff801134c8 ffff8807d6a569c0 0000000000000000 
       0000000000000122 ffffffff8014ee2d ffffffff80405580 0000000000009100 
       0000000000000122 ffff8807d6a569c0 
Call Trace: <IRQ> <ffffffff80113262>{smp_really_stop_cpu+0}
       <ffffffff801134c8>{smp_call_function_interrupt+79} <ffffffff8014ee2d>{handle_IRQ_event+78}
       <ffffffff8014ef10>{__do_IRQ+153} <ffffffff8010d661>{do_IRQ+60}
       <ffffffff8023834a>{evtchn_do_upcall+134} <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff801073aa>{hypercall_page+938} <ffffffff801073aa>{hypercall_page+938}
       <ffffffff801095de>{xen_idle+132} <ffffffff80109933>{cpu_idle+217}

Code: f4 eb fd 41 55 49 89 fd 48 c7 c7 c8 c9 3c 80 41 54 55 48 89 
RIP <ffffffff8011326b>{smp_really_stop_cpu+9} RSP <ffff88002c0cbea8>
Badness in do_unblank_screen at drivers/char/vt.c:2830

Call Trace: <IRQ> <ffffffff80228114>{do_unblank_screen+76}
       <ffffffff8011775b>{bust_spinlocks+28} <ffffffff8010bbbd>{oops_end+21}
       <ffffffff8010c25b>{die+58} <ffffffff8010d04a>{do_general_protection+291}
       <ffffffff80113262>{smp_really_stop_cpu+0} <ffffffff8010b5c7>{error_exit+0}
       <ffffffff80113262>{smp_really_stop_cpu+0} <ffffffff8011326b>{smp_really_stop_cpu+9}
       <ffffffff8011326b>{smp_really_stop_cpu+9} <ffffffff80113262>{smp_really_stop_cpu+0}
       <ffffffff801134c8>{smp_call_function_interrupt+79} <ffffffff8014ee2d>{handle_IRQ_event+78}
       <ffffffff8014ef10>{__do_IRQ+153} <ffffffff8010d661>{do_IRQ+60}
       <ffffffff8023834a>{evtchn_do_upcall+134} <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff801073aa>{hypercall_page+938} <ffffffff801073aa>{hypercall_page+938}
       <ffffffff801095de>{xen_idle+132} <ffffffff80109933>{cpu_idle+217}
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
 <0>general protection fault: 0000 [15] SMP 
CPU 10 
Modules linked in: ipv6 bridge button battery ac dm_mod e1000 ext3 jbd reiserfs fan thermal processor sg mptspi mptscsih mptbase piix sd_mod scsi_mod
Pid: 5972, comm: nautilus Tainted: GF     2.6.16-xen #1
RIP: e030:[<ffffffff8011326b>] <ffffffff8011326b>{smp_really_stop_cpu+9}
RSP: e02b:ffff88002c0bfea8  EFLAGS: 00010002
RAX: ffffffffff578280 RBX: 0000000000000000 RCX: 0000000000000001
RDX: ffffffffff578000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 0000000000000000 R08: ffff8800240db608 R09: ffff8807c6ca1b48
R10: 000000000000001e R11: 0000000000000246 R12: ffffffff80113262
R13: 0000000000000000 R14: ffff8807c6ca18f8 R15: ffff8807c6ca18f8
FS:  00002aede0373370(0000) GS:ffffffff803fb500(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000
Process nautilus (pid: 5972, threadinfo ffff8807c6ca0000, task ffff8807d5120820)
Stack: 0000000000000000 ffffffff801134c8 ffff8807d6a56bc0 0000000000000000 
       000000000000011f ffffffff8014ee2d ffffffff80405400 0000000000008f80 
       000000000000011f ffff8807d6a56bc0 
Call Trace: <IRQ> <ffffffff801134c8>{smp_call_function_interrupt+79}
       <ffffffff8014ee2d>{handle_IRQ_event+78} <ffffffff8014ef10>{__do_IRQ+153}
       <ffffffff8010d661>{do_IRQ+60} <ffffffff8023834a>{evtchn_do_upcall+134}
       <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff8010722a>{hypercall_page+554} <ffffffff8010722a>{hypercall_page+554}
       <ffffffff80237bfd>{force_evtchn_callback+10} <ffffffff801585f6>{shrink_zone+1774}
       <ffffffff802cd0ab>{thread_return+0} <ffffffff80134fb6>{lock_timer_base+27}
       <ffffffff80135028>{try_to_del_timer_sync+81} <ffffffff802cd47c>{io_schedule_timeout+50}
       <ffffffff801cd57c>{blk_congestion_wait+132} <ffffffff8013fa4d>{autoremove_wake_function+0}
       <ffffffff80159715>{try_to_free_pages+342} <ffffffff8015335c>{out_of_memory+368}
       <ffffffff80154cdf>{__alloc_pages+422} <ffffffff8015667c>{__do_page_cache_readahead+245}
       <ffffffff8015016c>{sync_page+0} <ffffffff802cd4b9>{io_schedule+50}
       <ffffffff802cdcfd>{__wait_on_bit_lock+91} <ffffffff80152893>{filemap_nopage+323}
       <ffffffff8015d2e3>{__handle_mm_fault+1415} <ffffffff801812b2>{__user_walk_fd+65}
       <ffffffff801180d0>{do_page_fault+1938} <ffffffff8018aa7b>{mntput_no_expire+25}
       <ffffffff8014dd09>{audit_syscall_exit+747} <ffffffff8010b5c7>{error_exit+0}

Code: f4 eb fd 41 55 49 89 fd 48 c7 c7 c8 c9 3c 80 41 54 55 48 89 
RIP <ffffffff8011326b>{smp_really_stop_cpu+9} RSP <ffff88002c0bfea8>
Badness in do_unblank_screen at drivers/char/vt.c:2830

Call Trace: <IRQ> <ffffffff80228114>{do_unblank_screen+76}
       <ffffffff8011775b>{bust_spinlocks+28} <ffffffff8010bbbd>{oops_end+21}
       <ffffffff8010c25b>{die+58} <ffffffff8010d04a>{do_general_protection+291}
       <ffffffff80113262>{smp_really_stop_cpu+0} <ffffffff8010b5c7>{error_exit+0}
       <ffffffff80113262>{smp_really_stop_cpu+0} <ffffffff8011326b>{smp_really_stop_cpu+9}
       <ffffffff8011326b>{smp_really_stop_cpu+9} <ffffffff801134c8>{smp_call_function_interrupt+79}
       <ffffffff8014ee2d>{handle_IRQ_event+78} <ffffffff8014ef10>{__do_IRQ+153}
       <ffffffff8010d661>{do_IRQ+60} <ffffffff8023834a>{evtchn_do_upcall+134}
       <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff8010722a>{hypercall_page+554} <ffffffff8010722a>{hypercall_page+554}
       <ffffffff80237bfd>{force_evtchn_callback+10} <ffffffff801585f6>{shrink_zone+1774}
       <ffffffff802cd0ab>{thread_return+0} <ffffffff80134fb6>{lock_timer_base+27}
       <ffffffff80135028>{try_to_del_timer_sync+81} <ffffffff802cd47c>{io_schedule_timeout+50}
       <ffffffff801cd57c>{blk_congestion_wait+132} <ffffffff8013fa4d>{autoremove_wake_function+0}
       <ffffffff80159715>{try_to_free_pages+342} <ffffffff8015335c>{out_of_memory+368}
       <ffffffff80154cdf>{__alloc_pages+422} <ffffffff8015667c>{__do_page_cache_readahead+245}
       <ffffffff8015016c>{sync_page+0} <ffffffff802cd4b9>{io_schedule+50}
       <ffffffff802cdcfd>{__wait_on_bit_lock+91} <ffffffff80152893>{filemap_nopage+323}
       <ffffffff8015d2e3>{__handle_mm_fault+1415} <ffffffff801812b2>{__user_walk_fd+65}
       <ffffffff801180d0>{do_page_fault+1938} <ffffffff8018aa7b>{mntput_no_expire+25}
       <ffffffff8014dd09>{audit_syscall_exit+747} <ffffffff8010b5c7>{error_exit+0}
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
 <0>general protection fault: 0000 [16] SMP 
CPU 2 
Modules linked in: ipv6 bridge button battery ac dm_mod e1000 ext3 jbd reiserfs fan thermal processor sg mptspi mptscsih mptbase piix sd_mod scsi_mod
Pid: 5632, comm: master Tainted: GF     2.6.16-xen #1
RIP: e030:[<ffffffff8011326b>] <ffffffff8011326b>{smp_really_stop_cpu+9}
RSP: e02b:ffff88000007bea8  EFLAGS: 00010002
RAX: ffffffffff578080 RBX: 0000000000000000 RCX: 0000000000000001
RDX: ffffffffff578000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 0000000000000000 R08: ffff880023ff0fc8 R09: ffff8807cf869ae8
R10: 000000000000001d R11: 0000000000000246 R12: ffffffff80113262
R13: 0000000000000000 R14: ffff8807cf869898 R15: ffff8807cf869898
FS:  00002abbf927f370(0000) GS:ffffffff803fb100(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000
Process master (pid: 5632, threadinfo ffff8807cf868000, task ffff8807d4d0b040)
Stack: 0000000000000100 ffffffff801134c8 ffff880000044e00 0000000000000000 
       0000000000000107 ffffffff8014ee2d ffffffff80404800 0000000000008380 
       0000000000000107 ffff880000044e00 
Call Trace: <IRQ> <ffffffff801134c8>{smp_call_function_interrupt+79}
       <ffffffff8014ee2d>{handle_IRQ_event+78} <ffffffff8014ef10>{__do_IRQ+153}
       <ffffffff8010d661>{do_IRQ+60} <ffffffff8023834a>{evtchn_do_upcall+134}
       <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff8010722a>{hypercall_page+554} <ffffffff8010722a>{hypercall_page+554}
       <ffffffff80237bfd>{force_evtchn_callback+10} <ffffffff80158e15>{shrink_zone+3853}
       <ffffffff802cd0ab>{thread_return+0} <ffffffff80134fb6>{lock_timer_base+27}
       <ffffffff80135028>{try_to_del_timer_sync+81} <ffffffff802cd47c>{io_schedule_timeout+50}
       <ffffffff801cd57c>{blk_congestion_wait+132} <ffffffff8013fa4d>{autoremove_wake_function+0}
       <ffffffff80159715>{try_to_free_pages+342} <ffffffff80154cdf>{__alloc_pages+422}
       <ffffffff802c8ed4>{unix_write_space+69} <ffffffff80167143>{read_swap_cache_async+66}
       <ffffffff8015b333>{swapin_readahead+78} <ffffffff8015d670>{__handle_mm_fault+2324}
       <ffffffff8015499e>{get_page_from_freelist+561} <ffffffff801180d0>{do_page_fault+1938}
       <ffffffff80135028>{try_to_del_timer_sync+81} <ffffffff8013503d>{del_timer_sync+12}
       <ffffffff8015407e>{free_hot_cold_page+234} <ffffffff8010b5c7>{error_exit+0}
       <ffffffff802c644c>{unix_poll+0} <ffffffff801dac33>{copy_user_generic+147}
       <ffffffff80183b84>{sys_select+788} <ffffffff8010b05c>{tracesys+167}

Code: f4 eb fd 41 55 49 89 fd 48 c7 c7 c8 c9 3c 80 41 54 55 48 89 
RIP <ffffffff8011326b>{smp_really_stop_cpu+9} RSP <ffff88000007bea8>
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
 <0>general protection fault: 0000 [17] SMP 
CPU 30 
Modules linked in: ipv6 bridge button battery ac dm_mod e1000 ext3 jbd reiserfs fan thermal processor sg mptspi mptscsih mptbase piix sd_mod scsi_mod
Pid: 6296, comm: mono Tainted: GF     2.6.16-xen #1
RIP: e030:[<ffffffff8011326b>] <ffffffff8011326b>{smp_really_stop_cpu+9}
RSP: e02b:ffff880000cf3ea8  EFLAGS: 00010002
RAX: ffffffffff578780 RBX: 0000000000000000 RCX: 0000000000000001
RDX: ffffffffff578000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 0000000000000000 R08: ffff8807bea56000 R09: 000000000000001e
R10: ffff8807d4bf7b70 R11: 0000000000000246 R12: ffffffff80113262
R13: 0000000000000000 R14: ffff8807bea579e8 R15: ffff8807bea579e8
FS:  00002b660bc42ce0(0063) GS:ffffffff803fbf00(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000
Process mono (pid: 6296, threadinfo ffff8807bea56000, task ffff8807bd5db8a0)
Stack: 0000000000000100 ffffffff801134c8 ffff880000cbfd80 0000000000000000 
       000000000000015b ffffffff8014ee2d ffffffff80407200 000000000000ad80 
       000000000000015b ffff880000cbfd80 
Call Trace: <IRQ> <ffffffff801134c8>{smp_call_function_interrupt+79}
       <ffffffff8014ee2d>{handle_IRQ_event+78} <ffffffff8014ef10>{__do_IRQ+153}
       <ffffffff8010d661>{do_IRQ+60} <ffffffff8023834a>{evtchn_do_upcall+134}
       <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff8010722a>{hypercall_page+554} <ffffffff8010722a>{hypercall_page+554}
       <ffffffff80237bfd>{force_evtchn_callback+10} <ffffffff802cd129>{thread_return+126}
       <ffffffff801daa07>{find_next_bit+85} <ffffffff8012408f>{move_tasks+183}
       <ffffffff80134fb6>{lock_timer_base+27} <ffffffff802cdc55>{schedule_timeout+138}
       <ffffffff80134b67>{process_timeout+0} <ffffffff802cd47c>{io_schedule_timeout+50}
       <ffffffff801cd56e>{blk_congestion_wait+118} <ffffffff801564bf>{pdflush_operation+71}
       <ffffffff8013fa4d>{autoremove_wake_function+0} <ffffffff801597ba>{try_to_free_pages+507}
       <ffffffff8015335c>{out_of_memory+368} <ffffffff80154cdf>{__alloc_pages+422}
       <ffffffff80167143>{read_swap_cache_async+66} <ffffffff8015b333>{swapin_readahead+78}
       <ffffffff8015d670>{__handle_mm_fault+2324} <ffffffff801180d0>{do_page_fault+1938}
       <ffffffff802ce0ad>{schedule_hrtimer+49} <ffffffff80142162>{hrtimer_nanosleep+76}
       <ffffffff8014da85>{audit_syscall_exit+103} <ffffffff8010b5c7>{error_exit+0}

Code: f4 eb fd 41 55 49 89 fd 48 c7 c7 c8 c9 3c 80 41 54 55 48 89 
RIP <ffffffff8011326b>{smp_really_stop_cpu+9} RSP <ffff880000cf3ea8>
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
 <0>general protection fault: 0000 [18] SMP 
CPU 28 
Modules linked in: ipv6 bridge button battery ac dm_mod e1000 ext3 jbd reiserfs fan thermal processor sg mptspi mptscsih mptbase piix sd_mod scsi_mod
Pid: 5346, comm: xinetd Tainted: GF     2.6.16-xen #1
RIP: e030:[<ffffffff8011326b>] <ffffffff8011326b>{smp_really_stop_cpu+9}
RSP: e02b:ffff880000cbbea8  EFLAGS: 00010002
RAX: ffffffffff578700 RBX: 0000000000000000 RCX: 0000000000000001
RDX: ffffffffff578000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 0000000000000000 R08: 00000000ffffffff R09: ffff8807d0109a28
R10: 0000000000000000 R11: 0000000000000246 R12: ffffffff80113262
R13: 0000000000000000 R14: ffff8807d01097d8 R15: ffff8807d01097d8
FS:  00002b46375cfae0(0000) GS:ffffffff803fbe00(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000
Process xinetd (pid: 5346, threadinfo ffff8807d0108000, task ffff8807d6886140)
Stack: 0000000000000040 ffffffff801134c8 ffff8800041a9300 0000000000000000 
       0000000000000155 ffffffff8014ee2d ffffffff80406f00 000000000000aa80 
       0000000000000155 ffff8800041a9300 
Call Trace: <IRQ> <ffffffff801134c8>{smp_call_function_interrupt+79}
       <ffffffff8014ee2d>{handle_IRQ_event+78} <ffffffff8014ef10>{__do_IRQ+153}
       <ffffffff8010d661>{do_IRQ+60} <ffffffff8023834a>{evtchn_do_upcall+134}
       <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff8010722a>{hypercall_page+554} <ffffffff8010722a>{hypercall_page+554}
       <ffffffff80237bfd>{force_evtchn_callback+10} <ffffffff801585f6>{shrink_zone+1774}
       <ffffffff802cd0ab>{thread_return+0} <ffffffff80134fb6>{lock_timer_base+27}
       <ffffffff80135028>{try_to_del_timer_sync+81} <ffffffff8013503d>{del_timer_sync+12}
       <ffffffff802cdc5d>{schedule_timeout+146} <ffffffff80134b67>{process_timeout+0}
       <ffffffff802cd47c>{io_schedule_timeout+50} <ffffffff801cd57c>{blk_congestion_wait+132}
       <ffffffff8013fa4d>{autoremove_wake_function+0} <ffffffff80159715>{try_to_free_pages+342}
       <ffffffff8015335c>{out_of_memory+368} <ffffffff80154cdf>{__alloc_pages+422}
       <ffffffff80167143>{read_swap_cache_async+66} <ffffffff8015b333>{swapin_readahead+78}
       <ffffffff8015d670>{__handle_mm_fault+2324} <ffffffff801d9298>{__up_read+19}
       <ffffffff801180d0>{do_page_fault+1938} <ffffffff802cd0ab>{thread_return+0}
       <ffffffff8010b5c7>{error_exit+0} <ffffffff8010a400>{do_signal+630}
       <ffffffff8010a38e>{do_signal+516} <ffffffff8014da85>{audit_syscall_exit+103}
       <ffffffff8010af9d>{sysret_signal+56} <ffffffff8010b271>{ptregscall_common+61}

Code: f4 eb fd 41 55 49 89 fd 48 c7 c7 c8 c9 3c 80 41 54 55 48 89 
RIP <ffffffff8011326b>{smp_really_stop_cpu+9} RSP <ffff880000cbbea8>
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
 <0>general protection fault: 0000 [19] SMP 
CPU 18 
Modules linked in: ipv6 bridge button battery ac dm_mod e1000 ext3 jbd reiserfs fan thermal processor sg mptspi mptscsih mptbase piix sd_mod scsi_mod
Pid: 6305, comm: gnome-power-man Tainted: GF     2.6.16-xen #1
RIP: e030:[<ffffffff8011326b>] <ffffffff8011326b>{smp_really_stop_cpu+9}
RSP: e02b:ffff880004963ea8  EFLAGS: 00010002
RAX: ffffffffff578480 RBX: 0000000000000000 RCX: 0000000000000001
RDX: ffffffffff578000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 0000000000000000 R08: ffff880023bec1f8 R09: ffff8807c0925b48
R10: 0000000000000020 R11: 0000000000000246 R12: ffffffff80113262
R13: 0000000000000000 R14: ffff8807c09258f8 R15: ffff8807c09258f8
FS:  00002afcbc5701f0(0000) GS:ffffffff803fb900(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000
Process gnome-power-man (pid: 6305, threadinfo ffff8807c0924000, task ffff8807be9db8a0)
Stack: 0000000000000000 ffffffff801134c8 ffff88002c0f1980 0000000000000000 
       0000000000000137 ffffffff8014ee2d ffffffff80406000 0000000000009b80 
       0000000000000137 ffff88002c0f1980 
Call Trace: <IRQ> <ffffffff801134c8>{smp_call_function_interrupt+79}
       <ffffffff8014ee2d>{handle_IRQ_event+78} <ffffffff8014ef10>{__do_IRQ+153}
       <ffffffff8010d661>{do_IRQ+60} <ffffffff8023834a>{evtchn_do_upcall+134}
       <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff8010722a>{hypercall_page+554} <ffffffff8010722a>{hypercall_page+554}
       <ffffffff80237bfd>{force_evtchn_callback+10} <ffffffff801585f6>{shrink_zone+1774}
       <ffffffff802cd0ab>{thread_return+0} <ffffffff80134fb6>{lock_timer_base+27}
       <ffffffff80135028>{try_to_del_timer_sync+81} <ffffffff802cdc5d>{schedule_timeout+146}
       <ffffffff80134b67>{process_timeout+0} <ffffffff801cd57c>{blk_congestion_wait+132}
       <ffffffff8013fa4d>{autoremove_wake_function+0} <ffffffff80159715>{try_to_free_pages+342}
       <ffffffff80154cdf>{__alloc_pages+422} <ffffffff8015667c>{__do_page_cache_readahead+245}
       <ffffffff8015016c>{sync_page+0} <ffffffff802cd4b9>{io_schedule+50}
       <ffffffff802cdcfd>{__wait_on_bit_lock+91} <ffffffff80152893>{filemap_nopage+323}
       <ffffffff8015d2e3>{__handle_mm_fault+1415} <ffffffff801180d0>{do_page_fault+1938}
       <ffffffff8016d9b0>{__kmalloc+69} <ffffffff8014da85>{audit_syscall_exit+103}
       <ffffffff8010b5c7>{error_exit+0}

Code: f4 eb fd 41 55 49 89 fd 48 c7 c7 c8 c9 3c 80 41 54 55 48 89 
RIP <ffffffff8011326b>{smp_really_stop_cpu+9} RSP <ffff880004963ea8>
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
 <0>general protection fault: 0000 [20] SMP 
CPU 16 
Modules linked in: ipv6 bridge button battery ac dm_mod e1000 ext3 jbd reiserfs fan thermal processor sg mptspi mptscsih mptbase piix sd_mod scsi_mod
Pid: 0, comm: swapper Tainted: GF     2.6.16-xen #1
RIP: e030:[<ffffffff8011326b>] <ffffffff8011326b>{smp_really_stop_cpu+9}
RSP: e02b:ffff880004947ea8  EFLAGS: 00010002
RAX: ffffffffff578400 RBX: 0000000000000000 RCX: 0000000000000001
RDX: ffffffffff578000 RSI: 0000000000000002 RDI: 0000000000000000
RBP: 0000000000000000 R08: 0000000100073335 R09: 0000000000000010
R10: 0000000100073398 R11: 0000000000000246 R12: ffffffff80113262
R13: 0000000000000000 R14: ffff880000c4de58 R15: ffff880000c4de58
FS:  00002b76e9a3c1b0(0000) GS:ffffffff803fb800(0000) knlGS:0000000000000000
CS:  e033 DS: 002b ES: 002b
Process swapper (pid: 0, threadinfo ffff880000c4c000, task ffff8800049a97e0)
Stack: ffffffff80113262 ffffffff801134c8 ffff88002c0f1dc0 0000000000000000 
       0000000000000131 ffffffff8014ee2d ffffffff80405d00 0000000000009880 
       0000000000000131 ffff88002c0f1dc0 
Call Trace: <IRQ> <ffffffff80113262>{smp_really_stop_cpu+0}
       <ffffffff801134c8>{smp_call_function_interrupt+79} <ffffffff8014ee2d>{handle_IRQ_event+78}
       <ffffffff8014ef10>{__do_IRQ+153} <ffffffff8010d661>{do_IRQ+60}
       <ffffffff8023834a>{evtchn_do_upcall+134} <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff801073aa>{hypercall_page+938} <ffffffff801073aa>{hypercall_page+938}
       <ffffffff801095de>{xen_idle+132} <ffffffff80109933>{cpu_idle+217}

Code: f4 eb fd 41 55 49 89 fd 48 c7 c7 c8 c9 3c 80 41 54 55 48 89 
RIP <ffffffff8011326b>{smp_really_stop_cpu+9} RSP <ffff880004947ea8>
Badness in do_unblank_screen at drivers/char/vt.c:2830

Call Trace: <IRQ> <ffffffff80228114>{do_unblank_screen+76}
       <ffffffff8011775b>{bust_spinlocks+28} <ffffffff8010bbbd>{oops_end+21}
       <ffffffff8010c25b>{die+58} <ffffffff8010d04a>{do_general_protection+291}
       <ffffffff80113262>{smp_really_stop_cpu+0} <ffffffff8010b5c7>{error_exit+0}
       <ffffffff80113262>{smp_really_stop_cpu+0} <ffffffff8011326b>{smp_really_stop_cpu+9}
       <ffffffff8011326b>{smp_really_stop_cpu+9} <ffffffff80113262>{smp_really_stop_cpu+0}
       <ffffffff801134c8>{smp_call_function_interrupt+79} <ffffffff8014ee2d>{handle_IRQ_event+78}
       <ffffffff8014ef10>{__do_IRQ+153} <ffffffff8010d661>{do_IRQ+60}
       <ffffffff8023834a>{evtchn_do_upcall+134} <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff801073aa>{hypercall_page+938} <ffffffff801073aa>{hypercall_page+938}
       <ffffffff801095de>{xen_idle+132} <ffffffff80109933>{cpu_idle+217}
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
 <0>general protection fault: 0000 [21] SMP 
CPU 20 
Modules linked in: ipv6 bridge button battery ac dm_mod e1000 ext3 jbd reiserfs fan thermal processor sg mptspi mptscsih mptbase piix sd_mod scsi_mod
Pid: 6283, comm: gnome-panel Tainted: GF     2.6.16-xen #1
RIP: e030:[<ffffffff8011326b>] <ffffffff8011326b>{smp_really_stop_cpu+9}
RSP: e02b:ffff880004197ea8  EFLAGS: 00010002
RAX: ffffffffff578500 RBX: 0000000000000000 RCX: 0000000000000001
RDX: ffffffffff578000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 0000000000000000 R08: ffff8807c2b699d8 R09: 0000000000000010
R10: ffff88002c13e2d8 R11: 0000000000000246 R12: ffffffff80113262
R13: 0000000000000000 R14: ffff8807c2b699d8 R15: ffff8807c2b699d8
FS:  00002b2e6d783e30(0000) GS:ffffffff803fba00(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000
Process gnome-panel (pid: 6283, threadinfo ffff8807c2b68000, task ffff8807c261b8a0)
Stack: 0000000000000000 ffffffff801134c8 ffff88002c0f1540 0000000000000000 
       000000000000013d ffffffff8014ee2d ffffffff80406300 0000000000009e80 
       000000000000013d ffff88002c0f1540 
Call Trace: <IRQ> <ffffffff801134c8>{smp_call_function_interrupt+79}
       <ffffffff8014ee2d>{handle_IRQ_event+78} <ffffffff8014ef10>{__do_IRQ+153}
       <ffffffff8010d661>{do_IRQ+60} <ffffffff8023834a>{evtchn_do_upcall+134}
       <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff8010722a>{hypercall_page+554} <ffffffff8010722a>{hypercall_page+554}
       <ffffffff80237bfd>{force_evtchn_callback+10} <ffffffff80158e84>{shrink_zone+3964}
       <ffffffff802cd0ab>{thread_return+0} <ffffffff80134fb6>{lock_timer_base+27}
       <ffffffff80135028>{try_to_del_timer_sync+81} <ffffffff802cd47c>{io_schedule_timeout+50}
       <ffffffff801cd57c>{blk_congestion_wait+132} <ffffffff8013fa4d>{autoremove_wake_function+0}
       <ffffffff80159715>{try_to_free_pages+342} <ffffffff8015016c>{sync_page+0}
       <ffffffff80154cdf>{__alloc_pages+422} <ffffffff80167143>{read_swap_cache_async+66}
       <ffffffff8015b333>{swapin_readahead+78} <ffffffff8015d670>{__handle_mm_fault+2324}
       <ffffffff801180d0>{do_page_fault+1938} <ffffffff801830b3>{poll_freewait+64}
       <ffffffff8014da85>{audit_syscall_exit+103} <ffffffff8010b5c7>{error_exit+0}

Code: f4 eb fd 41 55 49 89 fd 48 c7 c7 c8 c9 3c 80 41 54 55 48 89 
RIP <ffffffff8011326b>{smp_really_stop_cpu+9} RSP <ffff880004197ea8>
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
 <1>Unable to handle kernel NULL pointer dereference at 0000000000000001 RIP: 
[<0000000000000001>]
PGD 7d230f067 PUD 7d2b0e067 PMD 0 
Oops: 0010 [22] SMP 
CPU 29 
Modules linked in: ipv6 bridge button battery ac dm_mod e1000 ext3 jbd reiserfs fan thermal processor sg mptspi mptscsih mptbase piix sd_mod scsi_mod
Pid: 5941, comm: gpg-agent Tainted: GF     2.6.16-xen #1
RIP: e030:[<0000000000000001>] [<0000000000000001>]
RSP: e02b:ffff880000ce7eb0  EFLAGS: 00010006
RAX: ffff8807c7873fd8 RBX: ffffffff8011011e RCX: 0000000000000158
RDX: 0000000000001735 RSI: 0000000000000000 RDI: ffffffff8011011e
RBP: 0000000000000000 R08: ffff88002405d7c8 R09: ffff8807c7873c28
R10: 0000000000000000 R11: 0000000000000246 R12: 0000000000000001
R13: 0000000000000000 R14: ffff8807c78739d8 R15: ffff8807c78739d8
FS:  00002b5688465fc0(0000) GS:ffffffff803fbe80(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000
Process gpg-agent (pid: 5941, threadinfo ffff8807c7872000, task ffff8800006448a0)
Stack: ffffffff801134c8 ffff880000cbffc0 0000000000000000 0000000000000158 
       ffffffff8014ee2d ffffffff80407080 000000000000ac00 0000000000000158 
       ffff880000cbffc0 ffffffff804070a8 
Call Trace: <IRQ> <ffffffff801134c8>{smp_call_function_interrupt+79}
       <ffffffff8014ee2d>{handle_IRQ_event+78} <ffffffff8014ef10>{__do_IRQ+153}
       <ffffffff8010d661>{do_IRQ+60} <ffffffff8023834a>{evtchn_do_upcall+134}
       <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff8010722a>{hypercall_page+554} <ffffffff8010722a>{hypercall_page+554}
       <ffffffff80237bfd>{force_evtchn_callback+10} <ffffffff801585f6>{shrink_zone+1774}
       <ffffffff802cd0ab>{thread_return+0} <ffffffff8019f6c7>{mb_cache_shrink_fn+36}
       <ffffffff80159715>{try_to_free_pages+342} <ffffffff8015335c>{out_of_memory+368}
       <ffffffff80154cdf>{__alloc_pages+422} <ffffffff80167143>{read_swap_cache_async+66}
       <ffffffff8015b333>{swapin_readahead+78} <ffffffff8015d670>{__handle_mm_fault+2324}
       <ffffffff801180d0>{do_page_fault+1938} <ffffffff8014da85>{audit_syscall_exit+103}
       <ffffffff8010b5c7>{error_exit+0}

Code:  Bad RIP value.
RIP [<0000000000000001>] RSP <ffff880000ce7eb0>
CR2: 0000000000000001
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
(XEN) Domain 0 crashed: rebooting machine in 5 seconds.
general protection fault: 0000 [23] SMP 
CPU 24 
Modules linked in: ipv6 bridge button battery ac dm_mod e1000 ext3 jbd reiserfs fan thermal processor sg mptspi mptscsih mptbase piix sd_mod scsi_mod
Pid: 5900, comm: gnome-session Tainted: GF     2.6.16-xen #1
RIP: e030:[<ffffffff8011326b>] <ffffffff8011326b>{smp_really_stop_cpu+9}
RSP: e02b:ffff8800041ebea8  EFLAGS: 00010002
RAX: ffffffffff578600 RBX: 0000000000000000 RCX: 0000000000000001
RDX: ffffffffff578000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 0000000000000000 R08: ffffffff80333000 R09: 00000000007d72ca
R10: ffff8800089ee000 R11: 0000000000000000 R12: ffffffff80113262
R13: 0000000000000000 R14: ffff8807cfb29b28 R15: ffff8807cfb29b28
FS:  00002b2d4633aa60(0000) GS:ffffffff803fbc00(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000
Process gnome-session (pid: 5900, threadinfo ffff8807cfb28000, task ffff8807d3d7f180)
Stack: ffff8800010e6000 ffffffff801134c8 ffff8800041a9b80 0000000000000000 
       0000000000000149 ffffffff8014ee2d ffffffff80406900 000000000000a480 
       0000000000000149 ffff8800041a9b80 
Call Trace: <IRQ> <ffffffff801134c8>{smp_call_function_interrupt+79}
       <ffffffff8014ee2d>{handle_IRQ_event+78} <ffffffff8014ef10>{__do_IRQ+153}
       <ffffffff8010d661>{do_IRQ+60} <ffffffff8023834a>{evtchn_do_upcall+134}
       <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff80117379>{show_mem+132} <ffffffff8011732c>{show_mem+55}
       <ffffffff80153221>{out_of_memory+53} <ffffffff80154d56>{__alloc_pages+541}
       <ffffffff8015667c>{__do_page_cache_readahead+245} <ffffffff8015016c>{sync_page+0}
       <ffffffff802cd4b9>{io_schedule+50} <ffffffff802cdcfd>{__wait_on_bit_lock+91}
       <ffffffff80152893>{filemap_nopage+323} <ffffffff8015d2e3>{__handle_mm_fault+1415}
       <ffffffff801180d0>{do_page_fault+1938} <ffffffff801830b3>{poll_freewait+64}
       <ffffffff8014da85>{audit_syscall_exit+103} <ffffffff8010b5c7>{error_exit+0}

Code: f4 eb fd 41 55 49 89 fd 48 c7 c7 c8 c9 3c 80 41 54 55 48 89 
RIP <ffffffff8011326b>{smp_really_stop_cpu+9} RSP <ffff8800041ebea8>
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
 <0>general protection fault: 0000 [24] SMP 
CPU 31 
Modules linked in: ipv6 bridge button battery ac dm_mod e1000 ext3 jbd reiserfs fan thermal processor sg mptspi mptscsih mptbase piix sd_mod scsi_mod
Pid: 0, comm: swapper Tainted: GF     2.6.16-xen #1
RIP: e030:[<ffffffff8011326b>] <ffffffff8011326b>{smp_really_stop_cpu+9}
RSP: e02b:ffff880000cffea8  EFLAGS: 00010002
RAX: ffffffffff5787c0 RBX: 0000000000000000 RCX: 0000000000000001
RDX: ffffffffff578000 RSI: 0000000000000002 RDI: 0000000000000000
RBP: 0000000000000000 R08: 000000010007333b R09: 000000000000001f
R10: 0000000100073384 R11: 0000000000000246 R12: ffffffff80113262
R13: 0000000000000000 R14: ffff880000055e58 R15: ffff880000055e58
FS:  00002b2e6d783e30(0000) GS:ffffffff803fbf80(0000) knlGS:0000000000000000
CS:  e033 DS: 002b ES: 002b
Process swapper (pid: 0, threadinfo ffff880000054000, task ffff880000c74100)
Stack: ffffffff80113262 ffffffff801134c8 ffff880000cbfb80 0000000000000000 
       000000000000015e ffffffff8014ee2d ffffffff80407380 000000000000af00 
       000000000000015e ffff880000cbfb80 
Call Trace: <IRQ> <ffffffff80113262>{smp_really_stop_cpu+0}
       <ffffffff801134c8>{smp_call_function_interrupt+79} <ffffffff8014ee2d>{handle_IRQ_event+78}
       <ffffffff8014ef10>{__do_IRQ+153} <ffffffff8010d661>{do_IRQ+60}
       <ffffffff8023834a>{evtchn_do_upcall+134} <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff801073aa>{hypercall_page+938} <ffffffff801073aa>{hypercall_page+938}
       <ffffffff801095de>{xen_idle+132} <ffffffff80109933>{cpu_idle+217}

Code: f4 eb fd 41 55 49 89 fd 48 c7 c7 c8 c9 3c 80 41 54 55 48 89 
RIP <ffffffff8011326b>{smp_really_stop_cpu+9} RSP <ffff880000cffea8>
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
 <0>general protection fault: 0000 [25] SMP 
CPU 25 
Modules linked in: ipv6 bridge button battery ac dm_mod e1000 ext3 jbd reiserfs fan thermal processor sg mptspi mptscsih mptbase piix sd_mod scsi_mod
Pid: 6086, comm: gnome-terminal Tainted: GF     2.6.16-xen #1
RIP: e030:[<ffffffff8011326b>] <ffffffff8011326b>{smp_really_stop_cpu+9}
RSP: e02b:ffff8800041f7ea8  EFLAGS: 00010002
RAX: ffffffffff578640 RBX: 0000000000000000 RCX: 0000000000000001
RDX: ffffffffff578000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 0000000000000000 R08: ffff880023f3c0d8 R09: ffff8807bde15c28
R10: 0000000000000002 R11: 0000000000000246 R12: ffffffff80113262
R13: 0000000000000000 R14: ffff8807bde159d8 R15: ffff8807bde159d8
FS:  00002ae071177fc0(0000) GS:ffffffff803fbc80(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000
Process gnome-terminal (pid: 6086, threadinfo ffff8807bde14000, task ffff880000710040)
Stack: 0000000000000040 ffffffff801134c8 ffff8800041a9980 0000000000000000 
       000000000000014c ffffffff8014ee2d ffffffff80406a80 000000000000a600 
       000000000000014c ffff8800041a9980 
Call Trace: <IRQ> <ffffffff801134c8>{smp_call_function_interrupt+79}
       <ffffffff8014ee2d>{handle_IRQ_event+78} <ffffffff8014ef10>{__do_IRQ+153}
       <ffffffff8010d661>{do_IRQ+60} <ffffffff8023834a>{evtchn_do_upcall+134}
       <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff8010722a>{hypercall_page+554} <ffffffff8010722a>{hypercall_page+554}
       <ffffffff80237bfd>{force_evtchn_callback+10} <ffffffff801585f6>{shrink_zone+1774}
       <ffffffff802cd0ab>{thread_return+0} <ffffffff801cba4c>{generic_make_request+339}
       <ffffffff80134fb6>{lock_timer_base+27} <ffffffff80135028>{try_to_del_timer_sync+81}
       <ffffffff802cd47c>{io_schedule_timeout+50} <ffffffff801cd57c>{blk_congestion_wait+132}
       <ffffffff8013fa4d>{autoremove_wake_function+0} <ffffffff80159715>{try_to_free_pages+342}
       <ffffffff8013fa56>{autoremove_wake_function+9} <ffffffff80154cdf>{__alloc_pages+422}
       <ffffffff80167143>{read_swap_cache_async+66} <ffffffff8015b333>{swapin_readahead+78}
       <ffffffff8015d670>{__handle_mm_fault+2324} <ffffffff801180d0>{do_page_fault+1938}
       <ffffffff801830b3>{poll_freewait+64} <ffffffff8014da85>{audit_syscall_exit+103}
       <ffffffff8010b5c7>{error_exit+0}

Code: f4 eb fd 41 55 49 89 fd 48 c7 c7 c8 c9 3c 80 41 54 55 48 89 
RIP <ffffffff8011326b>{smp_really_stop_cpu+9} RSP <ffff8800041f7ea8>
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
 <0>general protection fault: 0000 [26] SMP 
CPU 22 
Modules linked in: ipv6 bridge button battery ac dm_mod e1000 ext3 jbd reiserfs fan thermal processor sg mptspi mptscsih mptbase piix sd_mod scsi_mod
Pid: 6275, comm: gnome-settings- Tainted: GF     2.6.16-xen #1
RIP: e030:[<ffffffff8011326b>] <ffffffff8011326b>{smp_really_stop_cpu+9}
RSP: e02b:ffff8800041d3ea8  EFLAGS: 00010002
RAX: ffffffffff578580 RBX: 0000000000000000 RCX: 0000000000000001
RDX: ffffffffff578000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 0000000000000000 R08: ffff880023b0b1c8 R09: ffff8807bc82fb48
R10: 0000000000000012 R11: 0000000000000246 R12: ffffffff80113262
R13: 0000000000000000 R14: ffff8807bc82f8f8 R15: ffff8807bc82f8f8
FS:  00002aae5150ffc0(0000) GS:ffffffff803fbb00(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000
Process gnome-settings- (pid: 6275, threadinfo ffff8807bc82e000, task ffff8807d0f0e140)
Stack: 0000000000000040 ffffffff801134c8 ffff8800041a9fc0 0000000000000000 
       0000000000000143 ffffffff8014ee2d ffffffff80406600 000000000000a180 
       0000000000000143 ffff8800041a9fc0 
Call Trace: <IRQ> <ffffffff801134c8>{smp_call_function_interrupt+79}
       <ffffffff8014ee2d>{handle_IRQ_event+78} <ffffffff8014ef10>{__do_IRQ+153}
       <ffffffff8010d661>{do_IRQ+60} <ffffffff8023834a>{evtchn_do_upcall+134}
       <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff8010722a>{hypercall_page+554} <ffffffff8010722a>{hypercall_page+554}
       <ffffffff80237bfd>{force_evtchn_callback+10} <ffffffff801585f6>{shrink_zone+1774}
       <ffffffff802cd0ab>{thread_return+0} <ffffffff80113fc1>{xen_send_IPI_mask+0}
       <ffffffff80134fb6>{lock_timer_base+27} <ffffffff80135028>{try_to_del_timer_sync+81}
       <ffffffff801cd57c>{blk_congestion_wait+132} <ffffffff8013fa4d>{autoremove_wake_function+0}
       <ffffffff80159715>{try_to_free_pages+342} <ffffffff8015335c>{out_of_memory+368}
       <ffffffff80154cdf>{__alloc_pages+422} <ffffffff8015667c>{__do_page_cache_readahead+245}
       <ffffffff8015016c>{sync_page+0} <ffffffff802cd4b9>{io_schedule+50}
       <ffffffff802cdcfd>{__wait_on_bit_lock+91} <ffffffff80152893>{filemap_nopage+323}
       <ffffffff8015d2e3>{__handle_mm_fault+1415} <ffffffff801180d0>{do_page_fault+1938}
       <ffffffff801830b3>{poll_freewait+64} <ffffffff8014da85>{audit_syscall_exit+103}
       <ffffffff8010b5c7>{error_exit+0}

Code: f4 eb fd 41 55 49 89 fd 48 c7 c7 c8 c9 3c 80 41 54 55 48 89 
RIP <ffffffff8011326b>{smp_really_stop_cpu+9} RSP <ffff8800041d3ea8>
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
 <0>general protection fault: 0000 [27] SMP 
CPU 17 
Modules linked in: ipv6 bridge button battery ac dm_mod e1000 ext3 jbd reiserfs fan thermal processor sg mptspi mptscsih mptbase piix sd_mod scsi_mod
Pid: 0, comm: swapper Tainted: GF     2.6.16-xen #1
RIP: e030:[<ffffffff8011326b>] <ffffffff8011326b>{smp_really_stop_cpu+9}
RSP: e02b:ffff880004953ea8  EFLAGS: 00010002
RAX: ffffffffff578440 RBX: 0000000000000000 RCX: 0000000000000001
RDX: ffffffffff578000 RSI: 0000000000000002 RDI: 0000000000000000
RBP: 0000000000000000 R08: 000000010007334e R09: 0000000000000011
R10: 00000001000733af R11: 0000000000000246 R12: ffffffff80113262
R13: 0000000000000000 R14: ffff880000c51e58 R15: ffff880000c51e58
FS:  00002b103f96a490(0000) GS:ffffffff803fb880(0000) knlGS:0000000000000000
CS:  e033 DS: 002b ES: 002b
Process swapper (pid: 0, threadinfo ffff880000c50000, task ffff8800049a90c0)
Stack: ffffffff80113262 ffffffff801134c8 ffff88002c0f1bc0 0000000000000000 
       0000000000000134 ffffffff8014ee2d ffffffff80405e80 0000000000009a00 
       0000000000000134 ffff88002c0f1bc0 
Call Trace: <IRQ> <ffffffff80113262>{smp_really_stop_cpu+0}
       <ffffffff801134c8>{smp_call_function_interrupt+79} <ffffffff8014ee2d>{handle_IRQ_event+78}
       <ffffffff8014ef10>{__do_IRQ+153} <ffffffff8010d661>{do_IRQ+60}
       <ffffffff8023834a>{evtchn_do_upcall+134} <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff801073aa>{hypercall_page+938} <ffffffff801073aa>{hypercall_page+938}
       <ffffffff801095de>{xen_idle+132} <ffffffff80109933>{cpu_idle+217}

Code: f4 eb fd 41 55 49 89 fd 48 c7 c7 c8 c9 3c 80 41 54 55 48 89 
RIP <ffffffff8011326b>{smp_really_stop_cpu+9} RSP <ffff880004953ea8>
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
 <0>general protection fault: 0000 [28] SMP 
CPU 21 
Modules linked in: ipv6 bridge button battery ac dm_mod e1000 ext3 jbd reiserfs fan thermal processor sg mptspi mptscsih mptbase piix sd_mod scsi_mod
Pid: 0, comm: swapper Tainted: GF     2.6.16-xen #1
RIP: e030:[<ffffffff8011326b>] <ffffffff8011326b>{smp_really_stop_cpu+9}
RSP: e02b:ffff8800041a3ea8  EFLAGS: 00010002
RAX: ffffffffff578540 RBX: 0000000000000000 RCX: 0000000000000001
RDX: ffffffffff578000 RSI: 0000000000000002 RDI: 0000000000000000
RBP: 0000000000000000 R08: 000000010007333b R09: 0000000000000015
R10: 000000010007339e R11: 0000000000000246 R12: ffffffff80113262
R13: 0000000000000000 R14: ffff880000c5be58 R15: ffff880000c5be58
FS:  00002aab92f75d80(0000) GS:ffffffff803fba80(0000) knlGS:0000000000000000
CS:  e033 DS: 002b ES: 002b
Process swapper (pid: 0, threadinfo ffff880000c5a000, task ffff8800049b0140)
Stack: ffffffff80113262 ffffffff801134c8 ffff88002c0f1340 0000000000000000 
       0000000000000140 ffffffff8014ee2d ffffffff80406480 000000000000a000 
       0000000000000140 ffff88002c0f1340 
Call Trace: <IRQ> <ffffffff80113262>{smp_really_stop_cpu+0}
       <ffffffff801134c8>{smp_call_function_interrupt+79} <ffffffff8014ee2d>{handle_IRQ_event+78}
       <ffffffff8014ef10>{__do_IRQ+153} <ffffffff8010d661>{do_IRQ+60}
       <ffffffff8023834a>{evtchn_do_upcall+134} <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff801073aa>{hypercall_page+938} <ffffffff801073aa>{hypercall_page+938}
       <ffffffff801095de>{xen_idle+132} <ffffffff80109933>{cpu_idle+217}

Code: f4 eb fd 41 55 49 89 fd 48 c7 c7 c8 c9 3c 80 41 54 55 48 89 
RIP <ffffffff8011326b>{smp_really_stop_cpu+9} RSP <ffff8800041a3ea8>
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
 <0>general protection fault: 0000 [29] SMP 
CPU 19 
Modules linked in: ipv6 bridge button battery ac dm_mod e1000 ext3 jbd reiserfs fan thermal processor sg mptspi mptscsih mptbase piix sd_mod scsi_mod
Pid: 5667, comm: gdm Tainted: GF     2.6.16-xen #1
RIP: e030:[<ffffffff8011326b>] <ffffffff8011326b>{smp_really_stop_cpu+9}
RSP: e02b:ffff88000418bea8  EFLAGS: 00010002
RAX: ffffffffff5784c0 RBX: 0000000000000000 RCX: 0000000000000001
RDX: ffffffffff578000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 0000000000000000 R08: ffff880023f2d6c8 R09: ffff8807ceb73b48
R10: 0000000000000020 R11: 0000000000000246 R12: ffffffff80113262
R13: 0000000000000000 R14: ffff8807ceb738f8 R15: ffff8807ceb738f8
FS:  00002aab92f75d80(0000) GS:ffffffff803fb980(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000
Process gdm (pid: 5667, threadinfo ffff8807ceb72000, task ffff8807d69d97e0)
Stack: 0000000000000000 ffffffff801134c8 ffff88002c0f1780 0000000000000000 
       000000000000013a ffffffff8014ee2d ffffffff80406180 0000000000009d00 
       000000000000013a ffff88002c0f1780 
Call Trace: <IRQ> <ffffffff801134c8>{smp_call_function_interrupt+79}
       <ffffffff8014ee2d>{handle_IRQ_event+78} <ffffffff8014ef10>{__do_IRQ+153}
       <ffffffff8010d661>{do_IRQ+60} <ffffffff8023834a>{evtchn_do_upcall+134}
       <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff8010722a>{hypercall_page+554} <ffffffff8010722a>{hypercall_page+554}
       <ffffffff80237bfd>{force_evtchn_callback+10} <ffffffff80158e15>{shrink_zone+3853}
       <ffffffff802cd0ab>{thread_return+0} <ffffffff80113fc1>{xen_send_IPI_mask+0}
       <ffffffff80237cf1>{notify_remote_via_irq+44} <ffffffff80134fb6>{lock_timer_base+27}
       <ffffffff801236c3>{__wake_up_common+62} <ffffffff8019f6c7>{mb_cache_shrink_fn+36}
       <ffffffff80159715>{try_to_free_pages+342} <ffffffff8015335c>{out_of_memory+368}
       <ffffffff80154cdf>{__alloc_pages+422} <ffffffff8015667c>{__do_page_cache_readahead+245}
       <ffffffff8015016c>{sync_page+0} <ffffffff802cd4b9>{io_schedule+50}
       <ffffffff802cdcfd>{__wait_on_bit_lock+91} <ffffffff80152893>{filemap_nopage+323}
       <ffffffff8015d2e3>{__handle_mm_fault+1415} <ffffffff801180d0>{do_page_fault+1938}
       <ffffffff8014da85>{audit_syscall_exit+103} <ffffffff8010b5c7>{error_exit+0}

Code: f4 eb fd 41 55 49 89 fd 48 c7 c7 c8 c9 3c 80 41 54 55 48 89 
RIP <ffffffff8011326b>{smp_really_stop_cpu+9} RSP <ffff88000418bea8>
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
 <0>general protection fault: 0000 [30] SMP 
CPU 26 
Modules linked in: ipv6 bridge button battery ac dm_mod e1000 ext3 jbd reiserfs fan thermal processor sg mptspi mptscsih mptbase piix sd_mod scsi_mod
Pid: 0, comm: swapper Tainted: GF     2.6.16-xen #1
RIP: e030:[<ffffffff8011326b>] <ffffffff8011326b>{smp_really_stop_cpu+9}
RSP: e02b:ffff880000ca3ea8  EFLAGS: 00010002
RAX: ffffffffff578680 RBX: 0000000000000000 RCX: 0000000000000001
RDX: ffffffffff578000 RSI: 0000000000000002 RDI: 0000000000000000
RBP: 0000000000000000 R08: 000000010007333b R09: 000000000000001a
R10: 000000010007339f R11: 0000000000000246 R12: ffffffff80113262
R13: 0000000000000000 R14: ffff880000c6be58 R15: ffff880000c6be58
FS:  00002b2e6d783e30(0000) GS:ffffffff803fbd00(0000) knlGS:0000000000000000
CS:  e033 DS: 002b ES: 002b
Process swapper (pid: 0, threadinfo ffff880000c6a000, task ffff8800049bb7a0)
Stack: ffffffff80113262 ffffffff801134c8 ffff8800041a9740 0000000000000000 
       000000000000014f ffffffff8014ee2d ffffffff80406c00 000000000000a780 
       000000000000014f ffff8800041a9740 
Call Trace: <IRQ> <ffffffff80113262>{smp_really_stop_cpu+0}
       <ffffffff801134c8>{smp_call_function_interrupt+79} <ffffffff8014ee2d>{handle_IRQ_event+78}
       <ffffffff8014ef10>{__do_IRQ+153} <ffffffff8010d661>{do_IRQ+60}
       <ffffffff8023834a>{evtchn_do_upcall+134} <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff801073aa>{hypercall_page+938} <ffffffff801073aa>{hypercall_page+938}
       <ffffffff801095de>{xen_idle+132} <ffffffff80109933>{cpu_idle+217}

Code: f4 eb fd 41 55 49 89 fd 48 c7 c7 c8 c9 3c 80 41 54 55 48 89 
RIP <ffffffff8011326b>{smp_really_stop_cpu+9} RSP <ffff880000ca3ea8>
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
general protection fault: 0000 [31]  SMP 
CPU 23 
Modules linked in: ipv6 bridge button battery ac dm_mod e1000 ext3 jbd reiserfs fan thermal processor sg mptspi mptscsih mptbase piix sd_mod scsi_mod
Pid: 3568, comm: klogd Tainted: GF     2.6.16-xen #1
RIP: e030:[<ffffffff8011326b>] <ffffffff8011326b>{smp_really_stop_cpu+9}
RSP: e02b:ffff8800041dfea8  EFLAGS: 00010002
RAX: ffffffffff5785c0 RBX: 0000000000000000 RCX: 0000000000000001
RDX: ffffffffff578000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 0000000000000000 R08: ffff88002401ec98 R09: ffff8807d4619b48
R10: 0000000000000002 R11: 0000000000000246 R12: ffffffff80113262
R13: 0000000000000000 R14: ffff8807d46198f8 R15: ffff8807d46198f8
FS:  00002b5863f0d6d0(0000) GS:ffffffff803fbb80(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000
Process klogd (pid: 3568, threadinfo ffff8807d4618000, task ffff8807d56f4820)
Stack: 0000000000000040 ffffffff801134c8 ffff8800041a9dc0 0000000000000000 
       0000000000000146 ffffffff8014ee2d ffffffff80406780 000000000000a300 
       0000000000000146 ffff8800041a9dc0 
Call Trace: <IRQ> <ffffffff801134c8>{smp_call_function_interrupt+79}
       <ffffffff8014ee2d>{handle_IRQ_event+78} <ffffffff8014ef10>{__do_IRQ+153}
       <ffffffff8010d661>{do_IRQ+60} <ffffffff8023834a>{evtchn_do_upcall+134}
       <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff8010722a>{hypercall_page+554} <ffffffff8010722a>{hypercall_page+554}
       <ffffffff8015208c>{__remove_from_page_cache+28} <ffffffff80237bfd>{force_evtchn_callback+10}
       <ffffffff801585f6>{shrink_zone+1774} <ffffffff80113fc1>{xen_send_IPI_mask+0}
       <ffffffff80113fc1>{xen_send_IPI_mask+0} <ffffffff80237cf1>{notify_remote_via_irq+44}
       <ffffffff801236c3>{__wake_up_common+62} <ffffffff8019f6c7>{mb_cache_shrink_fn+36}
       <ffffffff80159715>{try_to_free_pages+342} <ffffffff8015335c>{out_of_memory+368}
       <ffffffff80154cdf>{__alloc_pages+422} <ffffffff8015667c>{__do_page_cache_readahead+245}
       <ffffffff8015016c>{sync_page+0} <ffffffff802cd4b9>{io_schedule+50}
       <ffffffff802cdcfd>{__wait_on_bit_lock+91} <ffffffff80152893>{filemap_nopage+323}
       <ffffffff8015d2e3>{__handle_mm_fault+1415} <ffffffff801180d0>{do_page_fault+1938}
       <ffffffff8013fa4d>{autoremove_wake_function+0} <ffffffff801611cd>{unmap_region+274}
       <ffffffff8010b5c7>{error_exit+0}

Code: f4 eb fd 41 55 49 89 fd 48 c7 c7 c8 c9 3c 80 41 54 55 48 89 
RIP <ffffffff8011326b>{smp_really_stop_cpu+9} RSP <ffff8800041dfea8>
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
general protection fault: 0000 [32] SMP 
CPU 27  
Modules linked in: ipv6 bridge button battery ac dm_mod e1000 ext3 jbd reiserfs fan thermal processor sg mptspi mptscsih mptbase piix sd_mod scsi_mod
Pid: 6498, comm: xentop Tainted: GF     2.6.16-xen #1
RIP: e030:[<ffffffff8011326b>] <ffffffff8011326b>{smp_really_stop_cpu+9}
RSP: e02b:ffff880000cafea8  EFLAGS: 00010002
RAX: ffffffffff5786c0 RBX: 0000000000000000 RCX: 0000000000000001
RDX: ffffffffff578000 RSI: 0000000000000000 RDI: 0000000000000000
RBP: 0000000000000000 R08: 0000000000000000 R09: ffff8807c1e524a8
R10: ffffffff80172daf R11: 0000000000000246 R12: ffffffff80113262
R13: 0000000000000000 R14: ffff88078f01f8f8 R15: ffff88078f01f8f8
FS:  00002b8a13c69b70(0000) GS:ffffffff803fbd80(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000
Process xentop (pid: 6498, threadinfo ffff88078f01e000, task ffff8807be9db180)
Stack: 0000000000000040 ffffffff801134c8 ffff8800041a9540 0000000000000000 
       0000000000000152 ffffffff8014ee2d ffffffff80406d80 000000000000a900 
       0000000000000152 ffff8800041a9540 
Call Trace: <IRQ> <ffffffff801134c8>{smp_call_function_interrupt+79}
       <ffffffff8014ee2d>{handle_IRQ_event+78} <ffffffff8014ef10>{__do_IRQ+153}
       <ffffffff8010d661>{do_IRQ+60} <ffffffff8023834a>{evtchn_do_upcall+134}
       <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff80172daf>{try_to_free_buffers+96} <ffffffff8010722a>{hypercall_page+554}
       <ffffffff8010722a>{hypercall_page+554} <ffffffff80237bfd>{force_evtchn_callback+10}
       <ffffffff80158e15>{shrink_zone+3853} <ffffffff802cd0ab>{thread_return+0}
       <ffffffff80113fc1>{xen_send_IPI_mask+0} <ffffffff80237cf1>{notify_remote_via_irq+44}
       <ffffffff80134fb6>{lock_timer_base+27} <ffffffff80135028>{try_to_del_timer_sync+81}
       <ffffffff802cd47c>{io_schedule_timeout+50} <ffffffff801cd57c>{blk_congestion_wait+132}
       <ffffffff8013fa4d>{autoremove_wake_function+0} <ffffffff80159715>{try_to_free_pages+342}
       <ffffffff80154cdf>{__alloc_pages+422} <ffffffff8015667c>{__do_page_cache_readahead+245}
       <ffffffff8015016c>{sync_page+0} <ffffffff802cd4b9>{io_schedule+50}
       <ffffffff802cdcfd>{__wait_on_bit_lock+91} <ffffffff80152893>{filemap_nopage+323}
       <ffffffff8015d2e3>{__handle_mm_fault+1415} <ffffffff801180d0>{do_page_fault+1938}
       <ffffffff80183bb7>{__pollwait+0} <ffffffff8014da85>{audit_syscall_exit+103}
       <ffffffff8010b5c7>{error_exit+0}

Code: f4 eb fd 41 55 49 89 fd 48 c7 c7 c8 c9 3c 80 41 54 55 48 89 
RIP <ffffffff8011326b>{smp_really_stop_cpu+9} RSP <ffff880000cafea8>
 <0>Kernel panic - not syncing: Aiee, killing interrupt handler!
 Badness in do_unblank_screen at drivers/char/vt.c:2830

Call Trace: <IRQ> <ffffffff80228114>{do_unblank_screen+76}
       <ffffffff8011775b>{bust_spinlocks+28} <ffffffff8012b880>{panic+136}
       <ffffffff8012e91c>{do_exit+124} <ffffffff801183cc>{do_page_fault+2702}
       <ffffffff8010e483>{monotonic_clock+53} <ffffffff801daa07>{find_next_bit+85}
       <ffffffff8012408f>{move_tasks+183} <ffffffff8011011e>{xen_panic_event+30}
       <ffffffff8010b5c7>{error_exit+0} <ffffffff8011011e>{xen_panic_event+30}
       <ffffffff8011011e>{xen_panic_event+30} <ffffffff801134c8>{smp_call_function_interrupt+79}
       <ffffffff8014ee2d>{handle_IRQ_event+78} <ffffffff8014ef10>{__do_IRQ+153}
       <ffffffff8010d661>{do_IRQ+60} <ffffffff8023834a>{evtchn_do_upcall+134}
       <ffffffff8010b672>{do_hypervisor_callback+30} <EOI>
       <ffffffff8010722a>{hypercall_page+554} <ffffffff8010722a>{hypercall_page+554}
       <ffffffff80237bfd>{force_evtchn_callback+10} <ffffffff801585f6>{shrink_zone+1774}
       <ffffffff802cd0ab>{thread_return+0} <ffffffff8019f6c7>{mb_cache_shrink_fn+36}
       <ffffffff80159715>{try_to_free_pages+342} <ffffffff8015335c>{out_of_memory+368}
       <ffffffff80154cdf>{__alloc_pages+422} <ffffffff80167143>{read_swap_cache_async+66}
       <ffffffff8015b333>{swapin_readahead+78} <ffffffff8015d670>{__handle_mm_fault+2324}
       <ffffffff801180d0>{do_page_fault+1938} <ffffffff8014da85>{audit_syscall_exit+103}
       <ffffffff8010b5c7>{error_exit+0}
 (XEN) Domain 0 crashed: rebooting machine in 5 seconds.

[-- Attachment #3: Type: text/plain, Size: 138 bytes --]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

^ permalink raw reply	[flat|nested] 7+ messages in thread
end of thread, other threads:[~2006-05-04 15:49 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2006-05-03 20:58 xm mem-set causes kernel panic Krysan, Susan
2006-05-03 21:29 ` Andrew Theurer
  -- strict thread matches above, loose matches on Subject: below --
2006-05-04 15:49 Puthiyaparambil, Aravindh
2006-05-03 21:15 Ian Pratt
2006-05-04 10:03 ` Keir Fraser
2006-05-03 20:18 Ian Pratt
2006-05-03 19:01 Krysan, Susan

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.