out of memory page request

out of memory page request

[David Brown]

I just noticed something bad, I forgot to pass the -m argument when
starting my kvm session. So I tried to load an initrd that was 100+M
into the default memory size which is smaller using an etherboot image
for the rtl8139 card. Obviously this is my bad but I got a kernel
oopse and it froze the qemu process and wouldn't release the memory
back to the system.

Could this be handled a little more cleanly? Maybe kill the qemu
process and hand the memory back to the system saying that it couldn't
allocate the memory or something?

[ 5487.411029] Unable to handle kernel paging request at ffff810213a9ad04 RIP:
[ 5487.411036]  [<ffffffff88ae75b9>] :kvm:ioapic_mmio_write+0x59/0xa0
[ 5487.411052] PGD 8063 PUD 0
[ 5487.411055] Oops: 0002 [1] SMP
[ 5487.411057] CPU 1
[ 5487.411058] Modules linked in: tun bitrev crc32 kvm_intel kvm
nls_cp437 vfat fat udf nls_iso8859_1 ntfs nls_base ipv6 binfmt_misc
snd_pcm_oss snd_mixer_oss snd_seq_midi snd_emu10k1_synth
snd_emux_synth snd_seq_virmidi snd_seq_midi_event snd_seq_midi_emul
snd_seq bridge usb_storage libusual usbmouse usbkbd usbhid hid
ff_memless snd_emu10k1 snd_rawmidi snd_ac97_codec ohci1394 ac97_bus
snd_pcm snd_seq_device ieee1394 snd_timer snd_page_alloc snd_util_mem
nvidia(P) firewire_ohci firewire_core snd_hwdep snd crc_itu_t
i2c_nforce2 generic amd74xx soundcore tg3 emu10k1_gp gameport rtc
i2c_core ide_core 8250_pnp 8250 serial_core serio_raw ehci_hcd
ohci_hcd usbcore pcspkr shpchp pci_hotplug psmouse sr_mod cdrom sg
evdev tsdev ext3 mbcache jbd ata_generic pata_amd sata_nv libata
sd_mod scsi_mod dm_crypt aes sha512 sha256 dm_mod
[ 5487.411097] Pid: 25075, comm: qemu-system-x86 Tainted: P       2.6.22.9 #2
[ 5487.411099] RIP: 0010:[<ffffffff88ae75b9>]  [<ffffffff88ae75b9>]
:kvm:ioapic_mmio_write+0x59/0xa0
[ 5487.411106] RSP: 0018:ffff8100321d9870  EFLAGS: 00010202
[ 5487.411108] RAX: 0000000037e835dd RBX: ffff810040a80000 RCX: 0000000053b9fb0f
[ 5487.411110] RDX: 0000000000000001 RSI: 0000000037e835dd RDI: ffff81005467fe00
[ 5487.411112] RBP: 00000000000fec00 R08: 0000000000000004 R09: 0000000000000004
[ 5487.411114] R10: ffff810040a80000 R11: ffffffff88ad89f0 R12: 00000000fec00010
[ 5487.411116] R13: 0000000000000004 R14: ffff810040a810e8 R15: 0000000000000000
[ 5487.411118] FS:  00002ba48158ad30(0000) GS:ffff81007f968280(0000)
knlGS:0000000000000000
[ 5487.411120] CS:  0010 DS: 002b ES: 002b CR0: 0000000080050033
[ 5487.411122] CR2: ffff810213a9ad04 CR3: 0000000025c87000 CR4: 00000000000026e0
[ 5487.411125] Process qemu-system-x86 (pid: 25075, threadinfo
ffff8100321d8000, task ffff8100168314a0)
[ 5487.411126] Stack:  ffffffff88ada2d2 0000000000000004
00000000fec00010 ffff810040a810b8
[ 5487.411131]  ffff810040a81068 00000000fec00010 ffff810040a81110
ffffffff88af1ee0
[ 5487.411134]  ffffffff88ae1638 0000000000000000 ffff810040a81110
0000000000000000
[ 5487.411137] Call Trace:
[ 5487.411143]  [<ffffffff88ada2d2>]
:kvm:emulator_write_emulated_onepage+0xc2/0x190
[ 5487.411152]  [<ffffffff88ae1638>] :kvm:x86_emulate_insn+0x618/0x4190
[ 5487.411161]  [<ffffffff88af5467>] :kvm_intel:vmcs_readl+0x17/0x20
[ 5487.411169]  [<ffffffff88ada0d1>] :kvm:emulate_instruction+0x1a1/0x2e0
[ 5487.411174]  [<ffffffff88af83b3>] :kvm_intel:handle_exception+0x1f3/0x2c0
[ 5487.411182]  [<ffffffff88ada6d2>] :kvm:kvm_vcpu_ioctl_run+0x102/0x460
[ 5487.411188]  [<ffffffff88adc6c1>] :kvm:kvm_vm_ioctl+0x511/0xa70
[ 5487.411195]  [<ffffffff88adafcb>] :kvm:kvm_vcpu_ioctl+0x59b/0xf40
[ 5487.411203]  [<ffffffff810915e0>] fget_light+0x30/0x90
[ 5487.411208]  [<ffffffff8109f0b8>] do_select+0x518/0x540
[ 5487.411215]  [<ffffffff8109f6f0>] __pollwait+0x0/0x130
[ 5487.411219]  [<ffffffff810272bc>] task_rq_lock+0x4c/0x90
[ 5487.411223]  [<ffffffff810299ed>] try_to_wake_up+0x5d/0x4b0
[ 5487.411228]  [<ffffffff81026b97>] __wake_up_common+0x47/0x70
[ 5487.411232]  [<ffffffff810271f3>] __wake_up+0x43/0x70
[ 5487.411237]  [<ffffffff8118f3f1>] sock_def_readable+0x61/0x70
[ 5487.411241]  [<ffffffff81201cb4>] unix_stream_sendmsg+0x1c4/0x390
[ 5487.411244]  [<ffffffff8111c8ad>] copy_user_generic_string+0xd/0x40
[ 5487.411252]  [<ffffffff8118a031>] sock_aio_write+0x171/0x180
[ 5487.411260]  [<ffffffff810ef20a>] avc_has_perm+0x5a/0x80
[ 5487.411266]  [<ffffffff810f027b>] inode_has_perm+0x5b/0xa0
[ 5487.411268]  [<ffffffff81039dc9>] __dequeue_signal+0x19/0x1d0
[ 5487.411272]  [<ffffffff8103962e>] recalc_sigpending+0xe/0x30
[ 5487.411274]  [<ffffffff8103b78c>] dequeue_signal+0x5c/0x190
[ 5487.411279]  [<ffffffff8109dc95>] do_ioctl+0x35/0xe0
[ 5487.411282]  [<ffffffff8109ddb4>] vfs_ioctl+0x74/0x2d0
[ 5487.411286]  [<ffffffff8109e0a5>] sys_ioctl+0x95/0xb0
[ 5487.411290]  [<ffffffff8100a0fe>] system_call+0x7e/0x83
[ 5487.411295]
[ 5487.411296]
[ 5487.411296] Code: 89 4c c7 1c b8 01 00 00 00 89 f1 d3 e0 85 47 10
74 c3 e9 b0
[ 5487.411303] RIP  [<ffffffff88ae75b9>] :kvm:ioapic_mmio_write+0x59/0xa0
[ 5487.411310]  RSP <ffff8100321d9870>
[ 5487.411312] CR2: ffff810213a9ad04
[ 5487.421581] br0: port 2(tap0) entering disabled state
[ 5487.427775] br0: port 2(tap0) entering disabled state

This is using kvm 45 and linux kernel 2.6.22.9.

Thanks,
- David Brown

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/

Re: out of memory page request

[Avi Kivity]

David Brown wrote:
> I just noticed something bad, I forgot to pass the -m argument when
> starting my kvm session. So I tried to load an initrd that was 100+M
> into the default memory size which is smaller using an etherboot image
> for the rtl8139 card. Obviously this is my bad but I got a kernel
> oopse and it froze the qemu process and wouldn't release the memory
> back to the system.
>
> Could this be handled a little more cleanly? Maybe kill the qemu
> process and hand the memory back to the system saying that it couldn't
> allocate the memory or something?
>
> [ 5487.411029] Unable to handle kernel paging request at ffff810213a9ad04 RIP:
> [ 5487.411036]  [<ffffffff88ae75b9>] :kvm:ioapic_mmio_write+0x59/0xa0
>   

Can you run 'gdb /path/to/kvm.ko' and post the output of 'disassemble 
ioapic_mmio_write' please?

-- 
Any sufficiently difficult bug is indistinguishable from a feature.


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/

Re: out of memory page request

[David Brown]

> Can you run 'gdb /path/to/kvm.ko' and post the output of 'disassemble
> ioapic_mmio_write' please?
>

Sorry for not getting back sooner.

(gdb) disassemble ioapic_mmio_write
Dump of assembler code for function ioapic_mmio_write:
0x000000000000f540 <ioapic_mmio_write+0>:       mov    %edx,%r8d
0x000000000000f543 <ioapic_mmio_write+3>:       cmp    $0x4,%edx
0x000000000000f546 <ioapic_mmio_write+6>:       mov    0x20(%rdi),%rdi
0x000000000000f54a <ioapic_mmio_write+10>:      sete   %dl
0x000000000000f54d <ioapic_mmio_write+13>:      cmp    $0x8,%r8d
0x000000000000f551 <ioapic_mmio_write+17>:      sete   %al
0x000000000000f554 <ioapic_mmio_write+20>:      or     %al,%dl
0x000000000000f556 <ioapic_mmio_write+22>:      je     0xf570
<ioapic_mmio_write+48>
0x000000000000f558 <ioapic_mmio_write+24>:      mov    %rsi,%rax
0x000000000000f55b <ioapic_mmio_write+27>:      mov    (%rcx),%ecx
0x000000000000f55d <ioapic_mmio_write+29>:      and    $0xff,%eax
0x000000000000f562 <ioapic_mmio_write+34>:      jne    0xf568
<ioapic_mmio_write+40>
0x000000000000f564 <ioapic_mmio_write+36>:      mov    %ecx,0x8(%rdi)
0x000000000000f567 <ioapic_mmio_write+39>:      retq
0x000000000000f568 <ioapic_mmio_write+40>:      cmp    $0x10,%rax
0x000000000000f56c <ioapic_mmio_write+44>:      je     0xf581
<ioapic_mmio_write+65>
0x000000000000f56e <ioapic_mmio_write+46>:      repz retq
0x000000000000f570 <ioapic_mmio_write+48>:      mov    %r8d,%esi
0x000000000000f573 <ioapic_mmio_write+51>:      mov    $0x0,%rdi
---Type <return> to continue, or q <return> to quit---
0x000000000000f57a <ioapic_mmio_write+58>:      xor    %eax,%eax
0x000000000000f57c <ioapic_mmio_write+60>:      jmpq   0xf581
<ioapic_mmio_write+65>
0x000000000000f581 <ioapic_mmio_write+65>:      mov    0x8(%rdi),%eax
0x000000000000f584 <ioapic_mmio_write+68>:      test   %eax,%eax
0x000000000000f586 <ioapic_mmio_write+70>:      je     0xf5b0
<ioapic_mmio_write+112>
0x000000000000f588 <ioapic_mmio_write+72>:      cmp    $0x2,%eax
0x000000000000f58b <ioapic_mmio_write+75>:      jbe    0xf5ba
<ioapic_mmio_write+122>
0x000000000000f58d <ioapic_mmio_write+77>:      lea
0xfffffffffffffff0(%rax),%esi
0x000000000000f590 <ioapic_mmio_write+80>:      shr    %esi
0x000000000000f592 <ioapic_mmio_write+82>:      test   $0x1,%al
0x000000000000f594 <ioapic_mmio_write+84>:      je     0xf5bc
<ioapic_mmio_write+124>
0x000000000000f596 <ioapic_mmio_write+86>:      movslq %esi,%rax
0x000000000000f599 <ioapic_mmio_write+89>:      mov    %ecx,0x1c(%rdi,%rax,8)
0x000000000000f59d <ioapic_mmio_write+93>:      mov    $0x1,%eax
0x000000000000f5a2 <ioapic_mmio_write+98>:      mov    %esi,%ecx
0x000000000000f5a4 <ioapic_mmio_write+100>:     shl    %cl,%eax
0x000000000000f5a6 <ioapic_mmio_write+102>:     test   %eax,0x10(%rdi)
0x000000000000f5a9 <ioapic_mmio_write+105>:     je     0xf56e
<ioapic_mmio_write---Type <return> to continue, or q <return> to
quit---
+46>
0x000000000000f5ab <ioapic_mmio_write+107>:     jmpq   0xf460 <ioapic_service>
0x000000000000f5b0 <ioapic_mmio_write+112>:     shr    $0x18,%ecx
0x000000000000f5b3 <ioapic_mmio_write+115>:     and    $0xf,%ecx
0x000000000000f5b6 <ioapic_mmio_write+118>:     mov    %ecx,0xc(%rdi)
0x000000000000f5b9 <ioapic_mmio_write+121>:     retq
0x000000000000f5ba <ioapic_mmio_write+122>:     repz retq
0x000000000000f5bc <ioapic_mmio_write+124>:     movslq %esi,%rax
0x000000000000f5bf <ioapic_mmio_write+127>:     mov    $0xffffffff00000000,%rdx
0x000000000000f5c9 <ioapic_mmio_write+137>:     mov    %ecx,%ecx
0x000000000000f5cb <ioapic_mmio_write+139>:     and    0x18(%rdi,%rax,8),%rdx
0x000000000000f5d0 <ioapic_mmio_write+144>:     or     %rcx,%rdx
0x000000000000f5d3 <ioapic_mmio_write+147>:     mov    %rdx,0x18(%rdi,%rax,8)
0x000000000000f5d8 <ioapic_mmio_write+152>:     andb   $0xbf,0x19(%rdi,%rax,8)
0x000000000000f5dd <ioapic_mmio_write+157>:     jmp    0xf59d
<ioapic_mmio_write+93>
End of assembler dump.
(gdb)

I'm using gcc 4.1.2 and glibc kernel headers for 2.6.22.6 currently.

Thanks,
- David Brown

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/

Re: out of memory page request

[Avi Kivity]

[-- Attachment #1: Type: text/plain, Size: 439 bytes --]

David Brown wrote:
>> Can you run 'gdb /path/to/kvm.ko' and post the output of 'disassemble
>> ioapic_mmio_write' please?
>>
>>     
>
> Sorry for not getting back sooner.
>
> (gdb) disassemble ioapic_mmio_write
> Dump of assembler code for function ioapic_mmio_write:
>   

The attached patch should fix, though there's probably another problem 
lurking in there.

-- 
Any sufficiently difficult bug is indistinguishable from a feature.


[-- Attachment #2: ioapic-indirect-write-fix.patch --]
[-- Type: text/x-patch, Size: 1092 bytes --]

commit f1b8c28f1886c9375361d7f2ebca1f742ea6bc5f
Author: Avi Kivity <avi-atKUWr5tajBWk0Htik3J/w@public.gmane.org>
Date:   Mon Oct 8 10:01:45 2007 +0200

    KVM: Check I/O APIC indirect index before writing
    
    Signed-off-by: Avi Kivity <avi-atKUWr5tajBWk0Htik3J/w@public.gmane.org>

diff --git a/drivers/kvm/ioapic.c b/drivers/kvm/ioapic.c
index 9eb5058..c7992e6 100644
--- a/drivers/kvm/ioapic.c
+++ b/drivers/kvm/ioapic.c
@@ -96,7 +96,7 @@ static void ioapic_service(struct kvm_ioapic *ioapic, unsigned int idx)
 
 static void ioapic_write_indirect(struct kvm_ioapic *ioapic, u32 val)
 {
-	int index;
+	unsigned index;
 
 	switch (ioapic->ioregsel) {
 	case IOAPIC_REG_VERSION:
@@ -114,7 +114,8 @@ static void ioapic_write_indirect(struct kvm_ioapic *ioapic, u32 val)
 		index = (ioapic->ioregsel - 0x10) >> 1;
 
 		ioapic_debug("change redir index %x val %x", index, val);
-		ASSERT(irq < IOAPIC_NUM_PINS);
+		if (index >= IOAPIC_NUM_PINS)
+			return;
 		if (ioapic->ioregsel & 1) {
 			ioapic->redirtbl[index].bits &= 0xffffffff;
 			ioapic->redirtbl[index].bits |= (u64) val << 32;

[-- Attachment #3: Type: text/plain, Size: 314 bytes --]

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/

[-- Attachment #4: Type: text/plain, Size: 186 bytes --]

_______________________________________________
kvm-devel mailing list
kvm-devel-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org
https://lists.sourceforge.net/lists/listinfo/kvm-devel

Re: out of memory page request

[Dong, Eddie]

kvm-devel-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org wrote:
> David Brown wrote:
>>> Can you run 'gdb /path/to/kvm.ko' and post the output of
>>> 'disassemble ioapic_mmio_write' please? 
>>> 
>>> 
>> 
>> Sorry for not getting back sooner.
>> 
>> (gdb) disassemble ioapic_mmio_write
>> Dump of assembler code for function ioapic_mmio_write:
>> 
> 
> The attached patch should fix, though there's probably another
> problem lurking in there. 
> 
Should we printk something to alert something error already?
Eddie

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/

Re: out of memory page request

[Avi Kivity]

Dong, Eddie wrote:
> kvm-devel-bounces-5NWGOfrQmneRv+LV9MX5uipxlwaOVQ5f@public.gmane.org wrote:
>   
>> David Brown wrote:
>>     
>>>> Can you run 'gdb /path/to/kvm.ko' and post the output of
>>>> 'disassemble ioapic_mmio_write' please? 
>>>>
>>>>
>>>>         
>>> Sorry for not getting back sooner.
>>>
>>> (gdb) disassemble ioapic_mmio_write
>>> Dump of assembler code for function ioapic_mmio_write:
>>>
>>>       
>> The attached patch should fix, though there's probably another
>> problem lurking in there. 
>>
>>     
> Should we printk something to alert something error already?
> Eddie
>   

It needs to be pr_debug() to avoid allowing a guest to spam the host 
logs.  But I don't think it will help in this case.  There's probably a 
bug under low memory conditions that causes the guest to execute 
incorrectly, and it just happened to hit this bug.  I don't think it's 
really related to the ioapic.


-- 
error compiling committee.c: too many arguments to function


-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/

Re: out of memory page request

[David Brown]

> The attached patch should fix, though there's probably another problem
> lurking in there.
>
> --
> Any sufficiently difficult bug is indistinguishable from a feature.
>

Thanks, the patch worked it didn't hoze the host anymore.

- David Brown

-------------------------------------------------------------------------
This SF.net email is sponsored by: Splunk Inc.
Still grepping through log files to find problems?  Stop.
Now Search log events and configuration files using AJAX and a browser.
Download your FREE copy of Splunk now >> http://get.splunk.com/