* order 1 page allocation failures
@ 2011-09-27 22:10 Thomas Fjellstrom
2011-09-29 17:26 ` Marcelo Tosatti
0 siblings, 1 reply; 2+ messages in thread
From: Thomas Fjellstrom @ 2011-09-27 22:10 UTC (permalink / raw)
To: kvm
Hi,
I've been having some issues with KVM recently where one or more vms will cause page allocation failure messages, usually with the backtrace including networking functions, example follows:
[362409.429944] kvm: page allocation failure: order:1, mode:0x20
[362409.429957] Pid: 3453, comm: kvm Not tainted 3.0.0-1-amd64 #1
[362409.429965] Call Trace:
[362409.429970] <IRQ> [<ffffffff810b9c90>] ? warn_alloc_failed+0x108/0x11b
[362409.429998] [<ffffffff810bcd78>] ? __alloc_pages_nodemask+0x6e6/0x75c
[362409.430012] [<ffffffff810ec0c0>] ? kmem_getpages+0x55/0xf0
[362409.430022] [<ffffffff810ec87a>] ? fallback_alloc+0x129/0x1c1
[362409.430035] [<ffffffff8100e28d>] ? paravirt_read_tsc+0x5/0x8
[362409.430045] [<ffffffff810ed10e>] ? kmem_cache_alloc+0x73/0xf0
[362409.430057] [<ffffffff812707a2>] ? sk_prot_alloc+0x2b/0x128
[362409.430067] [<ffffffff81270965>] ? sk_clone+0x14/0x2bd
[362409.430077] [<ffffffff812ade7d>] ? inet_csk_clone+0x10/0x91
[362409.430088] [<ffffffff812c1aae>] ? tcp_create_openreq_child+0x21/0x41a
[362409.430099] [<ffffffff812bf98a>] ? tcp_v4_syn_recv_sock+0x33/0x208
[362409.430110] [<ffffffff812c2441>] ? tcp_check_req+0x1ff/0x2dd
[362409.430122] [<ffffffff812adc06>] ? inet_csk_search_req+0x35/0xa7
[362409.430132] [<ffffffff812bf4f1>] ? tcp_v4_do_rcv+0x206/0x32c
[362409.430144] [<ffffffff812c15d4>] ? tcp_v4_rcv+0x419/0x66c
[362409.430154] [<ffffffff8100e74a>] ? native_sched_clock+0x28/0x30
[362409.430173] [<ffffffff812a5a0c>] ? ip_local_deliver_finish+0x14b/0x1bb
[362409.430186] [<ffffffff8127cc8f>] ? __netif_receive_skb+0x3d7/0x40b
[362409.430197] [<ffffffff8127d74b>] ? netif_receive_skb+0x52/0x58
[362409.430220] [<ffffffffa04b5af6>] ? br_nf_pre_routing_finish+0x1d4/0x1e1 [bridge]
[362409.430241] [<ffffffffa04b5111>] ? NF_HOOK_THRESH+0x3b/0x55 [bridge]
[362409.430260] [<ffffffffa04b60ed>] ? br_nf_pre_routing+0x3be/0x3cb [bridge]
[362409.430272] [<ffffffff8129fb69>] ? nf_iterate+0x41/0x77
[362409.430288] [<ffffffffa04b13a3>] ? NF_HOOK.clone.4+0x56/0x56 [bridge]
[362409.430305] [<ffffffffa04b13a3>] ? NF_HOOK.clone.4+0x56/0x56 [bridge]
[362409.430315] [<ffffffff8129fc12>] ? nf_hook_slow+0x73/0x111
[362409.430330] [<ffffffffa04b13a3>] ? NF_HOOK.clone.4+0x56/0x56 [bridge]
[362409.430342] [<ffffffff8103f0a4>] ? try_to_wake_up+0x199/0x199
[362409.430358] [<ffffffffa04b13a3>] ? NF_HOOK.clone.4+0x56/0x56 [bridge]
[362409.430375] [<ffffffffa04b1389>] ? NF_HOOK.clone.4+0x3c/0x56 [bridge]
[362409.430392] [<ffffffffa04b1745>] ? br_handle_frame+0x1af/0x1c6 [bridge]
[362409.430408] [<ffffffffa04b1596>] ? br_handle_frame_finish+0x1f3/0x1f3 [bridge]
[362409.430420] [<ffffffff8127cb7c>] ? __netif_receive_skb+0x2c4/0x40b
[362409.430432] [<ffffffff8127cd3b>] ? process_backlog+0x78/0x157
[362409.430443] [<ffffffff8127dd68>] ? net_rx_action+0xa4/0x1b2
[362409.430454] [<ffffffff81038189>] ? test_tsk_need_resched+0xe/0x17
[362409.430465] [<ffffffff8104bdd4>] ? __do_softirq+0xb9/0x178
[362409.430476] [<ffffffff8133cf1c>] ? call_softirq+0x1c/0x30
[362409.430481] <EOI> [<ffffffff8100a9ef>] ? do_softirq+0x3f/0x84
[362409.430498] [<ffffffff8127e051>] ? netif_rx_ni+0x1e/0x27
[362409.430509] [<ffffffffa06af7a8>] ? tun_get_user+0x390/0x3b8 [tun]
[362409.430520] [<ffffffff81060000>] ? bit_waitqueue+0x71/0xa4
[362409.430529] [<ffffffff81023145>] ? _flat_send_IPI_mask+0x6a/0x7c
[362409.430541] [<ffffffffa06afbad>] ? tun_get_socket+0x3b/0x3b [tun]
[362409.430552] [<ffffffffa06afc0b>] ? tun_chr_aio_write+0x5e/0x79 [tun]
[362409.430563] [<ffffffff810fc193>] ? do_sync_readv_writev+0x9a/0xd5
[362409.430574] [<ffffffff810383fc>] ? need_resched+0x1a/0x23
[362409.430585] [<ffffffff813358d7>] ? _cond_resched+0x9/0x20
[362409.430596] [<ffffffff810fb603>] ? copy_from_user+0x18/0x30
[362409.430608] [<ffffffff811653e9>] ? security_file_permission+0x18/0x33
[362409.430618] [<ffffffff810fc3e1>] ? do_readv_writev+0xa4/0x11a
[362409.430627] [<ffffffff810fcb94>] ? fput+0x1a/0x1a2
[362409.430636] [<ffffffff810fc5c5>] ? sys_writev+0x45/0x90
[362409.430647] [<ffffffff8133bd12>] ? system_call_fastpath+0x16/0x1b
[362409.430654] Mem-Info:
[362409.430659] Node 0 DMA per-cpu:
[362409.430667] CPU 0: hi: 0, btch: 1 usd: 0
[362409.430673] CPU 1: hi: 0, btch: 1 usd: 0
[362409.430679] CPU 2: hi: 0, btch: 1 usd: 0
[362409.430686] CPU 3: hi: 0, btch: 1 usd: 0
[362409.430691] Node 0 DMA32 per-cpu:
[362409.430699] CPU 0: hi: 186, btch: 31 usd: 147
[362409.430705] CPU 1: hi: 186, btch: 31 usd: 82
[362409.430712] CPU 2: hi: 186, btch: 31 usd: 171
[362409.430718] CPU 3: hi: 186, btch: 31 usd: 86
[362409.430724] Node 0 Normal per-cpu:
[362409.430730] CPU 0: hi: 186, btch: 31 usd: 157
[362409.430737] CPU 1: hi: 186, btch: 31 usd: 96
[362409.430743] CPU 2: hi: 186, btch: 31 usd: 163
[362409.430749] CPU 3: hi: 186, btch: 31 usd: 173
[362409.430764] active_anon:576670 inactive_anon:110462 isolated_anon:0
[362409.430769] active_file:615766 inactive_file:638287 isolated_file:0
[362409.430774] unevictable:0 dirty:3203 writeback:0 unstable:0
[362409.430778] free:31113 slab_reclaimable:36977 slab_unreclaimable:11009
[362409.430783] mapped:11738 shmem:226 pagetables:9104 bounce:0
[362409.430791] Node 0 DMA free:15912kB min:128kB low:160kB high:192kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15688kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
[362409.430827] lowmem_reserve[]: 0 3254 8051 8051
[362409.430837] Node 0 DMA32 free:57588kB min:27260kB low:34072kB high:40888kB active_anon:454620kB inactive_anon:92920kB active_file:1279680kB inactive_file:1348708kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:3332192kB mlocked:0kB dirty:5928kB writeback:0kB mapped:17404kB shmem:572kB slab_reclaimable:71792kB slab_unreclaimable:5488kB kernel_stack:296kB pagetables:2420kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
[362409.430877] lowmem_reserve[]: 0 0 4797 4797
[362409.430886] Node 0 Normal free:51448kB min:40192kB low:50240kB high:60288kB active_anon:1852060kB inactive_anon:348928kB active_file:1183384kB inactive_file:1204152kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:4912640kB mlocked:0kB dirty:6884kB writeback:0kB mapped:29548kB shmem:332kB slab_reclaimable:76116kB slab_unreclaimable:38548kB kernel_stack:3184kB pagetables:33996kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:9 all_unreclaimable? no
[362409.430926] lowmem_reserve[]: 0 0 0 0
[362409.430935] Node 0 DMA: 0*4kB 1*8kB 0*16kB 1*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15912kB
[362409.430959] Node 0 DMA32: 13011*4kB 198*8kB 7*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 57836kB
[362409.430983] Node 0 Normal: 11740*4kB 49*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 1*4096kB = 51448kB
[362409.431007] 1269832 total pagecache pages
[362409.431012] 15734 pages in swap cache
[362409.431018] Swap cache stats: add 129275, delete 113541, find 78987/83931
[362409.431025] Free swap = 7546528kB
[362409.431030] Total swap = 7811068kB
[362409.433896] 2097136 pages RAM
[362409.433896] 47949 pages reserved
[362409.433896] 681769 pages shared
[362409.433896] 1367619 pages non-shared
The server has 8G of ram, and usually never uses more than about 4G (sitting at 3.4G right now).
--
Thomas Fjellstrom
thomas@fjellstrom.ca
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: order 1 page allocation failures
2011-09-27 22:10 order 1 page allocation failures Thomas Fjellstrom
@ 2011-09-29 17:26 ` Marcelo Tosatti
0 siblings, 0 replies; 2+ messages in thread
From: Marcelo Tosatti @ 2011-09-29 17:26 UTC (permalink / raw)
To: Thomas Fjellstrom; +Cc: kvm
On Tue, Sep 27, 2011 at 04:10:20PM -0600, Thomas Fjellstrom wrote:
> Hi,
>
> I've been having some issues with KVM recently where one or more vms will cause page allocation failure messages, usually with the backtrace including networking functions, example follows:
>
> [362409.429944] kvm: page allocation failure: order:1, mode:0x20
> [362409.429957] Pid: 3453, comm: kvm Not tainted 3.0.0-1-amd64 #1
> [362409.429965] Call Trace:
> [362409.429970] <IRQ> [<ffffffff810b9c90>] ? warn_alloc_failed+0x108/0x11b
> [362409.429998] [<ffffffff810bcd78>] ? __alloc_pages_nodemask+0x6e6/0x75c
> [362409.430012] [<ffffffff810ec0c0>] ? kmem_getpages+0x55/0xf0
> [362409.430022] [<ffffffff810ec87a>] ? fallback_alloc+0x129/0x1c1
> [362409.430035] [<ffffffff8100e28d>] ? paravirt_read_tsc+0x5/0x8
> [362409.430045] [<ffffffff810ed10e>] ? kmem_cache_alloc+0x73/0xf0
> [362409.430057] [<ffffffff812707a2>] ? sk_prot_alloc+0x2b/0x128
> [362409.430067] [<ffffffff81270965>] ? sk_clone+0x14/0x2bd
> [362409.430077] [<ffffffff812ade7d>] ? inet_csk_clone+0x10/0x91
> [362409.430088] [<ffffffff812c1aae>] ? tcp_create_openreq_child+0x21/0x41a
> [362409.430099] [<ffffffff812bf98a>] ? tcp_v4_syn_recv_sock+0x33/0x208
> [362409.430110] [<ffffffff812c2441>] ? tcp_check_req+0x1ff/0x2dd
> [362409.430122] [<ffffffff812adc06>] ? inet_csk_search_req+0x35/0xa7
> [362409.430132] [<ffffffff812bf4f1>] ? tcp_v4_do_rcv+0x206/0x32c
> [362409.430144] [<ffffffff812c15d4>] ? tcp_v4_rcv+0x419/0x66c
> [362409.430154] [<ffffffff8100e74a>] ? native_sched_clock+0x28/0x30
> [362409.430173] [<ffffffff812a5a0c>] ? ip_local_deliver_finish+0x14b/0x1bb
> [362409.430186] [<ffffffff8127cc8f>] ? __netif_receive_skb+0x3d7/0x40b
> [362409.430197] [<ffffffff8127d74b>] ? netif_receive_skb+0x52/0x58
> [362409.430220] [<ffffffffa04b5af6>] ? br_nf_pre_routing_finish+0x1d4/0x1e1 [bridge]
> [362409.430241] [<ffffffffa04b5111>] ? NF_HOOK_THRESH+0x3b/0x55 [bridge]
> [362409.430260] [<ffffffffa04b60ed>] ? br_nf_pre_routing+0x3be/0x3cb [bridge]
> [362409.430272] [<ffffffff8129fb69>] ? nf_iterate+0x41/0x77
> [362409.430288] [<ffffffffa04b13a3>] ? NF_HOOK.clone.4+0x56/0x56 [bridge]
> [362409.430305] [<ffffffffa04b13a3>] ? NF_HOOK.clone.4+0x56/0x56 [bridge]
> [362409.430315] [<ffffffff8129fc12>] ? nf_hook_slow+0x73/0x111
> [362409.430330] [<ffffffffa04b13a3>] ? NF_HOOK.clone.4+0x56/0x56 [bridge]
> [362409.430342] [<ffffffff8103f0a4>] ? try_to_wake_up+0x199/0x199
> [362409.430358] [<ffffffffa04b13a3>] ? NF_HOOK.clone.4+0x56/0x56 [bridge]
> [362409.430375] [<ffffffffa04b1389>] ? NF_HOOK.clone.4+0x3c/0x56 [bridge]
> [362409.430392] [<ffffffffa04b1745>] ? br_handle_frame+0x1af/0x1c6 [bridge]
> [362409.430408] [<ffffffffa04b1596>] ? br_handle_frame_finish+0x1f3/0x1f3 [bridge]
> [362409.430420] [<ffffffff8127cb7c>] ? __netif_receive_skb+0x2c4/0x40b
> [362409.430432] [<ffffffff8127cd3b>] ? process_backlog+0x78/0x157
> [362409.430443] [<ffffffff8127dd68>] ? net_rx_action+0xa4/0x1b2
> [362409.430454] [<ffffffff81038189>] ? test_tsk_need_resched+0xe/0x17
> [362409.430465] [<ffffffff8104bdd4>] ? __do_softirq+0xb9/0x178
> [362409.430476] [<ffffffff8133cf1c>] ? call_softirq+0x1c/0x30
>
> The server has 8G of ram, and usually never uses more than about 4G (sitting at 3.4G right now).
This is a guest problem, please report it to the netfilter/lkml lists
(if its not a known issue with particular kernel version already).
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2011-09-29 17:42 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-09-27 22:10 order 1 page allocation failures Thomas Fjellstrom
2011-09-29 17:26 ` Marcelo Tosatti
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox