* devel/next-2.6.39 SLUB: Unable to allocate memory on node -1 (gfp=0x20)
@ 2011-05-23 7:12 Sander Eikelenboom
2011-05-24 13:48 ` Konrad Rzeszutek Wilk
0 siblings, 1 reply; 3+ messages in thread
From: Sander Eikelenboom @ 2011-05-23 7:12 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel
Hi Konrad,
It seems i have another problem with the devel/next-2.6.39 as dom0, don't know if it's xen related or not.
It doesn't crash the machine, also netwerk still seems to work, but these warns keep coming say one till 3 a minute.
I haven't changed the rest of the parameters/setup from the stable configuration i was running before with a 2.6.32 from jeremy.
The panic i reported earlier hasn't hit me again yet (after attaching serial console).
--
Sander
May 23 09:01:58 serveerstertje kernel: [128112.912479] swapper: page allocation failure. order:0, mode:0x4020
May 23 09:01:58 serveerstertje kernel: [128112.916453] Pid: 0, comm: swapper Not tainted 2.6.39-xen-next-2.6.39-konrad+ #5
May 23 09:01:58 serveerstertje kernel: [128112.916453] Call Trace:
May 23 09:01:58 serveerstertje kernel: [128112.916453] <IRQ> [<ffffffff8112213c>] __alloc_pages_nodemask+0x64c/0x7e0
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8132bdf6>] ? security_sock_rcv_skb+0x16/0x20
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff819899b8>] ? tcp_v4_rcv+0x828/0x930
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff811588c3>] new_slab+0x233/0x250
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff81158f34>] T.1026+0x194/0x440
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8190a014>] ? __netdev_alloc_skb+0x24/0x50
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8115afcc>] __kmalloc_track_caller+0x15c/0x1e0
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8190a014>] ? __netdev_alloc_skb+0x24/0x50
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff81908d67>] __alloc_skb+0x77/0x160
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8190a014>] __netdev_alloc_skb+0x24/0x50
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff815ea2fb>] rtl8169_rx_interrupt+0x20b/0x420
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff815eb51e>] ? rtl8169_interrupt+0xfe/0x3e0
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff815ed06c>] rtl8169_poll+0x3c/0x1c0
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8104c21d>] ? xen_force_evtchn_callback+0xd/0x10
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8191313e>] net_rx_action+0x14e/0x2d0
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8109ec61>] __do_softirq+0xd1/0x230
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff810e7b2d>] ? handle_edge_irq+0x6d/0x120
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff81a8cbdc>] call_softirq+0x1c/0x30
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8105231d>] do_softirq+0x8d/0xf0
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8109e96e>] irq_exit+0x9e/0xc0
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8140e675>] xen_evtchn_do_upcall+0x35/0x50
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff81a8cc2e>] xen_do_hypervisor_callback+0x1e/0x30
May 23 09:01:58 serveerstertje kernel: [128112.916453] <EOI> [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff810013aa>] ? hypercall_page+0x3aa/0x1000
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8104c280>] ? xen_safe_halt+0x10/0x20
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff81058f2a>] ? default_idle+0x5a/0x110
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8104f80b>] ? cpu_idle+0x7b/0xd0
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff81a4f22d>] ? rest_init+0x6d/0x80
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff821a5dde>] ? start_kernel+0x48d/0x498
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff821a5346>] ? x86_64_start_reservations+0x131/0x135
May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff821a9dcd>] ? xen_start_kernel+0x5d8/0x5df
May 23 09:01:58 serveerstertje kernel: [128112.916453] Mem-Info:
May 23 09:01:58 serveerstertje kernel: [128112.916453] DMA per-cpu:
May 23 09:01:58 serveerstertje kernel: [128112.916453] CPU 0: hi: 0, btch: 1 usd: 0
May 23 09:01:58 serveerstertje kernel: [128112.916453] CPU 1: hi: 0, btch: 1 usd: 0
May 23 09:01:58 serveerstertje kernel: [128112.916453] CPU 2: hi: 0, btch: 1 usd: 0
May 23 09:01:58 serveerstertje kernel: [128112.916453] CPU 3: hi: 0, btch: 1 usd: 0
May 23 09:01:58 serveerstertje kernel: [128112.916453] CPU 4: hi: 0, btch: 1 usd: 0
May 23 09:01:58 serveerstertje kernel: [128112.916453] CPU 5: hi: 0, btch: 1 usd: 0
May 23 09:01:58 serveerstertje kernel: [128112.916453] DMA32 per-cpu:
May 23 09:01:58 serveerstertje kernel: [128112.916453] CPU 0: hi: 186, btch: 31 usd: 197
May 23 09:01:58 serveerstertje kernel: [128112.916453] CPU 1: hi: 186, btch: 31 usd: 0
May 23 09:01:58 serveerstertje kernel: [128112.916453] CPU 2: hi: 186, btch: 31 usd: 0
May 23 09:01:58 serveerstertje kernel: [128112.916453] CPU 3: hi: 186, btch: 31 usd: 0
May 23 09:01:58 serveerstertje kernel: [128112.916453] CPU 4: hi: 186, btch: 31 usd: 0
May 23 09:01:58 serveerstertje kernel: [128112.916453] CPU 5: hi: 186, btch: 31 usd: 0
May 23 09:01:58 serveerstertje kernel: [128112.916453] Normal per-cpu:
May 23 09:01:58 serveerstertje kernel: [128112.916453] CPU 0: hi: 186, btch: 31 usd: 0
May 23 09:01:58 serveerstertje kernel: [128112.916453] CPU 1: hi: 186, btch: 31 usd: 0
May 23 09:01:58 serveerstertje kernel: [128112.916453] CPU 2: hi: 186, btch: 31 usd: 0
May 23 09:01:58 serveerstertje kernel: [128112.916453] CPU 3: hi: 186, btch: 31 usd: 0
May 23 09:01:58 serveerstertje kernel: [128112.916453] CPU 4: hi: 186, btch: 31 usd: 0
May 23 09:01:58 serveerstertje kernel: [128112.916453] CPU 5: hi: 186, btch: 31 usd: 0
May 23 09:01:58 serveerstertje kernel: [128112.916453] active_anon:2892 inactive_anon:13225 isolated_anon:0
May 23 09:01:58 serveerstertje kernel: [128112.916453] active_file:46371 inactive_file:46406 isolated_file:0
May 23 09:01:58 serveerstertje kernel: [128112.916453] unevictable:497 dirty:10074 writeback:0 unstable:0
May 23 09:01:58 serveerstertje kernel: [128112.916453] free:10892 slab_reclaimable:7833 slab_unreclaimable:4902
May 23 09:01:58 serveerstertje kernel: [128112.916453] mapped:1519 shmem:341 pagetables:573 bounce:0
May 23 09:01:58 serveerstertje kernel: [128112.916453] DMA free:15924kB min:20kB low:24kB high:28kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15700kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
May 23 09:01:58 serveerstertje kernel: [128112.916453] lowmem_reserve[]: 0 696 7513 7513
May 23 09:01:58 serveerstertje kernel: [128112.916453] DMA32 free:27644kB min:1024kB low:1280kB high:1536kB active_anon:11568kB inactive_anon:52900kB active_file:185484kB inactive_file:185624kB unevictable:1988kB isolated(anon):0kB isolated(file):0kB present:712928kB mlocked:1988kB dirty:40296kB writeback:0kB mapped:6076kB shmem:1364kB slab_reclaimable:31332kB slab_unreclaimable:19608kB kernel_stack:1680kB pagetables:2292kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:8 all_unreclaimable? no
May 23 09:01:58 serveerstertje kernel: [128112.916453] lowmem_reserve[]: 0 0 6817 6817
May 23 09:01:58 serveerstertje kernel: [128112.916453] Normal free:0kB min:10052kB low:12564kB high:15076kB active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:6981120kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:0kB slab_unreclaimable:0kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
May 23 09:01:58 serveerstertje kernel: [128112.916453] lowmem_reserve[]: 0 0 0 0
May 23 09:01:58 serveerstertje kernel: [128112.916453] DMA: 1*4kB 0*8kB 1*16kB 1*32kB 2*64kB 1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 3*4096kB = 15924kB
May 23 09:01:58 serveerstertje kernel: [128112.916453] DMA32: 545*4kB 182*8kB 374*16kB 340*32kB 88*64kB 10*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 27668kB
May 23 09:01:58 serveerstertje kernel: [128112.916453] Normal: 0*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 0kB
May 23 09:01:58 serveerstertje kernel: [128112.916453] 94198 total pagecache pages
May 23 09:01:58 serveerstertje kernel: [128112.916453] 667 pages in swap cache
May 23 09:01:58 serveerstertje kernel: [128112.916453] Swap cache stats: add 8251, delete 7584, find 211335/211728
May 23 09:01:58 serveerstertje kernel: [128112.916453] Free swap = 2078352kB
May 23 09:01:58 serveerstertje kernel: [128112.916453] Total swap = 2097148kB
May 23 09:01:58 serveerstertje kernel: [128112.916453] 1966064 pages RAM
May 23 09:01:58 serveerstertje kernel: [128112.916453] 1825240 pages reserved
May 23 09:01:58 serveerstertje kernel: [128112.916453] 94876 pages shared
May 23 09:01:58 serveerstertje kernel: [128112.916453] 38994 pages non-shared
May 23 09:01:58 serveerstertje kernel: [128112.916453] SLUB: Unable to allocate memory on node -1 (gfp=0x20)
May 23 09:01:58 serveerstertje kernel: [128112.916453] cache: kmalloc-2048, object size: 2048, buffer size: 2048, default order: 3, min order: 0
May 23 09:01:58 serveerstertje kernel: [128112.916453] node 0: slabs: 55, objs: 866, free: 0
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: devel/next-2.6.39 SLUB: Unable to allocate memory on node -1 (gfp=0x20)
2011-05-23 7:12 devel/next-2.6.39 SLUB: Unable to allocate memory on node -1 (gfp=0x20) Sander Eikelenboom
@ 2011-05-24 13:48 ` Konrad Rzeszutek Wilk
2011-05-24 14:09 ` Sander Eikelenboom
0 siblings, 1 reply; 3+ messages in thread
From: Konrad Rzeszutek Wilk @ 2011-05-24 13:48 UTC (permalink / raw)
To: Sander Eikelenboom; +Cc: xen-devel
On Mon, May 23, 2011 at 09:12:08AM +0200, Sander Eikelenboom wrote:
> Hi Konrad,
>
> It seems i have another problem with the devel/next-2.6.39 as dom0, don't know if it's xen related or not.
> It doesn't crash the machine, also netwerk still seems to work, but these warns keep coming say one till 3 a minute.
> I haven't changed the rest of the parameters/setup from the stable configuration i was running before with a 2.6.32 from jeremy.
This is quite curious. There is this nasty issue that the folks at Virtual Computer found with
the rtl8169 where it would scribble over the DMA pages. It would only happen with large amounts of data.
But this looks to be memory related.
> The panic i reported earlier hasn't hit me again yet (after attaching serial console).
I think that was due to kernel mismatch. The 'per_cpu' failure you saw is similar to not having
a specific patch. And that specific patch got removed from the #master branch as a similar
patch for it is being submitted upstream by tglrx.
Specific patch being: xen: do not implement irq_mask and irq_unmask in xen_percpu_chip
(48d036199309516f4e08f3cf256bab76c078864c)
> --
> Sander
>
> May 23 09:01:58 serveerstertje kernel: [128112.912479] swapper: page allocation failure. order:0, mode:0x4020
> May 23 09:01:58 serveerstertje kernel: [128112.916453] Pid: 0, comm: swapper Not tainted 2.6.39-xen-next-2.6.39-konrad+ #5
> May 23 09:01:58 serveerstertje kernel: [128112.916453] Call Trace:
> May 23 09:01:58 serveerstertje kernel: [128112.916453] <IRQ> [<ffffffff8112213c>] __alloc_pages_nodemask+0x64c/0x7e0
> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8132bdf6>] ? security_sock_rcv_skb+0x16/0x20
> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff819899b8>] ? tcp_v4_rcv+0x828/0x930
> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff811588c3>] new_slab+0x233/0x250
> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff81158f34>] T.1026+0x194/0x440
> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8190a014>] ? __netdev_alloc_skb+0x24/0x50
> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8115afcc>] __kmalloc_track_caller+0x15c/0x1e0
> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8190a014>] ? __netdev_alloc_skb+0x24/0x50
> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff81908d67>] __alloc_skb+0x77/0x160
> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8190a014>] __netdev_alloc_skb+0x24/0x50
> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff815ea2fb>] rtl8169_rx_interrupt+0x20b/0x420
> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff815eb51e>] ? rtl8169_interrupt+0xfe/0x3e0
> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff815ed06c>] rtl8169_poll+0x3c/0x1c0
> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8104c21d>] ? xen_force_evtchn_callback+0xd/0x10
> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8191313e>] net_rx_action+0x14e/0x2d0
> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8109ec61>] __do_softirq+0xd1/0x230
> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff810e7b2d>] ? handle_edge_irq+0x6d/0x120
Hmm, edge. So this card is doing MSI, right?
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: devel/next-2.6.39 SLUB: Unable to allocate memory on node -1 (gfp=0x20)
2011-05-24 13:48 ` Konrad Rzeszutek Wilk
@ 2011-05-24 14:09 ` Sander Eikelenboom
0 siblings, 0 replies; 3+ messages in thread
From: Sander Eikelenboom @ 2011-05-24 14:09 UTC (permalink / raw)
To: Konrad Rzeszutek Wilk; +Cc: xen-devel
Tuesday, May 24, 2011, 3:48:47 PM, you wrote:
> On Mon, May 23, 2011 at 09:12:08AM +0200, Sander Eikelenboom wrote:
>> Hi Konrad,
>>
>> It seems i have another problem with the devel/next-2.6.39 as dom0, don't know if it's xen related or not.
>> It doesn't crash the machine, also netwerk still seems to work, but these warns keep coming say one till 3 a minute.
>> I haven't changed the rest of the parameters/setup from the stable configuration i was running before with a 2.6.32 from jeremy.
> This is quite curious. There is this nasty issue that the folks at Virtual Computer found with
> the rtl8169 where it would scribble over the DMA pages. It would only happen with large amounts of data.
> But this looks to be memory related.
Hmm i think it's the same nasty issue then, at that moment i was indeed downloading a large file at high speed.
But i never saw it with the 2.6.32 series, so it must be introduced later.
>> The panic i reported earlier hasn't hit me again yet (after attaching serial console).
> I think that was due to kernel mismatch. The 'per_cpu' failure you saw is similar to not having
> a specific patch. And that specific patch got removed from the #master branch as a similar
> patch for it is being submitted upstream by tglrx.
> Specific patch being: xen: do not implement irq_mask and irq_unmask in xen_percpu_chip
> (48d036199309516f4e08f3cf256bab76c078864c)
Ah ok, i will lookup that patch and apply to my devel/next-2.6.39 clone !
But i haven't been able to trigger this one again so far ..
Thx
--
Sander
>> --
>> Sander
>>
>> May 23 09:01:58 serveerstertje kernel: [128112.912479] swapper: page allocation failure. order:0, mode:0x4020
>> May 23 09:01:58 serveerstertje kernel: [128112.916453] Pid: 0, comm: swapper Not tainted 2.6.39-xen-next-2.6.39-konrad+ #5
>> May 23 09:01:58 serveerstertje kernel: [128112.916453] Call Trace:
>> May 23 09:01:58 serveerstertje kernel: [128112.916453] <IRQ> [<ffffffff8112213c>] __alloc_pages_nodemask+0x64c/0x7e0
>> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8132bdf6>] ? security_sock_rcv_skb+0x16/0x20
>> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff819899b8>] ? tcp_v4_rcv+0x828/0x930
>> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff811588c3>] new_slab+0x233/0x250
>> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff81158f34>] T.1026+0x194/0x440
>> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8190a014>] ? __netdev_alloc_skb+0x24/0x50
>> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8115afcc>] __kmalloc_track_caller+0x15c/0x1e0
>> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8190a014>] ? __netdev_alloc_skb+0x24/0x50
>> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff81908d67>] __alloc_skb+0x77/0x160
>> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8190a014>] __netdev_alloc_skb+0x24/0x50
>> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff815ea2fb>] rtl8169_rx_interrupt+0x20b/0x420
>> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff815eb51e>] ? rtl8169_interrupt+0xfe/0x3e0
>> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff815ed06c>] rtl8169_poll+0x3c/0x1c0
>> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8104c21d>] ? xen_force_evtchn_callback+0xd/0x10
>> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8191313e>] net_rx_action+0x14e/0x2d0
>> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff8109ec61>] __do_softirq+0xd1/0x230
>> May 23 09:01:58 serveerstertje kernel: [128112.916453] [<ffffffff810e7b2d>] ? handle_edge_irq+0x6d/0x120
> Hmm, edge. So this card is doing MSI, right?
--
Best regards,
Sander mailto:linux@eikelenboom.it
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2011-05-24 14:09 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2011-05-23 7:12 devel/next-2.6.39 SLUB: Unable to allocate memory on node -1 (gfp=0x20) Sander Eikelenboom
2011-05-24 13:48 ` Konrad Rzeszutek Wilk
2011-05-24 14:09 ` Sander Eikelenboom
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.