On Tue, Dec 07, 2010 at 10:33:51PM +0800, Wu Fengguang wrote: > > In a simple dd test on a 8p system with "mem=256M", I find the light > > When increasing to 10 concurrent dd tasks, I managed to crash ext4.. > (2 concurrent dd's are OK, with very good write performance.) > > Here is the dmesg, which contains some ext4 routines. > > It looks like a memory leak problem. Look at the attached > vmstat-dirty.png, which is based on recorded vmstat numbers before > system hang. The global dirty/background thresholds keep decreasing. > Their values are derived from determine_dirtyable_memory(). The > decline means less and less reclaimable memory in the system. > > ext3 and ext2 are fine, as you can see from the attached graphs. > > The kernel is based on 2.6.37-rc4 with some additional writeback > patches. Here are more messages on vanilla 2.6.37-rc3. Attached is the kconfig. Thanks, Fengguang --- [ 158.713193] python invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0 [ 158.713642] python cpuset=/ mems_allowed=0 [ 158.713863] Pid: 2760, comm: python Not tainted 2.6.37-rc3 #154 [ 158.714155] Call Trace: [ 158.714320] [] ? _raw_spin_unlock+0x2b/0x40 [ 158.714605] [] dump_header+0x83/0x200 [ 158.714878] [] ? trace_hardirqs_on+0xd/0x10 [ 158.715170] [] ? ___ratelimit+0xab/0x150 [ 158.715457] [] ? oom_badness+0x77/0x190 [ 158.715732] [] oom_kill_process+0x8d/0x270 [ 158.716019] [] out_of_memory+0x1d8/0x410 [ 158.716299] [] __alloc_pages_nodemask+0x81b/0x830 [ 158.716608] [] alloc_pages_current+0x99/0x110 [ 158.716904] [] new_slab+0x294/0x2a0 [ 158.717169] [] ? __slab_alloc+0x232/0x450 [ 158.717451] [] __slab_alloc+0x247/0x450 [ 158.717725] [] ? nfs_create_request+0x41/0x160 [ 158.718021] [] ? nfs_create_request+0x41/0x160 [ 158.718321] [] kmem_cache_alloc+0x17c/0x190 [ 158.718609] [] ? _raw_spin_unlock_irq+0x30/0x40 [ 158.718907] [] ? _raw_spin_unlock_irq+0x30/0x40 [ 158.719203] [] nfs_create_request+0x41/0x160 [ 158.719499] [] ? _raw_spin_unlock_irq+0x30/0x40 [ 158.719801] [] ? readpage_async_filler+0x0/0x150 [ 158.720102] [] readpage_async_filler+0x65/0x150 [ 158.720399] [] ? readpage_async_filler+0x0/0x150 [ 158.720699] [] read_cache_pages+0xa2/0xf0 [ 158.720983] [] ? local_clock+0x6f/0x80 [ 158.721252] [] nfs_readpages+0xe2/0x1b0 [ 158.721524] [] ? nfs_pagein_one+0x0/0x100 [ 158.721802] [] __do_page_cache_readahead+0x18c/0x260 [ 158.722116] [] ? __do_page_cache_readahead+0xad/0x260 [ 158.722438] [] ? lock_release_holdtime+0x3d/0x180 [ 158.722745] [] ra_submit+0x21/0x30 [ 158.723003] [] filemap_fault+0x4dc/0x4f0 [ 158.723287] [] ? __lock_acquire+0x53e/0x1e70 [ 158.723595] [] __do_fault+0x54/0x540 [ 158.723871] [] handle_mm_fault+0x1af/0xc20 [ 158.724163] [] ? do_page_fault+0xf6/0x560 [ 158.724454] [] ? lock_release_holdtime+0x3d/0x180 [ 158.724764] [] do_page_fault+0x16b/0x560 [ 158.725039] [] ? might_fault+0xa5/0xb0 [ 158.725306] [] ? might_fault+0x5c/0xb0 [ 158.725572] [] ? poll_select_copy_remaining+0x101/0x150 [ 158.725893] [] ? error_sti+0x5/0x6 [ 158.726148] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 158.726456] [] page_fault+0x25/0x30 [ 158.726712] Mem-Info: [ 158.726861] Node 0 DMA per-cpu: [ 158.727076] CPU 0: hi: 0, btch: 1 usd: 0 [ 158.727326] CPU 1: hi: 0, btch: 1 usd: 0 [ 158.727570] CPU 2: hi: 0, btch: 1 usd: 0 [ 158.727814] CPU 3: hi: 0, btch: 1 usd: 0 [ 158.728057] CPU 4: hi: 0, btch: 1 usd: 0 [ 158.728300] CPU 5: hi: 0, btch: 1 usd: 0 [ 158.728543] CPU 6: hi: 0, btch: 1 usd: 0 [ 158.728786] CPU 7: hi: 0, btch: 1 usd: 0 [ 158.729027] Node 0 DMA32 per-cpu: [ 158.729250] CPU 0: hi: 90, btch: 15 usd: 6 [ 158.729493] CPU 1: hi: 90, btch: 15 usd: 3 [ 158.729736] CPU 2: hi: 90, btch: 15 usd: 0 [ 158.729979] CPU 3: hi: 90, btch: 15 usd: 14 [ 158.730222] CPU 4: hi: 90, btch: 15 usd: 27 [ 158.730465] CPU 5: hi: 90, btch: 15 usd: 4 [ 158.730707] CPU 6: hi: 90, btch: 15 usd: 0 [ 158.730950] CPU 7: hi: 90, btch: 15 usd: 3 [ 158.731195] active_anon:7304 inactive_anon:112 isolated_anon:0 [ 158.731196] active_file:405 inactive_file:2450 isolated_file:0 [ 158.731197] unevictable:0 dirty:198 writeback:810 unstable:0 [ 158.731197] free:761 slab_reclaimable:8191 slab_unreclaimable:5281 [ 158.731198] mapped:355 shmem:152 pagetables:557 bounce:0 [ 158.732603] Node 0 DMA free:1056kB min:120kB low:148kB high:180kB active_anon:4316kB inactive_anon:0kB active_file:0kB inactive_file:3284kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:260kB writeback:416kB mapped:8kB shmem:0kB slab_reclaimable:6392kB slab_unreclaimable:392kB kernel_stack:8kB pagetables:40kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no [ 158.734263] lowmem_reserve[]: 0 236 236 236 [ 158.734649] Node 0 DMA32 free:1852kB min:1904kB low:2380kB high:2856kB active_anon:24900kB inactive_anon:448kB active_file:1620kB inactive_file:6516kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:532kB writeback:2824kB mapped:1412kB shmem:608kB slab_reclaimable:26372kB slab_unreclaimable:20732kB kernel_stack:1224kB pagetables:2188kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no [ 158.736406] lowmem_reserve[]: 0 0 0 0 [ 158.736777] Node 0 DMA: 32*4kB 55*8kB 32*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1080kB [ 158.737720] Node 0 DMA32: 3*4kB 28*8kB 99*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1852kB [ 158.738658] 3061 total pagecache pages [ 158.738860] 0 pages in swap cache [ 158.739046] Swap cache stats: add 0, delete 0, find 0/0 [ 158.739301] Free swap = 0kB [ 158.739476] Total swap = 0kB [ 158.740314] 65520 pages RAM [ 158.740483] 16492 pages reserved [ 158.740667] 5167 pages shared [ 158.740840] 44889 pages non-shared [ 158.741030] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 158.741410] [ 1995] 0 1995 4237 169 1 -17 -1000 udevd [ 158.741785] [ 2082] 0 2082 4264 179 2 -17 -1000 udevd [ 158.742158] [ 2083] 0 2083 4264 169 2 -17 -1000 udevd [ 158.742527] [ 2269] 0 2269 12303 170 1 -17 -1000 sshd [ 158.742894] [ 2343] 1 2343 2162 71 3 0 0 portmap [ 158.743278] [ 2355] 0 2355 3606 87 0 0 0 rpc.statd [ 158.743691] [ 2428] 0 2428 1633 93 1 0 0 syslogd [ 158.744080] [ 2437] 0 2437 1107 99 6 0 0 klogd [ 158.744462] [ 2448] 104 2448 5864 102 4 0 0 dbus-daemon [ 158.744864] [ 2490] 106 2490 11105 317 2 0 0 hald [ 158.745244] [ 2491] 0 2491 5552 101 1 0 0 hald-runner [ 158.745646] [ 2520] 0 2520 19756 246 4 0 0 sshd [ 158.746020] [ 2524] 0 2524 6081 83 5 0 0 hald-addon-inpu [ 158.746428] [ 2541] 0 2541 6083 82 0 0 0 hald-addon-cpuf [ 158.746835] [ 2542] 106 2542 6535 103 0 0 0 hald-addon-acpi [ 158.747244] [ 2551] 1000 2551 19756 322 4 0 0 sshd [ 158.747625] [ 2571] 1000 2571 11804 470 0 0 0 zsh [ 158.747997] [ 2632] 0 2632 1494 66 4 0 0 getty [ 158.748371] [ 2633] 0 2633 1494 67 0 0 0 getty [ 158.748744] [ 2634] 0 2634 1494 67 0 0 0 getty [ 158.749118] [ 2635] 0 2635 1494 67 0 0 0 getty [ 158.749492] [ 2636] 0 2636 1494 66 0 0 0 getty [ 158.750434] [ 2638] 0 2638 1494 66 0 0 0 getty [ 158.750811] [ 2689] 0 2689 13201 134 4 0 0 su [ 158.751175] [ 2692] 0 2692 11425 600 0 0 0 zsh [ 158.751588] [ 2720] 0 2720 3431 142 0 0 0 concurrent-dd.s [ 158.751997] [ 2749] 0 2749 3460 323 5 0 0 dd [ 158.752362] [ 2750] 0 2750 3460 322 1 0 0 dd [ 158.752727] [ 2751] 0 2751 3460 322 4 0 0 dd [ 158.753093] [ 2752] 0 2752 3460 323 3 0 0 dd [ 158.753459] [ 2753] 0 2753 3460 323 7 0 0 dd [ 158.753824] [ 2754] 0 2754 3460 322 3 0 0 dd [ 158.754189] [ 2755] 0 2755 3460 322 5 0 0 dd [ 158.754553] [ 2756] 0 2756 3460 323 0 0 0 dd [ 158.754914] [ 2757] 0 2757 3460 323 0 0 0 dd [ 158.755276] [ 2758] 0 2758 3460 323 3 0 0 dd [ 158.755647] [ 2759] 0 2759 3436 227 0 0 0 plot-written.sh [ 158.756054] [ 2760] 0 2760 11157 1726 2 0 0 python [ 158.756432] [ 2776] 0 2776 2144 140 4 0 0 iostat [ 158.756811] [ 7125] 0 7125 100 30 0 0 0 date [ 158.757184] Out of memory: Kill process 2571 (zsh) score 9 or sacrifice child [ 158.757515] Killed process 2689 (su) total-vm:52804kB, anon-rss:328kB, file-rss:208kB [ 158.862569] zsh invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 [ 158.862991] zsh cpuset=/ mems_allowed=0 [ 158.863209] Pid: 2571, comm: zsh Not tainted 2.6.37-rc3 #154 [ 158.863540] Call Trace: [ 158.863710] [] ? _raw_spin_unlock+0x2b/0x40 [ 158.864006] [] dump_header+0x83/0x200 [ 158.864282] [] ? trace_hardirqs_on+0xd/0x10 [ 158.864576] [] ? ___ratelimit+0xab/0x150 [ 158.864860] [] ? oom_badness+0x77/0x190 [ 158.865139] [] oom_kill_process+0x8d/0x270 [ 158.865428] [] out_of_memory+0x1d8/0x410 [ 158.865718] [] __alloc_pages_nodemask+0x81b/0x830 [ 158.866037] [] alloc_pages_current+0x99/0x110 [ 158.866345] [] __page_cache_alloc+0x87/0x90 [ 158.866644] [] __do_page_cache_readahead+0x120/0x260 [ 158.866972] [] ? __do_page_cache_readahead+0xad/0x260 [ 158.867305] [] ? lock_release_holdtime+0x3d/0x180 [ 158.867635] [] ra_submit+0x21/0x30 [ 158.867903] [] filemap_fault+0x4dc/0x4f0 [ 158.868193] [] ? __lock_acquire+0x53e/0x1e70 [ 158.868496] [] ? local_clock+0x6f/0x80 [ 158.868777] [] __do_fault+0x54/0x540 [ 158.869052] [] ? trace_hardirqs_off_caller+0x29/0x150 [ 158.869383] [] handle_mm_fault+0x1af/0xc20 [ 158.869676] [] ? do_page_fault+0xf6/0x560 [ 158.869969] [] ? lock_release_holdtime+0x3d/0x180 [ 158.870288] [] do_page_fault+0x16b/0x560 [ 158.870575] [] ? error_sti+0x5/0x6 [ 158.870842] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 158.871166] [] page_fault+0x25/0x30 [ 158.871446] Mem-Info: [ 158.871605] Node 0 DMA per-cpu: [ 158.871836] CPU 0: hi: 0, btch: 1 usd: 0 [ 158.872092] CPU 1: hi: 0, btch: 1 usd: 0 [ 158.872345] CPU 2: hi: 0, btch: 1 usd: 0 [ 158.872598] CPU 3: hi: 0, btch: 1 usd: 0 [ 158.872852] CPU 4: hi: 0, btch: 1 usd: 0 [ 158.873104] CPU 5: hi: 0, btch: 1 usd: 0 [ 158.873358] CPU 6: hi: 0, btch: 1 usd: 0 [ 158.873610] CPU 7: hi: 0, btch: 1 usd: 0 [ 158.873863] Node 0 DMA32 per-cpu: [ 158.874098] CPU 0: hi: 90, btch: 15 usd: 17 [ 158.874352] CPU 1: hi: 90, btch: 15 usd: 1 [ 158.874606] CPU 2: hi: 90, btch: 15 usd: 0 [ 158.874858] CPU 3: hi: 90, btch: 15 usd: 0 [ 158.875110] CPU 4: hi: 90, btch: 15 usd: 69 [ 158.875363] CPU 5: hi: 90, btch: 15 usd: 0 [ 158.875631] CPU 6: hi: 90, btch: 15 usd: 14 [ 158.875885] CPU 7: hi: 90, btch: 15 usd: 0 [ 158.876141] active_anon:7254 inactive_anon:95 isolated_anon:0 [ 158.876143] active_file:405 inactive_file:2484 isolated_file:0 [ 158.876144] unevictable:0 dirty:0 writeback:753 unstable:0 [ 158.876145] free:707 slab_reclaimable:8191 slab_unreclaimable:5231 [ 158.876146] mapped:380 shmem:152 pagetables:532 bounce:0 [ 158.877578] Node 0 DMA free:1056kB min:120kB low:148kB high:180kB active_anon:4316kB inactive_anon:0kB active_file:0kB inactive_file:2572kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:0kB writeback:780kB mapped:8kB shmem:0kB slab_reclaimable:6392kB slab_unreclaimable:464kB kernel_stack:8kB pagetables:40kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:4353 all_unreclaimable? yes [ 158.879330] lowmem_reserve[]: 0 236 236 236 [ 158.879758] Node 0 DMA32 free:1788kB min:1904kB low:2380kB high:2856kB active_anon:24700kB inactive_anon:380kB active_file:1620kB inactive_file:7364kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:0kB writeback:1724kB mapped:1512kB shmem:608kB slab_reclaimable:26372kB slab_unreclaimable:20460kB kernel_stack:1224kB pagetables:2088kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:16092 all_unreclaimable? yes [ 158.881595] lowmem_reserve[]: 0 0 0 0 [ 158.881987] Node 0 DMA: 43*4kB 90*8kB 11*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1068kB [ 158.882974] Node 0 DMA32: 91*4kB 39*8kB 85*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2068kB [ 158.883989] 2867 total pagecache pages [ 158.884205] 0 pages in swap cache [ 158.884404] Swap cache stats: add 0, delete 0, find 0/0 [ 158.884677] Free swap = 0kB [ 158.884859] Total swap = 0kB [ 158.885766] 65520 pages RAM [ 158.885945] 16492 pages reserved [ 158.886140] 5300 pages shared [ 158.886329] 44770 pages non-shared [ 158.886531] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 158.886928] [ 1995] 0 1995 4237 169 1 -17 -1000 udevd [ 158.887324] [ 2082] 0 2082 4264 179 2 -17 -1000 udevd [ 158.887737] [ 2083] 0 2083 4264 169 2 -17 -1000 udevd [ 158.888132] [ 2269] 0 2269 12303 170 1 -17 -1000 sshd [ 158.888523] [ 2343] 1 2343 2162 89 1 0 0 portmap [ 158.888923] [ 2355] 0 2355 3606 87 0 0 0 rpc.statd [ 158.889332] [ 2428] 0 2428 1633 118 2 0 0 syslogd [ 158.889734] [ 2437] 0 2437 1107 92 1 0 0 klogd [ 158.890131] [ 2448] 104 2448 5864 111 4 0 0 dbus-daemon [ 158.890542] [ 2490] 106 2490 11105 318 2 0 0 hald [ 158.890930] [ 2491] 0 2491 5552 101 1 0 0 hald-runner [ 158.891346] [ 2520] 0 2520 19756 246 4 0 0 sshd [ 158.891746] [ 2524] 0 2524 6081 83 5 0 0 hald-addon-inpu [ 158.892175] [ 2541] 0 2541 6083 82 0 0 0 hald-addon-cpuf [ 158.892600] [ 2542] 106 2542 6535 103 0 0 0 hald-addon-acpi [ 158.893024] [ 2551] 1000 2551 19756 323 4 0 0 sshd [ 158.893415] [ 2571] 1000 2571 11804 495 4 0 0 zsh [ 158.893805] [ 2632] 0 2632 1494 66 4 0 0 getty [ 158.894197] [ 2633] 0 2633 1494 67 0 0 0 getty [ 158.894591] [ 2634] 0 2634 1494 67 0 0 0 getty [ 158.894989] [ 2635] 0 2635 1494 67 0 0 0 getty [ 158.895381] [ 2636] 0 2636 1494 66 0 0 0 getty [ 158.895784] [ 2638] 0 2638 1494 66 0 0 0 getty [ 158.896179] [ 2692] 0 2692 11425 600 0 0 0 zsh [ 158.896568] [ 2720] 0 2720 3431 142 0 0 0 concurrent-dd.s [ 158.896996] [ 2749] 0 2749 3460 323 7 0 0 dd [ 158.897381] [ 2750] 0 2750 3460 322 1 0 0 dd [ 158.897766] [ 2751] 0 2751 3460 322 6 0 0 dd [ 158.898713] [ 2752] 0 2752 3460 323 7 0 0 dd [ 158.899099] [ 2753] 0 2753 3460 323 5 0 0 dd [ 158.899492] [ 2754] 0 2754 3460 322 7 0 0 dd [ 158.899877] [ 2755] 0 2755 3460 322 3 0 0 dd [ 158.900263] [ 2756] 0 2756 3460 323 4 0 0 dd [ 158.900648] [ 2757] 0 2757 3460 323 4 0 0 dd [ 158.901032] [ 2758] 0 2758 3460 323 1 0 0 dd [ 158.901418] [ 2759] 0 2759 3436 227 0 0 0 plot-written.sh [ 158.901840] [ 2760] 0 2760 11157 1726 6 0 0 python [ 158.902239] [ 2776] 0 2776 2144 134 4 0 0 iostat [ 158.902637] [ 7125] 0 7125 3202 138 5 0 0 date [ 158.903026] Out of memory: Kill process 2571 (zsh) score 10 or sacrifice child [ 158.903421] Killed process 2571 (zsh) total-vm:47216kB, anon-rss:1692kB, file-rss:288kB remote: Counting objects: 272, done. remote: Compressing objects: 100% (58/58), done. remote: Total 178 (delta 151), reused 136 (delta 119) Receiving objects: 100% (178/178), 25.77 KiB, done. Resolving deltas: 100% (151/151), completed with 65 local objects. >From git://proxy.jf.intel.com/pub/scm/linux/kernel/git/torvalds/linux-2.6 771f8bc..cf7d7e5 master -> origin/master * [new tag] v2.6.37-rc5 -> v2.6.37-rc5 [1] + 19413 done git remote update [ 206.063452] date invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 [ 206.063863] date cpuset=/ mems_allowed=0 [ 206.064074] Pid: 9396, comm: date Not tainted 2.6.37-rc3 #154 [ 206.064352] Call Trace: [ 206.064513] [] ? _raw_spin_unlock+0x2b/0x40 [ 206.064798] [] dump_header+0x83/0x200 [ 206.065063] [] ? trace_hardirqs_on+0xd/0x10 [ 206.065345] [] ? ___ratelimit+0xab/0x150 [ 206.065633] [] ? oom_badness+0x77/0x190 [ 206.065908] [] oom_kill_process+0x8d/0x270 [ 206.066193] [] out_of_memory+0x1d8/0x410 [ 206.066466] [] __alloc_pages_nodemask+0x81b/0x830 [ 206.066768] [] alloc_pages_current+0x99/0x110 [ 206.067059] [] __page_cache_alloc+0x87/0x90 [ 206.067342] [] __do_page_cache_readahead+0x120/0x260 [ 206.067652] [] ? __do_page_cache_readahead+0xad/0x260 [ 206.067968] [] ? lock_release_holdtime+0x3d/0x180 [ 206.068270] [] ra_submit+0x21/0x30 [ 206.068523] [] filemap_fault+0x4dc/0x4f0 [ 206.068795] [] ? __lock_acquire+0x53e/0x1e70 [ 206.069078] [] __do_fault+0x54/0x540 [ 206.069391] [] handle_mm_fault+0x1af/0xc20 [ 206.069685] [] ? do_page_fault+0xf6/0x560 [ 206.069961] [] ? lock_release_holdtime+0x3d/0x180 [ 206.070262] [] do_page_fault+0x16b/0x560 [ 206.070536] [] ? local_clock+0x6f/0x80 [ 206.070802] [] ? trace_hardirqs_off_caller+0x29/0x150 [ 206.071115] [] ? trace_hardirqs_off+0xd/0x10 [ 206.071399] [] ? local_clock+0x6f/0x80 [ 206.071665] [] ? error_sti+0x5/0x6 [ 206.071920] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 206.072227] [] page_fault+0x25/0x30 [ 206.072481] Mem-Info: [ 206.072631] Node 0 DMA per-cpu: [ 206.072846] CPU 0: hi: 0, btch: 1 usd: 0 [ 206.073090] CPU 1: hi: 0, btch: 1 usd: 0 [ 206.073331] CPU 2: hi: 0, btch: 1 usd: 0 [ 206.073582] CPU 3: hi: 0, btch: 1 usd: 0 [ 206.073825] CPU 4: hi: 0, btch: 1 usd: 0 [ 206.074068] CPU 5: hi: 0, btch: 1 usd: 0 [ 206.074311] CPU 6: hi: 0, btch: 1 usd: 0 [ 206.074553] CPU 7: hi: 0, btch: 1 usd: 0 [ 206.074795] Node 0 DMA32 per-cpu: [ 206.075016] CPU 0: hi: 90, btch: 15 usd: 0 [ 206.075258] CPU 1: hi: 90, btch: 15 usd: 0 [ 206.075502] CPU 2: hi: 90, btch: 15 usd: 0 [ 206.075743] CPU 3: hi: 90, btch: 15 usd: 0 [ 206.075985] CPU 4: hi: 90, btch: 15 usd: 0 [ 206.076228] CPU 5: hi: 90, btch: 15 usd: 0 [ 206.076471] CPU 6: hi: 90, btch: 15 usd: 0 [ 206.076714] CPU 7: hi: 90, btch: 15 usd: 0 [ 206.076957] active_anon:4989 inactive_anon:19 isolated_anon:0 [ 206.076958] active_file:246 inactive_file:1257 isolated_file:0 [ 206.076959] unevictable:0 dirty:0 writeback:0 unstable:0 [ 206.076959] free:725 slab_reclaimable:11315 slab_unreclaimable:5147 [ 206.076960] mapped:198 shmem:75 pagetables:429 bounce:0 [ 206.078351] Node 0 DMA free:1088kB min:120kB low:148kB high:180kB active_anon:576kB inactive_anon:0kB active_file:36kB inactive_file:848kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:0kB writeback:24kB mapped:0kB shmem:0kB slab_reclaimable:12824kB slab_unreclaimable:352kB kernel_stack:16kB pagetables:12kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1466 all_unreclaimable? yes [ 206.080021] lowmem_reserve[]: 0 236 236 236 [ 206.080410] Node 0 DMA32 free:1840kB min:1904kB low:2380kB high:2856kB active_anon:19372kB inactive_anon:28kB active_file:940kB inactive_file:4232kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:0kB writeback:0kB mapped:780kB shmem:300kB slab_reclaimable:32428kB slab_unreclaimable:20268kB kernel_stack:1200kB pagetables:1704kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:9452 all_unreclaimable? yes [ 206.082158] lowmem_reserve[]: 0 0 0 0 [ 206.082531] Node 0 DMA: 67*4kB 4*8kB 46*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1036kB [ 206.083464] Node 0 DMA32: 173*4kB 98*8kB 23*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1844kB [ 206.084413] 1660 total pagecache pages [ 206.084617] 0 pages in swap cache [ 206.084805] Swap cache stats: add 0, delete 0, find 0/0 [ 206.085063] Free swap = 0kB [ 206.085234] Total swap = 0kB [ 206.085981] 65520 pages RAM [ 206.086152] 16492 pages reserved [ 206.086337] 1938 pages shared [ 206.086511] 46189 pages non-shared [ 206.086703] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 206.087084] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 206.087459] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 206.087834] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 206.088210] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 206.088581] [ 2343] 1 2343 2162 65 4 0 0 portmap [ 206.088963] [ 2355] 0 2355 3606 43 0 0 0 rpc.statd [ 206.089351] [ 2428] 0 2428 1633 44 0 0 0 syslogd [ 206.089738] [ 2437] 0 2437 1107 25 2 0 0 klogd [ 206.090113] [ 2448] 104 2448 5864 83 0 0 0 dbus-daemon [ 206.090507] [ 2490] 106 2490 11105 276 2 0 0 hald [ 206.090877] [ 2491] 0 2491 5552 63 1 0 0 hald-runner [ 206.091270] [ 2524] 0 2524 6081 49 5 0 0 hald-addon-inpu [ 206.091679] [ 2541] 0 2541 6083 48 0 0 0 hald-addon-cpuf [ 206.092085] [ 2542] 106 2542 6535 60 0 0 0 hald-addon-acpi [ 206.092492] [ 2632] 0 2632 1494 26 4 0 0 getty [ 206.092866] [ 2633] 0 2633 1494 27 0 0 0 getty [ 206.093238] [ 2634] 0 2634 1494 27 0 0 0 getty [ 206.093622] [ 2635] 0 2635 1494 27 0 0 0 getty [ 206.093999] [ 2636] 0 2636 1494 26 0 0 0 getty [ 206.094372] [ 2638] 0 2638 1494 26 0 0 0 getty [ 206.094746] [ 2692] 0 2692 11425 554 0 0 0 zsh [ 206.095115] [ 2720] 0 2720 3431 63 4 0 0 concurrent-dd.s [ 206.095522] [ 2749] 0 2749 3460 290 5 0 0 dd [ 206.095887] [ 2750] 0 2750 3460 289 4 0 0 dd [ 206.096252] [ 2751] 0 2751 3460 289 6 0 0 dd [ 206.096618] [ 2752] 0 2752 3460 290 4 0 0 dd [ 206.096982] [ 2753] 0 2753 3460 290 6 0 0 dd [ 206.097347] [ 2754] 0 2754 3460 289 1 0 0 dd [ 206.097720] [ 2755] 0 2755 3460 289 3 0 0 dd [ 206.098086] [ 2756] 0 2756 3460 290 6 0 0 dd [ 206.098452] [ 2757] 0 2757 3460 290 0 0 0 dd [ 206.098817] [ 2758] 0 2758 3460 290 0 0 0 dd [ 206.099182] [ 2759] 0 2759 3437 185 1 0 0 plot-written.sh [ 206.099589] [ 2776] 0 2776 2145 71 3 0 0 iostat [ 206.100540] [ 9396] 0 9396 3202 134 6 0 0 date [ 206.100910] Out of memory: Kill process 2490 (hald) score 5 or sacrifice child [ 206.101276] Killed process 2491 (hald-runner) total-vm:22208kB, anon-rss:248kB, file-rss:4kB [ 206.494475] date invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 [ 206.494895] date cpuset=/ mems_allowed=0 [ 206.495112] Pid: 9396, comm: date Not tainted 2.6.37-rc3 #154 [ 206.495396] Call Trace: [ 206.495558] [] ? _raw_spin_unlock+0x2b/0x40 [ 206.495845] [] dump_header+0x83/0x200 [ 206.496112] [] ? trace_hardirqs_on+0xd/0x10 [ 206.496400] [] ? ___ratelimit+0xab/0x150 [ 206.496675] [] ? oom_badness+0x77/0x190 [ 206.496951] [] oom_kill_process+0x8d/0x270 [ 206.497234] [] out_of_memory+0x1d8/0x410 [ 206.497514] [] __alloc_pages_nodemask+0x81b/0x830 [ 206.497834] [] alloc_pages_current+0x99/0x110 [ 206.498131] [] __page_cache_alloc+0x87/0x90 [ 206.498421] [] __do_page_cache_readahead+0x120/0x260 [ 206.498740] [] ? __do_page_cache_readahead+0xad/0x260 [ 206.499062] [] ? lock_release_holdtime+0x3d/0x180 [ 206.499373] [] ra_submit+0x21/0x30 [ 206.499632] [] filemap_fault+0x4dc/0x4f0 [ 206.499911] [] ? __lock_acquire+0x53e/0x1e70 [ 206.500207] [] __do_fault+0x54/0x540 [ 206.500475] [] handle_mm_fault+0x1af/0xc20 [ 206.500762] [] ? do_page_fault+0xf6/0x560 [ 206.501045] [] ? lock_release_holdtime+0x3d/0x180 [ 206.501355] [] do_page_fault+0x16b/0x560 [ 206.501635] [] ? local_clock+0x6f/0x80 [ 206.501923] [] ? trace_hardirqs_off_caller+0x29/0x150 [ 206.502244] [] ? trace_hardirqs_off+0xd/0x10 [ 206.502535] [] ? local_clock+0x6f/0x80 [ 206.502809] [] ? error_sti+0x5/0x6 [ 206.503068] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 206.503384] [] page_fault+0x25/0x30 [ 206.503645] Mem-Info: [ 206.503798] Node 0 DMA per-cpu: [ 206.504019] CPU 0: hi: 0, btch: 1 usd: 0 [ 206.504269] CPU 1: hi: 0, btch: 1 usd: 0 [ 206.504518] CPU 2: hi: 0, btch: 1 usd: 0 [ 206.504767] CPU 3: hi: 0, btch: 1 usd: 0 [ 206.505069] CPU 4: hi: 0, btch: 1 usd: 0 [ 206.505317] CPU 5: hi: 0, btch: 1 usd: 0 [ 206.505566] CPU 6: hi: 0, btch: 1 usd: 0 [ 206.505825] CPU 7: hi: 0, btch: 1 usd: 0 [ 206.506074] Node 0 DMA32 per-cpu: [ 206.506302] CPU 0: hi: 90, btch: 15 usd: 14 [ 206.506550] CPU 1: hi: 90, btch: 15 usd: 0 [ 206.506798] CPU 2: hi: 90, btch: 15 usd: 0 [ 206.507048] CPU 3: hi: 90, btch: 15 usd: 0 [ 206.507297] CPU 4: hi: 90, btch: 15 usd: 0 [ 206.507545] CPU 5: hi: 90, btch: 15 usd: 0 [ 206.507795] CPU 6: hi: 90, btch: 15 usd: 14 [ 206.508045] CPU 7: hi: 90, btch: 15 usd: 0 [ 206.508296] active_anon:4928 inactive_anon:7 isolated_anon:0 [ 206.508297] active_file:142 inactive_file:571 isolated_file:0 [ 206.508298] unevictable:0 dirty:0 writeback:0 unstable:0 [ 206.508299] free:688 slab_reclaimable:11310 slab_unreclaimable:5150 [ 206.508301] mapped:121 shmem:75 pagetables:436 bounce:0 [ 206.509719] Node 0 DMA free:1044kB min:120kB low:148kB high:180kB active_anon:552kB inactive_anon:0kB active_file:32kB inactive_file:388kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:44kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:12836kB slab_unreclaimable:352kB kernel_stack:8kB pagetables:20kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:802 all_unreclaimable? yes [ 206.511425] lowmem_reserve[]: 0 236 236 236 [ 206.511830] Node 0 DMA32 free:1708kB min:1904kB low:2380kB high:2856kB active_anon:19160kB inactive_anon:28kB active_file:536kB inactive_file:1896kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:0kB writeback:0kB mapped:500kB shmem:300kB slab_reclaimable:32404kB slab_unreclaimable:20248kB kernel_stack:1176kB pagetables:1724kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:3774 all_unreclaimable? yes [ 206.513597] lowmem_reserve[]: 0 0 0 0 [ 206.513990] Node 0 DMA: 66*4kB 2*8kB 49*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1064kB [ 206.514957] Node 0 DMA32: 146*4kB 77*8kB 29*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1760kB [ 206.515940] 906 total pagecache pages [ 206.516146] 0 pages in swap cache [ 206.516339] Swap cache stats: add 0, delete 0, find 0/0 [ 206.516604] Free swap = 0kB [ 206.516781] Total swap = 0kB [ 206.517638] 65520 pages RAM [ 206.517821] 16492 pages reserved [ 206.518011] 778 pages shared [ 206.518186] 47393 pages non-shared [ 206.518383] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 206.518772] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 206.519158] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 206.519542] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 206.519927] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 206.520309] [ 2343] 1 2343 2162 62 4 0 0 portmap [ 206.520702] [ 2355] 0 2355 3606 43 0 0 0 rpc.statd [ 206.521098] [ 2428] 0 2428 1633 90 0 0 0 syslogd [ 206.521488] [ 2437] 0 2437 1107 67 0 0 0 klogd [ 206.521879] [ 2448] 104 2448 5864 83 0 0 0 dbus-daemon [ 206.522283] [ 2490] 106 2490 11105 294 1 0 0 hald [ 206.522664] [ 2524] 0 2524 6081 49 5 0 0 hald-addon-inpu [ 206.523082] [ 2541] 0 2541 6083 48 0 0 0 hald-addon-cpuf [ 206.523499] [ 2542] 106 2542 6535 59 0 0 0 hald-addon-acpi [ 206.523916] [ 2632] 0 2632 1494 26 4 0 0 getty [ 206.524300] [ 2633] 0 2633 1494 27 0 0 0 getty [ 206.524684] [ 2634] 0 2634 1494 27 0 0 0 getty [ 206.525069] [ 2635] 0 2635 1494 27 0 0 0 getty [ 206.525453] [ 2636] 0 2636 1494 26 0 0 0 getty [ 206.525845] [ 2638] 0 2638 1494 26 0 0 0 getty [ 206.526231] [ 2692] 0 2692 11425 554 0 0 0 zsh [ 206.526609] [ 2720] 0 2720 3431 63 4 0 0 concurrent-dd.s [ 206.527027] [ 2749] 0 2749 3460 290 1 0 0 dd [ 206.527401] [ 2750] 0 2750 3460 289 0 0 0 dd [ 206.527777] [ 2751] 0 2751 3460 289 4 0 0 dd [ 206.528152] [ 2752] 0 2752 3460 290 2 0 0 dd [ 206.528527] [ 2753] 0 2753 3460 290 7 0 0 dd [ 206.528902] [ 2754] 0 2754 3460 289 3 0 0 dd [ 206.529275] [ 2755] 0 2755 3460 289 4 0 0 dd [ 206.529650] [ 2756] 0 2756 3460 290 6 0 0 dd [ 206.530032] [ 2757] 0 2757 3460 290 3 0 0 dd [ 206.530409] [ 2758] 0 2758 3460 290 5 0 0 dd [ 206.530785] [ 2759] 0 2759 3437 102 1 0 0 plot-written.sh [ 206.531203] [ 2776] 0 2776 2145 68 3 0 0 iostat [ 206.531591] [ 9396] 0 9396 3202 117 2 0 0 date [ 206.531972] Out of memory: Kill process 2490 (hald) score 5 or sacrifice child [ 206.532346] Killed process 2490 (hald) total-vm:44420kB, anon-rss:1096kB, file-rss:80kB [ 224.490577] plot-written.sh invoked oom-killer: gfp_mask=0x200da, order=0, oom_adj=0, oom_score_adj=0 [ 224.491034] plot-written.sh cpuset=/ mems_allowed=0 [ 224.491283] Pid: 9920, comm: plot-written.sh Not tainted 2.6.37-rc3 #154 [ 224.491597] Call Trace: [ 224.491758] [] ? _raw_spin_unlock+0x2b/0x40 [ 224.492043] [] dump_header+0x83/0x200 [ 224.492311] [] ? trace_hardirqs_on+0xd/0x10 [ 224.492597] [] ? ___ratelimit+0xab/0x150 [ 224.492870] [] ? oom_badness+0x77/0x190 [ 224.493137] [] oom_kill_process+0x8d/0x270 [ 224.493411] [] out_of_memory+0x1d8/0x410 [ 224.493679] [] __alloc_pages_nodemask+0x81b/0x830 [ 224.493975] [] alloc_page_vma+0x88/0x160 [ 224.494258] [] do_wp_page+0xf1/0x880 [ 224.494517] [] handle_mm_fault+0x3e4/0xc20 [ 224.495345] [] do_page_fault+0x16b/0x560 [ 224.495614] [] ? might_fault+0x5c/0xb0 [ 224.495876] [] ? error_sti+0x5/0x6 [ 224.496125] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 224.496425] [] page_fault+0x25/0x30 [ 224.496676] Mem-Info: [ 224.496821] Node 0 DMA per-cpu: [ 224.497031] CPU 0: hi: 0, btch: 1 usd: 0 [ 224.497268] CPU 1: hi: 0, btch: 1 usd: 0 [ 224.497504] CPU 2: hi: 0, btch: 1 usd: 0 [ 224.497740] CPU 3: hi: 0, btch: 1 usd: 0 [ 224.497977] CPU 4: hi: 0, btch: 1 usd: 0 [ 224.498235] CPU 5: hi: 0, btch: 1 usd: 0 [ 224.498476] CPU 6: hi: 0, btch: 1 usd: 0 [ 224.498719] CPU 7: hi: 0, btch: 1 usd: 0 [ 224.498963] Node 0 DMA32 per-cpu: [ 224.499181] CPU 0: hi: 90, btch: 15 usd: 86 [ 224.499422] CPU 1: hi: 90, btch: 15 usd: 0 [ 224.499658] CPU 2: hi: 90, btch: 15 usd: 0 [ 224.499895] CPU 3: hi: 90, btch: 15 usd: 0 [ 224.500133] CPU 4: hi: 90, btch: 15 usd: 26 [ 224.500369] CPU 5: hi: 90, btch: 15 usd: 0 [ 224.500606] CPU 6: hi: 90, btch: 15 usd: 0 [ 224.500842] CPU 7: hi: 90, btch: 15 usd: 0 [ 224.501080] active_anon:4612 inactive_anon:31 isolated_anon:0 [ 224.501081] active_file:317 inactive_file:984 isolated_file:0 [ 224.501081] unevictable:0 dirty:0 writeback:699 unstable:0 [ 224.501082] free:1515 slab_reclaimable:12137 slab_unreclaimable:5131 [ 224.501082] mapped:201 shmem:75 pagetables:382 bounce:0 [ 224.502468] Node 0 DMA free:1124kB min:120kB low:148kB high:180kB active_anon:572kB inactive_anon:0kB active_file:16kB inactive_file:672kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:104kB writeback:572kB mapped:0kB shmem:0kB slab_reclaimable:12952kB slab_unreclaimable:292kB kernel_stack:8kB pagetables:20kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no [ 224.504105] lowmem_reserve[]: 0 236 236 236 [ 224.504494] Node 0 DMA32 free:5600kB min:1904kB low:2380kB high:2856kB active_anon:17976kB inactive_anon:124kB active_file:1252kB inactive_file:2744kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:0kB writeback:2072kB mapped:804kB shmem:300kB slab_reclaimable:35596kB slab_unreclaimable:20232kB kernel_stack:1160kB pagetables:1508kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:14 all_unreclaimable? no [ 224.506241] lowmem_reserve[]: 0 0 0 0 [ 224.506608] Node 0 DMA: 22*4kB 53*8kB 42*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1184kB [ 224.507546] Node 0 DMA32: 1016*4kB 165*8kB 6*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 5480kB [ 224.508496] 1394 total pagecache pages [ 224.508700] 0 pages in swap cache [ 224.508888] Swap cache stats: add 0, delete 0, find 0/0 [ 224.509146] Free swap = 0kB [ 224.509318] Total swap = 0kB [ 224.510110] 65520 pages RAM [ 224.510282] 16492 pages reserved [ 224.510468] 2424 pages shared [ 224.510642] 45531 pages non-shared [ 224.510834] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 224.511216] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 224.511592] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 224.511967] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 224.512343] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 224.512716] [ 2343] 1 2343 2162 61 4 0 0 portmap [ 224.513098] [ 2355] 0 2355 3606 43 0 0 0 rpc.statd [ 224.513486] [ 2428] 0 2428 1633 102 2 0 0 syslogd [ 224.513867] [ 2437] 0 2437 1107 79 6 0 0 klogd [ 224.514253] [ 2448] 104 2448 5864 95 4 0 0 dbus-daemon [ 224.514648] [ 2524] 0 2524 6081 79 5 0 0 hald-addon-inpu [ 224.515054] [ 2541] 0 2541 6083 69 1 0 0 hald-addon-cpuf [ 224.515462] [ 2542] 106 2542 6535 68 4 0 0 hald-addon-acpi [ 224.515869] [ 2632] 0 2632 1494 26 4 0 0 getty [ 224.516243] [ 2633] 0 2633 1494 27 0 0 0 getty [ 224.516614] [ 2634] 0 2634 1494 27 0 0 0 getty [ 224.516985] [ 2635] 0 2635 1494 27 0 0 0 getty [ 224.517357] [ 2636] 0 2636 1494 26 0 0 0 getty [ 224.517729] [ 2638] 0 2638 1494 26 0 0 0 getty [ 224.518112] [ 2692] 0 2692 11425 554 0 0 0 zsh [ 224.518477] [ 2720] 0 2720 3431 63 4 0 0 concurrent-dd.s [ 224.518880] [ 2749] 0 2749 3460 290 0 0 0 dd [ 224.519241] [ 2750] 0 2750 3460 289 6 0 0 dd [ 224.519608] [ 2751] 0 2751 3460 289 3 0 0 dd [ 224.519969] [ 2752] 0 2752 3460 290 3 0 0 dd [ 224.520337] [ 2753] 0 2753 3460 290 2 0 0 dd [ 224.520702] [ 2754] 0 2754 3460 289 4 0 0 dd [ 224.521065] [ 2755] 0 2755 3460 289 7 0 0 dd [ 224.521428] [ 2756] 0 2756 3460 290 2 0 0 dd [ 224.521790] [ 2757] 0 2757 3460 290 2 0 0 dd [ 224.522165] [ 2758] 0 2758 3460 290 6 0 0 dd [ 224.522529] [ 2759] 0 2759 3437 197 4 0 0 plot-written.sh [ 224.522936] [ 2776] 0 2776 2144 94 2 0 0 iostat [ 224.523310] [ 9920] 0 9920 3437 130 5 0 0 plot-written.sh [ 224.523713] Out of memory: Kill process 2343 (portmap) score 1 or sacrifice child [ 224.524084] Killed process 2343 (portmap) total-vm:8648kB, anon-rss:132kB, file-rss:112kB [ 225.411466] iostat invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 [ 225.411906] iostat cpuset=/ mems_allowed=0 [ 225.412137] Pid: 2776, comm: iostat Not tainted 2.6.37-rc3 #154 [ 225.412437] Call Trace: [ 225.412607] [] ? _raw_spin_unlock+0x2b/0x40 [ 225.412907] [] dump_header+0x83/0x200 [ 225.413183] [] ? trace_hardirqs_on+0xd/0x10 [ 225.413482] [] ? ___ratelimit+0xab/0x150 [ 225.413768] [] ? oom_badness+0x77/0x190 [ 225.414052] [] oom_kill_process+0x8d/0x270 [ 225.414346] [] out_of_memory+0x1d8/0x410 [ 225.414650] [] __alloc_pages_nodemask+0x81b/0x830 [ 225.414970] [] alloc_pages_current+0x99/0x110 [ 225.415272] [] __page_cache_alloc+0x87/0x90 [ 225.415565] [] __do_page_cache_readahead+0x120/0x260 [ 225.415884] [] ? __do_page_cache_readahead+0xad/0x260 [ 225.416206] [] ? lock_release_holdtime+0x3d/0x180 [ 225.416518] [] ra_submit+0x21/0x30 [ 225.416779] [] filemap_fault+0x4dc/0x4f0 [ 225.417059] [] ? __lock_acquire+0x53e/0x1e70 [ 225.417353] [] __do_fault+0x54/0x540 [ 225.417620] [] handle_mm_fault+0x1af/0xc20 [ 225.417907] [] ? do_page_fault+0xf6/0x560 [ 225.418190] [] ? lock_release_holdtime+0x3d/0x180 [ 225.418509] [] do_page_fault+0x16b/0x560 [ 225.418792] [] ? _raw_spin_unlock+0x2b/0x40 [ 225.419081] [] ? error_sti+0x5/0x6 [ 225.419342] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 225.419657] [] page_fault+0x25/0x30 [ 225.419919] Mem-Info: [ 225.420073] Node 0 DMA per-cpu: [ 225.420296] CPU 0: hi: 0, btch: 1 usd: 0 [ 225.420545] CPU 1: hi: 0, btch: 1 usd: 0 [ 225.420793] CPU 2: hi: 0, btch: 1 usd: 0 [ 225.421041] CPU 3: hi: 0, btch: 1 usd: 0 [ 225.421291] CPU 4: hi: 0, btch: 1 usd: 0 [ 225.421541] CPU 5: hi: 0, btch: 1 usd: 0 [ 225.421789] CPU 6: hi: 0, btch: 1 usd: 0 [ 225.422038] CPU 7: hi: 0, btch: 1 usd: 0 [ 225.422286] Node 0 DMA32 per-cpu: [ 225.422528] CPU 0: hi: 90, btch: 15 usd: 30 [ 225.422779] CPU 1: hi: 90, btch: 15 usd: 12 [ 225.423028] CPU 2: hi: 90, btch: 15 usd: 0 [ 225.423276] CPU 3: hi: 90, btch: 15 usd: 0 [ 225.423527] CPU 4: hi: 90, btch: 15 usd: 22 [ 225.423776] CPU 5: hi: 90, btch: 15 usd: 0 [ 225.424026] CPU 6: hi: 90, btch: 15 usd: 88 [ 225.424276] CPU 7: hi: 90, btch: 15 usd: 0 [ 225.424528] active_anon:4608 inactive_anon:48 isolated_anon:0 [ 225.424530] active_file:301 inactive_file:1753 isolated_file:0 [ 225.424531] unevictable:0 dirty:55 writeback:10 unstable:0 [ 225.424532] free:719 slab_reclaimable:12182 slab_unreclaimable:5137 [ 225.424533] mapped:226 shmem:75 pagetables:374 bounce:0 [ 225.426537] Node 0 DMA free:1036kB min:120kB low:148kB high:180kB active_anon:656kB inactive_anon:0kB active_file:16kB inactive_file:736kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:0kB writeback:32kB mapped:0kB shmem:0kB slab_reclaimable:12968kB slab_unreclaimable:300kB kernel_stack:16kB pagetables:52kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1138 all_unreclaimable? yes [ 225.428244] lowmem_reserve[]: 0 236 236 236 [ 225.428650] Node 0 DMA32 free:1868kB min:1904kB low:2380kB high:2856kB active_anon:17776kB inactive_anon:192kB active_file:1188kB inactive_file:6276kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:252kB writeback:8kB mapped:904kB shmem:300kB slab_reclaimable:35760kB slab_unreclaimable:20248kB kernel_stack:1152kB pagetables:1444kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:18485 all_unreclaimable? yes [ 225.430439] lowmem_reserve[]: 0 0 0 0 [ 225.430833] Node 0 DMA: 8*4kB 56*8kB 36*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1056kB [ 225.431806] Node 0 DMA32: 109*4kB 171*8kB 4*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1868kB [ 225.432790] 2123 total pagecache pages [ 225.432999] 0 pages in swap cache [ 225.433191] Swap cache stats: add 0, delete 0, find 0/0 [ 225.433456] Free swap = 0kB [ 225.433632] Total swap = 0kB [ 225.434528] 65520 pages RAM [ 225.434703] 16492 pages reserved [ 225.434893] 1459 pages shared [ 225.435072] 46692 pages non-shared [ 225.435269] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 225.435657] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 225.436042] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 225.436428] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 225.436814] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 225.437195] [ 2355] 0 2355 3606 43 0 0 0 rpc.statd [ 225.437593] [ 2428] 0 2428 1633 96 4 0 0 syslogd [ 225.437985] [ 2437] 0 2437 1107 73 7 0 0 klogd [ 225.438371] [ 2448] 104 2448 5864 95 4 0 0 dbus-daemon [ 225.438782] [ 2524] 0 2524 6081 79 5 0 0 hald-addon-inpu [ 225.439201] [ 2541] 0 2541 6083 69 1 0 0 hald-addon-cpuf [ 225.439617] [ 2542] 106 2542 6535 68 4 0 0 hald-addon-acpi [ 225.440037] [ 2632] 0 2632 1494 26 4 0 0 getty [ 225.440424] [ 2633] 0 2633 1494 27 0 0 0 getty [ 225.440809] [ 2634] 0 2634 1494 27 0 0 0 getty [ 225.441195] [ 2635] 0 2635 1494 27 0 0 0 getty [ 225.441579] [ 2636] 0 2636 1494 26 0 0 0 getty [ 225.441964] [ 2638] 0 2638 1494 26 0 0 0 getty [ 225.442349] [ 2692] 0 2692 11425 554 0 0 0 zsh [ 225.442735] [ 2720] 0 2720 3431 63 4 0 0 concurrent-dd.s [ 225.443154] [ 2749] 0 2749 3460 290 1 0 0 dd [ 225.443531] [ 2750] 0 2750 3460 289 6 0 0 dd [ 225.443906] [ 2751] 0 2751 3460 289 6 0 0 dd [ 225.444279] [ 2752] 0 2752 3460 290 7 0 0 dd [ 225.444655] [ 2753] 0 2753 3460 290 7 0 0 dd [ 225.445032] [ 2754] 0 2754 3460 289 3 0 0 dd [ 225.445407] [ 2755] 0 2755 3460 289 5 0 0 dd [ 225.445782] [ 2756] 0 2756 3460 290 0 0 0 dd [ 225.446158] [ 2757] 0 2757 3460 290 5 0 0 dd [ 225.446539] [ 2758] 0 2758 3460 290 6 0 0 dd [ 225.446916] [ 2759] 0 2759 3437 197 6 0 0 plot-written.sh [ 225.447334] [ 2776] 0 2776 2145 95 4 0 0 iostat [ 225.447722] [ 9922] 0 9922 3437 161 4 0 0 plot-written.sh [ 225.448141] [ 9923] 0 9923 1501 45 0 0 0 grep [ 225.448521] Out of memory: Kill process 2355 (rpc.statd) score 1 or sacrifice child [ 225.448913] Killed process 2355 (rpc.statd) total-vm:14424kB, anon-rss:168kB, file-rss:4kB [ 225.924637] iostat invoked oom-killer: gfp_mask=0xd0, order=0, oom_adj=0, oom_score_adj=0 [ 225.925045] iostat cpuset=/ mems_allowed=0 [ 225.925266] Pid: 2776, comm: iostat Not tainted 2.6.37-rc3 #154 [ 225.925551] Call Trace: [ 225.925710] [] ? _raw_spin_unlock+0x2b/0x40 [ 225.925996] [] dump_header+0x83/0x200 [ 225.926260] [] ? trace_hardirqs_on+0xd/0x10 [ 225.926541] [] ? ___ratelimit+0xab/0x150 [ 225.926833] [] ? oom_badness+0x77/0x190 [ 225.927106] [] oom_kill_process+0x8d/0x270 [ 225.927390] [] out_of_memory+0x1d8/0x410 [ 225.927673] [] __alloc_pages_nodemask+0x81b/0x830 [ 225.927984] [] alloc_pages_current+0x99/0x110 [ 225.928280] [] new_slab+0x294/0x2a0 [ 225.928544] [] ? __slab_alloc+0x232/0x450 [ 225.928827] [] __slab_alloc+0x247/0x450 [ 225.929104] [] ? nfs_create_request+0x41/0x160 [ 225.929403] [] ? nfs_create_request+0x41/0x160 [ 225.929704] [] kmem_cache_alloc+0x17c/0x190 [ 225.929993] [] ? _raw_spin_unlock_irq+0x30/0x40 [ 225.930296] [] ? _raw_spin_unlock_irq+0x30/0x40 [ 225.930598] [] nfs_create_request+0x41/0x160 [ 225.930902] [] ? _raw_spin_unlock_irq+0x30/0x40 [ 225.931206] [] ? readpage_async_filler+0x0/0x150 [ 225.931511] [] readpage_async_filler+0x65/0x150 [ 225.931815] [] ? readpage_async_filler+0x0/0x150 [ 225.932122] [] read_cache_pages+0xa2/0xf0 [ 225.932404] [] nfs_readpages+0xe2/0x1b0 [ 225.932678] [] ? nfs_pagein_one+0x0/0x100 [ 225.932963] [] __do_page_cache_readahead+0x18c/0x260 [ 225.933284] [] ? __do_page_cache_readahead+0xad/0x260 [ 225.933605] [] ? lock_release_holdtime+0x3d/0x180 [ 225.933915] [] ra_submit+0x21/0x30 [ 225.934175] [] filemap_fault+0x4dc/0x4f0 [ 225.934455] [] ? __lock_acquire+0x53e/0x1e70 [ 225.934758] [] __do_fault+0x54/0x540 [ 225.935023] [] handle_mm_fault+0x1af/0xc20 [ 225.935311] [] ? do_page_fault+0xf6/0x560 [ 225.935595] [] ? lock_release_holdtime+0x3d/0x180 [ 225.935957] [] do_page_fault+0x16b/0x560 [ 225.936237] [] ? fsnotify+0x7c/0x2d0 [ 225.936505] [] ? error_sti+0x5/0x6 [ 225.936767] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 225.937082] [] page_fault+0x25/0x30 [ 225.937344] Mem-Info: [ 225.937498] Node 0 DMA per-cpu: [ 225.937721] CPU 0: hi: 0, btch: 1 usd: 0 [ 225.937971] CPU 1: hi: 0, btch: 1 usd: 0 [ 225.938218] CPU 2: hi: 0, btch: 1 usd: 0 [ 225.938465] CPU 3: hi: 0, btch: 1 usd: 0 [ 225.938715] CPU 4: hi: 0, btch: 1 usd: 0 [ 225.938972] CPU 5: hi: 0, btch: 1 usd: 0 [ 225.939222] CPU 6: hi: 0, btch: 1 usd: 0 [ 225.939470] CPU 7: hi: 0, btch: 1 usd: 0 [ 225.939718] Node 0 DMA32 per-cpu: [ 225.939947] CPU 0: hi: 90, btch: 15 usd: 15 [ 225.940196] CPU 1: hi: 90, btch: 15 usd: 15 [ 225.940496] CPU 2: hi: 90, btch: 15 usd: 14 [ 225.940744] CPU 3: hi: 90, btch: 15 usd: 6 [ 225.940992] CPU 4: hi: 90, btch: 15 usd: 13 [ 225.941240] CPU 5: hi: 90, btch: 15 usd: 14 [ 225.941488] CPU 6: hi: 90, btch: 15 usd: 6 [ 225.941737] CPU 7: hi: 90, btch: 15 usd: 10 [ 225.941987] active_anon:4614 inactive_anon:4 isolated_anon:0 [ 225.941988] active_file:260 inactive_file:926 isolated_file:0 [ 225.941989] unevictable:0 dirty:43 writeback:0 unstable:0 [ 225.941990] free:755 slab_reclaimable:12199 slab_unreclaimable:5122 [ 225.941991] mapped:251 shmem:75 pagetables:374 bounce:0 [ 225.943408] Node 0 DMA free:1056kB min:120kB low:148kB high:180kB active_anon:656kB inactive_anon:0kB active_file:16kB inactive_file:164kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:20kB writeback:32kB mapped:0kB shmem:0kB slab_reclaimable:12968kB slab_unreclaimable:240kB kernel_stack:16kB pagetables:52kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:351 all_unreclaimable? yes [ 225.945113] lowmem_reserve[]: 0 236 236 236 [ 225.945566] Node 0 DMA32 free:1868kB min:1904kB low:2380kB high:2856kB active_anon:17800kB inactive_anon:16kB active_file:1024kB inactive_file:3540kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:152kB writeback:0kB mapped:1004kB shmem:300kB slab_reclaimable:35828kB slab_unreclaimable:20248kB kernel_stack:1152kB pagetables:1444kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:8696 all_unreclaimable? yes [ 225.947950] lowmem_reserve[]: 0 0 0 0 [ 225.948347] Node 0 DMA: 10*4kB 98*8kB 19*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1128kB [ 225.949349] Node 0 DMA32: 199*4kB 156*8kB 4*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2108kB [ 225.950362] 1343 total pagecache pages [ 225.950575] 0 pages in swap cache [ 225.950782] Swap cache stats: add 0, delete 0, find 0/0 [ 225.951053] Free swap = 0kB [ 225.951228] Total swap = 0kB [ 225.952026] 65520 pages RAM [ 225.952193] 16492 pages reserved [ 225.952375] 2225 pages shared [ 225.952548] 46398 pages non-shared [ 225.952739] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 225.953115] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 225.953488] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 225.953858] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 225.954230] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 225.954598] [ 2428] 0 2428 1633 96 4 0 0 syslogd [ 225.954982] [ 2437] 0 2437 1107 74 2 0 0 klogd [ 225.955355] [ 2448] 104 2448 5864 94 4 0 0 dbus-daemon [ 225.955750] [ 2524] 0 2524 6081 73 5 0 0 hald-addon-inpu [ 225.956153] [ 2541] 0 2541 6083 63 1 0 0 hald-addon-cpuf [ 225.956556] [ 2542] 106 2542 6535 65 4 0 0 hald-addon-acpi [ 225.956959] [ 2632] 0 2632 1494 26 4 0 0 getty [ 225.957377] [ 2633] 0 2633 1494 27 0 0 0 getty [ 225.957746] [ 2634] 0 2634 1494 27 0 0 0 getty [ 225.958120] [ 2635] 0 2635 1494 27 0 0 0 getty [ 225.958494] [ 2636] 0 2636 1494 26 0 0 0 getty [ 225.958877] [ 2638] 0 2638 1494 26 0 0 0 getty [ 225.959244] [ 2692] 0 2692 11425 554 0 0 0 zsh [ 225.959603] [ 2720] 0 2720 3431 63 4 0 0 concurrent-dd.s [ 225.960003] [ 2749] 0 2749 3460 290 7 0 0 dd [ 225.960361] [ 2750] 0 2750 3460 289 3 0 0 dd [ 225.960718] [ 2751] 0 2751 3460 289 6 0 0 dd [ 225.961078] [ 2752] 0 2752 3460 290 7 0 0 dd [ 225.961437] [ 2753] 0 2753 3460 290 6 0 0 dd [ 225.961805] [ 2754] 0 2754 3460 289 6 0 0 dd [ 225.962186] [ 2755] 0 2755 3460 289 5 0 0 dd [ 225.962570] [ 2756] 0 2756 3460 290 3 0 0 dd [ 225.962963] [ 2757] 0 2757 3460 290 7 0 0 dd [ 225.963346] [ 2758] 0 2758 3460 290 2 0 0 dd [ 225.963781] [ 2759] 0 2759 3437 196 6 0 0 plot-written.sh [ 225.964201] [ 2776] 0 2776 2144 98 1 0 0 iostat [ 225.964577] [ 9922] 0 9922 3437 161 4 0 0 plot-written.sh [ 225.964981] [ 9923] 0 9923 2675 128 2 0 0 grep [ 225.965347] Out of memory: Kill process 2428 (syslogd) score 1 or sacrifice child [ 225.965721] Killed process 2428 (syslogd) total-vm:6532kB, anon-rss:164kB, file-rss:220kB [ 226.294192] wc invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 [ 226.294607] wc cpuset=/ mems_allowed=0 [ 226.294815] Pid: 9926, comm: wc Not tainted 2.6.37-rc3 #154 [ 226.295395] Call Trace: [ 226.295968] [] ? _raw_spin_unlock+0x2b/0x40 [ 226.296265] [] dump_header+0x83/0x200 [ 226.296523] [] ? trace_hardirqs_on+0xd/0x10 [ 226.296805] [] ? ___ratelimit+0xab/0x150 [ 226.297109] [] ? oom_badness+0x77/0x190 [ 226.297381] [] oom_kill_process+0x8d/0x270 [ 226.297839] [] out_of_memory+0x1d8/0x410 [ 226.298114] [] __alloc_pages_nodemask+0x81b/0x830 [ 226.298415] [] alloc_pages_current+0x99/0x110 [ 226.298702] [] __page_cache_alloc+0x87/0x90 [ 226.299037] [] __do_page_cache_readahead+0x120/0x260 [ 226.299344] [] ? __do_page_cache_readahead+0xad/0x260 [ 226.299663] [] ? lock_release_holdtime+0x3d/0x180 [ 226.299970] [] ra_submit+0x21/0x30 [ 226.300228] [] filemap_fault+0x4dc/0x4f0 [ 226.300649] [] ? __lock_acquire+0x53e/0x1e70 [ 226.300936] [] __do_fault+0x54/0x540 [ 226.301208] [] handle_mm_fault+0x1af/0xc20 [ 226.301485] [] ? do_page_fault+0xf6/0x560 [ 226.301761] [] ? lock_release_holdtime+0x3d/0x180 [ 226.302067] [] do_page_fault+0x16b/0x560 [ 226.302346] [] ? kmem_cache_free+0x105/0x180 [ 226.302637] [] ? kmem_cache_free+0x105/0x180 [ 226.302990] [] ? error_sti+0x5/0x6 [ 226.303248] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 226.303561] [] page_fault+0x25/0x30 [ 226.303821] Mem-Info: [ 226.303977] Node 0 DMA per-cpu: [ 226.304203] CPU 0: hi: 0, btch: 1 usd: 0 [ 226.304451] CPU 1: hi: 0, btch: 1 usd: 0 [ 226.304699] CPU 2: hi: 0, btch: 1 usd: 0 [ 226.304947] CPU 3: hi: 0, btch: 1 usd: 0 [ 226.305194] CPU 4: hi: 0, btch: 1 usd: 0 [ 226.305441] CPU 5: hi: 0, btch: 1 usd: 0 [ 226.305690] CPU 6: hi: 0, btch: 1 usd: 0 [ 226.305938] CPU 7: hi: 0, btch: 1 usd: 0 [ 226.306186] Node 0 DMA32 per-cpu: [ 226.306418] CPU 0: hi: 90, btch: 15 usd: 0 [ 226.306666] CPU 1: hi: 90, btch: 15 usd: 0 [ 226.306924] CPU 2: hi: 90, btch: 15 usd: 0 [ 226.307175] CPU 3: hi: 90, btch: 15 usd: 0 [ 226.307423] CPU 4: hi: 90, btch: 15 usd: 0 [ 226.307671] CPU 5: hi: 90, btch: 15 usd: 13 [ 226.307916] CPU 6: hi: 90, btch: 15 usd: 0 [ 226.308162] CPU 7: hi: 90, btch: 15 usd: 0 [ 226.308409] active_anon:4552 inactive_anon:24 isolated_anon:0 [ 226.308410] active_file:287 inactive_file:1656 isolated_file:0 [ 226.308410] unevictable:0 dirty:1 writeback:1203 unstable:0 [ 226.308411] free:954 slab_reclaimable:12273 slab_unreclaimable:5181 [ 226.308411] mapped:252 shmem:75 pagetables:360 bounce:0 [ 226.309798] Node 0 DMA free:1168kB min:120kB low:148kB high:180kB active_anon:616kB inactive_anon:0kB active_file:12kB inactive_file:612kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:12kB writeback:520kB mapped:4kB shmem:0kB slab_reclaimable:12972kB slab_unreclaimable:288kB kernel_stack:8kB pagetables:40kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no [ 226.311750] lowmem_reserve[]: 0 236 236 236 [ 226.312156] Node 0 DMA32 free:2748kB min:1904kB low:2380kB high:2856kB active_anon:17592kB inactive_anon:96kB active_file:1136kB inactive_file:6012kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:0kB writeback:3792kB mapped:1004kB shmem:300kB slab_reclaimable:36120kB slab_unreclaimable:20436kB kernel_stack:1144kB pagetables:1400kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no [ 226.313900] lowmem_reserve[]: 0 0 0 0 [ 226.314289] Node 0 DMA: 33*4kB 54*8kB 38*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1172kB [ 226.315300] Node 0 DMA32: 275*4kB 194*8kB 6*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2748kB [ 226.316294] 2103 total pagecache pages [ 226.316505] 0 pages in swap cache [ 226.316697] Swap cache stats: add 0, delete 0, find 0/0 [ 226.316960] Free swap = 0kB [ 226.317136] Total swap = 0kB [ 226.317871] 65520 pages RAM [ 226.318049] 16492 pages reserved [ 226.318239] 3641 pages shared [ 226.318419] 45492 pages non-shared [ 226.318615] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 226.319059] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 226.319440] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 226.319818] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 226.320195] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 226.320569] [ 2437] 0 2437 1107 76 6 0 0 klogd [ 226.321511] [ 2448] 104 2448 5864 94 4 0 0 dbus-daemon [ 226.321906] [ 2524] 0 2524 6081 73 5 0 0 hald-addon-inpu [ 226.322313] [ 2541] 0 2541 6083 63 1 0 0 hald-addon-cpuf [ 226.322724] [ 2542] 106 2542 6535 65 4 0 0 hald-addon-acpi [ 226.323244] [ 2632] 0 2632 1494 26 4 0 0 getty [ 226.323628] [ 2633] 0 2633 1494 27 0 0 0 getty [ 226.324004] [ 2634] 0 2634 1494 27 0 0 0 getty [ 226.324380] [ 2635] 0 2635 1494 27 0 0 0 getty [ 226.324945] [ 2636] 0 2636 1494 26 0 0 0 getty [ 226.325325] [ 2638] 0 2638 1494 26 0 0 0 getty [ 226.325701] [ 2692] 0 2692 11425 554 0 0 0 zsh [ 226.326071] [ 2720] 0 2720 3431 63 4 0 0 concurrent-dd.s [ 226.326479] [ 2749] 0 2749 3460 290 1 0 0 dd [ 226.326844] [ 2750] 0 2750 3460 289 3 0 0 dd [ 226.327322] [ 2751] 0 2751 3460 289 1 0 0 dd [ 226.327688] [ 2752] 0 2752 3460 290 3 0 0 dd [ 226.328054] [ 2753] 0 2753 3460 290 2 0 0 dd [ 226.328421] [ 2754] 0 2754 3460 289 2 0 0 dd [ 226.328786] [ 2755] 0 2755 3460 289 1 0 0 dd [ 226.329153] [ 2756] 0 2756 3460 290 3 0 0 dd [ 226.329518] [ 2757] 0 2757 3460 290 3 0 0 dd [ 226.329884] [ 2758] 0 2758 3460 290 6 0 0 dd [ 226.330250] [ 2759] 0 2759 3437 197 0 0 0 plot-written.sh [ 226.330659] [ 2776] 0 2776 2144 95 7 0 0 iostat [ 226.331239] [ 9924] 0 9924 3437 150 1 0 0 plot-written.sh [ 226.331651] [ 9926] 0 9926 2137 127 0 0 0 wc [ 226.332019] Out of memory: Kill process 2437 (klogd) score 1 or sacrifice child [ 226.332392] Killed process 2437 (klogd) total-vm:4428kB, anon-rss:96kB, file-rss:208kB [ 227.419481] dd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 [ 227.419889] dd cpuset=/ mems_allowed=0 [ 227.420101] Pid: 2749, comm: dd Not tainted 2.6.37-rc3 #154 [ 227.420380] Call Trace: [ 227.420546] [] ? _raw_spin_unlock+0x2b/0x40 [ 227.420836] [] dump_header+0x83/0x200 [ 227.421107] [] ? trace_hardirqs_on+0xd/0x10 [ 227.421399] [] ? ___ratelimit+0xab/0x150 [ 227.421679] [] ? oom_badness+0x77/0x190 [ 227.421955] [] oom_kill_process+0x8d/0x270 [ 227.422242] [] out_of_memory+0x1d8/0x410 [ 227.422524] [] __alloc_pages_nodemask+0x81b/0x830 [ 227.422833] [] alloc_pages_current+0x99/0x110 [ 227.423129] [] __page_cache_alloc+0x87/0x90 [ 227.423420] [] __do_page_cache_readahead+0x120/0x260 [ 227.423750] [] ? __do_page_cache_readahead+0xad/0x260 [ 227.424070] [] ? lock_release_holdtime+0x3d/0x180 [ 227.424379] [] ra_submit+0x21/0x30 [ 227.424639] [] filemap_fault+0x4dc/0x4f0 [ 227.424916] [] ? __lock_acquire+0x53e/0x1e70 [ 227.425209] [] __do_fault+0x54/0x540 [ 227.425475] [] ? __mutex_unlock_slowpath+0xd5/0x170 [ 227.425790] [] ? __mutex_unlock_slowpath+0xd5/0x170 [ 227.426103] [] handle_mm_fault+0x1af/0xc20 [ 227.426390] [] ? do_page_fault+0xf6/0x560 [ 227.426673] [] ? local_clock+0x6f/0x80 [ 227.426943] [] do_page_fault+0x16b/0x560 [ 227.427222] [] ? fsnotify+0x7c/0x2d0 [ 227.427503] [] ? error_sti+0x5/0x6 [ 227.427763] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 227.428077] [] page_fault+0x25/0x30 [ 227.428336] Mem-Info: [ 227.428491] Node 0 DMA per-cpu: [ 227.428711] CPU 0: hi: 0, btch: 1 usd: 0 [ 227.428960] CPU 1: hi: 0, btch: 1 usd: 0 [ 227.429206] CPU 2: hi: 0, btch: 1 usd: 0 [ 227.429453] CPU 3: hi: 0, btch: 1 usd: 0 [ 227.429700] CPU 4: hi: 0, btch: 1 usd: 0 [ 227.429948] CPU 5: hi: 0, btch: 1 usd: 0 [ 227.430195] CPU 6: hi: 0, btch: 1 usd: 0 [ 227.430441] CPU 7: hi: 0, btch: 1 usd: 0 [ 227.430688] Node 0 DMA32 per-cpu: [ 227.430916] CPU 0: hi: 90, btch: 15 usd: 0 [ 227.431163] CPU 1: hi: 90, btch: 15 usd: 12 [ 227.431409] CPU 2: hi: 90, btch: 15 usd: 8 [ 227.431663] CPU 3: hi: 90, btch: 15 usd: 12 [ 227.431910] CPU 4: hi: 90, btch: 15 usd: 14 [ 227.432156] CPU 5: hi: 90, btch: 15 usd: 9 [ 227.432404] CPU 6: hi: 90, btch: 15 usd: 24 [ 227.432651] CPU 7: hi: 90, btch: 15 usd: 0 [ 227.432903] active_anon:4512 inactive_anon:22 isolated_anon:0 [ 227.432904] active_file:26 inactive_file:398 isolated_file:265 [ 227.432904] unevictable:0 dirty:0 writeback:0 unstable:0 [ 227.432905] free:817 slab_reclaimable:12343 slab_unreclaimable:5117 [ 227.432906] mapped:34 shmem:75 pagetables:343 bounce:0 [ 227.434316] Node 0 DMA free:1056kB min:120kB low:148kB high:180kB active_anon:572kB inactive_anon:0kB active_file:0kB inactive_file:204kB unevictable:0kB isolated(anon):0kB isolated(file):128kB present:15660kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:12984kB slab_unreclaimable:268kB kernel_stack:8kB pagetables:20kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:64 all_unreclaimable? no [ 227.436004] lowmem_reserve[]: 0 236 236 236 [ 227.436402] Node 0 DMA32 free:1876kB min:1904kB low:2380kB high:2856kB active_anon:17476kB inactive_anon:88kB active_file:104kB inactive_file:1740kB unevictable:0kB isolated(anon):0kB isolated(file):716kB present:242400kB mlocked:0kB dirty:96kB writeback:184kB mapped:136kB shmem:300kB slab_reclaimable:36388kB slab_unreclaimable:20200kB kernel_stack:1128kB pagetables:1352kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:834 all_unreclaimable? no [ 227.438203] lowmem_reserve[]: 0 0 0 0 [ 227.438599] Node 0 DMA: 10*4kB 44*8kB 39*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1016kB [ 227.439586] Node 0 DMA32: 156*4kB 155*8kB 4*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1928kB [ 227.440661] 1185 total pagecache pages [ 227.440867] 0 pages in swap cache [ 227.441062] Swap cache stats: add 0, delete 0, find 0/0 [ 227.441322] Free swap = 0kB [ 227.441496] Total swap = 0kB [ 227.442297] 65520 pages RAM [ 227.442471] 16492 pages reserved [ 227.442658] 1711 pages shared [ 227.442837] 46607 pages non-shared [ 227.443032] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 227.443425] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 227.443812] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 227.444196] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 227.444572] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 227.444946] [ 2448] 104 2448 5864 83 4 0 0 dbus-daemon [ 227.445345] [ 2524] 0 2524 6081 49 5 0 0 hald-addon-inpu [ 227.445761] [ 2541] 0 2541 6083 48 1 0 0 hald-addon-cpuf [ 227.446181] [ 2542] 106 2542 6535 53 0 0 0 hald-addon-acpi [ 227.446598] [ 2632] 0 2632 1494 26 4 0 0 getty [ 227.446982] [ 2633] 0 2633 1494 27 0 0 0 getty [ 227.447367] [ 2634] 0 2634 1494 27 0 0 0 getty [ 227.447762] [ 2635] 0 2635 1494 27 0 0 0 getty [ 227.448144] [ 2636] 0 2636 1494 26 0 0 0 getty [ 227.448531] [ 2638] 0 2638 1494 26 0 0 0 getty [ 227.448919] [ 2692] 0 2692 11425 554 0 0 0 zsh [ 227.449294] [ 2720] 0 2720 3431 63 4 0 0 concurrent-dd.s [ 227.449715] [ 2749] 0 2749 3460 288 2 0 0 dd [ 227.450086] [ 2750] 0 2750 3460 287 6 0 0 dd [ 227.450460] [ 2751] 0 2751 3460 287 1 0 0 dd [ 227.451407] [ 2752] 0 2752 3460 288 1 0 0 dd [ 227.451788] [ 2753] 0 2753 3460 288 7 0 0 dd [ 227.452160] [ 2754] 0 2754 3460 287 7 0 0 dd [ 227.452533] [ 2755] 0 2755 3460 287 4 0 0 dd [ 227.452908] [ 2756] 0 2756 3460 288 5 0 0 dd [ 227.453284] [ 2757] 0 2757 3460 288 3 0 0 dd [ 227.453660] [ 2758] 0 2758 3460 288 1 0 0 dd [ 227.454032] [ 2759] 0 2759 3437 84 1 0 0 plot-written.sh [ 227.454444] [ 2776] 0 2776 2145 39 4 0 0 iostat [ 227.454830] [ 9929] 0 9929 3202 39 5 0 0 date [ 227.455207] Out of memory: Kill process 2448 (dbus-daemon) score 1 or sacrifice child [ 227.455670] Killed process 2448 (dbus-daemon) total-vm:23456kB, anon-rss:328kB, file-rss:4kB [ 227.931323] iostat invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 [ 227.931753] iostat cpuset=/ mems_allowed=0 [ 227.931972] Pid: 2776, comm: iostat Not tainted 2.6.37-rc3 #154 [ 227.932255] Call Trace: [ 227.932414] [] ? _raw_spin_unlock+0x2b/0x40 [ 227.932697] [] dump_header+0x83/0x200 [ 227.932962] [] ? trace_hardirqs_on+0xd/0x10 [ 227.933245] [] ? ___ratelimit+0xab/0x150 [ 227.933518] [] ? oom_badness+0x77/0x190 [ 227.933787] [] oom_kill_process+0x8d/0x270 [ 227.934066] [] out_of_memory+0x1d8/0x410 [ 227.934339] [] __alloc_pages_nodemask+0x81b/0x830 [ 227.934644] [] alloc_pages_current+0x99/0x110 [ 227.934932] [] __page_cache_alloc+0x87/0x90 [ 227.935216] [] __do_page_cache_readahead+0x120/0x260 [ 227.935528] [] ? __do_page_cache_readahead+0xad/0x260 [ 227.935880] [] ? lock_release_holdtime+0x3d/0x180 [ 227.936183] [] ra_submit+0x21/0x30 [ 227.936435] [] filemap_fault+0x4dc/0x4f0 [ 227.936707] [] ? __lock_acquire+0x53e/0x1e70 [ 227.936993] [] __do_fault+0x54/0x540 [ 227.937251] [] handle_mm_fault+0x1af/0xc20 [ 227.937530] [] ? do_page_fault+0xf6/0x560 [ 227.937804] [] ? lock_release_holdtime+0x3d/0x180 [ 227.938104] [] do_page_fault+0x16b/0x560 [ 227.938375] [] ? error_sti+0x5/0x6 [ 227.938627] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 227.938935] [] page_fault+0x25/0x30 [ 227.939189] Mem-Info: [ 227.939339] Node 0 DMA per-cpu: [ 227.939556] CPU 0: hi: 0, btch: 1 usd: 0 [ 227.939833] CPU 1: hi: 0, btch: 1 usd: 0 [ 227.940078] CPU 2: hi: 0, btch: 1 usd: 0 [ 227.940322] CPU 3: hi: 0, btch: 1 usd: 0 [ 227.940564] CPU 4: hi: 0, btch: 1 usd: 0 [ 227.940806] CPU 5: hi: 0, btch: 1 usd: 0 [ 227.941049] CPU 6: hi: 0, btch: 1 usd: 0 [ 227.941291] CPU 7: hi: 0, btch: 1 usd: 0 [ 227.941533] Node 0 DMA32 per-cpu: [ 227.941755] CPU 0: hi: 90, btch: 15 usd: 0 [ 227.941998] CPU 1: hi: 90, btch: 15 usd: 0 [ 227.942241] CPU 2: hi: 90, btch: 15 usd: 0 [ 227.942484] CPU 3: hi: 90, btch: 15 usd: 0 [ 227.942728] CPU 4: hi: 90, btch: 15 usd: 17 [ 227.942969] CPU 5: hi: 90, btch: 15 usd: 7 [ 227.943212] CPU 6: hi: 90, btch: 15 usd: 14 [ 227.943455] CPU 7: hi: 90, btch: 15 usd: 0 [ 227.943735] active_anon:4462 inactive_anon:0 isolated_anon:0 [ 227.943737] active_file:26 inactive_file:1013 isolated_file:0 [ 227.943738] unevictable:0 dirty:11 writeback:0 unstable:0 [ 227.943739] free:728 slab_reclaimable:12283 slab_unreclaimable:5147 [ 227.943740] mapped:34 shmem:75 pagetables:343 bounce:0 [ 227.945124] Node 0 DMA free:1056kB min:120kB low:148kB high:180kB active_anon:572kB inactive_anon:0kB active_file:0kB inactive_file:148kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:12984kB slab_unreclaimable:268kB kernel_stack:8kB pagetables:20kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:252 all_unreclaimable? yes [ 227.946837] lowmem_reserve[]: 0 236 236 236 [ 227.947228] Node 0 DMA32 free:1880kB min:1904kB low:2380kB high:2856kB active_anon:17276kB inactive_anon:0kB active_file:104kB inactive_file:3904kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:96kB writeback:0kB mapped:136kB shmem:300kB slab_reclaimable:36148kB slab_unreclaimable:20320kB kernel_stack:1128kB pagetables:1352kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:10121 all_unreclaimable? yes [ 227.949004] lowmem_reserve[]: 0 0 0 0 [ 227.949374] Node 0 DMA: 21*4kB 30*8kB 48*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1092kB [ 227.950312] Node 0 DMA32: 156*4kB 105*8kB 16*16kB 5*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1880kB [ 227.951264] 1245 total pagecache pages [ 227.951469] 0 pages in swap cache [ 227.951656] Swap cache stats: add 0, delete 0, find 0/0 [ 227.951951] Free swap = 0kB [ 227.952124] Total swap = 0kB [ 227.952895] 65520 pages RAM [ 227.953065] 16492 pages reserved [ 227.953251] 1367 pages shared [ 227.953426] 46562 pages non-shared [ 227.953618] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 227.954000] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 227.954376] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 227.954752] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 227.955129] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 227.955501] [ 2524] 0 2524 6081 49 5 0 0 hald-addon-inpu [ 227.955922] [ 2541] 0 2541 6083 54 4 0 0 hald-addon-cpuf [ 227.956329] [ 2542] 106 2542 6535 53 0 0 0 hald-addon-acpi [ 227.956736] [ 2632] 0 2632 1494 26 4 0 0 getty [ 227.957110] [ 2633] 0 2633 1494 27 0 0 0 getty [ 227.957484] [ 2634] 0 2634 1494 27 0 0 0 getty [ 227.957858] [ 2635] 0 2635 1494 27 0 0 0 getty [ 227.958231] [ 2636] 0 2636 1494 26 0 0 0 getty [ 227.958606] [ 2638] 0 2638 1494 26 0 0 0 getty [ 227.958980] [ 2692] 0 2692 11425 554 0 0 0 zsh [ 227.959347] [ 2720] 0 2720 3431 63 4 0 0 concurrent-dd.s [ 227.959761] [ 2749] 0 2749 3460 286 4 0 0 dd [ 227.960126] [ 2750] 0 2750 3460 285 4 0 0 dd [ 227.960491] [ 2751] 0 2751 3460 287 4 0 0 dd [ 227.960855] [ 2752] 0 2752 3460 286 1 0 0 dd [ 227.961219] [ 2753] 0 2753 3460 288 3 0 0 dd [ 227.961585] [ 2754] 0 2754 3460 285 7 0 0 dd [ 227.961951] [ 2755] 0 2755 3460 287 7 0 0 dd [ 227.962317] [ 2756] 0 2756 3460 288 4 0 0 dd [ 227.962683] [ 2757] 0 2757 3460 286 1 0 0 dd [ 227.963047] [ 2758] 0 2758 3460 286 3 0 0 dd [ 227.963413] [ 2759] 0 2759 3437 80 1 0 0 plot-written.sh [ 227.963831] [ 2776] 0 2776 2144 48 6 0 0 iostat [ 227.964210] [ 9929] 0 9929 3202 40 5 0 0 date [ 227.964583] Out of memory: Kill process 2524 (hald-addon-inpu) score 1 or sacrifice child [ 227.964983] Killed process 2524 (hald-addon-inpu) total-vm:24324kB, anon-rss:192kB, file-rss:4kB [ 233.630941] grep invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 [ 233.631361] grep cpuset=/ mems_allowed=0 [ 233.631579] Pid: 10008, comm: grep Not tainted 2.6.37-rc3 #154 [ 233.631874] Call Trace: [ 233.632040] [] ? _raw_spin_unlock+0x2b/0x40 [ 233.632341] [] dump_header+0x83/0x200 [ 233.632616] [] ? trace_hardirqs_on+0xd/0x10 [ 233.632919] [] ? ___ratelimit+0xab/0x150 [ 233.633203] [] ? oom_badness+0x77/0x190 [ 233.633489] [] oom_kill_process+0x8d/0x270 [ 233.633787] [] out_of_memory+0x1d8/0x410 [ 233.634082] [] __alloc_pages_nodemask+0x81b/0x830 [ 233.634426] [] alloc_pages_current+0x99/0x110 [ 233.635279] [] __page_cache_alloc+0x87/0x90 [ 233.635576] [] __do_page_cache_readahead+0x120/0x260 [ 233.635903] [] ? __do_page_cache_readahead+0xad/0x260 [ 233.636235] [] ? lock_release_holdtime+0x3d/0x180 [ 233.636554] [] ra_submit+0x21/0x30 [ 233.636823] [] filemap_fault+0x4dc/0x4f0 [ 233.637109] [] ? __lock_acquire+0x53e/0x1e70 [ 233.637409] [] __do_fault+0x54/0x540 [ 233.637685] [] ? local_clock+0x6f/0x80 [ 233.637967] [] handle_mm_fault+0x1af/0xc20 [ 233.638261] [] ? do_page_fault+0xf6/0x560 [ 233.638559] [] ? lock_release_holdtime+0x3d/0x180 [ 233.638878] [] do_page_fault+0x16b/0x560 [ 233.639164] [] ? local_clock+0x6f/0x80 [ 233.639444] [] ? trace_hardirqs_off_caller+0x29/0x150 [ 233.639773] [] ? trace_hardirqs_off+0xd/0x10 [ 233.640075] [] ? local_clock+0x6f/0x80 [ 233.640357] [] ? error_sti+0x5/0x6 [ 233.640624] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 233.640945] [] page_fault+0x25/0x30 [ 233.641214] Mem-Info: [ 233.641372] Node 0 DMA per-cpu: [ 233.641603] CPU 0: hi: 0, btch: 1 usd: 0 [ 233.641859] CPU 1: hi: 0, btch: 1 usd: 0 [ 233.642114] CPU 2: hi: 0, btch: 1 usd: 0 [ 233.642377] CPU 3: hi: 0, btch: 1 usd: 0 [ 233.642632] CPU 4: hi: 0, btch: 1 usd: 0 [ 233.642888] CPU 5: hi: 0, btch: 1 usd: 0 [ 233.643145] CPU 6: hi: 0, btch: 1 usd: 0 [ 233.643400] CPU 7: hi: 0, btch: 1 usd: 0 [ 233.643655] Node 0 DMA32 per-cpu: [ 233.643890] CPU 0: hi: 90, btch: 15 usd: 0 [ 233.644148] CPU 1: hi: 90, btch: 15 usd: 23 [ 233.644406] CPU 2: hi: 90, btch: 15 usd: 0 [ 233.644659] CPU 3: hi: 90, btch: 15 usd: 0 [ 233.644915] CPU 4: hi: 90, btch: 15 usd: 0 [ 233.645171] CPU 5: hi: 90, btch: 15 usd: 7 [ 233.645428] CPU 6: hi: 90, btch: 15 usd: 14 [ 233.645685] CPU 7: hi: 90, btch: 15 usd: 8 [ 233.645942] active_anon:4396 inactive_anon:26 isolated_anon:0 [ 233.645943] active_file:26 inactive_file:1033 isolated_file:92 [ 233.645944] unevictable:0 dirty:368 writeback:61 unstable:0 [ 233.645945] free:754 slab_reclaimable:12613 slab_unreclaimable:5104 [ 233.645947] mapped:44 shmem:75 pagetables:328 bounce:0 [ 233.647410] Node 0 DMA free:1064kB min:120kB low:148kB high:180kB active_anon:572kB inactive_anon:0kB active_file:0kB inactive_file:268kB unevictable:0kB isolated(anon):0kB isolated(file):128kB present:15660kB mlocked:0kB dirty:20kB writeback:0kB mapped:48kB shmem:0kB slab_reclaimable:13092kB slab_unreclaimable:216kB kernel_stack:8kB pagetables:20kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:5473 all_unreclaimable? no [ 233.649164] lowmem_reserve[]: 0 236 236 236 [ 233.649582] Node 0 DMA32 free:1920kB min:1904kB low:2380kB high:2856kB active_anon:17012kB inactive_anon:36kB active_file:108kB inactive_file:3916kB unevictable:0kB isolated(anon):0kB isolated(file):288kB present:242400kB mlocked:0kB dirty:152kB writeback:1396kB mapped:128kB shmem:300kB slab_reclaimable:37360kB slab_unreclaimable:20200kB kernel_stack:1120kB pagetables:1292kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:445 all_unreclaimable? no [ 233.651403] lowmem_reserve[]: 0 0 0 0 [ 233.651800] Node 0 DMA: 48*4kB 100*8kB 8*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1120kB [ 233.652777] Node 0 DMA32: 104*4kB 156*8kB 2*16kB 5*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1856kB [ 233.653769] 1363 total pagecache pages [ 233.653983] 0 pages in swap cache [ 233.654175] Swap cache stats: add 0, delete 0, find 0/0 [ 233.654448] Free swap = 0kB [ 233.654625] Total swap = 0kB [ 233.655543] 65520 pages RAM [ 233.655721] 16492 pages reserved [ 233.655917] 1897 pages shared [ 233.656102] 46396 pages non-shared [ 233.656304] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 233.656706] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 233.657105] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 233.657503] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 233.657900] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 233.658294] [ 2541] 0 2541 6083 55 0 0 0 hald-addon-cpuf [ 233.658734] [ 2542] 106 2542 6535 51 4 0 0 hald-addon-acpi [ 233.659159] [ 2632] 0 2632 1494 26 4 0 0 getty [ 233.659546] [ 2633] 0 2633 1494 27 0 0 0 getty [ 233.659932] [ 2634] 0 2634 1494 27 0 0 0 getty [ 233.660332] [ 2635] 0 2635 1494 27 0 0 0 getty [ 233.660718] [ 2636] 0 2636 1494 26 0 0 0 getty [ 233.661110] [ 2638] 0 2638 1494 26 0 0 0 getty [ 233.661510] [ 2692] 0 2692 11425 554 0 0 0 zsh [ 233.661897] [ 2720] 0 2720 3431 63 4 0 0 concurrent-dd.s [ 233.662329] [ 2749] 0 2749 3460 289 5 0 0 dd [ 233.662723] [ 2750] 0 2750 3460 289 2 0 0 dd [ 233.663109] [ 2751] 0 2751 3460 288 2 0 0 dd [ 233.663496] [ 2752] 0 2752 3460 289 6 0 0 dd [ 233.663880] [ 2753] 0 2753 3460 290 7 0 0 dd [ 233.664267] [ 2754] 0 2754 3460 285 0 0 0 dd [ 233.664657] [ 2755] 0 2755 3460 288 5 0 0 dd [ 233.665042] [ 2756] 0 2756 3460 289 5 0 0 dd [ 233.665428] [ 2757] 0 2757 3460 290 5 0 0 dd [ 233.665815] [ 2758] 0 2758 3460 290 5 0 0 dd [ 233.666202] [ 2759] 0 2759 3437 70 4 0 0 plot-written.sh [ 233.666644] [ 2776] 0 2776 2145 63 3 0 0 iostat [ 233.667044] [10007] 0 10007 3437 70 5 0 0 plot-written.sh [ 233.667470] [10008] 0 10008 2675 33 4 0 0 grep [ 233.667864] Out of memory: Kill process 2541 (hald-addon-cpuf) score 1 or sacrifice child [ 233.668288] Killed process 2541 (hald-addon-cpuf) total-vm:24332kB, anon-rss:188kB, file-rss:32kB [ 236.216613] iostat invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 [ 236.217039] iostat cpuset=/ mems_allowed=0 [ 236.217262] Pid: 2776, comm: iostat Not tainted 2.6.37-rc3 #154 [ 236.217553] Call Trace: [ 236.217718] [] ? _raw_spin_unlock+0x2b/0x40 [ 236.218008] [] dump_header+0x83/0x200 [ 236.218277] [] ? trace_hardirqs_on+0xd/0x10 [ 236.218565] [] ? ___ratelimit+0xab/0x150 [ 236.218844] [] ? oom_badness+0x77/0x190 [ 236.219120] [] oom_kill_process+0x8d/0x270 [ 236.219408] [] out_of_memory+0x1d8/0x410 [ 236.219701] [] __alloc_pages_nodemask+0x81b/0x830 [ 236.220015] [] alloc_pages_current+0x99/0x110 [ 236.220312] [] __page_cache_alloc+0x87/0x90 [ 236.220601] [] __do_page_cache_readahead+0x120/0x260 [ 236.220921] [] ? __do_page_cache_readahead+0xad/0x260 [ 236.221243] [] ? lock_release_holdtime+0x3d/0x180 [ 236.221552] [] ra_submit+0x21/0x30 [ 236.221812] [] filemap_fault+0x4dc/0x4f0 [ 236.222090] [] ? __lock_acquire+0x53e/0x1e70 [ 236.222379] [] __do_fault+0x54/0x540 [ 236.222643] [] handle_mm_fault+0x1af/0xc20 [ 236.222929] [] ? do_page_fault+0xf6/0x560 [ 236.223209] [] ? lock_release_holdtime+0x3d/0x180 [ 236.223515] [] do_page_fault+0x16b/0x560 [ 236.223802] [] ? __call_rcu+0xd4/0x1a0 [ 236.224072] [] ? __call_rcu+0xd4/0x1a0 [ 236.224342] [] ? error_sti+0x5/0x6 [ 236.224600] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 236.224913] [] page_fault+0x25/0x30 [ 236.225172] Mem-Info: [ 236.225326] Node 0 DMA per-cpu: [ 236.225544] CPU 0: hi: 0, btch: 1 usd: 0 [ 236.225789] CPU 1: hi: 0, btch: 1 usd: 0 [ 236.226036] CPU 2: hi: 0, btch: 1 usd: 0 [ 236.226282] CPU 3: hi: 0, btch: 1 usd: 0 [ 236.226529] CPU 4: hi: 0, btch: 1 usd: 0 [ 236.226775] CPU 5: hi: 0, btch: 1 usd: 0 [ 236.227032] CPU 6: hi: 0, btch: 1 usd: 0 [ 236.227278] CPU 7: hi: 0, btch: 1 usd: 0 [ 236.227525] Node 0 DMA32 per-cpu: [ 236.227768] CPU 0: hi: 90, btch: 15 usd: 33 [ 236.228015] CPU 1: hi: 90, btch: 15 usd: 24 [ 236.228261] CPU 2: hi: 90, btch: 15 usd: 24 [ 236.228507] CPU 3: hi: 90, btch: 15 usd: 0 [ 236.228754] CPU 4: hi: 90, btch: 15 usd: 82 [ 236.229575] CPU 5: hi: 90, btch: 15 usd: 0 [ 236.229824] CPU 6: hi: 90, btch: 15 usd: 14 [ 236.230072] CPU 7: hi: 90, btch: 15 usd: 20 [ 236.230321] active_anon:4318 inactive_anon:0 isolated_anon:2 [ 236.230322] active_file:188 inactive_file:1707 isolated_file:0 [ 236.230323] unevictable:0 dirty:104 writeback:160 unstable:0 [ 236.230324] free:734 slab_reclaimable:12754 slab_unreclaimable:5107 [ 236.230325] mapped:117 shmem:75 pagetables:292 bounce:0 [ 236.231744] Node 0 DMA free:1044kB min:120kB low:148kB high:180kB active_anon:572kB inactive_anon:0kB active_file:36kB inactive_file:640kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:0kB writeback:40kB mapped:16kB shmem:0kB slab_reclaimable:13144kB slab_unreclaimable:256kB kernel_stack:8kB pagetables:20kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1480 all_unreclaimable? yes [ 236.233447] lowmem_reserve[]: 0 236 236 236 [ 236.233851] Node 0 DMA32 free:1892kB min:1904kB low:2380kB high:2856kB active_anon:16700kB inactive_anon:0kB active_file:716kB inactive_file:6188kB unevictable:0kB isolated(anon):8kB isolated(file):0kB present:242400kB mlocked:0kB dirty:416kB writeback:600kB mapped:452kB shmem:300kB slab_reclaimable:37872kB slab_unreclaimable:20172kB kernel_stack:1096kB pagetables:1148kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:15720 all_unreclaimable? yes [ 236.235637] lowmem_reserve[]: 0 0 0 0 [ 236.236017] Node 0 DMA: 10*4kB 57*8kB 37*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1088kB [ 236.236975] Node 0 DMA32: 74*4kB 185*8kB 0*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1872kB [ 236.237940] 2059 total pagecache pages [ 236.238146] 0 pages in swap cache [ 236.238339] Swap cache stats: add 0, delete 0, find 0/0 [ 236.238601] Free swap = 0kB [ 236.238776] Total swap = 0kB [ 236.239644] 65520 pages RAM [ 236.239819] 16492 pages reserved [ 236.240008] 1330 pages shared [ 236.240187] 46616 pages non-shared [ 236.240383] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 236.240771] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 236.241157] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 236.241542] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 236.241925] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 236.242304] [ 2542] 106 2542 6535 51 4 0 0 hald-addon-acpi [ 236.242720] [ 2632] 0 2632 1494 26 4 0 0 getty [ 236.243100] [ 2633] 0 2633 1494 27 0 0 0 getty [ 236.243482] [ 2634] 0 2634 1494 27 0 0 0 getty [ 236.243875] [ 2635] 0 2635 1494 27 0 0 0 getty [ 236.244259] [ 2636] 0 2636 1494 26 0 0 0 getty [ 236.244643] [ 2638] 0 2638 1494 26 0 0 0 getty [ 236.245029] [ 2692] 0 2692 11425 554 0 0 0 zsh [ 236.245407] [ 2720] 0 2720 3431 63 4 0 0 concurrent-dd.s [ 236.245825] [ 2749] 0 2749 3460 290 4 0 0 dd [ 236.246199] [ 2750] 0 2750 3460 289 4 0 0 dd [ 236.246573] [ 2751] 0 2751 3460 289 7 0 0 dd [ 236.246947] [ 2752] 0 2752 3460 290 1 0 0 dd [ 236.247322] [ 2753] 0 2753 3460 290 3 0 0 dd [ 236.247704] [ 2754] 0 2754 3460 289 7 0 0 dd [ 236.248079] [ 2755] 0 2755 3460 289 1 0 0 dd [ 236.248454] [ 2756] 0 2756 3460 290 2 0 0 dd [ 236.248828] [ 2757] 0 2757 3460 290 1 0 0 dd [ 236.249201] [ 2758] 0 2758 3460 290 7 0 0 dd [ 236.249575] [ 2759] 0 2759 3437 158 4 0 0 plot-written.sh [ 236.249993] [ 2776] 0 2776 2145 76 3 0 0 iostat [ 236.250381] [10015] 0 10015 3437 106 0 0 0 plot-written.sh [ 236.250796] Out of memory: Kill process 2542 (hald-addon-acpi) score 1 or sacrifice child [ 236.251205] Killed process 2542 (hald-addon-acpi) total-vm:26140kB, anon-rss:200kB, file-rss:4kB [ 237.068298] date invoked oom-killer: gfp_mask=0x84d0, order=0, oom_adj=0, oom_score_adj=0 [ 237.068716] date cpuset=/ mems_allowed=0 [ 237.068934] Pid: 10025, comm: date Not tainted 2.6.37-rc3 #154 [ 237.069223] Call Trace: [ 237.069388] [] ? _raw_spin_unlock+0x2b/0x40 [ 237.069679] [] dump_header+0x83/0x200 [ 237.069950] [] ? trace_hardirqs_on+0xd/0x10 [ 237.070240] [] ? ___ratelimit+0xab/0x150 [ 237.070518] [] ? oom_badness+0x77/0x190 [ 237.070793] [] oom_kill_process+0x8d/0x270 [ 237.071079] [] out_of_memory+0x1d8/0x410 [ 237.071361] [] __alloc_pages_nodemask+0x81b/0x830 [ 237.071671] [] alloc_pages_current+0x99/0x110 [ 237.071989] [] pte_alloc_one+0x1b/0x40 [ 237.072264] [] __pte_alloc+0x2f/0xf0 [ 237.072531] [] ? local_clock+0x6f/0x80 [ 237.072804] [] handle_mm_fault+0xa97/0xc20 [ 237.073089] [] ? do_page_fault+0xf6/0x560 [ 237.073372] [] do_page_fault+0x16b/0x560 [ 237.073652] [] ? local_clock+0x6f/0x80 [ 237.073926] [] ? trace_hardirqs_off_caller+0x29/0x150 [ 237.074247] [] ? trace_hardirqs_off+0xd/0x10 [ 237.074539] [] ? local_clock+0x6f/0x80 [ 237.074811] [] ? error_sti+0x5/0x6 [ 237.075070] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 237.075386] [] page_fault+0x25/0x30 [ 237.075649] Mem-Info: [ 237.075802] Node 0 DMA per-cpu: [ 237.076033] CPU 0: hi: 0, btch: 1 usd: 0 [ 237.076283] CPU 1: hi: 0, btch: 1 usd: 0 [ 237.076532] CPU 2: hi: 0, btch: 1 usd: 0 [ 237.076781] CPU 3: hi: 0, btch: 1 usd: 0 [ 237.077029] CPU 4: hi: 0, btch: 1 usd: 0 [ 237.077279] CPU 5: hi: 0, btch: 1 usd: 0 [ 237.077528] CPU 6: hi: 0, btch: 1 usd: 0 [ 237.077777] CPU 7: hi: 0, btch: 1 usd: 0 [ 237.078026] Node 0 DMA32 per-cpu: [ 237.078256] CPU 0: hi: 90, btch: 15 usd: 35 [ 237.078505] CPU 1: hi: 90, btch: 15 usd: 28 [ 237.078753] CPU 2: hi: 90, btch: 15 usd: 14 [ 237.079001] CPU 3: hi: 90, btch: 15 usd: 0 [ 237.079250] CPU 4: hi: 90, btch: 15 usd: 3 [ 237.079498] CPU 5: hi: 90, btch: 15 usd: 0 [ 237.079748] CPU 6: hi: 90, btch: 15 usd: 0 [ 237.080005] CPU 7: hi: 90, btch: 15 usd: 5 [ 237.080261] active_anon:4303 inactive_anon:0 isolated_anon:0 [ 237.080262] active_file:240 inactive_file:1785 isolated_file:0 [ 237.080263] unevictable:0 dirty:122 writeback:0 unstable:0 [ 237.080264] free:748 slab_reclaimable:12788 slab_unreclaimable:5100 [ 237.080265] mapped:250 shmem:75 pagetables:307 bounce:0 [ 237.081685] Node 0 DMA free:1036kB min:120kB low:148kB high:180kB active_anon:552kB inactive_anon:0kB active_file:36kB inactive_file:608kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:0kB writeback:48kB mapped:100kB shmem:0kB slab_reclaimable:13144kB slab_unreclaimable:224kB kernel_stack:24kB pagetables:24kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:982 all_unreclaimable? yes [ 237.083397] lowmem_reserve[]: 0 236 236 236 [ 237.083803] Node 0 DMA32 free:1920kB min:1904kB low:2380kB high:2856kB active_anon:16660kB inactive_anon:0kB active_file:924kB inactive_file:6532kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:496kB writeback:0kB mapped:900kB shmem:300kB slab_reclaimable:38008kB slab_unreclaimable:20176kB kernel_stack:1104kB pagetables:1204kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:13652 all_unreclaimable? yes [ 237.085592] lowmem_reserve[]: 0 0 0 0 [ 237.085979] Node 0 DMA: 2*4kB 48*8kB 44*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1096kB [ 237.086950] Node 0 DMA32: 130*4kB 143*8kB 7*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1808kB [ 237.087933] 2190 total pagecache pages [ 237.088150] 0 pages in swap cache [ 237.088344] Swap cache stats: add 0, delete 0, find 0/0 [ 237.088609] Free swap = 0kB [ 237.088785] Total swap = 0kB [ 237.089664] 65520 pages RAM [ 237.089838] 16492 pages reserved [ 237.090029] 1235 pages shared [ 237.090208] 46759 pages non-shared [ 237.090404] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 237.090793] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 237.091177] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 237.091562] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 237.091947] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 237.092910] [ 2632] 0 2632 1494 26 4 0 0 getty [ 237.093295] [ 2633] 0 2633 1494 27 0 0 0 getty [ 237.093679] [ 2634] 0 2634 1494 27 0 0 0 getty [ 237.094062] [ 2635] 0 2635 1494 27 0 0 0 getty [ 237.094446] [ 2636] 0 2636 1494 26 0 0 0 getty [ 237.094830] [ 2638] 0 2638 1494 26 0 0 0 getty [ 237.095215] [ 2692] 0 2692 11425 554 0 0 0 zsh [ 237.095594] [ 2720] 0 2720 3431 63 4 0 0 concurrent-dd.s [ 237.096018] [ 2749] 0 2749 3460 290 1 0 0 dd [ 237.096393] [ 2750] 0 2750 3460 289 6 0 0 dd [ 237.096768] [ 2751] 0 2751 3460 289 2 0 0 dd [ 237.097141] [ 2752] 0 2752 3460 290 2 0 0 dd [ 237.097516] [ 2753] 0 2753 3460 290 4 0 0 dd [ 237.097890] [ 2754] 0 2754 3460 289 5 0 0 dd [ 237.098264] [ 2755] 0 2755 3460 289 3 0 0 dd [ 237.098639] [ 2756] 0 2756 3460 290 2 0 0 dd [ 237.099013] [ 2757] 0 2757 3460 290 7 0 0 dd [ 237.099387] [ 2758] 0 2758 3460 290 5 0 0 dd [ 237.099762] [ 2759] 0 2759 3437 172 4 0 0 plot-written.sh [ 237.100186] [ 2776] 0 2776 2144 79 5 0 0 iostat [ 237.100574] [10025] 0 10025 3202 120 6 0 0 date [ 237.100954] Out of memory: Kill process 2632 (getty) score 1 or sacrifice child [ 237.101330] Killed process 2632 (getty) total-vm:5976kB, anon-rss:100kB, file-rss:4kB [ 237.225753] getconf invoked oom-killer: gfp_mask=0x84d0, order=0, oom_adj=0, oom_score_adj=0 [ 237.226177] getconf cpuset=/ mems_allowed=0 [ 237.226399] Pid: 10027, comm: getconf Not tainted 2.6.37-rc3 #154 [ 237.226690] Call Trace: [ 237.226850] [] ? _raw_spin_unlock+0x2b/0x40 [ 237.227134] [] dump_header+0x83/0x200 [ 237.227399] [] ? trace_hardirqs_on+0xd/0x10 [ 237.227683] [] ? ___ratelimit+0xab/0x150 [ 237.227956] [] ? oom_badness+0x77/0x190 [ 237.228238] [] oom_kill_process+0x8d/0x270 [ 237.228518] [] out_of_memory+0x1d8/0x410 [ 237.228792] [] __alloc_pages_nodemask+0x81b/0x830 [ 237.229096] [] alloc_pages_current+0x99/0x110 [ 237.229387] [] pte_alloc_one+0x1b/0x40 [ 237.229655] [] __pte_alloc+0x2f/0xf0 [ 237.229917] [] ? local_clock+0x6f/0x80 [ 237.230184] [] handle_mm_fault+0xa97/0xc20 [ 237.230465] [] ? do_page_fault+0xf6/0x560 [ 237.230742] [] do_page_fault+0x16b/0x560 [ 237.231015] [] ? local_clock+0x6f/0x80 [ 237.231282] [] ? trace_hardirqs_off_caller+0x29/0x150 [ 237.231598] [] ? trace_hardirqs_off+0xd/0x10 [ 237.231883] [] ? local_clock+0x6f/0x80 [ 237.232162] [] ? error_sti+0x5/0x6 [ 237.232417] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 237.232727] [] page_fault+0x25/0x30 [ 237.232984] Mem-Info: [ 237.233134] Node 0 DMA per-cpu: [ 237.233351] CPU 0: hi: 0, btch: 1 usd: 0 [ 237.233594] CPU 1: hi: 0, btch: 1 usd: 0 [ 237.233836] CPU 2: hi: 0, btch: 1 usd: 0 [ 237.234080] CPU 3: hi: 0, btch: 1 usd: 0 [ 237.234323] CPU 4: hi: 0, btch: 1 usd: 0 [ 237.234567] CPU 5: hi: 0, btch: 1 usd: 0 [ 237.234809] CPU 6: hi: 0, btch: 1 usd: 0 [ 237.235053] CPU 7: hi: 0, btch: 1 usd: 0 [ 237.235297] Node 0 DMA32 per-cpu: [ 237.235521] CPU 0: hi: 90, btch: 15 usd: 22 [ 237.235761] CPU 1: hi: 90, btch: 15 usd: 75 [ 237.236001] CPU 2: hi: 90, btch: 15 usd: 14 [ 237.236257] CPU 3: hi: 90, btch: 15 usd: 0 [ 237.236500] CPU 4: hi: 90, btch: 15 usd: 19 [ 237.236741] CPU 5: hi: 90, btch: 15 usd: 0 [ 237.236988] CPU 6: hi: 90, btch: 15 usd: 0 [ 237.237231] CPU 7: hi: 90, btch: 15 usd: 0 [ 237.237474] active_anon:4281 inactive_anon:7 isolated_anon:0 [ 237.237475] active_file:251 inactive_file:1703 isolated_file:0 [ 237.237476] unevictable:0 dirty:7 writeback:196 unstable:0 [ 237.237476] free:701 slab_reclaimable:12808 slab_unreclaimable:5119 [ 237.237477] mapped:197 shmem:75 pagetables:277 bounce:0 [ 237.238855] Node 0 DMA free:1036kB min:120kB low:148kB high:180kB active_anon:572kB inactive_anon:0kB active_file:8kB inactive_file:728kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:0kB writeback:0kB mapped:44kB shmem:0kB slab_reclaimable:13148kB slab_unreclaimable:216kB kernel_stack:8kB pagetables:20kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1114 all_unreclaimable? yes [ 237.240511] lowmem_reserve[]: 0 236 236 236 [ 237.240898] Node 0 DMA32 free:1868kB min:1904kB low:2380kB high:2856kB active_anon:16552kB inactive_anon:28kB active_file:996kB inactive_file:6084kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:88kB writeback:816kB mapped:744kB shmem:300kB slab_reclaimable:38084kB slab_unreclaimable:20260kB kernel_stack:1096kB pagetables:1088kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:13373 all_unreclaimable? yes [ 237.242627] lowmem_reserve[]: 0 0 0 0 [ 237.242994] Node 0 DMA: 6*4kB 36*8kB 46*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1048kB [ 237.243934] Node 0 DMA32: 107*4kB 168*8kB 4*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1868kB [ 237.244905] 2126 total pagecache pages [ 237.245109] 0 pages in swap cache [ 237.245297] Swap cache stats: add 0, delete 0, find 0/0 [ 237.245552] Free swap = 0kB [ 237.245723] Total swap = 0kB [ 237.246505] 65520 pages RAM [ 237.246672] 16492 pages reserved [ 237.246856] 1393 pages shared [ 237.247030] 46727 pages non-shared [ 237.247221] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 237.247596] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 237.247969] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 237.248354] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 237.248728] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 237.249102] [ 2633] 0 2633 1494 27 0 0 0 getty [ 237.249479] [ 2634] 0 2634 1494 27 0 0 0 getty [ 237.249856] [ 2635] 0 2635 1494 27 0 0 0 getty [ 237.250232] [ 2636] 0 2636 1494 26 0 0 0 getty [ 237.250608] [ 2638] 0 2638 1494 26 0 0 0 getty [ 237.250983] [ 2692] 0 2692 11425 554 0 0 0 zsh [ 237.251353] [ 2720] 0 2720 3431 63 4 0 0 concurrent-dd.s [ 237.251761] [ 2749] 0 2749 3460 290 6 0 0 dd [ 237.252138] [ 2750] 0 2750 3460 289 6 0 0 dd [ 237.252506] [ 2751] 0 2751 3460 289 7 0 0 dd [ 237.252874] [ 2752] 0 2752 3460 290 2 0 0 dd [ 237.253243] [ 2753] 0 2753 3460 290 3 0 0 dd [ 237.253608] [ 2754] 0 2754 3460 289 5 0 0 dd [ 237.253976] [ 2755] 0 2755 3460 289 7 0 0 dd [ 237.254342] [ 2756] 0 2756 3460 290 6 0 0 dd [ 237.254710] [ 2757] 0 2757 3460 290 1 0 0 dd [ 237.255076] [ 2758] 0 2758 3460 290 1 0 0 dd [ 237.255444] [ 2759] 0 2759 3437 171 4 0 0 plot-written.sh [ 237.255851] [ 2776] 0 2776 2144 79 5 0 0 iostat [ 237.256238] [10026] 0 10026 3437 156 4 0 0 plot-written.sh [ 237.256648] [10027] 0 10027 2135 106 4 0 0 getconf [ 237.257030] Out of memory: Kill process 2633 (getty) score 1 or sacrifice child [ 237.257399] Killed process 2633 (getty) total-vm:5976kB, anon-rss:104kB, file-rss:4kB [ 237.441342] init invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 [ 237.441747] init cpuset=/ mems_allowed=0 [ 237.441957] Pid: 10030, comm: init Not tainted 2.6.37-rc3 #154 [ 237.442236] Call Trace: [ 237.442393] [] ? _raw_spin_unlock+0x2b/0x40 [ 237.442673] [] dump_header+0x83/0x200 [ 237.442936] [] ? trace_hardirqs_on+0xd/0x10 [ 237.443216] [] ? ___ratelimit+0xab/0x150 [ 237.443486] [] ? oom_badness+0x77/0x190 [ 237.443753] [] oom_kill_process+0x8d/0x270 [ 237.444028] [] out_of_memory+0x1d8/0x410 [ 237.444317] [] __alloc_pages_nodemask+0x81b/0x830 [ 237.444618] [] alloc_pages_current+0x99/0x110 [ 237.444904] [] __page_cache_alloc+0x87/0x90 [ 237.445184] [] __do_page_cache_readahead+0x120/0x260 [ 237.445491] [] ? __do_page_cache_readahead+0xad/0x260 [ 237.445803] [] ? local_clock+0x6f/0x80 [ 237.446068] [] ra_submit+0x21/0x30 [ 237.446321] [] ondemand_readahead+0x10e/0x2b0 [ 237.446609] [] page_cache_sync_readahead+0x36/0x50 [ 237.446914] [] filemap_fault+0x444/0x4f0 [ 237.447188] [] ? __lock_acquire+0x53e/0x1e70 [ 237.447474] [] __do_fault+0x54/0x540 [ 237.447734] [] handle_mm_fault+0x1af/0xc20 [ 237.448014] [] ? do_page_fault+0xf6/0x560 [ 237.448344] [] ? lock_release_holdtime+0x3d/0x180 [ 237.448643] [] do_page_fault+0x16b/0x560 [ 237.448914] [] ? local_clock+0x6f/0x80 [ 237.449177] [] ? lock_release_holdtime+0x3d/0x180 [ 237.449476] [] ? error_sti+0x5/0x6 [ 237.449725] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 237.450029] [] page_fault+0x25/0x30 [ 237.450284] Mem-Info: [ 237.450431] Node 0 DMA per-cpu: [ 237.450645] CPU 0: hi: 0, btch: 1 usd: 0 [ 237.450884] CPU 1: hi: 0, btch: 1 usd: 0 [ 237.451125] CPU 2: hi: 0, btch: 1 usd: 0 [ 237.451366] CPU 3: hi: 0, btch: 1 usd: 0 [ 237.451606] CPU 4: hi: 0, btch: 1 usd: 0 [ 237.451846] CPU 5: hi: 0, btch: 1 usd: 0 [ 237.452087] CPU 6: hi: 0, btch: 1 usd: 0 [ 237.452379] CPU 7: hi: 0, btch: 1 usd: 0 [ 237.452623] Node 0 DMA32 per-cpu: [ 237.452846] CPU 0: hi: 90, btch: 15 usd: 23 [ 237.453088] CPU 1: hi: 90, btch: 15 usd: 0 [ 237.453330] CPU 2: hi: 90, btch: 15 usd: 76 [ 237.453573] CPU 3: hi: 90, btch: 15 usd: 0 [ 237.453816] CPU 4: hi: 90, btch: 15 usd: 29 [ 237.454058] CPU 5: hi: 90, btch: 15 usd: 28 [ 237.454301] CPU 6: hi: 90, btch: 15 usd: 0 [ 237.454544] CPU 7: hi: 90, btch: 15 usd: 0 [ 237.454787] active_anon:4247 inactive_anon:15 isolated_anon:0 [ 237.454788] active_file:251 inactive_file:565 isolated_file:0 [ 237.454788] unevictable:0 dirty:21 writeback:73 unstable:0 [ 237.454789] free:742 slab_reclaimable:12825 slab_unreclaimable:5119 [ 237.454790] mapped:197 shmem:75 pagetables:277 bounce:0 [ 237.456183] Node 0 DMA free:1036kB min:120kB low:148kB high:180kB active_anon:572kB inactive_anon:0kB active_file:8kB inactive_file:184kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:96kB writeback:176kB mapped:44kB shmem:0kB slab_reclaimable:13148kB slab_unreclaimable:216kB kernel_stack:8kB pagetables:20kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:288 all_unreclaimable? yes [ 237.457859] lowmem_reserve[]: 0 236 236 236 [ 237.458252] Node 0 DMA32 free:1932kB min:1904kB low:2380kB high:2856kB active_anon:16416kB inactive_anon:60kB active_file:996kB inactive_file:2076kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:0kB writeback:116kB mapped:744kB shmem:300kB slab_reclaimable:38152kB slab_unreclaimable:20260kB kernel_stack:1096kB pagetables:1088kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:4678 all_unreclaimable? yes [ 237.459983] lowmem_reserve[]: 0 0 0 0 [ 237.460384] Node 0 DMA: 2*4kB 38*8kB 46*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1048kB [ 237.461341] Node 0 DMA32: 79*4kB 173*8kB 4*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1860kB [ 237.462306] 1022 total pagecache pages [ 237.462513] 0 pages in swap cache [ 237.462704] Swap cache stats: add 0, delete 0, find 0/0 [ 237.462966] Free swap = 0kB [ 237.463143] Total swap = 0kB [ 237.463993] 65520 pages RAM [ 237.464176] 16492 pages reserved [ 237.464367] 1204 pages shared [ 237.464545] 46922 pages non-shared [ 237.464741] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 237.465128] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 237.465513] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 237.465897] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 237.466281] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 237.466661] [ 2634] 0 2634 1494 27 0 0 0 getty [ 237.467044] [ 2635] 0 2635 1494 27 0 0 0 getty [ 237.467429] [ 2636] 0 2636 1494 26 0 0 0 getty [ 237.467812] [ 2638] 0 2638 1494 26 0 0 0 getty [ 237.468204] [ 2692] 0 2692 11425 554 0 0 0 zsh [ 237.468585] [ 2720] 0 2720 3431 63 4 0 0 concurrent-dd.s [ 237.469003] [ 2749] 0 2749 3460 290 2 0 0 dd [ 237.469378] [ 2750] 0 2750 3460 289 0 0 0 dd [ 237.469752] [ 2751] 0 2751 3460 289 1 0 0 dd [ 237.470126] [ 2752] 0 2752 3460 290 6 0 0 dd [ 237.470500] [ 2753] 0 2753 3460 290 3 0 0 dd [ 237.470874] [ 2754] 0 2754 3460 289 4 0 0 dd [ 237.471250] [ 2755] 0 2755 3460 289 4 0 0 dd [ 237.471625] [ 2756] 0 2756 3460 290 4 0 0 dd [ 237.471998] [ 2757] 0 2757 3460 290 1 0 0 dd [ 237.472381] [ 2758] 0 2758 3460 290 1 0 0 dd [ 237.472755] [ 2759] 0 2759 3437 171 4 0 0 plot-written.sh [ 237.473169] [ 2776] 0 2776 2144 85 6 0 0 iostat [ 237.473558] [10028] 0 10028 3437 156 0 0 0 plot-written.sh [ 237.473974] [10029] 0 10029 2675 107 0 0 0 grep [ 237.474356] [10030] 0 10030 2611 51 5 0 0 init [ 237.474735] Out of memory: Kill process 2634 (getty) score 1 or sacrifice child [ 237.475112] Killed process 2634 (getty) total-vm:5976kB, anon-rss:104kB, file-rss:4kB [ 237.869084] dd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 [ 237.869486] dd cpuset=/ mems_allowed=0 [ 237.869691] Pid: 2750, comm: dd Not tainted 2.6.37-rc3 #154 [ 237.869964] Call Trace: [ 237.870123] [] ? _raw_spin_unlock+0x2b/0x40 [ 237.870406] [] dump_header+0x83/0x200 [ 237.870671] [] ? trace_hardirqs_on+0xd/0x10 [ 237.870952] [] ? ___ratelimit+0xab/0x150 [ 237.871224] [] ? oom_badness+0x77/0x190 [ 237.871492] [] oom_kill_process+0x8d/0x270 [ 237.871769] [] out_of_memory+0x1d8/0x410 [ 237.872042] [] __alloc_pages_nodemask+0x81b/0x830 [ 237.872355] [] alloc_pages_current+0x99/0x110 [ 237.872644] [] __page_cache_alloc+0x87/0x90 [ 237.872927] [] __do_page_cache_readahead+0x120/0x260 [ 237.873236] [] ? __do_page_cache_readahead+0xad/0x260 [ 237.873549] [] ? lock_release_holdtime+0x3d/0x180 [ 237.873848] [] ra_submit+0x21/0x30 [ 237.874100] [] filemap_fault+0x4dc/0x4f0 [ 237.874373] [] ? __lock_acquire+0x53e/0x1e70 [ 237.874656] [] __do_fault+0x54/0x540 [ 237.874916] [] ? __mutex_unlock_slowpath+0xd5/0x170 [ 237.875222] [] ? __mutex_unlock_slowpath+0xd5/0x170 [ 237.875525] [] handle_mm_fault+0x1af/0xc20 [ 237.875801] [] ? do_page_fault+0xf6/0x560 [ 237.876074] [] ? local_clock+0x6f/0x80 [ 237.876341] [] do_page_fault+0x16b/0x560 [ 237.876611] [] ? fsnotify+0x7c/0x2d0 [ 237.876867] [] ? error_sti+0x5/0x6 [ 237.877116] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 237.877419] [] page_fault+0x25/0x30 [ 237.877673] Mem-Info: [ 237.877820] Node 0 DMA per-cpu: [ 237.878032] CPU 0: hi: 0, btch: 1 usd: 0 [ 237.878273] CPU 1: hi: 0, btch: 1 usd: 0 [ 237.878515] CPU 2: hi: 0, btch: 1 usd: 0 [ 237.878756] CPU 3: hi: 0, btch: 1 usd: 0 [ 237.878998] CPU 4: hi: 0, btch: 1 usd: 0 [ 237.879819] CPU 5: hi: 0, btch: 1 usd: 0 [ 237.880067] CPU 6: hi: 0, btch: 1 usd: 0 [ 237.880313] CPU 7: hi: 0, btch: 1 usd: 0 [ 237.880567] Node 0 DMA32 per-cpu: [ 237.880801] CPU 0: hi: 90, btch: 15 usd: 8 [ 237.881053] CPU 1: hi: 90, btch: 15 usd: 14 [ 237.881307] CPU 2: hi: 90, btch: 15 usd: 0 [ 237.881558] CPU 3: hi: 90, btch: 15 usd: 36 [ 237.881809] CPU 4: hi: 90, btch: 15 usd: 1 [ 237.882063] CPU 5: hi: 90, btch: 15 usd: 14 [ 237.882312] CPU 6: hi: 90, btch: 15 usd: 77 [ 237.882563] CPU 7: hi: 90, btch: 15 usd: 0 [ 237.882816] active_anon:4255 inactive_anon:40 isolated_anon:0 [ 237.882817] active_file:170 inactive_file:670 isolated_file:0 [ 237.882818] unevictable:0 dirty:0 writeback:184 unstable:0 [ 237.882819] free:1245 slab_reclaimable:12817 slab_unreclaimable:5088 [ 237.882820] mapped:247 shmem:75 pagetables:302 bounce:0 [ 237.884267] Node 0 DMA free:1320kB min:120kB low:148kB high:180kB active_anon:572kB inactive_anon:0kB active_file:0kB inactive_file:164kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:0kB writeback:20kB mapped:44kB shmem:0kB slab_reclaimable:13184kB slab_unreclaimable:228kB kernel_stack:8kB pagetables:20kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no [ 237.885936] lowmem_reserve[]: 0 236 236 236 [ 237.886326] Node 0 DMA32 free:2604kB min:1904kB low:2380kB high:2856kB active_anon:16448kB inactive_anon:160kB active_file:728kB inactive_file:2228kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:0kB writeback:716kB mapped:944kB shmem:300kB slab_reclaimable:38084kB slab_unreclaimable:20124kB kernel_stack:1096kB pagetables:1188kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no [ 237.888061] lowmem_reserve[]: 0 0 0 0 [ 237.888438] Node 0 DMA: 14*4kB 53*8kB 38*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1120kB [ 237.889428] Node 0 DMA32: 217*4kB 178*8kB 8*16kB 2*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2484kB [ 237.890375] 1886 total pagecache pages [ 237.890579] 0 pages in swap cache [ 237.890765] Swap cache stats: add 0, delete 0, find 0/0 [ 237.891024] Free swap = 0kB [ 237.891246] Total swap = 0kB [ 237.891994] 65520 pages RAM [ 237.892162] 16492 pages reserved [ 237.892351] 2506 pages shared [ 237.892526] 45810 pages non-shared [ 237.892745] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 237.893157] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 237.893534] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 237.893909] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 237.894284] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 237.894659] [ 2635] 0 2635 1494 27 0 0 0 getty [ 237.895034] [ 2636] 0 2636 1494 26 0 0 0 getty [ 237.895407] [ 2638] 0 2638 1494 26 0 0 0 getty [ 237.895783] [ 2692] 0 2692 11425 554 0 0 0 zsh [ 237.896153] [ 2720] 0 2720 3431 63 4 0 0 concurrent-dd.s [ 237.896571] [ 2749] 0 2749 3460 288 6 0 0 dd [ 237.896939] [ 2750] 0 2750 3460 286 2 0 0 dd [ 237.897305] [ 2751] 0 2751 3460 286 5 0 0 dd [ 237.897669] [ 2752] 0 2752 3460 287 6 0 0 dd [ 237.898036] [ 2753] 0 2753 3460 288 3 0 0 dd [ 237.898401] [ 2754] 0 2754 3460 287 4 0 0 dd [ 237.898767] [ 2755] 0 2755 3460 286 6 0 0 dd [ 237.899134] [ 2756] 0 2756 3460 287 4 0 0 dd [ 237.899500] [ 2757] 0 2757 3460 288 1 0 0 dd [ 237.899864] [ 2758] 0 2758 3460 287 1 0 0 dd [ 237.900230] [ 2759] 0 2759 3437 160 4 0 0 plot-written.sh [ 237.900641] [ 2776] 0 2776 2144 71 6 0 0 iostat [ 237.901021] [10030] 0 10030 1493 106 0 0 0 getty [ 237.901397] [10031] 0 10031 3437 139 5 0 0 plot-written.sh [ 237.901804] [10033] 0 10033 2137 101 3 0 0 wc [ 237.902170] [10034] 0 10034 1493 106 5 0 0 getty [ 237.902543] Out of memory: Kill process 2635 (getty) score 1 or sacrifice child [ 237.902912] Killed process 2635 (getty) total-vm:5976kB, anon-rss:104kB, file-rss:4kB [ 238.261109] wc invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 [ 238.261533] wc cpuset=/ mems_allowed=0 [ 238.261751] Pid: 10033, comm: wc Not tainted 2.6.37-rc3 #154 [ 238.262040] Call Trace: [ 238.262208] [] ? _raw_spin_unlock+0x2b/0x40 [ 238.262505] [] dump_header+0x83/0x200 [ 238.262782] [] ? trace_hardirqs_on+0xd/0x10 [ 238.263077] [] ? ___ratelimit+0xab/0x150 [ 238.263363] [] ? oom_badness+0x77/0x190 [ 238.263645] [] oom_kill_process+0x8d/0x270 [ 238.263932] [] out_of_memory+0x1d8/0x410 [ 238.264214] [] __alloc_pages_nodemask+0x81b/0x830 [ 238.264540] [] alloc_pages_current+0x99/0x110 [ 238.264843] [] __page_cache_alloc+0x87/0x90 [ 238.265134] [] __do_page_cache_readahead+0x120/0x260 [ 238.265452] [] ? __do_page_cache_readahead+0xad/0x260 [ 238.265769] [] ? lock_release_holdtime+0x3d/0x180 [ 238.266073] [] ra_submit+0x21/0x30 [ 238.266328] [] filemap_fault+0x4dc/0x4f0 [ 238.266602] [] ? __lock_acquire+0x53e/0x1e70 [ 238.266890] [] __do_fault+0x54/0x540 [ 238.267151] [] handle_mm_fault+0x1af/0xc20 [ 238.267432] [] ? do_page_fault+0xf6/0x560 [ 238.267709] [] ? lock_release_holdtime+0x3d/0x180 [ 238.268012] [] do_page_fault+0x16b/0x560 [ 238.268286] [] ? error_sti+0x5/0x6 [ 238.268556] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 238.268874] [] page_fault+0x25/0x30 [ 238.269132] Mem-Info: [ 238.269284] Node 0 DMA per-cpu: [ 238.269504] CPU 0: hi: 0, btch: 1 usd: 0 [ 238.269748] CPU 1: hi: 0, btch: 1 usd: 0 [ 238.269992] CPU 2: hi: 0, btch: 1 usd: 0 [ 238.270235] CPU 3: hi: 0, btch: 1 usd: 0 [ 238.270478] CPU 4: hi: 0, btch: 1 usd: 0 [ 238.270721] CPU 5: hi: 0, btch: 1 usd: 0 [ 238.270962] CPU 6: hi: 0, btch: 1 usd: 0 [ 238.271203] CPU 7: hi: 0, btch: 1 usd: 0 [ 238.271444] Node 0 DMA32 per-cpu: [ 238.271667] CPU 0: hi: 90, btch: 15 usd: 19 [ 238.271909] CPU 1: hi: 90, btch: 15 usd: 76 [ 238.272152] CPU 2: hi: 90, btch: 15 usd: 14 [ 238.272396] CPU 3: hi: 90, btch: 15 usd: 13 [ 238.272649] CPU 4: hi: 90, btch: 15 usd: 0 [ 238.272900] CPU 5: hi: 90, btch: 15 usd: 0 [ 238.273145] CPU 6: hi: 90, btch: 15 usd: 6 [ 238.273389] CPU 7: hi: 90, btch: 15 usd: 13 [ 238.273634] active_anon:4240 inactive_anon:21 isolated_anon:0 [ 238.273635] active_file:200 inactive_file:1735 isolated_file:0 [ 238.273635] unevictable:0 dirty:43 writeback:303 unstable:0 [ 238.273636] free:726 slab_reclaimable:12860 slab_unreclaimable:5111 [ 238.273636] mapped:181 shmem:75 pagetables:276 bounce:0 [ 238.275034] Node 0 DMA free:1052kB min:120kB low:148kB high:180kB active_anon:568kB inactive_anon:0kB active_file:24kB inactive_file:540kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:0kB writeback:224kB mapped:12kB shmem:0kB slab_reclaimable:13168kB slab_unreclaimable:312kB kernel_stack:8kB pagetables:24kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:990 all_unreclaimable? yes [ 238.276718] lowmem_reserve[]: 0 236 236 236 [ 238.277129] Node 0 DMA32 free:1852kB min:1904kB low:2380kB high:2856kB active_anon:16392kB inactive_anon:84kB active_file:776kB inactive_file:6400kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:248kB writeback:988kB mapped:712kB shmem:300kB slab_reclaimable:38272kB slab_unreclaimable:20132kB kernel_stack:1088kB pagetables:1080kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:15241 all_unreclaimable? yes [ 238.278917] lowmem_reserve[]: 0 0 0 0 [ 238.279307] Node 0 DMA: 13*4kB 45*8kB 43*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1100kB [ 238.280275] Node 0 DMA32: 117*4kB 151*8kB 9*16kB 2*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1884kB [ 238.281262] 1896 total pagecache pages [ 238.281474] 0 pages in swap cache [ 238.281670] Swap cache stats: add 0, delete 0, find 0/0 [ 238.281938] Free swap = 0kB [ 238.282119] Total swap = 0kB [ 238.283135] 65520 pages RAM [ 238.283314] 16492 pages reserved [ 238.283507] 1889 pages shared [ 238.283693] 46297 pages non-shared [ 238.284473] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 238.284883] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 238.285276] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 238.285670] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 238.286067] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 238.286460] [ 2636] 0 2636 1494 26 0 0 0 getty [ 238.286854] [ 2638] 0 2638 1494 26 0 0 0 getty [ 238.287250] [ 2692] 0 2692 11425 554 0 0 0 zsh [ 238.287636] [ 2720] 0 2720 3431 63 4 0 0 concurrent-dd.s [ 238.288064] [ 2749] 0 2749 3460 290 1 0 0 dd [ 238.288536] [ 2750] 0 2750 3460 289 5 0 0 dd [ 238.288921] [ 2751] 0 2751 3460 289 2 0 0 dd [ 238.289305] [ 2752] 0 2752 3460 290 7 0 0 dd [ 238.289689] [ 2753] 0 2753 3460 290 7 0 0 dd [ 238.290074] [ 2754] 0 2754 3460 289 6 0 0 dd [ 238.290458] [ 2755] 0 2755 3460 289 1 0 0 dd [ 238.290840] [ 2756] 0 2756 3460 290 6 0 0 dd [ 238.291224] [ 2757] 0 2757 3460 290 1 0 0 dd [ 238.291612] [ 2758] 0 2758 3460 290 2 0 0 dd [ 238.291997] [ 2759] 0 2759 3437 160 4 0 0 plot-written.sh [ 238.292425] [ 2776] 0 2776 2144 70 6 0 0 iostat [ 238.292833] [10030] 0 10030 1493 108 6 0 0 getty [ 238.293275] [10031] 0 10031 3437 139 5 0 0 plot-written.sh [ 238.293697] [10033] 0 10033 2265 114 4 0 0 wc [ 238.294079] [10034] 0 10034 1493 110 0 0 0 getty [ 238.294474] Out of memory: Kill process 2636 (getty) score 1 or sacrifice child [ 238.294858] Killed process 2636 (getty) total-vm:5976kB, anon-rss:100kB, file-rss:4kB [ 239.413479] date invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 [ 239.413907] date cpuset=/ mems_allowed=0 [ 239.414130] Pid: 10040, comm: date Not tainted 2.6.37-rc3 #154 [ 239.414426] Call Trace: [ 239.414593] [] ? _raw_spin_unlock+0x2b/0x40 [ 239.414890] [] dump_header+0x83/0x200 [ 239.415166] [] ? trace_hardirqs_on+0xd/0x10 [ 239.415466] [] ? ___ratelimit+0xab/0x150 [ 239.415755] [] ? oom_badness+0x77/0x190 [ 239.416038] [] oom_kill_process+0x8d/0x270 [ 239.416331] [] out_of_memory+0x1d8/0x410 [ 239.416620] [] __alloc_pages_nodemask+0x81b/0x830 [ 239.416941] [] alloc_pages_current+0x99/0x110 [ 239.417254] [] __page_cache_alloc+0x87/0x90 [ 239.417553] [] __do_page_cache_readahead+0x120/0x260 [ 239.417878] [] ? __do_page_cache_readahead+0xad/0x260 [ 239.418210] [] ? lock_release_holdtime+0x3d/0x180 [ 239.418528] [] ra_submit+0x21/0x30 [ 239.418849] [] filemap_fault+0x4dc/0x4f0 [ 239.419137] [] ? __lock_acquire+0x53e/0x1e70 [ 239.419440] [] __do_fault+0x54/0x540 [ 239.419713] [] ? local_clock+0x6f/0x80 [ 239.419994] [] handle_mm_fault+0x1af/0xc20 [ 239.420288] [] ? do_page_fault+0xf6/0x560 [ 239.420580] [] do_page_fault+0x16b/0x560 [ 239.420868] [] ? lock_release_holdtime+0x3d/0x180 [ 239.421201] [] ? error_sti+0x5/0x6 [ 239.421468] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 239.421791] [] page_fault+0x25/0x30 [ 239.422063] Mem-Info: [ 239.422222] Node 0 DMA per-cpu: [ 239.422451] CPU 0: hi: 0, btch: 1 usd: 0 [ 239.422706] CPU 1: hi: 0, btch: 1 usd: 0 [ 239.422962] CPU 2: hi: 0, btch: 1 usd: 0 [ 239.423218] CPU 3: hi: 0, btch: 1 usd: 0 [ 239.423473] CPU 4: hi: 0, btch: 1 usd: 0 [ 239.423729] CPU 5: hi: 0, btch: 1 usd: 0 [ 239.423985] CPU 6: hi: 0, btch: 1 usd: 0 [ 239.424239] CPU 7: hi: 0, btch: 1 usd: 0 [ 239.424495] Node 0 DMA32 per-cpu: [ 239.424731] CPU 0: hi: 90, btch: 15 usd: 10 [ 239.424988] CPU 1: hi: 90, btch: 15 usd: 7 [ 239.425250] CPU 2: hi: 90, btch: 15 usd: 0 [ 239.425503] CPU 3: hi: 90, btch: 15 usd: 0 [ 239.425757] CPU 4: hi: 90, btch: 15 usd: 14 [ 239.426013] CPU 5: hi: 90, btch: 15 usd: 0 [ 239.426270] CPU 6: hi: 90, btch: 15 usd: 0 [ 239.426524] CPU 7: hi: 90, btch: 15 usd: 0 [ 239.426778] active_anon:4253 inactive_anon:0 isolated_anon:0 [ 239.426779] active_file:192 inactive_file:1605 isolated_file:0 [ 239.426780] unevictable:0 dirty:0 writeback:465 unstable:0 [ 239.426782] free:749 slab_reclaimable:12916 slab_unreclaimable:5130 [ 239.426783] mapped:199 shmem:75 pagetables:284 bounce:0 [ 239.428281] Node 0 DMA free:1044kB min:120kB low:148kB high:180kB active_anon:564kB inactive_anon:0kB active_file:12kB inactive_file:600kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:0kB writeback:0kB mapped:16kB shmem:0kB slab_reclaimable:13192kB slab_unreclaimable:324kB kernel_stack:8kB pagetables:24kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1083 all_unreclaimable? yes [ 239.430024] lowmem_reserve[]: 0 236 236 236 [ 239.430441] Node 0 DMA32 free:1952kB min:1904kB low:2380kB high:2856kB active_anon:16448kB inactive_anon:0kB active_file:756kB inactive_file:5820kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:0kB writeback:1888kB mapped:780kB shmem:300kB slab_reclaimable:38472kB slab_unreclaimable:20196kB kernel_stack:1096kB pagetables:1112kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:14500 all_unreclaimable? yes [ 239.432254] lowmem_reserve[]: 0 0 0 0 [ 239.432648] Node 0 DMA: 2*4kB 39*8kB 45*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1040kB [ 239.433651] Node 0 DMA32: 93*4kB 160*8kB 5*16kB 3*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1828kB [ 239.434660] 1995 total pagecache pages [ 239.434874] 0 pages in swap cache [ 239.435073] Swap cache stats: add 0, delete 0, find 0/0 [ 239.435344] Free swap = 0kB [ 239.435525] Total swap = 0kB [ 239.436391] 65520 pages RAM [ 239.436571] 16492 pages reserved [ 239.436764] 3000 pages shared [ 239.436949] 45889 pages non-shared [ 239.437203] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 239.437607] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 239.438003] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 239.438395] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 239.438790] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 239.439182] [ 2638] 0 2638 1494 26 0 0 0 getty [ 239.439577] [ 2692] 0 2692 11425 554 0 0 0 zsh [ 239.439964] [ 2720] 0 2720 3431 63 4 0 0 concurrent-dd.s [ 239.440392] [ 2749] 0 2749 3460 290 6 0 0 dd [ 239.440777] [ 2750] 0 2750 3460 289 1 0 0 dd [ 239.441222] [ 2751] 0 2751 3460 289 2 0 0 dd [ 239.441609] [ 2752] 0 2752 3460 290 2 0 0 dd [ 239.441991] [ 2753] 0 2753 3460 290 3 0 0 dd [ 239.442370] [ 2754] 0 2754 3460 289 4 0 0 dd [ 239.442752] [ 2755] 0 2755 3460 289 1 0 0 dd [ 239.443137] [ 2756] 0 2756 3460 290 0 0 0 dd [ 239.443518] [ 2757] 0 2757 3460 290 2 0 0 dd [ 239.443896] [ 2758] 0 2758 3460 290 2 0 0 dd [ 239.444267] [ 2759] 0 2759 3437 134 4 0 0 plot-written.sh [ 239.444696] [ 2776] 0 2776 2145 79 5 0 0 iostat [ 239.445107] [10030] 0 10030 1494 103 1 0 0 getty [ 239.446067] [10034] 0 10034 1494 103 0 0 0 getty [ 239.446460] [10035] 0 10035 1493 95 2 0 0 getty [ 239.446855] [10036] 0 10036 1493 95 5 0 0 getty [ 239.447250] [10038] 0 10038 1493 96 4 0 0 getty [ 239.447647] [10040] 0 10040 1485 37 4 0 0 date [ 239.448036] Out of memory: Kill process 2638 (getty) score 1 or sacrifice child [ 239.448417] Killed process 2638 (getty) total-vm:5976kB, anon-rss:100kB, file-rss:4kB [ 239.956444] getty invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 [ 239.956874] getty cpuset=/ mems_allowed=0 [ 239.957096] Pid: 10038, comm: getty Not tainted 2.6.37-rc3 #154 [ 239.957415] Call Trace: [ 239.957579] [] ? _raw_spin_unlock+0x2b/0x40 [ 239.957881] [] dump_header+0x83/0x200 [ 239.958158] [] ? trace_hardirqs_on+0xd/0x10 [ 239.958453] [] ? ___ratelimit+0xab/0x150 [ 239.958737] [] ? oom_badness+0x77/0x190 [ 239.959019] [] oom_kill_process+0x8d/0x270 [ 239.959309] [] out_of_memory+0x1d8/0x410 [ 239.959586] [] __alloc_pages_nodemask+0x81b/0x830 [ 239.959902] [] alloc_pages_current+0x99/0x110 [ 239.960203] [] __page_cache_alloc+0x87/0x90 [ 239.960490] [] __do_page_cache_readahead+0x120/0x260 [ 239.960815] [] ? __do_page_cache_readahead+0xad/0x260 [ 239.961141] [] ? lock_release_holdtime+0x3d/0x180 [ 239.961462] [] ra_submit+0x21/0x30 [ 239.961721] [] filemap_fault+0x4dc/0x4f0 [ 239.962010] [] ? __lock_acquire+0x53e/0x1e70 [ 239.962310] [] ? find_get_page+0x0/0x110 [ 239.962590] [] __do_fault+0x54/0x540 [ 239.962865] [] ? restore_args+0x0/0x30 [ 239.963138] [] handle_mm_fault+0x1af/0xc20 [ 239.963432] [] ? trace_hardirqs_on_thunk+0x3a/0x3f [ 239.963753] [] ? do_softirq+0x97/0xe0 [ 239.964030] [] ? restore_args+0x0/0x30 [ 239.964312] [] do_page_fault+0x16b/0x560 [ 239.964599] [] ? error_sti+0x5/0x6 [ 239.964866] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 239.965190] [] page_fault+0x25/0x30 [ 239.965469] Mem-Info: [ 239.965623] Node 0 DMA per-cpu: [ 239.965858] CPU 0: hi: 0, btch: 1 usd: 0 [ 239.966114] CPU 1: hi: 0, btch: 1 usd: 0 [ 239.966363] CPU 2: hi: 0, btch: 1 usd: 0 [ 239.966621] CPU 3: hi: 0, btch: 1 usd: 0 [ 239.966877] CPU 4: hi: 0, btch: 1 usd: 0 [ 239.967133] CPU 5: hi: 0, btch: 1 usd: 0 [ 239.967389] CPU 6: hi: 0, btch: 1 usd: 0 [ 239.967638] CPU 7: hi: 0, btch: 1 usd: 0 [ 239.967896] Node 0 DMA32 per-cpu: [ 239.968134] CPU 0: hi: 90, btch: 15 usd: 0 [ 239.968392] CPU 1: hi: 90, btch: 15 usd: 0 [ 239.968649] CPU 2: hi: 90, btch: 15 usd: 0 [ 239.968899] CPU 3: hi: 90, btch: 15 usd: 14 [ 239.969157] CPU 4: hi: 90, btch: 15 usd: 0 [ 239.969414] CPU 5: hi: 90, btch: 15 usd: 0 [ 239.969673] CPU 6: hi: 90, btch: 15 usd: 0 [ 239.969929] CPU 7: hi: 90, btch: 15 usd: 0 [ 239.970188] active_anon:4249 inactive_anon:13 isolated_anon:0 [ 239.970189] active_file:192 inactive_file:1679 isolated_file:32 [ 239.970190] unevictable:0 dirty:38 writeback:353 unstable:0 [ 239.970191] free:700 slab_reclaimable:12933 slab_unreclaimable:5138 [ 239.970192] mapped:174 shmem:75 pagetables:309 bounce:0 [ 239.971636] Node 0 DMA free:1044kB min:120kB low:148kB high:180kB active_anon:564kB inactive_anon:0kB active_file:12kB inactive_file:520kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:52kB writeback:24kB mapped:16kB shmem:0kB slab_reclaimable:13192kB slab_unreclaimable:288kB kernel_stack:8kB pagetables:24kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:812 all_unreclaimable? yes [ 239.973367] lowmem_reserve[]: 0 236 236 236 [ 239.973784] Node 0 DMA32 free:1872kB min:1904kB low:2380kB high:2856kB active_anon:16432kB inactive_anon:52kB active_file:756kB inactive_file:6196kB unevictable:0kB isolated(anon):0kB isolated(file):128kB present:242400kB mlocked:0kB dirty:100kB writeback:1388kB mapped:680kB shmem:300kB slab_reclaimable:38540kB slab_unreclaimable:20264kB kernel_stack:1096kB pagetables:1212kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:16855 all_unreclaimable? yes [ 239.975606] lowmem_reserve[]: 0 0 0 0 [ 239.976013] Node 0 DMA: 5*4kB 74*8kB 26*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1028kB [ 239.977021] Node 0 DMA32: 110*4kB 157*8kB 7*16kB 2*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1872kB [ 239.978059] 2132 total pagecache pages [ 239.978277] 0 pages in swap cache [ 239.978471] Swap cache stats: add 0, delete 0, find 0/0 [ 239.978745] Free swap = 0kB [ 239.978929] Total swap = 0kB [ 239.979822] 65520 pages RAM [ 239.979997] 16492 pages reserved [ 239.980195] 3174 pages shared [ 239.980381] 45788 pages non-shared [ 239.980577] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 239.980972] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 239.981372] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 239.981765] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 239.982158] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 239.982547] [ 2692] 0 2692 11425 554 0 0 0 zsh [ 239.982933] [ 2720] 0 2720 3431 63 4 0 0 concurrent-dd.s [ 239.983359] [ 2749] 0 2749 3460 290 7 0 0 dd [ 239.983741] [ 2750] 0 2750 3460 289 5 0 0 dd [ 239.984123] [ 2751] 0 2751 3460 289 5 0 0 dd [ 239.984505] [ 2752] 0 2752 3460 290 3 0 0 dd [ 239.984887] [ 2753] 0 2753 3460 290 1 0 0 dd [ 239.985270] [ 2754] 0 2754 3460 289 7 0 0 dd [ 239.985660] [ 2755] 0 2755 3460 289 5 0 0 dd [ 239.986042] [ 2756] 0 2756 3460 290 4 0 0 dd [ 239.986425] [ 2757] 0 2757 3460 290 2 0 0 dd [ 239.986807] [ 2758] 0 2758 3460 290 2 0 0 dd [ 239.987189] [ 2759] 0 2759 3437 134 4 0 0 plot-written.sh [ 239.987614] [ 2776] 0 2776 2145 79 5 0 0 iostat [ 239.988013] [10030] 0 10030 1494 103 1 0 0 getty [ 239.988404] [10034] 0 10034 1494 103 0 0 0 getty [ 239.988796] [10035] 0 10035 1622 106 0 0 0 getty [ 239.989189] [10036] 0 10036 1494 114 5 0 0 getty [ 239.989589] [10038] 0 10038 1621 106 0 0 0 getty [ 239.989980] [10040] 0 10040 3202 115 2 0 0 date [ 239.990368] [10041] 0 10041 1493 104 7 0 0 getty [ 239.990759] Out of memory: Kill process 2692 (zsh) score 1 or sacrifice child [ 239.991107] Killed process 2720 (concurrent-dd.s) total-vm:13724kB, anon-rss:248kB, file-rss:4kB [ 240.414356] zsh invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 [ 240.414762] zsh cpuset=/ mems_allowed=0 [ 240.414971] Pid: 2692, comm: zsh Not tainted 2.6.37-rc3 #154 [ 240.415247] Call Trace: [ 240.415409] [] ? _raw_spin_unlock+0x2b/0x40 [ 240.415697] [] dump_header+0x83/0x200 [ 240.415959] [] ? trace_hardirqs_on+0xd/0x10 [ 240.416240] [] ? ___ratelimit+0xab/0x150 [ 240.416511] [] ? oom_badness+0x77/0x190 [ 240.416782] [] oom_kill_process+0x8d/0x270 [ 240.417058] [] out_of_memory+0x1d8/0x410 [ 240.417332] [] __alloc_pages_nodemask+0x81b/0x830 [ 240.417652] [] alloc_pages_current+0x99/0x110 [ 240.417942] [] __page_cache_alloc+0x87/0x90 [ 240.418224] [] __do_page_cache_readahead+0x120/0x260 [ 240.418534] [] ? __do_page_cache_readahead+0xad/0x260 [ 240.418847] [] ? lock_release_holdtime+0x3d/0x180 [ 240.419148] [] ra_submit+0x21/0x30 [ 240.419401] [] filemap_fault+0x4dc/0x4f0 [ 240.419672] [] ? __lock_acquire+0x53e/0x1e70 [ 240.419957] [] __do_fault+0x54/0x540 [ 240.420217] [] handle_mm_fault+0x1af/0xc20 [ 240.420496] [] ? do_page_fault+0xf6/0x560 [ 240.420769] [] ? lock_release_holdtime+0x3d/0x180 [ 240.421067] [] do_page_fault+0x16b/0x560 [ 240.421337] [] ? do_wait+0x1f7/0x250 [ 240.421603] [] ? do_vfs_ioctl+0x9d/0x5a0 [ 240.422449] [] ? error_sti+0x5/0x6 [ 240.422703] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 240.423011] [] page_fault+0x25/0x30 [ 240.423268] Mem-Info: [ 240.423418] Node 0 DMA per-cpu: [ 240.423634] CPU 0: hi: 0, btch: 1 usd: 0 [ 240.423876] CPU 1: hi: 0, btch: 1 usd: 0 [ 240.424119] CPU 2: hi: 0, btch: 1 usd: 0 [ 240.424363] CPU 3: hi: 0, btch: 1 usd: 0 [ 240.424606] CPU 4: hi: 0, btch: 1 usd: 0 [ 240.424849] CPU 5: hi: 0, btch: 1 usd: 0 [ 240.425091] CPU 6: hi: 0, btch: 1 usd: 0 [ 240.425335] CPU 7: hi: 0, btch: 1 usd: 0 [ 240.425585] Node 0 DMA32 per-cpu: [ 240.425807] CPU 0: hi: 90, btch: 15 usd: 0 [ 240.426051] CPU 1: hi: 90, btch: 15 usd: 0 [ 240.426293] CPU 2: hi: 90, btch: 15 usd: 14 [ 240.426536] CPU 3: hi: 90, btch: 15 usd: 0 [ 240.426778] CPU 4: hi: 90, btch: 15 usd: 87 [ 240.427021] CPU 5: hi: 90, btch: 15 usd: 0 [ 240.427265] CPU 6: hi: 90, btch: 15 usd: 88 [ 240.427508] CPU 7: hi: 90, btch: 15 usd: 0 [ 240.427752] active_anon:4218 inactive_anon:0 isolated_anon:0 [ 240.427753] active_file:84 inactive_file:662 isolated_file:0 [ 240.427753] unevictable:0 dirty:0 writeback:484 unstable:0 [ 240.427754] free:1812 slab_reclaimable:12954 slab_unreclaimable:5122 [ 240.427754] mapped:86 shmem:75 pagetables:273 bounce:0 [ 240.429138] Node 0 DMA free:1072kB min:120kB low:148kB high:180kB active_anon:584kB inactive_anon:0kB active_file:20kB inactive_file:524kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:0kB writeback:300kB mapped:8kB shmem:0kB slab_reclaimable:13196kB slab_unreclaimable:324kB kernel_stack:8kB pagetables:32kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:198 all_unreclaimable? no [ 240.430797] lowmem_reserve[]: 0 236 236 236 [ 240.431185] Node 0 DMA32 free:6176kB min:1904kB low:2380kB high:2856kB active_anon:16288kB inactive_anon:0kB active_file:316kB inactive_file:2124kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:0kB writeback:1636kB mapped:336kB shmem:300kB slab_reclaimable:38620kB slab_unreclaimable:20164kB kernel_stack:1096kB pagetables:1060kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1127 all_unreclaimable? no [ 240.432923] lowmem_reserve[]: 0 0 0 0 [ 240.433292] Node 0 DMA: 19*4kB 72*8kB 27*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1084kB [ 240.434290] Node 0 DMA32: 155*4kB 226*8kB 8*16kB 2*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2620kB [ 240.435236] 989 total pagecache pages [ 240.435436] 0 pages in swap cache [ 240.435624] Swap cache stats: add 0, delete 0, find 0/0 [ 240.435881] Free swap = 0kB [ 240.436053] Total swap = 0kB [ 240.436840] 65520 pages RAM [ 240.437011] 16492 pages reserved [ 240.437193] 1663 pages shared [ 240.437368] 46900 pages non-shared [ 240.437567] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 240.437948] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 240.438324] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 240.438700] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 240.439073] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 240.439442] [ 2692] 0 2692 11425 564 3 0 0 zsh [ 240.439811] [ 2749] 0 2749 3460 288 4 0 0 dd [ 240.440178] [ 2750] 0 2750 3460 287 6 0 0 dd [ 240.440543] [ 2751] 0 2751 3460 287 1 0 0 dd [ 240.440908] [ 2752] 0 2752 3460 288 5 0 0 dd [ 240.441274] [ 2753] 0 2753 3460 288 1 0 0 dd [ 240.441646] [ 2754] 0 2754 3460 287 6 0 0 dd [ 240.442011] [ 2755] 0 2755 3460 287 6 0 0 dd [ 240.442376] [ 2756] 0 2756 3460 288 5 0 0 dd [ 240.442741] [ 2757] 0 2757 3460 288 3 0 0 dd [ 240.443105] [ 2758] 0 2758 3460 288 6 0 0 dd [ 240.443470] [ 2759] 0 2759 3437 117 4 0 0 plot-written.sh [ 240.443878] [ 2776] 0 2776 2145 55 4 0 0 iostat [ 240.444256] [10030] 0 10030 1494 46 1 0 0 getty [ 240.444631] [10034] 0 10034 1494 46 0 0 0 getty [ 240.445005] [10035] 0 10035 1494 56 4 0 0 getty [ 240.445380] [10036] 0 10036 1494 55 5 0 0 getty [ 240.445762] [10038] 0 10038 1494 57 1 0 0 getty [ 240.446136] [10040] 0 10040 3330 70 5 0 0 date [ 240.446505] [10041] 0 10041 1493 60 0 0 0 getty [ 240.446874] Out of memory: Kill process 2692 (zsh) score 1 or sacrifice child [ 240.447202] Killed process 2692 (zsh) total-vm:45700kB, anon-rss:2212kB, file-rss:44kB [ 247.945070] date invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 [ 247.945502] date cpuset=/ mems_allowed=0 [ 247.945725] Pid: 10248, comm: date Not tainted 2.6.37-rc3 #154 [ 247.946023] Call Trace: [ 247.946191] [] ? _raw_spin_unlock+0x2b/0x40 [ 247.946490] [] dump_header+0x83/0x200 [ 247.946768] [] ? trace_hardirqs_on+0xd/0x10 [ 247.947066] [] ? ___ratelimit+0xab/0x150 [ 247.947352] [] ? oom_badness+0x77/0x190 [ 247.947634] [] oom_kill_process+0x8d/0x270 [ 247.947927] [] out_of_memory+0x1d8/0x410 [ 247.948216] [] __alloc_pages_nodemask+0x81b/0x830 [ 247.948535] [] alloc_pages_current+0x99/0x110 [ 247.948835] [] __page_cache_alloc+0x87/0x90 [ 247.949144] [] filemap_fault+0x1d7/0x4f0 [ 247.949433] [] __do_fault+0x54/0x540 [ 247.949708] [] handle_mm_fault+0x1af/0xc20 [ 247.950001] [] ? do_page_fault+0xf6/0x560 [ 247.950291] [] do_page_fault+0x16b/0x560 [ 247.950580] [] ? lock_release_holdtime+0x3d/0x180 [ 247.950896] [] ? error_sti+0x5/0x6 [ 247.951162] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 247.951484] [] page_fault+0x25/0x30 [ 247.951751] Mem-Info: [ 247.951909] Node 0 DMA per-cpu: [ 247.952137] CPU 0: hi: 0, btch: 1 usd: 0 [ 247.952393] CPU 1: hi: 0, btch: 1 usd: 0 [ 247.952645] CPU 2: hi: 0, btch: 1 usd: 0 [ 247.952897] CPU 3: hi: 0, btch: 1 usd: 0 [ 247.953157] CPU 4: hi: 0, btch: 1 usd: 0 [ 247.953411] CPU 5: hi: 0, btch: 1 usd: 0 [ 247.953668] CPU 6: hi: 0, btch: 1 usd: 0 [ 247.953921] CPU 7: hi: 0, btch: 1 usd: 0 [ 247.954176] Node 0 DMA32 per-cpu: [ 247.954409] CPU 0: hi: 90, btch: 15 usd: 1 [ 247.954666] CPU 1: hi: 90, btch: 15 usd: 0 [ 247.954923] CPU 2: hi: 90, btch: 15 usd: 0 [ 247.955178] CPU 3: hi: 90, btch: 15 usd: 14 [ 247.955433] CPU 4: hi: 90, btch: 15 usd: 20 [ 247.955688] CPU 5: hi: 90, btch: 15 usd: 77 [ 247.955944] CPU 6: hi: 90, btch: 15 usd: 25 [ 247.956200] CPU 7: hi: 90, btch: 15 usd: 0 [ 247.956450] active_anon:3680 inactive_anon:42 isolated_anon:0 [ 247.956452] active_file:142 inactive_file:1673 isolated_file:0 [ 247.956453] unevictable:0 dirty:0 writeback:199 unstable:0 [ 247.956454] free:751 slab_reclaimable:13557 slab_unreclaimable:5125 [ 247.956455] mapped:139 shmem:75 pagetables:260 bounce:0 [ 247.957919] Node 0 DMA free:1048kB min:120kB low:148kB high:180kB active_anon:576kB inactive_anon:0kB active_file:12kB inactive_file:476kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:13304kB slab_unreclaimable:252kB kernel_stack:16kB pagetables:32kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:802 all_unreclaimable? yes [ 247.959713] lowmem_reserve[]: 0 236 236 236 [ 247.960130] Node 0 DMA32 free:1956kB min:1904kB low:2380kB high:2856kB active_anon:14144kB inactive_anon:168kB active_file:556kB inactive_file:6216kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:0kB writeback:844kB mapped:568kB shmem:300kB slab_reclaimable:40924kB slab_unreclaimable:20248kB kernel_stack:1080kB pagetables:1008kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:12451 all_unreclaimable? yes [ 247.961965] lowmem_reserve[]: 0 0 0 0 [ 247.962356] Node 0 DMA: 1*4kB 54*8kB 45*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1156kB [ 247.963350] Node 0 DMA32: 108*4kB 147*8kB 7*16kB 4*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1848kB [ 247.964353] 2006 total pagecache pages [ 247.964568] 0 pages in swap cache [ 247.964766] Swap cache stats: add 0, delete 0, find 0/0 [ 247.965045] Free swap = 0kB [ 247.965228] Total swap = 0kB [ 247.966122] 65520 pages RAM [ 247.966302] 16492 pages reserved [ 247.967066] 2455 pages shared [ 247.967251] 45714 pages non-shared [ 247.967450] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 247.967851] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 247.968245] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 247.968642] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 247.969036] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 247.969434] [ 2749] 0 2749 3460 290 6 0 0 dd [ 247.969820] [ 2750] 0 2750 3460 289 1 0 0 dd [ 247.970205] [ 2751] 0 2751 3460 289 0 0 0 dd [ 247.970588] [ 2752] 0 2752 3460 290 6 0 0 dd [ 247.970971] [ 2753] 0 2753 3460 290 5 0 0 dd [ 247.971357] [ 2754] 0 2754 3460 289 6 0 0 dd [ 247.971740] [ 2755] 0 2755 3460 289 2 0 0 dd [ 247.972125] [ 2756] 0 2756 3460 290 5 0 0 dd [ 247.972511] [ 2757] 0 2757 3460 290 0 0 0 dd [ 247.972893] [ 2758] 0 2758 3460 290 0 0 0 dd [ 247.973284] [ 2759] 0 2759 3437 137 1 0 0 plot-written.sh [ 247.973711] [ 2776] 0 2776 2144 82 1 0 0 iostat [ 247.974109] [10030] 0 10030 1494 26 1 0 0 getty [ 247.974501] [10034] 0 10034 1494 26 0 0 0 getty [ 247.974888] [10035] 0 10035 1494 26 4 0 0 getty [ 247.975279] [10036] 0 10036 1494 26 5 0 0 getty [ 247.975672] [10038] 0 10038 1494 27 1 0 0 getty [ 247.976065] [10041] 0 10041 1494 29 4 0 0 getty [ 247.976460] [10248] 0 10248 3202 70 4 0 0 date [ 247.976851] Out of memory: Kill process 2749 (dd) score 1 or sacrifice child [ 247.977202] Killed process 2749 (dd) total-vm:13840kB, anon-rss:1140kB, file-rss:20kB [ 247.977652] dd: page allocation failure. order:0, mode:0x2005a [ 247.977947] Pid: 2749, comm: dd Not tainted 2.6.37-rc3 #154 [ 247.978227] Call Trace: [ 247.978393] [] __alloc_pages_nodemask+0x680/0x830 [ 247.978703] [] alloc_pages_current+0x99/0x110 [ 247.979002] [] __page_cache_alloc+0x87/0x90 [ 247.979292] [] grab_cache_page_write_begin+0x7f/0xc0 [ 247.979612] [] ext4_da_write_begin+0x143/0x280 [ 247.979911] [] generic_file_buffered_write+0x109/0x260 [ 247.980238] [] ? ext4_dirty_inode+0x54/0x60 [ 247.980530] [] ? mutex_lock_nested+0x280/0x350 [ 247.980829] [] __generic_file_aio_write+0x244/0x450 [ 247.981161] [] ? mutex_lock_nested+0x292/0x350 [ 247.981462] [] ? generic_file_aio_write+0x58/0xd0 [ 247.981771] [] ? generic_file_aio_write+0x58/0xd0 [ 247.982080] [] generic_file_aio_write+0x6b/0xd0 [ 247.982385] [] ext4_file_write+0x42/0xc0 [ 247.982665] [] do_sync_write+0xda/0x120 [ 247.982940] [] ? might_fault+0xa5/0xb0 [ 247.983216] [] ? read_zero+0xe5/0x120 [ 247.983487] [] vfs_write+0xce/0x190 [ 247.983750] [] sys_write+0x54/0x90 [ 247.984008] [] system_call_fastpath+0x16/0x1b [ 247.984304] Mem-Info: [ 247.984458] Node 0 DMA per-cpu: [ 247.984682] CPU 0: hi: 0, btch: 1 usd: 0 [ 247.984930] CPU 1: hi: 0, btch: 1 usd: 0 [ 247.985187] CPU 2: hi: 0, btch: 1 usd: 0 [ 247.985436] CPU 3: hi: 0, btch: 1 usd: 0 [ 247.985687] CPU 4: hi: 0, btch: 1 usd: 0 [ 247.985936] CPU 5: hi: 0, btch: 1 usd: 0 [ 247.986184] CPU 6: hi: 0, btch: 1 usd: 0 [ 247.986434] CPU 7: hi: 0, btch: 1 usd: 0 [ 247.986683] Node 0 DMA32 per-cpu: [ 247.986912] CPU 0: hi: 90, btch: 15 usd: 1 [ 247.987159] CPU 1: hi: 90, btch: 15 usd: 0 [ 247.987407] CPU 2: hi: 90, btch: 15 usd: 0 [ 247.987658] CPU 3: hi: 90, btch: 15 usd: 14 [ 247.987906] CPU 4: hi: 90, btch: 15 usd: 20 [ 247.988154] CPU 5: hi: 90, btch: 15 usd: 77 [ 247.988402] CPU 6: hi: 90, btch: 15 usd: 25 [ 247.988651] CPU 7: hi: 90, btch: 15 usd: 0 [ 247.988903] active_anon:3680 inactive_anon:42 isolated_anon:0 [ 247.988904] active_file:142 inactive_file:1673 isolated_file:0 [ 247.988905] unevictable:0 dirty:0 writeback:99 unstable:0 [ 247.988906] free:751 slab_reclaimable:13557 slab_unreclaimable:5125 [ 247.988907] mapped:139 shmem:75 pagetables:260 bounce:0 [ 247.990386] Node 0 DMA free:1048kB min:120kB low:148kB high:180kB active_anon:576kB inactive_anon:0kB active_file:12kB inactive_file:476kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:13304kB slab_unreclaimable:252kB kernel_stack:16kB pagetables:32kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:802 all_unreclaimable? yes [ 247.992091] lowmem_reserve[]: 0 236 236 236 [ 247.992496] Node 0 DMA32 free:1956kB min:1904kB low:2380kB high:2856kB active_anon:14144kB inactive_anon:168kB active_file:556kB inactive_file:6216kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:0kB writeback:444kB mapped:568kB shmem:300kB slab_reclaimable:40924kB slab_unreclaimable:20248kB kernel_stack:1080kB pagetables:1008kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:12451 all_unreclaimable? yes [ 247.994300] lowmem_reserve[]: 0 0 0 0 [ 247.994687] Node 0 DMA: 1*4kB 54*8kB 45*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1156kB [ 247.995656] Node 0 DMA32: 108*4kB 147*8kB 7*16kB 4*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1848kB [ 247.996688] 2006 total pagecache pages [ 247.996898] 0 pages in swap cache [ 247.997133] Swap cache stats: add 0, delete 0, find 0/0 [ 247.997398] Free swap = 0kB [ 247.997575] Total swap = 0kB [ 247.998351] 65520 pages RAM [ 247.998527] 16492 pages reserved [ 247.998717] 2335 pages shared [ 247.998898] 45714 pages non-shared [ 248.259785] date invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 [ 248.260191] date cpuset=/ mems_allowed=0 [ 248.260401] Pid: 10248, comm: date Not tainted 2.6.37-rc3 #154 [ 248.260682] Call Trace: [ 248.260843] [] ? _raw_spin_unlock+0x2b/0x40 [ 248.261126] [] dump_header+0x83/0x200 [ 248.261405] [] ? trace_hardirqs_on+0xd/0x10 [ 248.261689] [] ? ___ratelimit+0xab/0x150 [ 248.261961] [] ? oom_badness+0x77/0x190 [ 248.262230] [] oom_kill_process+0x8d/0x270 [ 248.262508] [] out_of_memory+0x1d8/0x410 [ 248.262781] [] __alloc_pages_nodemask+0x81b/0x830 [ 248.263084] [] alloc_pages_current+0x99/0x110 [ 248.263373] [] __page_cache_alloc+0x87/0x90 [ 248.263656] [] __do_page_cache_readahead+0x120/0x260 [ 248.263966] [] ? __do_page_cache_readahead+0xad/0x260 [ 248.264280] [] ? lock_release_holdtime+0x3d/0x180 [ 248.264582] [] ra_submit+0x21/0x30 [ 248.264835] [] filemap_fault+0x4dc/0x4f0 [ 248.265107] [] ? __lock_acquire+0x53e/0x1e70 [ 248.265442] [] __do_fault+0x54/0x540 [ 248.265702] [] ? local_clock+0x6f/0x80 [ 248.265968] [] handle_mm_fault+0x1af/0xc20 [ 248.266248] [] ? do_page_fault+0xf6/0x560 [ 248.266523] [] ? lock_release_holdtime+0x3d/0x180 [ 248.266825] [] do_page_fault+0x16b/0x560 [ 248.267096] [] ? local_clock+0x6f/0x80 [ 248.267362] [] ? trace_hardirqs_off_caller+0x29/0x150 [ 248.267676] [] ? trace_hardirqs_off+0xd/0x10 [ 248.267961] [] ? local_clock+0x6f/0x80 [ 248.268227] [] ? error_sti+0x5/0x6 [ 248.268479] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 248.268786] [] page_fault+0x25/0x30 [ 248.269043] Mem-Info: [ 248.269242] Node 0 DMA per-cpu: [ 248.269460] CPU 0: hi: 0, btch: 1 usd: 0 [ 248.269703] CPU 1: hi: 0, btch: 1 usd: 0 [ 248.269947] CPU 2: hi: 0, btch: 1 usd: 0 [ 248.270186] CPU 3: hi: 0, btch: 1 usd: 0 [ 248.270426] CPU 4: hi: 0, btch: 1 usd: 0 [ 248.270666] CPU 5: hi: 0, btch: 1 usd: 0 [ 248.270906] CPU 6: hi: 0, btch: 1 usd: 0 [ 248.271146] CPU 7: hi: 0, btch: 1 usd: 0 [ 248.271387] Node 0 DMA32 per-cpu: [ 248.271606] CPU 0: hi: 90, btch: 15 usd: 15 [ 248.271847] CPU 1: hi: 90, btch: 15 usd: 0 [ 248.272088] CPU 2: hi: 90, btch: 15 usd: 14 [ 248.272328] CPU 3: hi: 90, btch: 15 usd: 10 [ 248.272569] CPU 4: hi: 90, btch: 15 usd: 0 [ 248.272810] CPU 5: hi: 90, btch: 15 usd: 0 [ 248.273050] CPU 6: hi: 90, btch: 15 usd: 0 [ 248.273342] CPU 7: hi: 90, btch: 15 usd: 0 [ 248.274149] active_anon:3376 inactive_anon:12 isolated_anon:0 [ 248.274150] active_file:157 inactive_file:2089 isolated_file:0 [ 248.274150] unevictable:0 dirty:24 writeback:603 unstable:0 [ 248.274151] free:726 slab_reclaimable:13593 slab_unreclaimable:5131 [ 248.274151] mapped:128 shmem:75 pagetables:238 bounce:0 [ 248.275553] Node 0 DMA free:1052kB min:120kB low:148kB high:180kB active_anon:568kB inactive_anon:0kB active_file:12kB inactive_file:616kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:48kB writeback:136kB mapped:4kB shmem:0kB slab_reclaimable:13296kB slab_unreclaimable:252kB kernel_stack:8kB pagetables:32kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1000 all_unreclaimable? yes [ 248.277365] lowmem_reserve[]: 0 236 236 236 [ 248.277751] Node 0 DMA32 free:1852kB min:1904kB low:2380kB high:2856kB active_anon:12936kB inactive_anon:48kB active_file:616kB inactive_file:7740kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:48kB writeback:1776kB mapped:508kB shmem:300kB slab_reclaimable:41076kB slab_unreclaimable:20272kB kernel_stack:1072kB pagetables:920kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:12633 all_unreclaimable? yes [ 248.279490] lowmem_reserve[]: 0 0 0 0 [ 248.279860] Node 0 DMA: 7*4kB 39*8kB 45*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1060kB [ 248.280795] Node 0 DMA32: 122*4kB 118*8kB 16*16kB 6*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1880kB [ 248.281822] 2409 total pagecache pages [ 248.282030] 0 pages in swap cache [ 248.282224] Swap cache stats: add 0, delete 0, find 0/0 [ 248.282488] Free swap = 0kB [ 248.282664] Total swap = 0kB [ 248.283515] 65520 pages RAM [ 248.283689] 16492 pages reserved [ 248.283878] 1944 pages shared [ 248.284057] 46568 pages non-shared [ 248.284252] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 248.284636] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 248.285019] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 248.285457] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 248.285867] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 248.286274] [ 2750] 0 2750 3460 289 3 0 0 dd [ 248.286673] [ 2751] 0 2751 3460 289 1 0 0 dd [ 248.287046] [ 2752] 0 2752 3460 290 6 0 0 dd [ 248.287418] [ 2753] 0 2753 3460 290 5 0 0 dd [ 248.287792] [ 2754] 0 2754 3460 289 6 0 0 dd [ 248.288164] [ 2755] 0 2755 3460 289 2 0 0 dd [ 248.288538] [ 2756] 0 2756 3460 290 1 0 0 dd [ 248.288912] [ 2757] 0 2757 3460 290 4 0 0 dd [ 248.289340] [ 2758] 0 2758 3460 290 7 0 0 dd [ 248.289714] [ 2759] 0 2759 3437 136 1 0 0 plot-written.sh [ 248.290130] [ 2776] 0 2776 2144 82 1 0 0 iostat [ 248.290519] [10030] 0 10030 1494 26 1 0 0 getty [ 248.290901] [10034] 0 10034 1494 26 0 0 0 getty [ 248.291284] [10035] 0 10035 1494 26 4 0 0 getty [ 248.291667] [10036] 0 10036 1494 26 5 0 0 getty [ 248.292051] [10038] 0 10038 1494 27 1 0 0 getty [ 248.292433] [10041] 0 10041 1494 29 4 0 0 getty [ 248.292816] [10248] 0 10248 3202 75 1 0 0 date [ 248.293247] Out of memory: Kill process 2750 (dd) score 1 or sacrifice child [ 248.293580] Killed process 2750 (dd) total-vm:13840kB, anon-rss:1136kB, file-rss:20kB [ 251.665672] sleep invoked oom-killer: gfp_mask=0x84d0, order=0, oom_adj=0, oom_score_adj=0 [ 251.666091] sleep cpuset=/ mems_allowed=0 [ 251.666314] Pid: 10269, comm: sleep Not tainted 2.6.37-rc3 #154 [ 251.666606] Call Trace: [ 251.666768] [] ? _raw_spin_unlock+0x2b/0x40 [ 251.667072] [] dump_header+0x83/0x200 [ 251.667341] [] ? trace_hardirqs_on+0xd/0x10 [ 251.667623] [] ? ___ratelimit+0xab/0x150 [ 251.667894] [] ? oom_badness+0x77/0x190 [ 251.668162] [] oom_kill_process+0x8d/0x270 [ 251.668439] [] out_of_memory+0x1d8/0x410 [ 251.668712] [] __alloc_pages_nodemask+0x81b/0x830 [ 251.669012] [] alloc_pages_current+0x99/0x110 [ 251.669300] [] pte_alloc_one+0x1b/0x40 [ 251.669566] [] __pte_alloc+0x2f/0xf0 [ 251.669825] [] ? local_clock+0x6f/0x80 [ 251.670089] [] handle_mm_fault+0xa97/0xc20 [ 251.670367] [] ? do_page_fault+0xf6/0x560 [ 251.670641] [] do_page_fault+0x16b/0x560 [ 251.670921] [] ? local_clock+0x6f/0x80 [ 251.671188] [] ? trace_hardirqs_off_caller+0x29/0x150 [ 251.671499] [] ? trace_hardirqs_off+0xd/0x10 [ 251.671785] [] ? local_clock+0x6f/0x80 [ 251.672055] [] ? error_sti+0x5/0x6 [ 251.672307] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 251.672615] [] page_fault+0x25/0x30 [ 251.672872] Mem-Info: [ 251.673024] Node 0 DMA per-cpu: [ 251.673243] CPU 0: hi: 0, btch: 1 usd: 0 [ 251.673489] CPU 1: hi: 0, btch: 1 usd: 0 [ 251.673733] CPU 2: hi: 0, btch: 1 usd: 0 [ 251.673974] CPU 3: hi: 0, btch: 1 usd: 0 [ 251.674215] CPU 4: hi: 0, btch: 1 usd: 0 [ 251.674456] CPU 5: hi: 0, btch: 1 usd: 0 [ 251.674699] CPU 6: hi: 0, btch: 1 usd: 0 [ 251.674948] CPU 7: hi: 0, btch: 1 usd: 0 [ 251.675191] Node 0 DMA32 per-cpu: [ 251.675413] CPU 0: hi: 90, btch: 15 usd: 18 [ 251.675655] CPU 1: hi: 90, btch: 15 usd: 27 [ 251.675897] CPU 2: hi: 90, btch: 15 usd: 22 [ 251.676138] CPU 3: hi: 90, btch: 15 usd: 0 [ 251.676380] CPU 4: hi: 90, btch: 15 usd: 0 [ 251.676624] CPU 5: hi: 90, btch: 15 usd: 1 [ 251.676872] CPU 6: hi: 90, btch: 15 usd: 29 [ 251.677117] CPU 7: hi: 90, btch: 15 usd: 12 [ 251.677363] active_anon:3075 inactive_anon:35 isolated_anon:0 [ 251.677363] active_file:236 inactive_file:2005 isolated_file:0 [ 251.677364] unevictable:0 dirty:212 writeback:0 unstable:0 [ 251.677365] free:770 slab_reclaimable:13787 slab_unreclaimable:5155 [ 251.677365] mapped:180 shmem:75 pagetables:223 bounce:0 [ 251.678759] Node 0 DMA free:1060kB min:120kB low:148kB high:180kB active_anon:572kB inactive_anon:0kB active_file:12kB inactive_file:552kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:13260kB slab_unreclaimable:252kB kernel_stack:8kB pagetables:32kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1419 all_unreclaimable? yes [ 251.680436] lowmem_reserve[]: 0 236 236 236 [ 251.680834] Node 0 DMA32 free:2020kB min:1904kB low:2380kB high:2856kB active_anon:11728kB inactive_anon:140kB active_file:932kB inactive_file:7468kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:848kB writeback:0kB mapped:720kB shmem:300kB slab_reclaimable:41888kB slab_unreclaimable:20368kB kernel_stack:1064kB pagetables:860kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:26694 all_unreclaimable? yes [ 251.682579] lowmem_reserve[]: 0 0 0 0 [ 251.682963] Node 0 DMA: 2*4kB 38*8kB 45*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1064kB [ 251.683914] Node 0 DMA32: 191*4kB 88*8kB 24*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1884kB [ 251.684877] 2396 total pagecache pages [ 251.685082] 0 pages in swap cache [ 251.685272] Swap cache stats: add 0, delete 0, find 0/0 [ 251.685531] Free swap = 0kB [ 251.685704] Total swap = 0kB [ 251.686473] 65520 pages RAM [ 251.686643] 16492 pages reserved [ 251.686838] 1452 pages shared [ 251.687016] 46496 pages non-shared [ 251.687209] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 251.687587] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 251.687963] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 251.688339] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 251.688716] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 251.689090] [ 2751] 0 2751 3460 289 3 0 0 dd [ 251.689458] [ 2752] 0 2752 3460 290 1 0 0 dd [ 251.689824] [ 2753] 0 2753 3460 290 6 0 0 dd [ 251.690190] [ 2754] 0 2754 3460 289 2 0 0 dd [ 251.690556] [ 2755] 0 2755 3460 289 5 0 0 dd [ 251.690931] [ 2756] 0 2756 3460 290 6 0 0 dd [ 251.691873] [ 2757] 0 2757 3460 290 6 0 0 dd [ 251.692239] [ 2758] 0 2758 3460 290 5 0 0 dd [ 251.692605] [ 2759] 0 2759 3437 175 0 0 0 plot-written.sh [ 251.693013] [ 2776] 0 2776 2144 91 1 0 0 iostat [ 251.693392] [10030] 0 10030 1494 26 1 0 0 getty [ 251.693769] [10034] 0 10034 1494 26 0 0 0 getty [ 251.694145] [10035] 0 10035 1494 26 4 0 0 getty [ 251.694521] [10036] 0 10036 1494 26 5 0 0 getty [ 251.694914] [10038] 0 10038 1494 27 1 0 0 getty [ 251.695291] [10041] 0 10041 1494 29 4 0 0 getty [ 251.695668] [10269] 0 10269 2135 88 4 0 0 sleep [ 251.696043] Out of memory: Kill process 2751 (dd) score 1 or sacrifice child [ 251.696372] Killed process 2751 (dd) total-vm:13840kB, anon-rss:1136kB, file-rss:20kB [ 251.696805] dd: page allocation failure. order:0, mode:0x2005a [ 251.697098] Pid: 2751, comm: dd Not tainted 2.6.37-rc3 #154 [ 251.697376] Call Trace: [ 251.697541] [] __alloc_pages_nodemask+0x680/0x830 [ 251.697852] [] alloc_pages_current+0x99/0x110 [ 251.698149] [] __page_cache_alloc+0x87/0x90 [ 251.698439] [] grab_cache_page_write_begin+0x7f/0xc0 [ 251.698757] [] ext4_da_write_begin+0x143/0x280 [ 251.699067] [] generic_file_buffered_write+0x109/0x260 [ 251.699394] [] ? ext4_dirty_inode+0x54/0x60 [ 251.699686] [] ? mutex_lock_nested+0x280/0x350 [ 251.699984] [] __generic_file_aio_write+0x244/0x450 [ 251.700301] [] ? mutex_lock_nested+0x292/0x350 [ 251.700600] [] ? generic_file_aio_write+0x58/0xd0 [ 251.700909] [] ? generic_file_aio_write+0x58/0xd0 [ 251.701218] [] generic_file_aio_write+0x6b/0xd0 [ 251.701521] [] ext4_file_write+0x42/0xc0 [ 251.701801] [] do_sync_write+0xda/0x120 [ 251.702077] [] ? might_fault+0xa5/0xb0 [ 251.702352] [] ? read_zero+0xe5/0x120 [ 251.702622] [] vfs_write+0xce/0x190 [ 251.702901] [] sys_write+0x54/0x90 [ 251.703162] [] system_call_fastpath+0x16/0x1b [ 251.703457] Mem-Info: [ 251.703612] Node 0 DMA per-cpu: [ 251.703835] CPU 0: hi: 0, btch: 1 usd: 0 [ 251.704084] CPU 1: hi: 0, btch: 1 usd: 0 [ 251.704334] CPU 2: hi: 0, btch: 1 usd: 0 [ 251.704584] CPU 3: hi: 0, btch: 1 usd: 0 [ 251.704833] CPU 4: hi: 0, btch: 1 usd: 0 [ 251.705082] CPU 5: hi: 0, btch: 1 usd: 0 [ 251.705332] CPU 6: hi: 0, btch: 1 usd: 0 [ 251.705581] CPU 7: hi: 0, btch: 1 usd: 0 [ 251.705829] Node 0 DMA32 per-cpu: [ 251.706059] CPU 0: hi: 90, btch: 15 usd: 19 [ 251.706308] CPU 1: hi: 90, btch: 15 usd: 27 [ 251.706557] CPU 2: hi: 90, btch: 15 usd: 22 [ 251.706814] CPU 3: hi: 90, btch: 15 usd: 0 [ 251.707064] CPU 4: hi: 90, btch: 15 usd: 0 [ 251.707312] CPU 5: hi: 90, btch: 15 usd: 1 [ 251.707561] CPU 6: hi: 90, btch: 15 usd: 29 [ 251.707810] CPU 7: hi: 90, btch: 15 usd: 12 [ 251.708061] active_anon:3075 inactive_anon:35 isolated_anon:0 [ 251.708062] active_file:236 inactive_file:2005 isolated_file:0 [ 251.708063] unevictable:0 dirty:212 writeback:0 unstable:0 [ 251.708064] free:770 slab_reclaimable:13787 slab_unreclaimable:5155 [ 251.708065] mapped:180 shmem:75 pagetables:223 bounce:0 [ 251.709486] Node 0 DMA free:1060kB min:120kB low:148kB high:180kB active_anon:572kB inactive_anon:0kB active_file:12kB inactive_file:552kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:13260kB slab_unreclaimable:252kB kernel_stack:8kB pagetables:32kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:1419 all_unreclaimable? yes [ 251.711194] lowmem_reserve[]: 0 236 236 236 [ 251.711602] Node 0 DMA32 free:2020kB min:1904kB low:2380kB high:2856kB active_anon:11728kB inactive_anon:140kB active_file:932kB inactive_file:7468kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:848kB writeback:0kB mapped:720kB shmem:300kB slab_reclaimable:41888kB slab_unreclaimable:20368kB kernel_stack:1064kB pagetables:860kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:26694 all_unreclaimable? yes [ 251.713383] lowmem_reserve[]: 0 0 0 0 [ 251.713769] Node 0 DMA: 2*4kB 38*8kB 45*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1064kB [ 251.714739] Node 0 DMA32: 191*4kB 88*8kB 24*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1884kB [ 251.715732] 2396 total pagecache pages [ 251.715941] 0 pages in swap cache [ 251.716134] Swap cache stats: add 0, delete 0, find 0/0 [ 251.716399] Free swap = 0kB [ 251.716575] Total swap = 0kB [ 251.717304] 65520 pages RAM [ 251.717478] 16492 pages reserved [ 251.717668] 1452 pages shared [ 251.717847] 46496 pages non-shared [ 261.765242] sleep invoked oom-killer: gfp_mask=0x84d0, order=0, oom_adj=0, oom_score_adj=0 [ 261.765664] sleep cpuset=/ mems_allowed=0 [ 261.765887] Pid: 10422, comm: sleep Not tainted 2.6.37-rc3 #154 [ 261.766179] Call Trace: [ 261.766342] [] ? _raw_spin_unlock+0x2b/0x40 [ 261.766628] [] dump_header+0x83/0x200 [ 261.766895] [] ? trace_hardirqs_on+0xd/0x10 [ 261.767179] [] ? ___ratelimit+0xab/0x150 [ 261.767453] [] ? oom_badness+0x77/0x190 [ 261.767736] [] oom_kill_process+0x8d/0x270 [ 261.768017] [] out_of_memory+0x1d8/0x410 [ 261.768292] [] __alloc_pages_nodemask+0x81b/0x830 [ 261.768595] [] alloc_pages_current+0x99/0x110 [ 261.768886] [] pte_alloc_one+0x1b/0x40 [ 261.769153] [] __pte_alloc+0x2f/0xf0 [ 261.769415] [] ? local_clock+0x6f/0x80 [ 261.769683] [] handle_mm_fault+0xa97/0xc20 [ 261.769965] [] ? do_page_fault+0xf6/0x560 [ 261.770242] [] do_page_fault+0x16b/0x560 [ 261.770516] [] ? local_clock+0x6f/0x80 [ 261.770783] [] ? trace_hardirqs_off_caller+0x29/0x150 [ 261.771098] [] ? trace_hardirqs_off+0xd/0x10 [ 261.771385] [] ? local_clock+0x6f/0x80 [ 261.771661] [] ? error_sti+0x5/0x6 [ 261.771917] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 261.772228] [] page_fault+0x25/0x30 [ 261.772485] Mem-Info: [ 261.772636] Node 0 DMA per-cpu: [ 261.772854] CPU 0: hi: 0, btch: 1 usd: 0 [ 261.773095] CPU 1: hi: 0, btch: 1 usd: 0 [ 261.773336] CPU 2: hi: 0, btch: 1 usd: 0 [ 261.773578] CPU 3: hi: 0, btch: 1 usd: 0 [ 261.773818] CPU 4: hi: 0, btch: 1 usd: 0 [ 261.774060] CPU 5: hi: 0, btch: 1 usd: 0 [ 261.774302] CPU 6: hi: 0, btch: 1 usd: 0 [ 261.774543] CPU 7: hi: 0, btch: 1 usd: 0 [ 261.774786] Node 0 DMA32 per-cpu: [ 261.775011] CPU 0: hi: 90, btch: 15 usd: 31 [ 261.775255] CPU 1: hi: 90, btch: 15 usd: 0 [ 261.775512] CPU 2: hi: 90, btch: 15 usd: 0 [ 261.775760] CPU 3: hi: 90, btch: 15 usd: 0 [ 261.776005] CPU 4: hi: 90, btch: 15 usd: 83 [ 261.776246] CPU 5: hi: 90, btch: 15 usd: 28 [ 261.776487] CPU 6: hi: 90, btch: 15 usd: 0 [ 261.776728] CPU 7: hi: 90, btch: 15 usd: 15 [ 261.776971] active_anon:2800 inactive_anon:10 isolated_anon:0 [ 261.776971] active_file:280 inactive_file:1696 isolated_file:0 [ 261.776972] unevictable:0 dirty:0 writeback:351 unstable:0 [ 261.776972] free:732 slab_reclaimable:14332 slab_unreclaimable:5150 [ 261.776973] mapped:242 shmem:75 pagetables:188 bounce:0 [ 261.778361] Node 0 DMA free:1040kB min:120kB low:148kB high:180kB active_anon:572kB inactive_anon:0kB active_file:12kB inactive_file:536kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:0kB writeback:0kB mapped:8kB shmem:0kB slab_reclaimable:13392kB slab_unreclaimable:228kB kernel_stack:8kB pagetables:36kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:866 all_unreclaimable? yes [ 261.780066] lowmem_reserve[]: 0 236 236 236 [ 261.780459] Node 0 DMA32 free:1888kB min:1904kB low:2380kB high:2856kB active_anon:10628kB inactive_anon:40kB active_file:1108kB inactive_file:6248kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:28kB writeback:992kB mapped:960kB shmem:300kB slab_reclaimable:43936kB slab_unreclaimable:20372kB kernel_stack:1056kB pagetables:716kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:13066 all_unreclaimable? yes [ 261.782197] lowmem_reserve[]: 0 0 0 0 [ 261.782574] Node 0 DMA: 1*4kB 52*8kB 40*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1060kB [ 261.783583] Node 0 DMA32: 100*4kB 151*8kB 13*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1848kB [ 261.784550] 2137 total pagecache pages [ 261.784756] 0 pages in swap cache [ 261.784945] Swap cache stats: add 0, delete 0, find 0/0 [ 261.785204] Free swap = 0kB [ 261.785377] Total swap = 0kB [ 261.786140] 65520 pages RAM [ 261.786311] 16492 pages reserved [ 261.786497] 1556 pages shared [ 261.786674] 46628 pages non-shared [ 261.786867] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 261.787874] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 261.788252] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 261.788628] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 261.789004] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 261.789376] [ 2752] 0 2752 3460 290 2 0 0 dd [ 261.789743] [ 2753] 0 2753 3460 290 7 0 0 dd [ 261.790107] [ 2754] 0 2754 3460 289 5 0 0 dd [ 261.790471] [ 2755] 0 2755 3460 289 4 0 0 dd [ 261.790839] [ 2756] 0 2756 3460 290 2 0 0 dd [ 261.791207] [ 2757] 0 2757 3460 290 6 0 0 dd [ 261.791624] [ 2758] 0 2758 3460 290 1 0 0 dd [ 261.791989] [ 2759] 0 2759 3437 194 0 0 0 plot-written.sh [ 261.792397] [ 2776] 0 2776 2144 95 5 0 0 iostat [ 261.792774] [10030] 0 10030 1494 26 1 0 0 getty [ 261.793151] [10034] 0 10034 1494 26 0 0 0 getty [ 261.793527] [10035] 0 10035 1494 26 4 0 0 getty [ 261.793903] [10036] 0 10036 1494 26 5 0 0 getty [ 261.794276] [10038] 0 10038 1494 27 1 0 0 getty [ 261.794648] [10041] 0 10041 1494 29 4 0 0 getty [ 261.795020] [10422] 0 10422 2135 116 4 0 0 sleep [ 261.795393] Out of memory: Kill process 2752 (dd) score 1 or sacrifice child [ 261.795769] Killed process 2752 (dd) total-vm:13840kB, anon-rss:1140kB, file-rss:20kB [ 265.845137] grep invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 [ 265.845587] grep cpuset=/ mems_allowed=0 [ 265.845814] Pid: 10562, comm: grep Not tainted 2.6.37-rc3 #154 [ 265.846109] Call Trace: [ 265.846279] [] ? _raw_spin_unlock+0x2b/0x40 [ 265.846573] [] dump_header+0x83/0x200 [ 265.846844] [] ? trace_hardirqs_on+0xd/0x10 [ 265.847138] [] ? ___ratelimit+0xab/0x150 [ 265.847415] [] ? oom_badness+0x77/0x190 [ 265.847685] [] oom_kill_process+0x8d/0x270 [ 265.847965] [] out_of_memory+0x1d8/0x410 [ 265.848240] [] __alloc_pages_nodemask+0x81b/0x830 [ 265.848545] [] alloc_pages_current+0x99/0x110 [ 265.848835] [] __page_cache_alloc+0x87/0x90 [ 265.849119] [] __do_page_cache_readahead+0x120/0x260 [ 265.849439] [] ? __do_page_cache_readahead+0xad/0x260 [ 265.849755] [] ? local_clock+0x6f/0x80 [ 265.850023] [] ra_submit+0x21/0x30 [ 265.850278] [] ondemand_readahead+0x10e/0x2b0 [ 265.850569] [] page_cache_sync_readahead+0x36/0x50 [ 265.850875] [] filemap_fault+0x444/0x4f0 [ 265.851149] [] ? __lock_acquire+0x53e/0x1e70 [ 265.851436] [] __do_fault+0x54/0x540 [ 265.851695] [] handle_mm_fault+0x1af/0xc20 [ 265.851973] [] ? do_page_fault+0xf6/0x560 [ 265.852247] [] ? lock_release_holdtime+0x3d/0x180 [ 265.852546] [] do_page_fault+0x16b/0x560 [ 265.852818] [] ? local_clock+0x6f/0x80 [ 265.853083] [] ? trace_hardirqs_off_caller+0x29/0x150 [ 265.853396] [] ? trace_hardirqs_off+0xd/0x10 [ 265.853685] [] ? local_clock+0x6f/0x80 [ 265.853951] [] ? error_sti+0x5/0x6 [ 265.854203] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 265.854509] [] page_fault+0x25/0x30 [ 265.854764] Mem-Info: [ 265.854913] Node 0 DMA per-cpu: [ 265.855130] CPU 0: hi: 0, btch: 1 usd: 0 [ 265.855377] CPU 1: hi: 0, btch: 1 usd: 0 [ 265.855621] CPU 2: hi: 0, btch: 1 usd: 0 [ 265.855864] CPU 3: hi: 0, btch: 1 usd: 0 [ 265.856105] CPU 4: hi: 0, btch: 1 usd: 0 [ 265.856346] CPU 5: hi: 0, btch: 1 usd: 0 [ 265.856588] CPU 6: hi: 0, btch: 1 usd: 0 [ 265.856830] CPU 7: hi: 0, btch: 1 usd: 0 [ 265.857070] Node 0 DMA32 per-cpu: [ 265.857293] CPU 0: hi: 90, btch: 15 usd: 12 [ 265.857541] CPU 1: hi: 90, btch: 15 usd: 14 [ 265.857783] CPU 2: hi: 90, btch: 15 usd: 0 [ 265.858024] CPU 3: hi: 90, btch: 15 usd: 2 [ 265.858266] CPU 4: hi: 90, btch: 15 usd: 14 [ 265.858507] CPU 5: hi: 90, btch: 15 usd: 4 [ 265.858749] CPU 6: hi: 90, btch: 15 usd: 0 [ 265.858990] CPU 7: hi: 90, btch: 15 usd: 0 [ 265.859233] active_anon:2582 inactive_anon:52 isolated_anon:0 [ 265.859234] active_file:258 inactive_file:1714 isolated_file:0 [ 265.859234] unevictable:0 dirty:0 writeback:312 unstable:0 [ 265.859235] free:781 slab_reclaimable:14537 slab_unreclaimable:5120 [ 265.859236] mapped:255 shmem:75 pagetables:222 bounce:0 [ 265.860624] Node 0 DMA free:1056kB min:120kB low:148kB high:180kB active_anon:416kB inactive_anon:0kB active_file:8kB inactive_file:484kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:13408kB slab_unreclaimable:228kB kernel_stack:8kB pagetables:84kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:837 all_unreclaimable? yes [ 265.862279] lowmem_reserve[]: 0 236 236 236 [ 265.862676] Node 0 DMA32 free:2068kB min:1904kB low:2380kB high:2856kB active_anon:9912kB inactive_anon:208kB active_file:1024kB inactive_file:6372kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:0kB writeback:800kB mapped:1020kB shmem:300kB slab_reclaimable:44740kB slab_unreclaimable:20252kB kernel_stack:1056kB pagetables:804kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:11414 all_unreclaimable? yes [ 265.864406] lowmem_reserve[]: 0 0 0 0 [ 265.864778] Node 0 DMA: 13*4kB 105*8kB 12*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1084kB [ 265.865740] Node 0 DMA32: 150*4kB 129*8kB 15*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1904kB [ 265.866708] 2141 total pagecache pages [ 265.866913] 0 pages in swap cache [ 265.867102] Swap cache stats: add 0, delete 0, find 0/0 [ 265.867362] Free swap = 0kB [ 265.867535] Total swap = 0kB [ 265.868316] 65520 pages RAM [ 265.868487] 16492 pages reserved [ 265.868673] 1572 pages shared [ 265.868849] 46772 pages non-shared [ 265.869041] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 265.869427] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 265.869807] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 265.870184] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 265.870559] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 265.870928] [ 2753] 0 2753 3460 290 2 0 0 dd [ 265.871292] [ 2754] 0 2754 3460 289 7 0 0 dd [ 265.871655] [ 2755] 0 2755 3460 289 7 0 0 dd [ 265.872021] [ 2756] 0 2756 3460 290 5 0 0 dd [ 265.872384] [ 2757] 0 2757 3460 290 6 0 0 dd [ 265.872751] [ 2758] 0 2758 3460 290 6 0 0 dd [ 265.873116] [ 2759] 0 2759 3437 194 0 0 0 plot-written.sh [ 265.873532] [ 2776] 0 2776 2144 96 4 0 0 iostat [ 265.873912] [10030] 0 10030 1494 26 1 0 0 getty [ 265.874289] [10034] 0 10034 1494 26 0 0 0 getty [ 265.874665] [10035] 0 10035 1494 26 4 0 0 getty [ 265.875040] [10036] 0 10036 1494 26 5 0 0 getty [ 265.875415] [10038] 0 10038 1494 27 1 0 0 getty [ 265.875791] [10041] 0 10041 1494 29 4 0 0 getty [ 265.876168] [10561] 0 10561 3437 163 5 0 0 plot-written.sh [ 265.876577] [10562] 0 10562 2675 139 4 0 0 grep [ 265.876950] Out of memory: Kill process 2753 (dd) score 1 or sacrifice child [ 265.877277] Killed process 2753 (dd) total-vm:13840kB, anon-rss:1140kB, file-rss:20kB [ 265.877713] dd: page allocation failure. order:0, mode:0x2005a [ 265.878005] Pid: 2753, comm: dd Not tainted 2.6.37-rc3 #154 [ 265.878282] Call Trace: [ 265.878446] [] __alloc_pages_nodemask+0x680/0x830 [ 265.878756] [] alloc_pages_current+0x99/0x110 [ 265.879052] [] __page_cache_alloc+0x87/0x90 [ 265.879914] [] grab_cache_page_write_begin+0x7f/0xc0 [ 265.880233] [] ext4_da_write_begin+0x143/0x280 [ 265.880532] [] generic_file_buffered_write+0x109/0x260 [ 265.880857] [] ? _raw_spin_unlock+0x2b/0x40 [ 265.881147] [] ? mutex_lock_nested+0x280/0x350 [ 265.881462] [] __generic_file_aio_write+0x244/0x450 [ 265.881779] [] ? mutex_lock_nested+0x292/0x350 [ 265.882077] [] ? generic_file_aio_write+0x58/0xd0 [ 265.882383] [] ? generic_file_aio_write+0x58/0xd0 [ 265.882689] [] generic_file_aio_write+0x6b/0xd0 [ 265.882991] [] ext4_file_write+0x42/0xc0 [ 265.883269] [] do_sync_write+0xda/0x120 [ 265.883543] [] ? might_fault+0xa5/0xb0 [ 265.883814] [] ? read_zero+0xe5/0x120 [ 265.884082] [] vfs_write+0xce/0x190 [ 265.884343] [] sys_write+0x54/0x90 [ 265.884602] [] system_call_fastpath+0x16/0x1b [ 265.884894] Mem-Info: [ 265.885045] Node 0 DMA per-cpu: [ 265.885266] CPU 0: hi: 0, btch: 1 usd: 0 [ 265.885521] CPU 1: hi: 0, btch: 1 usd: 0 [ 265.885767] CPU 2: hi: 0, btch: 1 usd: 0 [ 265.886013] CPU 3: hi: 0, btch: 1 usd: 0 [ 265.886259] CPU 4: hi: 0, btch: 1 usd: 0 [ 265.886504] CPU 5: hi: 0, btch: 1 usd: 0 [ 265.886750] CPU 6: hi: 0, btch: 1 usd: 0 [ 265.886997] CPU 7: hi: 0, btch: 1 usd: 0 [ 265.887243] Node 0 DMA32 per-cpu: [ 265.887470] CPU 0: hi: 90, btch: 15 usd: 12 [ 265.887717] CPU 1: hi: 90, btch: 15 usd: 14 [ 265.887965] CPU 2: hi: 90, btch: 15 usd: 0 [ 265.888211] CPU 3: hi: 90, btch: 15 usd: 2 [ 265.888458] CPU 4: hi: 90, btch: 15 usd: 14 [ 265.888704] CPU 5: hi: 90, btch: 15 usd: 4 [ 265.888949] CPU 6: hi: 90, btch: 15 usd: 0 [ 265.889195] CPU 7: hi: 90, btch: 15 usd: 0 [ 265.889454] active_anon:2582 inactive_anon:52 isolated_anon:0 [ 265.889456] active_file:258 inactive_file:1714 isolated_file:0 [ 265.889457] unevictable:0 dirty:0 writeback:0 unstable:0 [ 265.889457] free:781 slab_reclaimable:14537 slab_unreclaimable:5120 [ 265.889459] mapped:255 shmem:75 pagetables:222 bounce:0 [ 265.890867] Node 0 DMA free:1056kB min:120kB low:148kB high:180kB active_anon:416kB inactive_anon:0kB active_file:8kB inactive_file:484kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:13408kB slab_unreclaimable:228kB kernel_stack:8kB pagetables:84kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:837 all_unreclaimable? yes [ 265.892556] lowmem_reserve[]: 0 236 236 236 [ 265.892961] Node 0 DMA32 free:2068kB min:1904kB low:2380kB high:2856kB active_anon:9912kB inactive_anon:208kB active_file:1024kB inactive_file:6372kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:0kB writeback:0kB mapped:1020kB shmem:300kB slab_reclaimable:44740kB slab_unreclaimable:20252kB kernel_stack:1056kB pagetables:804kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:11414 all_unreclaimable? yes [ 265.894744] lowmem_reserve[]: 0 0 0 0 [ 265.895132] Node 0 DMA: 13*4kB 105*8kB 12*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1084kB [ 265.896109] Node 0 DMA32: 150*4kB 129*8kB 15*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1904kB [ 265.897099] 2141 total pagecache pages [ 265.897308] 0 pages in swap cache [ 265.897515] Swap cache stats: add 0, delete 0, find 0/0 [ 265.897780] Free swap = 0kB [ 265.897956] Total swap = 0kB [ 265.898676] 65520 pages RAM [ 265.898851] 16492 pages reserved [ 265.899040] 1319 pages shared [ 265.899219] 46771 pages non-shared [ 268.591037] date invoked oom-killer: gfp_mask=0x84d0, order=0, oom_adj=0, oom_score_adj=0 [ 268.591469] date cpuset=/ mems_allowed=0 [ 268.591694] Pid: 10627, comm: date Not tainted 2.6.37-rc3 #154 [ 268.591988] Call Trace: [ 268.592156] [] ? _raw_spin_unlock+0x2b/0x40 [ 268.592454] [] dump_header+0x83/0x200 [ 268.592730] [] ? trace_hardirqs_on+0xd/0x10 [ 268.593025] [] ? ___ratelimit+0xab/0x150 [ 268.593309] [] ? oom_badness+0x77/0x190 [ 268.593585] [] oom_kill_process+0x8d/0x270 [ 268.593870] [] out_of_memory+0x1d8/0x410 [ 268.594151] [] __alloc_pages_nodemask+0x81b/0x830 [ 268.594462] [] alloc_pages_current+0x99/0x110 [ 268.594783] [] pte_alloc_one+0x1b/0x40 [ 268.595060] [] __pte_alloc+0x2f/0xf0 [ 268.595333] [] ? local_clock+0x6f/0x80 [ 268.595612] [] handle_mm_fault+0xa97/0xc20 [ 268.595905] [] ? do_page_fault+0xf6/0x560 [ 268.596195] [] do_page_fault+0x16b/0x560 [ 268.596480] [] ? local_clock+0x6f/0x80 [ 268.596760] [] ? trace_hardirqs_off_caller+0x29/0x150 [ 268.597088] [] ? trace_hardirqs_off+0xd/0x10 [ 268.597379] [] ? local_clock+0x6f/0x80 [ 268.597647] [] ? error_sti+0x5/0x6 [ 268.597902] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 268.598210] [] page_fault+0x25/0x30 [ 268.598467] Mem-Info: [ 268.598618] Node 0 DMA per-cpu: [ 268.598851] CPU 0: hi: 0, btch: 1 usd: 0 [ 268.599094] CPU 1: hi: 0, btch: 1 usd: 0 [ 268.599338] CPU 2: hi: 0, btch: 1 usd: 0 [ 268.599582] CPU 3: hi: 0, btch: 1 usd: 0 [ 268.599826] CPU 4: hi: 0, btch: 1 usd: 0 [ 268.600069] CPU 5: hi: 0, btch: 1 usd: 0 [ 268.600314] CPU 6: hi: 0, btch: 1 usd: 0 [ 268.600557] CPU 7: hi: 0, btch: 1 usd: 0 [ 268.600801] Node 0 DMA32 per-cpu: [ 268.601027] CPU 0: hi: 90, btch: 15 usd: 27 [ 268.601271] CPU 1: hi: 90, btch: 15 usd: 33 [ 268.601514] CPU 2: hi: 90, btch: 15 usd: 49 [ 268.601758] CPU 3: hi: 90, btch: 15 usd: 38 [ 268.602002] CPU 4: hi: 90, btch: 15 usd: 0 [ 268.602246] CPU 5: hi: 90, btch: 15 usd: 0 [ 268.602489] CPU 6: hi: 90, btch: 15 usd: 0 [ 268.602740] CPU 7: hi: 90, btch: 15 usd: 13 [ 268.602985] active_anon:2211 inactive_anon:13 isolated_anon:0 [ 268.602986] active_file:323 inactive_file:1806 isolated_file:0 [ 268.602986] unevictable:0 dirty:8 writeback:800 unstable:0 [ 268.602987] free:709 slab_reclaimable:14675 slab_unreclaimable:5134 [ 268.602988] mapped:249 shmem:75 pagetables:197 bounce:0 [ 268.604368] Node 0 DMA free:1040kB min:120kB low:148kB high:180kB active_anon:312kB inactive_anon:0kB active_file:12kB inactive_file:540kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:0kB writeback:0kB mapped:4kB shmem:0kB slab_reclaimable:13428kB slab_unreclaimable:224kB kernel_stack:8kB pagetables:32kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no [ 268.606013] lowmem_reserve[]: 0 236 236 236 [ 268.606408] Node 0 DMA32 free:2388kB min:1904kB low:2380kB high:2856kB active_anon:8532kB inactive_anon:52kB active_file:1280kB inactive_file:6684kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:84kB writeback:3000kB mapped:992kB shmem:300kB slab_reclaimable:45272kB slab_unreclaimable:20312kB kernel_stack:1048kB pagetables:756kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no [ 268.608186] lowmem_reserve[]: 0 0 0 0 [ 268.608555] Node 0 DMA: 1*4kB 63*8kB 39*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1132kB [ 268.609492] Node 0 DMA32: 167*4kB 153*8kB 31*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2388kB [ 268.610441] 2297 total pagecache pages [ 268.610644] 0 pages in swap cache [ 268.610846] Swap cache stats: add 0, delete 0, find 0/0 [ 268.611104] Free swap = 0kB [ 268.611275] Total swap = 0kB [ 268.612057] 65520 pages RAM [ 268.612227] 16492 pages reserved [ 268.612410] 2213 pages shared [ 268.612585] 46407 pages non-shared [ 268.612776] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 268.613160] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 268.613551] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 268.613944] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 268.614338] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 268.614747] [ 2754] 0 2754 3460 289 1 0 0 dd [ 268.615133] [ 2755] 0 2755 3460 289 2 0 0 dd [ 268.615516] [ 2756] 0 2756 3460 290 2 0 0 dd [ 268.615901] [ 2757] 0 2757 3460 290 1 0 0 dd [ 268.616285] [ 2758] 0 2758 3460 290 2 0 0 dd [ 268.616667] [ 2759] 0 2759 3438 197 0 0 0 plot-written.sh [ 268.617098] [ 2776] 0 2776 2144 101 0 0 0 iostat [ 268.617497] [10030] 0 10030 1494 26 1 0 0 getty [ 268.617893] [10034] 0 10034 1494 26 0 0 0 getty [ 268.618290] [10035] 0 10035 1494 26 4 0 0 getty [ 268.618684] [10036] 0 10036 1494 26 5 0 0 getty [ 268.619087] [10038] 0 10038 1494 27 1 0 0 getty [ 268.620052] [10041] 0 10041 1494 29 4 0 0 getty [ 268.620446] [10627] 0 10627 3202 133 4 0 0 date [ 268.620840] Out of memory: Kill process 2754 (dd) score 1 or sacrifice child [ 268.621188] Killed process 2754 (dd) total-vm:13840kB, anon-rss:1136kB, file-rss:20kB [ 275.637877] wc invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 [ 275.638373] wc cpuset=/ mems_allowed=0 [ 275.638580] Pid: 10781, comm: wc Not tainted 2.6.37-rc3 #154 [ 275.638858] Call Trace: [ 275.639016] [] ? _raw_spin_unlock+0x2b/0x40 [ 275.639302] [] dump_header+0x83/0x200 [ 275.639567] [] ? trace_hardirqs_on+0xd/0x10 [ 275.639852] [] ? ___ratelimit+0xab/0x150 [ 275.640120] [] ? oom_badness+0x77/0x190 [ 275.640391] [] oom_kill_process+0x8d/0x270 [ 275.640666] [] out_of_memory+0x1d8/0x410 [ 275.640941] [] __alloc_pages_nodemask+0x81b/0x830 [ 275.641239] [] alloc_pages_current+0x99/0x110 [ 275.641530] [] __page_cache_alloc+0x87/0x90 [ 275.641809] [] __do_page_cache_readahead+0x120/0x260 [ 275.642192] [] ? __do_page_cache_readahead+0xad/0x260 [ 275.642647] [] ? lock_release_holdtime+0x3d/0x180 [ 275.642951] [] ra_submit+0x21/0x30 [ 275.643200] [] filemap_fault+0x4dc/0x4f0 [ 275.643475] [] ? __lock_acquire+0x53e/0x1e70 [ 275.643759] [] __do_fault+0x54/0x540 [ 275.644020] [] handle_mm_fault+0x1af/0xc20 [ 275.644300] [] ? do_page_fault+0xf6/0x560 [ 275.644576] [] ? lock_release_holdtime+0x3d/0x180 [ 275.644879] [] do_page_fault+0x16b/0x560 [ 275.645153] [] ? lock_release_holdtime+0x3d/0x180 [ 275.645458] [] ? error_sti+0x5/0x6 [ 275.645714] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 275.646100] [] page_fault+0x25/0x30 [ 275.646366] Mem-Info: [ 275.646520] Node 0 DMA per-cpu: [ 275.646740] CPU 0: hi: 0, btch: 1 usd: 0 [ 275.646979] CPU 1: hi: 0, btch: 1 usd: 0 [ 275.647223] CPU 2: hi: 0, btch: 1 usd: 0 [ 275.647463] CPU 3: hi: 0, btch: 1 usd: 0 [ 275.647708] CPU 4: hi: 0, btch: 1 usd: 0 [ 275.647952] CPU 5: hi: 0, btch: 1 usd: 0 [ 275.648196] CPU 6: hi: 0, btch: 1 usd: 0 [ 275.648434] CPU 7: hi: 0, btch: 1 usd: 0 [ 275.648680] Node 0 DMA32 per-cpu: [ 275.648904] CPU 0: hi: 90, btch: 15 usd: 19 [ 275.649148] CPU 1: hi: 90, btch: 15 usd: 0 [ 275.649393] CPU 2: hi: 90, btch: 15 usd: 0 [ 275.649632] CPU 3: hi: 90, btch: 15 usd: 0 [ 275.649877] CPU 4: hi: 90, btch: 15 usd: 38 [ 275.650401] CPU 5: hi: 90, btch: 15 usd: 0 [ 275.650641] CPU 6: hi: 90, btch: 15 usd: 0 [ 275.650888] CPU 7: hi: 90, btch: 15 usd: 0 [ 275.651134] active_anon:1962 inactive_anon:21 isolated_anon:0 [ 275.651134] active_file:213 inactive_file:2025 isolated_file:0 [ 275.651135] unevictable:0 dirty:0 writeback:379 unstable:1 [ 275.651136] free:715 slab_reclaimable:14999 slab_unreclaimable:5127 [ 275.651136] mapped:160 shmem:75 pagetables:185 bounce:0 [ 275.652514] Node 0 DMA free:1044kB min:120kB low:148kB high:180kB active_anon:364kB inactive_anon:0kB active_file:4kB inactive_file:556kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:13396kB slab_unreclaimable:252kB kernel_stack:8kB pagetables:84kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:888 all_unreclaimable? yes [ 275.654231] lowmem_reserve[]: 0 236 236 236 [ 275.654638] Node 0 DMA32 free:1816kB min:1904kB low:2380kB high:2856kB active_anon:7484kB inactive_anon:84kB active_file:848kB inactive_file:7544kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:0kB writeback:1516kB mapped:640kB shmem:300kB slab_reclaimable:46600kB slab_unreclaimable:20256kB kernel_stack:1056kB pagetables:656kB unstable:4kB bounce:0kB writeback_tmp:0kB pages_scanned:16113 all_unreclaimable? yes [ 275.656374] lowmem_reserve[]: 0 0 0 0 [ 275.656745] Node 0 DMA: 0*4kB 73*8kB 31*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1080kB [ 275.657696] Node 0 DMA32: 125*4kB 115*8kB 27*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1884kB [ 275.658920] 2336 total pagecache pages [ 275.659127] 0 pages in swap cache [ 275.659312] Swap cache stats: add 0, delete 0, find 0/0 [ 275.659572] Free swap = 0kB [ 275.659747] Total swap = 0kB [ 275.660536] 65520 pages RAM [ 275.660703] 16492 pages reserved [ 275.660891] 1796 pages shared [ 275.661063] 46520 pages non-shared [ 275.661258] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 275.661634] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 275.662087] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 275.662473] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 275.662907] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 275.663291] [ 2755] 0 2755 3460 289 7 0 0 dd [ 275.663657] [ 2756] 0 2756 3460 290 7 0 0 dd [ 275.664022] [ 2757] 0 2757 3460 290 2 0 0 dd [ 275.664387] [ 2758] 0 2758 3460 290 4 0 0 dd [ 275.664757] [ 2759] 0 2759 3438 196 4 0 0 plot-written.sh [ 275.665167] [ 2776] 0 2776 2145 97 6 0 0 iostat [ 275.665548] [10030] 0 10030 1494 26 1 0 0 getty [ 275.666208] [10034] 0 10034 1494 26 0 0 0 getty [ 275.666586] [10035] 0 10035 1494 26 4 0 0 getty [ 275.666962] [10036] 0 10036 1494 26 5 0 0 getty [ 275.667337] [10038] 0 10038 1494 27 1 0 0 getty [ 275.667712] [10041] 0 10041 1494 29 4 0 0 getty [ 275.668087] [10779] 0 10779 3438 150 0 0 0 plot-written.sh [ 275.668493] [10781] 0 10781 2137 77 0 0 0 wc [ 275.668859] Out of memory: Kill process 2755 (dd) score 1 or sacrifice child [ 275.669186] Killed process 2755 (dd) total-vm:13840kB, anon-rss:1136kB, file-rss:20kB [ 282.955149] iostat invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 [ 282.955563] iostat cpuset=/ mems_allowed=0 [ 282.955778] Pid: 2776, comm: iostat Not tainted 2.6.37-rc3 #154 [ 282.956061] Call Trace: [ 282.956218] [] ? _raw_spin_unlock+0x2b/0x40 [ 282.956499] [] dump_header+0x83/0x200 [ 282.956760] [] ? trace_hardirqs_on+0xd/0x10 [ 282.957039] [] ? ___ratelimit+0xab/0x150 [ 282.957308] [] ? oom_badness+0x77/0x190 [ 282.957596] [] oom_kill_process+0x8d/0x270 [ 282.957871] [] out_of_memory+0x1d8/0x410 [ 282.958142] [] __alloc_pages_nodemask+0x81b/0x830 [ 282.958442] [] alloc_pages_current+0x99/0x110 [ 282.958728] [] __page_cache_alloc+0x87/0x90 [ 282.959010] [] __do_page_cache_readahead+0x120/0x260 [ 282.959319] [] ? __do_page_cache_readahead+0xad/0x260 [ 282.959633] [] ? lock_release_holdtime+0x3d/0x180 [ 282.959935] [] ra_submit+0x21/0x30 [ 282.960185] [] filemap_fault+0x4dc/0x4f0 [ 282.960455] [] ? __lock_acquire+0x53e/0x1e70 [ 282.960738] [] __do_fault+0x54/0x540 [ 282.960994] [] handle_mm_fault+0x1af/0xc20 [ 282.961272] [] ? do_page_fault+0xf6/0x560 [ 282.961553] [] ? lock_release_holdtime+0x3d/0x180 [ 282.961853] [] do_page_fault+0x16b/0x560 [ 282.962123] [] ? _raw_spin_unlock+0x2b/0x40 [ 282.962405] [] ? error_sti+0x5/0x6 [ 282.962658] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 282.962965] [] page_fault+0x25/0x30 [ 282.963219] Mem-Info: [ 282.963368] Node 0 DMA per-cpu: [ 282.963584] CPU 0: hi: 0, btch: 1 usd: 0 [ 282.963828] CPU 1: hi: 0, btch: 1 usd: 0 [ 282.964070] CPU 2: hi: 0, btch: 1 usd: 0 [ 282.964313] CPU 3: hi: 0, btch: 1 usd: 0 [ 282.964556] CPU 4: hi: 0, btch: 1 usd: 0 [ 282.964798] CPU 5: hi: 0, btch: 1 usd: 0 [ 282.965042] CPU 6: hi: 0, btch: 1 usd: 0 [ 282.965285] CPU 7: hi: 0, btch: 1 usd: 0 [ 282.965535] Node 0 DMA32 per-cpu: [ 282.965759] CPU 0: hi: 90, btch: 15 usd: 35 [ 282.966001] CPU 1: hi: 90, btch: 15 usd: 14 [ 282.966244] CPU 2: hi: 90, btch: 15 usd: 32 [ 282.966487] CPU 3: hi: 90, btch: 15 usd: 0 [ 282.966730] CPU 4: hi: 90, btch: 15 usd: 14 [ 282.966971] CPU 5: hi: 90, btch: 15 usd: 81 [ 282.967213] CPU 6: hi: 90, btch: 15 usd: 28 [ 282.967457] CPU 7: hi: 90, btch: 15 usd: 0 [ 282.967701] active_anon:1656 inactive_anon:28 isolated_anon:0 [ 282.967702] active_file:36 inactive_file:2139 isolated_file:0 [ 282.967703] unevictable:0 dirty:274 writeback:109 unstable:0 [ 282.967703] free:760 slab_reclaimable:15317 slab_unreclaimable:5085 [ 282.967704] mapped:0 shmem:75 pagetables:181 bounce:0 [ 282.969672] Node 0 DMA free:1060kB min:120kB low:148kB high:180kB active_anon:324kB inactive_anon:0kB active_file:0kB inactive_file:556kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:140kB writeback:132kB mapped:0kB shmem:0kB slab_reclaimable:13364kB slab_unreclaimable:308kB kernel_stack:8kB pagetables:32kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:886 all_unreclaimable? yes [ 282.971334] lowmem_reserve[]: 0 236 236 236 [ 282.971721] Node 0 DMA32 free:1980kB min:1904kB low:2380kB high:2856kB active_anon:6300kB inactive_anon:112kB active_file:152kB inactive_file:8000kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:956kB writeback:304kB mapped:0kB shmem:300kB slab_reclaimable:47904kB slab_unreclaimable:20032kB kernel_stack:1016kB pagetables:692kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:20046 all_unreclaimable? yes [ 282.973466] lowmem_reserve[]: 0 0 0 0 [ 282.973837] Node 0 DMA: 1*4kB 87*8kB 25*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1132kB [ 282.974773] Node 0 DMA32: 136*4kB 111*8kB 19*16kB 4*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1864kB [ 282.975724] 2323 total pagecache pages [ 282.975927] 0 pages in swap cache [ 282.976115] Swap cache stats: add 0, delete 0, find 0/0 [ 282.976373] Free swap = 0kB [ 282.976544] Total swap = 0kB [ 282.977336] 65520 pages RAM [ 282.977515] 16492 pages reserved [ 282.977702] 1382 pages shared [ 282.977876] 46536 pages non-shared [ 282.978067] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 282.978444] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 282.978819] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 282.979193] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 282.979568] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 282.979939] [ 2756] 0 2756 3460 290 6 0 0 dd [ 282.980305] [ 2757] 0 2757 3460 290 1 0 0 dd [ 282.980670] [ 2758] 0 2758 3460 290 2 0 0 dd [ 282.981036] [ 2759] 0 2759 3438 79 4 0 0 plot-written.sh [ 282.981449] [ 2776] 0 2776 2145 66 3 0 0 iostat [ 282.981826] [10030] 0 10030 1494 26 1 0 0 getty [ 282.982197] [10034] 0 10034 1494 26 0 0 0 getty [ 282.982569] [10035] 0 10035 1494 26 4 0 0 getty [ 282.982939] [10036] 0 10036 1494 26 5 0 0 getty [ 282.983311] [10038] 0 10038 1494 27 1 0 0 getty [ 282.983683] [10041] 0 10041 1494 27 4 0 0 getty [ 282.984054] [10932] 0 10932 3438 78 5 0 0 plot-written.sh [ 282.984459] Out of memory: Kill process 2756 (dd) score 1 or sacrifice child [ 282.984782] Killed process 2756 (dd) total-vm:13840kB, anon-rss:1140kB, file-rss:20kB [ 282.985208] dd: page allocation failure. order:0, mode:0x2005a [ 282.985507] Pid: 2756, comm: dd Not tainted 2.6.37-rc3 #154 [ 282.985785] Call Trace: [ 282.985948] [] __alloc_pages_nodemask+0x680/0x830 [ 282.986261] [] alloc_pages_current+0x99/0x110 [ 282.986558] [] __page_cache_alloc+0x87/0x90 [ 282.986847] [] grab_cache_page_write_begin+0x7f/0xc0 [ 282.987166] [] ext4_da_write_begin+0x143/0x280 [ 282.987465] [] generic_file_buffered_write+0x109/0x260 [ 282.987790] [] ? ext4_dirty_inode+0x54/0x60 [ 282.988081] [] ? mutex_lock_nested+0x280/0x350 [ 282.988380] [] __generic_file_aio_write+0x244/0x450 [ 282.988695] [] ? mutex_lock_nested+0x292/0x350 [ 282.988993] [] ? generic_file_aio_write+0x58/0xd0 [ 282.989301] [] ? generic_file_aio_write+0x58/0xd0 [ 282.989624] [] generic_file_aio_write+0x6b/0xd0 [ 282.989928] [] ext4_file_write+0x42/0xc0 [ 282.990209] [] do_sync_write+0xda/0x120 [ 282.990485] [] ? might_fault+0xa5/0xb0 [ 282.990757] [] ? read_zero+0xe5/0x120 [ 282.991027] [] vfs_write+0xce/0x190 [ 282.991289] [] sys_write+0x54/0x90 [ 282.991549] [] system_call_fastpath+0x16/0x1b [ 282.991843] Mem-Info: [ 282.991996] Node 0 DMA per-cpu: [ 282.992219] CPU 0: hi: 0, btch: 1 usd: 0 [ 282.992468] CPU 1: hi: 0, btch: 1 usd: 0 [ 282.992716] CPU 2: hi: 0, btch: 1 usd: 0 [ 282.992964] CPU 3: hi: 0, btch: 1 usd: 0 [ 282.993213] CPU 4: hi: 0, btch: 1 usd: 0 [ 282.993470] CPU 5: hi: 0, btch: 1 usd: 0 [ 282.993719] CPU 6: hi: 0, btch: 1 usd: 0 [ 282.993968] CPU 7: hi: 0, btch: 1 usd: 0 [ 282.994216] Node 0 DMA32 per-cpu: [ 282.994447] CPU 0: hi: 90, btch: 15 usd: 36 [ 282.994697] CPU 1: hi: 90, btch: 15 usd: 14 [ 282.994946] CPU 2: hi: 90, btch: 15 usd: 32 [ 282.995196] CPU 3: hi: 90, btch: 15 usd: 0 [ 282.995445] CPU 4: hi: 90, btch: 15 usd: 14 [ 282.995693] CPU 5: hi: 90, btch: 15 usd: 76 [ 282.995942] CPU 6: hi: 90, btch: 15 usd: 28 [ 282.996192] CPU 7: hi: 90, btch: 15 usd: 1 [ 282.996444] active_anon:1656 inactive_anon:28 isolated_anon:0 [ 282.996445] active_file:36 inactive_file:2139 isolated_file:0 [ 282.996446] unevictable:0 dirty:274 writeback:0 unstable:0 [ 282.996447] free:760 slab_reclaimable:15300 slab_unreclaimable:5085 [ 282.996448] mapped:0 shmem:75 pagetables:181 bounce:0 [ 282.997866] Node 0 DMA free:1060kB min:120kB low:148kB high:180kB active_anon:324kB inactive_anon:0kB active_file:0kB inactive_file:556kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:140kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:13364kB slab_unreclaimable:308kB kernel_stack:8kB pagetables:32kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:886 all_unreclaimable? yes [ 282.999565] lowmem_reserve[]: 0 236 236 236 [ 282.999969] Node 0 DMA32 free:1980kB min:1904kB low:2380kB high:2856kB active_anon:6300kB inactive_anon:112kB active_file:152kB inactive_file:8000kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:956kB writeback:0kB mapped:0kB shmem:300kB slab_reclaimable:47836kB slab_unreclaimable:20032kB kernel_stack:1016kB pagetables:692kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:20046 all_unreclaimable? yes [ 283.001746] lowmem_reserve[]: 0 0 0 0 [ 283.002132] Node 0 DMA: 1*4kB 87*8kB 25*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1132kB [ 283.003104] Node 0 DMA32: 135*4kB 104*8kB 19*16kB 4*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1804kB [ 283.004090] 2323 total pagecache pages [ 283.004298] 0 pages in swap cache [ 283.004490] Swap cache stats: add 0, delete 0, find 0/0 [ 283.004755] Free swap = 0kB [ 283.004930] Total swap = 0kB [ 283.005666] 65520 pages RAM [ 283.005841] 16492 pages reserved [ 283.006030] 1242 pages shared [ 283.006209] 46544 pages non-shared [ 293.931092] iostat invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0, oom_score_adj=0 [ 293.931515] iostat cpuset=/ mems_allowed=0 [ 293.931739] Pid: 2776, comm: iostat Not tainted 2.6.37-rc3 #154 [ 293.932058] Call Trace: [ 293.932219] [] ? _raw_spin_unlock+0x2b/0x40 [ 293.932536] [] dump_header+0x83/0x200 [ 293.932804] [] ? trace_hardirqs_on+0xd/0x10 [ 293.933149] [] ? ___ratelimit+0xab/0x150 [ 293.933426] [] ? oom_badness+0x77/0x190 [ 293.933729] [] oom_kill_process+0x8d/0x270 [ 293.934038] [] out_of_memory+0x1d8/0x410 [ 293.934316] [] __alloc_pages_nodemask+0x81b/0x830 [ 293.934634] [] alloc_pages_current+0x99/0x110 [ 293.934927] [] __page_cache_alloc+0x87/0x90 [ 293.935214] [] __do_page_cache_readahead+0x120/0x260 [ 293.935529] [] ? __do_page_cache_readahead+0xad/0x260 [ 293.935849] [] ? lock_release_holdtime+0x3d/0x180 [ 293.936155] [] ra_submit+0x21/0x30 [ 293.936414] [] filemap_fault+0x4dc/0x4f0 [ 293.936693] [] ? __lock_acquire+0x53e/0x1e70 [ 293.936984] [] __do_fault+0x54/0x540 [ 293.937250] [] handle_mm_fault+0x1af/0xc20 [ 293.937536] [] ? do_page_fault+0xf6/0x560 [ 293.937815] [] ? lock_release_holdtime+0x3d/0x180 [ 293.938120] [] do_page_fault+0x16b/0x560 [ 293.938397] [] ? _raw_spin_unlock+0x2b/0x40 [ 293.938698] [] ? error_sti+0x5/0x6 [ 293.938957] [] ? trace_hardirqs_off_thunk+0x3a/0x3c [ 293.939269] [] page_fault+0x25/0x30 [ 293.939528] Mem-Info: [ 293.939680] Node 0 DMA per-cpu: [ 293.939898] CPU 0: hi: 0, btch: 1 usd: 0 [ 293.940146] CPU 1: hi: 0, btch: 1 usd: 0 [ 293.940394] CPU 2: hi: 0, btch: 1 usd: 0 [ 293.940643] CPU 3: hi: 0, btch: 1 usd: 0 [ 293.940891] CPU 4: hi: 0, btch: 1 usd: 0 [ 293.941140] CPU 5: hi: 0, btch: 1 usd: 0 [ 293.941389] CPU 6: hi: 0, btch: 1 usd: 0 [ 293.942212] CPU 7: hi: 0, btch: 1 usd: 0 [ 293.942461] Node 0 DMA32 per-cpu: [ 293.942697] CPU 0: hi: 90, btch: 15 usd: 19 [ 293.942946] CPU 1: hi: 90, btch: 15 usd: 0 [ 293.943194] CPU 2: hi: 90, btch: 15 usd: 27 [ 293.943442] CPU 3: hi: 90, btch: 15 usd: 0 [ 293.943690] CPU 4: hi: 90, btch: 15 usd: 83 [ 293.943939] CPU 5: hi: 90, btch: 15 usd: 14 [ 293.944187] CPU 6: hi: 90, btch: 15 usd: 0 [ 293.944435] CPU 7: hi: 90, btch: 15 usd: 0 [ 293.944685] active_anon:1452 inactive_anon:23 isolated_anon:0 [ 293.944686] active_file:257 inactive_file:1826 isolated_file:0 [ 293.944687] unevictable:0 dirty:25 writeback:145 unstable:0 [ 293.944688] free:737 slab_reclaimable:15674 slab_unreclaimable:5093 [ 293.944689] mapped:161 shmem:75 pagetables:141 bounce:0 [ 293.946109] Node 0 DMA free:1040kB min:120kB low:148kB high:180kB active_anon:316kB inactive_anon:0kB active_file:4kB inactive_file:592kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:13412kB slab_unreclaimable:260kB kernel_stack:8kB pagetables:32kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:977 all_unreclaimable? yes [ 293.947801] lowmem_reserve[]: 0 236 236 236 [ 293.948206] Node 0 DMA32 free:1908kB min:1904kB low:2380kB high:2856kB active_anon:5492kB inactive_anon:92kB active_file:1024kB inactive_file:6712kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:100kB writeback:584kB mapped:644kB shmem:300kB slab_reclaimable:49284kB slab_unreclaimable:20112kB kernel_stack:1024kB pagetables:532kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:13132 all_unreclaimable? yes [ 293.949987] lowmem_reserve[]: 0 0 0 0 [ 293.950371] Node 0 DMA: 18*4kB 46*8kB 42*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1112kB [ 293.951354] Node 0 DMA32: 131*4kB 100*8kB 30*16kB 2*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1868kB [ 293.952338] 2303 total pagecache pages [ 293.952547] 0 pages in swap cache [ 293.952739] Swap cache stats: add 0, delete 0, find 0/0 [ 293.953004] Free swap = 0kB [ 293.953179] Total swap = 0kB [ 293.954046] 65520 pages RAM [ 293.954219] 16492 pages reserved [ 293.954409] 1590 pages shared [ 293.954599] 46579 pages non-shared [ 293.954796] [ pid ] uid tgid total_vm rss cpu oom_adj oom_score_adj name [ 293.955182] [ 1995] 0 1995 4237 150 1 -17 -1000 udevd [ 293.955566] [ 2082] 0 2082 4264 162 2 -17 -1000 udevd [ 293.955950] [ 2083] 0 2083 4264 153 2 -17 -1000 udevd [ 293.956334] [ 2269] 0 2269 12303 138 5 -17 -1000 sshd [ 293.956714] [ 2757] 0 2757 3460 290 6 0 0 dd [ 293.957087] [ 2758] 0 2758 3460 290 1 0 0 dd [ 293.957457] [ 2759] 0 2759 3438 198 1 0 0 plot-written.sh [ 293.957871] [ 2776] 0 2776 2145 96 2 0 0 iostat [ 293.958254] [10030] 0 10030 1494 26 1 0 0 getty [ 293.958642] [10034] 0 10034 1494 26 0 0 0 getty [ 293.959022] [10035] 0 10035 1494 26 4 0 0 getty [ 293.959405] [10036] 0 10036 1494 26 5 0 0 getty [ 293.959786] [10038] 0 10038 1494 27 1 0 0 getty [ 293.960170] [10041] 0 10041 1494 27 4 0 0 getty [ 293.960555] [11259] 0 11259 3438 164 3 0 0 plot-written.sh [ 293.960972] [11260] 0 11260 3438 112 3 0 0 plot-written.sh [ 293.961388] Out of memory: Kill process 2757 (dd) score 1 or sacrifice child [ 293.961723] Killed process 2757 (dd) total-vm:13840kB, anon-rss:1140kB, file-rss:20kB [ 293.962146] dd: page allocation failure. order:0, mode:0x2005a [ 293.962430] Pid: 2757, comm: dd Not tainted 2.6.37-rc3 #154 [ 293.962722] Call Trace: [ 293.962887] [] __alloc_pages_nodemask+0x680/0x830 [ 293.963197] [] alloc_pages_current+0x99/0x110 [ 293.963493] [] __page_cache_alloc+0x87/0x90 [ 293.963782] [] grab_cache_page_write_begin+0x7f/0xc0 [ 293.964102] [] ext4_da_write_begin+0x143/0x280 [ 293.964402] [] generic_file_buffered_write+0x109/0x260 [ 293.964727] [] ? ext4_dirty_inode+0x54/0x60 [ 293.965017] [] ? mutex_lock_nested+0x280/0x350 [ 293.965316] [] __generic_file_aio_write+0x244/0x450 [ 293.965631] [] ? mutex_lock_nested+0x292/0x350 [ 293.965931] [] ? generic_file_aio_write+0x58/0xd0 [ 293.966240] [] ? generic_file_aio_write+0x58/0xd0 [ 293.966557] [] generic_file_aio_write+0x6b/0xd0 [ 293.966880] [] ext4_file_write+0x42/0xc0 [ 293.967160] [] do_sync_write+0xda/0x120 [ 293.967436] [] ? might_fault+0xa5/0xb0 [ 293.967710] [] ? read_zero+0xe5/0x120 [ 293.967979] [] ? rw_verify_area+0x19/0xb0 [ 293.968262] [] vfs_write+0xce/0x190 [ 293.968524] [] sys_write+0x54/0x90 [ 293.968784] [] system_call_fastpath+0x16/0x1b [ 293.969078] Mem-Info: [ 293.969232] Node 0 DMA per-cpu: [ 293.969454] CPU 0: hi: 0, btch: 1 usd: 0 [ 293.969704] CPU 1: hi: 0, btch: 1 usd: 0 [ 293.969952] CPU 2: hi: 0, btch: 1 usd: 0 [ 293.970200] CPU 3: hi: 0, btch: 1 usd: 0 [ 293.970449] CPU 4: hi: 0, btch: 1 usd: 0 [ 293.970708] CPU 5: hi: 0, btch: 1 usd: 0 [ 293.970958] CPU 6: hi: 0, btch: 1 usd: 0 [ 293.971206] CPU 7: hi: 0, btch: 1 usd: 0 [ 293.971450] Node 0 DMA32 per-cpu: [ 293.971674] CPU 0: hi: 90, btch: 15 usd: 19 [ 293.971916] CPU 1: hi: 90, btch: 15 usd: 0 [ 293.972158] CPU 2: hi: 90, btch: 15 usd: 27 [ 293.972401] CPU 3: hi: 90, btch: 15 usd: 14 [ 293.972643] CPU 4: hi: 90, btch: 15 usd: 83 [ 293.972885] CPU 5: hi: 90, btch: 15 usd: 14 [ 293.973128] CPU 6: hi: 90, btch: 15 usd: 0 [ 293.973371] CPU 7: hi: 90, btch: 15 usd: 0 [ 293.973615] active_anon:1452 inactive_anon:23 isolated_anon:0 [ 293.973616] active_file:257 inactive_file:1826 isolated_file:0 [ 293.973616] unevictable:0 dirty:25 writeback:0 unstable:0 [ 293.973617] free:737 slab_reclaimable:15674 slab_unreclaimable:5093 [ 293.973617] mapped:161 shmem:75 pagetables:141 bounce:0 [ 293.975022] Node 0 DMA free:1040kB min:120kB low:148kB high:180kB active_anon:316kB inactive_anon:0kB active_file:4kB inactive_file:592kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15660kB mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_reclaimable:13412kB slab_unreclaimable:260kB kernel_stack:8kB pagetables:32kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:977 all_unreclaimable? yes [ 293.976677] lowmem_reserve[]: 0 236 236 236 [ 293.977069] Node 0 DMA32 free:1908kB min:1904kB low:2380kB high:2856kB active_anon:5492kB inactive_anon:92kB active_file:1024kB inactive_file:6712kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:242400kB mlocked:0kB dirty:100kB writeback:0kB mapped:644kB shmem:300kB slab_reclaimable:49284kB slab_unreclaimable:20112kB kernel_stack:1024kB pagetables:532kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:13132 all_unreclaimable? yes [ 293.978822] lowmem_reserve[]: 0 0 0 0 [ 293.979200] Node 0 DMA: 18*4kB 46*8kB 42*16kB 0*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1112kB [ 293.980138] Node 0 DMA32: 116*4kB 100*8kB 30*16kB 2*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1808kB [ 293.981088] 2303 total pagecache pages [ 293.981292] 0 pages in swap cache [ 293.981480] Swap cache stats: add 0, delete 0, find 0/0 [ 293.981738] Free swap = 0kB [ 293.981910] Total swap = 0kB [ 293.982420] 65520 pages RAM [ 293.982599] 16492 pages reserved [ 293.982790] 1432 pages shared [ 293.982970] 46580 pages non-shared [ 293.983878] dd used greatest stack depth: 2400 bytes left