Re: 2.6.23-mm1

Torsten Kaiser
2007-10-14T11:54:41Z

[-- Attachment #1: Type: text/plain, Size: 23671 bytes --]

On 10/13/07, Andrew Morton <akpm@linux-foundation.org> wrote:
> Please send /proc/meminfo and /proc/slabinfo after the leak has been
> happening for a while.
>
> Sometimes `echo m > /proc/sysrq_trigger ; dmesg -s 1000000' will
> provide useful info.

As I'm using SLUB there is no /proc/slabinfo.

I have attached any files that looked remotely related.
After I have seen ~2Gb leaked again, I took the first set of outputs.
atop showed at that point:
MEM | tot 3.9G | free 648.7M | cache 814.5M | buff 0.0M | slab 131.5M |
SWP | tot 9.3G | free 9.3G | | vmcom 16.7M | vmlim 11.3G |

free showed:
total used free shared buffers cached
Mem: 4061808 3396852 664956 0 28 834012
-/+ buffers/cache: 2562812 1498996
Swap: 9775416 60 9775356

I then tried to build 2.6.23-mm1 with PAGE_OWNER enabled. During
modpost the system started swapping like mad, I needed to abort the
build, but was still able to shut down the system normal.
At that point I took the secound set of outputs (files called *.end)
free showed at that time:
total used free shared buffers cached
Mem: 4061808 3777084 284724 0 0 2032
-/+ buffers/cache: 3775052 286756
Swap: 9775416 4904 9770512

The slubinfo-files are the output of Documentation/vm/slabinfo.c

> The page-owner code can pinpoint a leak source. See
> ftp://ftp.kernel.org/pub/linux/kernel/people/akpm/patches/2.6/2.6.23/2.6.23-mm1/broken-out/page-owner-tracking-leak-detector.patch
>
> Enable CONFIG_DEBUG_SLAB_LEAK, check out /proc/slab_allocators

Did that. The output of /proc/page_owner is ~350Mb, gzipped still ~7Mb.

Taking only the first line from each stackdump it shows the following counts:
73 [0xffffffff8020a13e] __switch_to+430
6 [0xffffffff8020a236] __switch_to+678
8 [0xffffffff8020ac30] default_idle+0
1 [0xffffffff8020ba3f] sys_rt_sigreturn+879
3 [0xffffffff8020bbbe] system_call+126
115 [0xffffffff802132e1] dma_alloc_pages+177
1 [0xffffffff802191fb] __smp_call_function_mask+235
1 [0xffffffff8021970b] flush_tlb_page+75
8 [0xffffffff8021c1b7] mp_register_gsi+71
3 [0xffffffff8021c25b] mp_register_gsi+235
26 [0xffffffff8021ec6d] flush_gart+13
2 [0xffffffff8021ec7f] flush_gart+31
28 [0xffffffff8021ecda] gart_map_simple+58
1 [0xffffffff8021f048] gart_map_sg+680
15 [0xffffffff8021f2af] k8_flush_garts+191
4 [0xffffffff8022078b] do_page_fault+683
1 [0xffffffff80220cc8] ioremap_change_attr+88
1 [0xffffffff80220db1] __ioremap+225
2 [0xffffffff8022123d] __change_page_attr+525
1 [0xffffffff80222d2e] twofish_dec_blk+1557
20 [0xffffffff80227e15] load_elf32_binary+181
123 [0xffffffff8022c223] enqueue_task+19
2 [0xffffffff8022c2a2] activate_task+50
31 [0xffffffff8022c3f4] source_load+52
54 [0xffffffff8022c464] target_load+52
25 [0xffffffff8022c5b6] move_tasks+118
4 [0xffffffff8022c684] __wake_up_common+20
23 [0xffffffff8022c6ca] __wake_up_common+90
38 [0xffffffff8022cb31] update_curr+97
5 [0xffffffff8022cb78] update_curr+168
1 [0xffffffff8022cd35] balance_tasks+117
12 [0xffffffff8022cde3] balance_tasks+291
2 [0xffffffff8022cee8] load_balance_fair+88
3 [0xffffffff8022cff0] load_balance_next_fair+0
137 [0xffffffff8022d04d] __dequeue_entity+61
4 [0xffffffff8022d083] set_next_entity+35
1101 [0xffffffff8022d0f5] pick_next_task_fair+69
319 [0xffffffff8022d3e3] __wake_up+67
6 [0xffffffff8022d4bc] task_rq_lock+76
17 [0xffffffff8022d693] try_to_wake_up+99
211 [0xffffffff8022d910] default_wake_function+0
575 [0xffffffff8022df58] find_busiest_group+472
1 [0xffffffff80230844] finish_task_switch+164
217 [0xffffffff802318f4] enqueue_task_fair+132
335 [0xffffffff80231981] dequeue_task_fair+81
8 [0xffffffff80231ca4] check_preempt_wakeup+212
6 [0xffffffff80231cba] check_preempt_wakeup+234
2 [0xffffffff8023252c] dup_fd+44
31 [0xffffffff8023273b] dup_fd+571
1 [0xffffffff80233393] copy_process+1811
1 [0xffffffff8023549e] release_console_sem+494
5 [0xffffffff80235b0e] printk+78
1 [0xffffffff80237dd6] do_wait+1558
1 [0xffffffff8023a2a6] on_each_cpu+38
5 [0xffffffff8023e844] lock_timer_base+52
1 [0xffffffff8023e8d2] try_to_del_timer_sync+82
5 [0xffffffff8023e8f0] del_timer_sync+16
10 [0xffffffff8023ea12] __mod_timer+194
65 [0xffffffff8023ec4d] mod_timer+29
1 [0xffffffff8023ecee] del_timer+110
2 [0xffffffff8023ff98] __dequeue_signal+344
1 [0xffffffff8024003e] signal_wake_up+30
3 [0xffffffff80242090] sys_rt_sigprocmask+128
2 [0xffffffff802457f9] call_usermodehelper_exec+153
525 [0xffffffff802462c0] __queue_work+48
28 [0xffffffff80246347] queue_work+55
4 [0xffffffff802467b0] worker_thread+0
1 [0xffffffff80246845] worker_thread+149
132 [0xffffffff80247e99] __kernel_text_address+9
151 [0xffffffff80247eaa] __kernel_text_address+26
62 [0xffffffff8024a2c0] bit_waitqueue+16
23 [0xffffffff8024a398] __wake_up_bit+40
24 [0xffffffff8024a3b8] wake_up_bit+24
254 [0xffffffff8024a3e0] autoremove_wake_function+0
106 [0xffffffff8024a3e9] autoremove_wake_function+9
1 [0xffffffff8024a6b9] remove_wait_queue+41
2 [0xffffffff8024f245] sysctl_check_lookup+197
4 [0xffffffff8024f554] sysctl_check_table+756
1 [0xffffffff8024f5bc] sysctl_check_table+860
1 [0xffffffff80254603] get_futex_key+83
21 [0xffffffff8025595c] futex_wake+204
6 [0xffffffff80257eeb] __module_text_address+91
1221 [0xffffffff80263734] file_read_actor+324
89 [0xffffffff80263821] find_get_page+33
1 [0xffffffff80263850] find_get_page+80
2009 [0xffffffff802638bf] __lock_page+95
57 [0xffffffff802638e0] sync_page+0
1 [0xffffffff80263a92] find_lock_page+2
206 [0xffffffff80263ab9] find_lock_page+41
1 [0xffffffff80263b03] find_lock_page+115
13762 [0xffffffff80263cda] add_to_page_cache+186
33836 [0xffffffff80263d93] __grab_cache_page+99
4 [0xffffffff802640f9] do_generic_mapping_read+489
7431 [0xffffffff802641b2] do_generic_mapping_read+674
1 [0xffffffff8026421d] do_generic_mapping_read+781
678 [0xffffffff802643a6] find_or_create_page+54
12884 [0xffffffff802643b6] find_or_create_page+70
1 [0xffffffff80265bbe] generic_file_aio_read+254
122 [0xffffffff80265e13] filemap_fault+419
227 [0xffffffff80266240] mempool_free_slab+0
354042 [0xffffffff80266373] mempool_alloc+83
113 [0xffffffff802689e8] __rmqueue_smallest+216
76 [0xffffffff80268a39] __rmqueue_smallest+297
3 [0xffffffff80269043] free_pages_bulk+611
6 [0xffffffff80269253] __pagevec_free+35
249 [0xffffffff802693fe] __get_free_pages+14
2002 [0xffffffff80269457] get_zeroed_page+23
221 [0xffffffff802694bf] __rmqueue+47
1 [0xffffffff802695dc] __rmqueue+332
40 [0xffffffff8026977e] rmqueue_bulk+110
41 [0xffffffff80269b30] get_page_from_freelist+896
7323 [0xffffffff80269b3b] get_page_from_freelist+907
2 [0xffffffff80269d13] get_page_from_freelist+1379
11 [0xffffffff80269d26] get_page_from_freelist+1398
6 [0xffffffff80269eb2] __alloc_pages+2
1 [0xffffffff80269eba] __alloc_pages+10
1 [0xffffffff80269ec2] __alloc_pages+18
1 [0xffffffff8026a0fe] __alloc_pages+590
24 [0xffffffff8026a205] __alloc_pages+853
105887 [0xffffffff8026c246] __do_page_cache_readahead+230
15 [0xffffffff8026ca6f] release_pages+367
6 [0xffffffff8026cb84] __pagevec_lru_add_active+212
638 [0xffffffff8026cc79] __pagevec_lru_add+201
1 [0xffffffff8026ccf4] activate_page+100
165 [0xffffffff8026cd3f] activate_page+175
2 [0xffffffff8026d07b] mark_page_accessed+43
78 [0xffffffff8026d1d7] pagevec_lookup+23
79 [0xffffffff8026d9a8] truncate_inode_pages_range+392
128 [0xffffffff80270f62] zone_statistics+66
2426 [0xffffffff80270f9d] zone_statistics+125
236 [0xffffffff80272c60] __do_fault+480
1 [0xffffffff80272c85] __do_fault+517
5233 [0xffffffff80272cb8] __do_fault+568
32 [0xffffffff80272d62] __do_fault+738
4744 [0xffffffff80273233] do_wp_page+211
97 [0xffffffff80273cd5] unmap_vmas+933
1 [0xffffffff802743dc] unmap_mapping_range+140
43 [0xffffffff80274978] handle_mm_fault+456
103412 [0xffffffff80274ae4] handle_mm_fault+820
4 [0xffffffff80274baa] handle_mm_fault+1018
15 [0xffffffff80274d81] handle_mm_fault+1489
2 [0xffffffff80277f6a] unmap_region+250
5 [0xffffffff802783b4] find_extend_vma+36
1 [0xffffffff8027886e] vma_adjust+606
4 [0xffffffff80278ef6] do_munmap+662
732 [0xffffffff80279292] vma_link+130
1 [0xffffffff80279c8e] mmap_region+750
343 [0xffffffff8027cb45] page_dup_rmap+21
3613 [0xffffffff8027db83] __vmalloc_area_node+371
4096 [0xffffffff8027dbc8] __vmalloc_area_node+440
1 [0xffffffff80283348] zonelist_policy+8
8 [0xffffffff8028339a] zonelist_policy+90
1 [0xffffffff802833a2] zonelist_policy+98
3 [0xffffffff802833c3] zonelist_policy+131
1 [0xffffffff802833e3] zonelist_policy+163
12 [0xffffffff802833ef] zonelist_policy+175
1 [0xffffffff802833f8] zonelist_policy+184
1040 [0xffffffff80284256] alloc_page_interleave+86
9 [0xffffffff802842cd] alloc_page_interleave+205
4 [0xffffffff80285411] alloc_pages_current+1
2 [0xffffffff80285423] alloc_pages_current+19
19 [0xffffffff8028542a] alloc_pages_current+26
13 [0xffffffff8028542d] alloc_pages_current+29
3 [0xffffffff80285452] alloc_pages_current+66
4 [0xffffffff8028545b] alloc_pages_current+75
2 [0xffffffff8028545f] alloc_pages_current+79
1 [0xffffffff80285464] alloc_pages_current+84
10948 [0xffffffff80287c7a] shmem_getpage+394
37 [0xffffffff80287d30] shmem_getpage+576
51 [0xffffffff8028a66b] add_partial+43
26 [0xffffffff8028bddc] new_slab+172
26 [0xffffffff8028befa] new_slab+458
41349 [0xffffffff8028bf67] new_slab+567
62 [0xffffffff8028c008] get_partial_node+104
1332 [0xffffffff8028c217] __slab_free+487
157 [0xffffffff8028c4f9] __slab_alloc+441
34 [0xffffffff8028c561] __slab_alloc+545
2 [0xffffffff8028c67f] __slab_alloc+831
341 [0xffffffff8028c697] __slab_alloc+855
4 [0xffffffff8028cbfb] kmem_cache_alloc+155
1 [0xffffffff8028dfbe] sysfs_slab_alias+46
1 [0xffffffff802905d4] filp_close+84
19 [0xffffffff80290687] get_unused_fd_flags+119
1 [0xffffffff80291ae0] vfs_statfs_native+64
1 [0xffffffff80292449] do_sync_write+217
2 [0xffffffff802958c0] chrdev_open+0
2 [0xffffffff802961e7] sys_newlstat+39
15 [0xffffffff802961f6] sys_newlstat+54
1 [0xffffffff80296387] sys_newstat+39
71 [0xffffffff80296396] sys_newstat+54
2 [0xffffffff80298f70] pipe_ioctl+128
11 [0xffffffff802998e6] pipe_write+742
23 [0xffffffff80299e5f] pipe_read+687
1225 [0xffffffff80299eae] pipe_read+766
10 [0xffffffff8029a3f3] generic_permission+99
3 [0xffffffff8029a869] __follow_mount+41
8501 [0xffffffff8029aabf] do_lookup+143
42 [0xffffffff8029c325] deny_write_access+21
12 [0xffffffff8029c3bb] lookup_one_len+91
10 [0xffffffff8029c5d6] __link_path_walk+134
5 [0xffffffff8029c829] __link_path_walk+729
508 [0xffffffff8029d056] __link_path_walk+2822
3 [0xffffffff8029d269] __link_path_walk+3353
486 [0xffffffff8029d460] link_path_walk+128
29 [0xffffffff8029d739] do_path_lookup+137
15 [0xffffffff8029ff94] do_ioctl+132
7 [0xffffffff802a08e1] free_poll_entry+17
1 [0xffffffff802a0c68] do_sys_poll+744
1 [0xffffffff802a14f2] core_sys_select+562
10 [0xffffffff802a1870] __pollwait+0
77 [0xffffffff802a4b21] d_rehash+33
48 [0xffffffff802a4db5] d_kill+53
23 [0xffffffff802a588f] __d_lookup+175
1 [0xffffffff802a5908] __d_lookup+296
472 [0xffffffff802a5ef6] dput+38
8 [0xffffffff802a725f] __iget+79
1 [0xffffffff802a796e] alloc_inode+318
2 [0xffffffff802a7ed4] iget_locked+68
1 [0xffffffff802a863c] new_inode+92
4 [0xffffffff802a88af] touch_atime+31
3 [0xffffffff802a8978] touch_atime+232
1 [0xffffffff802a8e52] notify_change+466
29 [0xffffffff802aa80f] mntput_no_expire+31
76 [0xffffffff802ac485] mnt_drop_write+37
2 [0xffffffff802ac8cd] mnt_want_write+45
7 [0xffffffff802afd0d] seq_open+77
1 [0xffffffff802b33b5] __mark_inode_dirty+101
68796 [0xffffffff802b7d7b] __set_page_dirty+219
1 [0xffffffff802b7e00] mark_buffer_dirty+32
1 [0xffffffff802b7edc] __block_commit_write+92
1 [0xffffffff802b7f08] __block_commit_write+136
1 [0xffffffff802b7f10] __block_commit_write+144
311 [0xffffffff802b7f15] __block_commit_write+149
1 [0xffffffff802b8346] unmap_underlying_metadata+6
3 [0xffffffff802ba04e] generic_write_end+94
2 [0xffffffff802bc347] check_disk_change+39
5 [0xffffffff802bffb0] mpage_end_io_read+0
3 [0xffffffff802d6ed2] proc_alloc_inode+18
1 [0xffffffff802d6f40] pde_users_dec+16
1 [0xffffffff802d7860] proc_reg_open+0
1 [0xffffffff802d94fd] proc_flush_task_mnt+285
132 [0xffffffff802d9b10] proc_fd_instantiate+0
3 [0xffffffff802db798] create_proc_entry+88
1 [0xffffffff802dc1a7] proc_lookup+247
1 [0xffffffff802de5a8] show_stat+1064
1 [0xffffffff802e289a] sysfs_add_file+106
8 [0xffffffff802e2c30] sysfs_ilookup_test+0
2 [0xffffffff802e2ed9] sysfs_find_dirent+41
1 [0xffffffff802e327d] sysfs_addrm_finish+29
1 [0xffffffff802e3641] sysfs_create_dir+49
272 [0xffffffff802e8cc3] nfs_lookup_revalidate+579
1 [0xffffffff802e9d41] nfs_do_access+241
1615 [0xffffffff802ea2f0] nfs_readdir_filler+0
1 [0xffffffff802f9ded] _nfs4_proc_lookupfh+125
14 [0xffffffff802ff415] decode_op_hdr+21
1 [0xffffffff8032190d] xfs_alloc_vextent+637
1 [0xffffffff80321c15] xfs_alloc_log_block+69
5 [0xffffffff80325622] xfs_attr_leaf_get+98
28 [0xffffffff8032575b] xfs_attr_fetch+235
8 [0xffffffff8032bec1] xfs_bmap_last_offset+273
1 [0xffffffff8032c5a5] xfs_bmap_search_multi_extents+117
8 [0xffffffff8032c5c6] xfs_bmap_search_multi_extents+150
92 [0xffffffff8032c5df] xfs_bmap_search_multi_extents+175
569 [0xffffffff8032c6bd] xfs_bmap_search_extents+109
5 [0xffffffff8033358c] xfs_bmapi+540
1 [0xffffffff80333a54] xfs_bmapi+1764
1 [0xffffffff803393b5] xfs_btree_init_cursor+69
49 [0xffffffff8033a084] xfs_buf_item_init+100
1 [0xffffffff8033ab3f] xfs_da_brelse+111
5066 [0xffffffff8033ad57] xfs_da_buf_make+231
7 [0xffffffff8033ba64] xfs_da_do_buf+1444
5 [0xffffffff8033e76b] xfs_dir_lookup+299
12 [0xffffffff8033e79f] xfs_dir_lookup+351
4 [0xffffffff8033eb9b] xfs_dir_createname+379
2 [0xffffffff8033ec37] xfs_dir_init+103
1 [0xffffffff8033ee1a] xfs_dir2_block_lookup_int+90
1 [0xffffffff8033eeb9] xfs_dir2_block_lookup_int+249
1 [0xffffffff803416d4] xfs_dir2_leaf_lookup_int+516
1 [0xffffffff80343234] xfs_dir2_leafn_order+4
63 [0xffffffff8034e2bc] xfs_iunlock+92
24 [0xffffffff8034e31c] xfs_iunlock+188
1 [0xffffffff8034ed57] xfs_iget_core+1063
511 [0xffffffff8034f96b] xfs_iext_bno_to_ext+155
5 [0xffffffff80350869] xfs_iext_add+361
11 [0xffffffff803512cd] xfs_itobp+381
5 [0xffffffff80351657] xfs_itruncate_finish+679
810 [0xffffffff80351a22] xfs_iformat_local+258
886 [0xffffffff803528d1] xfs_iformat_extents+177
1 [0xffffffff803529c7] xfs_iformat_extents+423
1 [0xffffffff80354a7e] xfs_inode_item_init+30
2924 [0xffffffff803555b6] xfs_iomap+406
3 [0xffffffff80355e55] xfs_iomap_write_delay+389
140 [0xffffffff8035a4d6] xlog_state_release_iclog+38
195 [0xffffffff8035a59b] xlog_state_get_iclog_space+59
4 [0xffffffff8035aac8] xlog_write+920
248 [0xffffffff8035aea0] xfs_log_release_iclog+16
1 [0xffffffff803620a9] xfs_icsb_sync_counters_flags+121
4 [0xffffffff80362931] xfs_icsb_modify_counters+113
1 [0xffffffff80362b01] xfs_mod_incore_sb_batch+161
3 [0xffffffff8036600f] xfs_trans_unreserve_and_mod_sb+207
10 [0xffffffff80366967] _xfs_trans_commit+823
2 [0xffffffff80366bbb] xfs_trans_unlocked_item+59
1016 [0xffffffff80367654] xfs_trans_read_buf+100
1 [0xffffffff80367657] xfs_trans_read_buf+103
12 [0xffffffff80367786] xfs_trans_read_buf+406
1 [0xffffffff80367c4d] xfs_trans_ijoin+61
5 [0xffffffff80367e94] xfs_trans_unlock_chunk+132
1 [0xffffffff8036b284] xfs_statvfs+292
2 [0xffffffff80370db3] xfs_release+339
1143 [0xffffffff80371592] xfs_readlink+130
366 [0xffffffff80371f6c] kmem_zone_alloc+92
128 [0xffffffff803720bd] kmem_alloc+93
1812 [0xffffffff80374156] xfs_buf_iowait+70
1 [0xffffffff80374a0e] xfs_buf_rele+46
1 [0xffffffff80374cbe] xfs_buf_delwri_queue+94
4252 [0xffffffff803751fd] _xfs_buf_lookup_pages+637
31 [0xffffffff80376170] _xfs_buf_find+256
666 [0xffffffff80376357] xfs_buf_get_flags+183
118 [0xffffffff80376442] xfs_buf_read_flags+18
2 [0xffffffff80376495] xfs_buf_read_flags+101
278 [0xffffffff803790d5] xfs_vn_permission+21
24 [0xffffffff80379872] xfs_vn_lookup+66
21 [0xffffffff8037a9c5] xfs_read+405
14 [0xffffffff80386c60] dummy_inode_alloc_security+0
1 [0xffffffff80386d10] dummy_inode_readlink+0
36 [0xffffffff80386d30] dummy_inode_permission+0
1 [0xffffffff80388e60] cap_capable+0
1 [0xffffffff80388e6f] cap_capable+15
1 [0xffffffff80388fca] cap_syslog+42
32 [0xffffffff8038e045] crypto_cbc_init_tfm+69
49 [0xffffffff8039b14d] blk_recount_segments+61
5 [0xffffffff8039c230] generic_make_request+464
2 [0xffffffff8039c486] submit_bio+102
23 [0xffffffff8039f108] generic_unplug_device+24
2 [0xffffffff803a16ec] exact_lock+12
1 [0xffffffff803a3e70] cfq_queue_empty+0
5 [0xffffffff803a8908] _atomic_dec_and_lock+72
5 [0xffffffff803a8f85] ida_get_new_above+165
63 [0xffffffff803a90f9] idr_get_new_above_int+73
1 [0xffffffff803a911b] idr_get_new+11
1 [0xffffffff803a9a50] kobject_release+0
2 [0xffffffff803a9ac2] kobject_get+18
7 [0xffffffff803aa006] kobject_set_name+278
2 [0xffffffff803aa73a] kobject_uevent_env+298
2 [0xffffffff803ab2e3] prio_tree_insert+35
5 [0xffffffff803ab33c] prio_tree_insert+124
4 [0xffffffff803ab4be] prio_tree_insert+510
18 [0xffffffff803abeb2] radix_tree_node_alloc+50
1 [0xffffffff803ac6fa] radix_tree_delete+426
14 [0xffffffff803ac94d] rb_insert_color+141
14 [0xffffffff803ac99e] rb_insert_color+222
2699 [0xffffffff803ad047] __up_read+39
4 [0xffffffff803ad0e7] __up_write+39
10 [0xffffffff803ae2ea] number+682
20 [0xffffffff803aee3b] vsnprintf+811
3 [0xffffffff803aee7b] vsnprintf+875
4 [0xffffffff803af241] sprintf+81
1 [0xffffffff803af304] snprintf+68
1 [0xffffffff803af535] clear_page_c+5
4 [0xffffffff803af537] clear_page_c+7
11 [0xffffffff803c8218] vgacon_init+200
1 [0xffffffff803c8981] vgacon_scroll+289
3 [0xffffffff803c8e80] vgacon_cursor+0
2 [0xffffffff803c9450] acpi_os_release_object+9
1 [0xffffffff803ce3b5] acpi_ev_address_space_dispatch+360
1 [0xffffffff803d26ba] acpi_ex_field_datum_io+231
2 [0xffffffff803d282e] acpi_ex_extract_from_field+224
11 [0xffffffff803d7412] acpi_ns_lookup+578
1 [0xffffffff803d7858] acpi_ns_search_one_scope+28
10 [0xffffffff803d7944] acpi_ns_search_and_enter+196
1 [0xffffffff803ddddd] acpi_ut_evaluate_object+390
1 [0xffffffff803de372] acpi_ut_valid_acpi_name+39
1 [0xffffffff803df43c] acpi_ut_update_ref_count+82
2 [0xffffffff803df564] acpi_ut_update_object_reference+219
1 [0xffffffff803dfbcd] acpi_ut_allocate_object_desc_dbg+52
1 [0xffffffff803dfe89] acpi_ut_create_generic_state+65
1 [0xffffffff803e4250] acpi_pci_link_get_current+116
28 [0xffffffff803f14c8] secure_ip_id+72
2 [0xffffffff803f20e1] tty_ldisc_deref+81
1 [0xffffffff803f30c3] init_dev+675
1 [0xffffffff803f312d] init_dev+781
1 [0xffffffff8040e6db] uart_console_write+43
1 [0xffffffff8041224c] serial8250_start_tx+124
1 [0xffffffff80431560] scsi_done+0
1 [0xffffffff804352d6] scsi_error_handler+102
1 [0xffffffff80439041] scsi_target_reap+49
5 [0xffffffff80440cb3] ata_dev_init+83
6 [0xffffffff804424f8] ata_qc_issue+312
1 [0xffffffff80449376] ata_scsi_qc_complete+246
2 [0xffffffff8044c79f] ata_exec_command+31
22 [0xffffffff8044cc6c] ata_tf_load+236
1 [0xffffffff80466b87] dma_trm_flush+359
1 [0xffffffff8046a445] ether1394_data_handler+661
1 [0xffffffff8047388b] usb_start_wait_urb+139
1 [0xffffffff80473b4f] usb_control_msg+239
669 [0xffffffff804b5c66] __release_stripe+198
96 [0xffffffff804b7f49] grow_one_stripe+153
5 [0xffffffff804c115d] md_import_device+125
1 [0xffffffff804c18e4] mddev_find+372
12 [0xffffffff804c9103] read_sb_page+35
191 [0xffffffff804cb74d] __map_bio+61
135 [0xffffffff804cc4ba] __split_bio+906
13 [0xffffffff804cc8ad] dm_request+269
44 [0xffffffff804ce33e] dm_table_any_congested+46
23 [0xffffffff804d2589] crypt_map+105
1 [0xffffffff804ddf85] snd_card_file_remove+101
1 [0xffffffff804f6f4d] snd_hda_codec_write+109
8 [0xffffffff8050ab1c] pci_conf1_read+204
12 [0xffffffff8050d8e1] sockfd_lookup_light+65
35 [0xffffffff8050e1bb] sock_sendmsg+203
14 [0xffffffff8051044f] sk_reset_timer+15
203 [0xffffffff80513f82] skb_dequeue+82
1 [0xffffffff80515006] __alloc_skb+150
18 [0xffffffff8051687d] memcpy_toiovec+77
3 [0xffffffff805168cf] verify_iovec+47
2 [0xffffffff80519630] dev_seq_open+96
2 [0xffffffff8051cd1d] netdev_run_todo+333
3 [0xffffffff8051d2fa] dev_queue_xmit+170
1 [0xffffffff8052a2e8] __qdisc_run+152
1 [0xffffffff8053c39f] ip_output+447
5 [0xffffffff80543710] tcp_sendmsg+1904
1 [0xffffffff80580318] unix_release_sock+408
1 [0xffffffff805813e3] unix_stream_sendmsg+483
2 [0xffffffff8058e069] xprt_end_transmit+57
2 [0xffffffff80593207] rpc_wake_up_task+103
17 [0xffffffff80593993] rpc_wake_up_next+147
2 [0xffffffff805ac844] wait_for_common+292
57 [0xffffffff805acb3f] __switch_to_end+24
12 [0xffffffff805aced9] __switch_to_end+946
11861 [0xffffffff805ad128] io_schedule+40
8 [0xffffffff805ad235] schedule_timeout+149
9 [0xffffffff805ad2d4] __wait_on_bit_lock+36
2 [0xffffffff805ad315] __wait_on_bit_lock+101
15 [0xffffffff805ad43b] __wait_on_bit+107
46337 [0xffffffff805ae2c2] __down_read+18
206 [0xffffffff805ae388] __down_failed+53
17 [0xffffffff805ae725] __down+229
2 [0xffffffff805ae8ec] _read_lock_irq+12
25 [0xffffffff805aeaed] error_exit+0
1 [0xffffffff807c2000] _sinittext+0
1 [0xffffffff88047000] __videobuf_mmap_free+0
1 [0xffffffff88096000] tda9855_volume+0
1 [0xffffffff880bb000] w83792d_detach_client+0
1 [0xffffffff880da570] ndisc_ifinfo_sysctl_change+0
1 [0xffffffff8810e000] nfsd_vers+0
1 [0xffffffff8814fa1a] drm_addmap_core+1098
2 [0xffffffff88157650] drm_sg_alloc_ioctl+0

Torsten

[-- Attachment #2: buddyinfo.txt --]
[-- Type: text/plain, Size: 400 bytes --]

Node 0, zone DMA 10 10 5 8 4 4 6 3 3 1 0
Node 0, zone DMA32 6808 8403 9564 4697 1296 100 24 7 0 1 0
Node 1, zone DMA32 4441 1388 618 167 72 26 12 10 4 3 2
Node 1, zone Normal 824 472 333 225 112 26 14 12 11 6 2

[-- Attachment #3: buddyinfo.end.txt --]
[-- Type: text/plain, Size: 400 bytes --]

Node 0, zone DMA 6 7 4 6 4 2 4 2 1 2 0
Node 0, zone DMA32 66334 23 8 1 0 0 0 0 0 1 0
Node 1, zone DMA32 417 4 2 3 1 2 0 1 1 1 0
Node 1, zone Normal 120 30 6 5 3 0 0 0 0 0 0

[-- Attachment #4: meminfo.txt --]
[-- Type: text/plain, Size: 748 bytes --]

MemTotal: 4061808 kB
MemFree: 664880 kB
Buffers: 28 kB
Cached: 833996 kB
SwapCached: 28 kB
Active: 241576 kB
Inactive: 601436 kB
SwapTotal: 9775416 kB
SwapFree: 9775356 kB
Dirty: 4 kB
Writeback: 0 kB
AnonPages: 8996 kB
Mapped: 4776 kB
Slab: 134612 kB
SReclaimable: 100168 kB
SUnreclaim: 34444 kB
PageTables: 1272 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
CommitLimit: 11806320 kB
Committed_AS: 16580 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 26672 kB
VmallocChunk: 34359710643 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB

[-- Attachment #5: meminfo.end.txt --]
[-- Type: text/plain, Size: 748 bytes --]

MemTotal: 4061808 kB
MemFree: 285032 kB
Buffers: 0 kB
Cached: 2156 kB
SwapCached: 408 kB
Active: 100 kB
Inactive: 2072 kB
SwapTotal: 9775416 kB
SwapFree: 9770508 kB
Dirty: 0 kB
Writeback: 0 kB
AnonPages: 0 kB
Mapped: 396 kB
Slab: 7884 kB
SReclaimable: 1084 kB
SUnreclaim: 6800 kB
PageTables: 588 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
CommitLimit: 11806320 kB
Committed_AS: 8536 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 26672 kB
VmallocChunk: 34359710643 kB
HugePages_Total: 0
HugePages_Free: 0
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB

[-- Attachment #6: pagetypeinfo.txt --]
[-- Type: text/plain, Size: 3302 bytes --]

Page block order: 9
Pages per block: 512

Free pages count per migrate type at order 0 1 2 3 4 5 6 7 8 9 10
Node 0, zone DMA, type Unmovable 3 2 0 2 0 1 1 1 1 0 0
Node 0, zone DMA, type Reclaimable 1 1 1 1 1 1 1 1 1 0 0
Node 0, zone DMA, type Movable 6 7 4 5 3 2 4 1 1 0 0
Node 0, zone DMA, type Reserve 0 0 0 0 0 0 0 0 0 1 0
Node 0, zone DMA, type <NULL> 0 0 0 0 0 0 0 0 0 0 0
Node 0, zone DMA32, type Unmovable 326 66 43 40 32 7 1 0 0 0 0
Node 0, zone DMA32, type Reclaimable 32 22 5 3 2 0 0 0 0 0 0
Node 0, zone DMA32, type Movable 6423 8296 9508 4653 1262 93 23 7 0 0 0
Node 0, zone DMA32, type Reserve 27 19 8 1 0 0 0 0 0 1 0
Node 0, zone DMA32, type <NULL> 0 0 0 0 0 0 0 0 0 0 0

Number of blocks type Unmovable Reclaimable Movable Reserve <NULL>
Node 0, zone DMA 1 1 5 1 0
Node 0, zone DMA32 453 28 533 2 0
Page block order: 9
Pages per block: 512

Free pages count per migrate type at order 0 1 2 3 4 5 6 7 8 9 10
Node 1, zone DMA32, type Unmovable 1503 431 229 71 30 12 2 1 1 0 0
Node 1, zone DMA32, type Reclaimable 56 23 12 2 1 0 0 0 0 0 0
Node 1, zone DMA32, type Movable 2882 934 377 94 41 14 10 9 3 3 1
Node 1, zone DMA32, type Reserve 0 0 0 0 0 0 0 0 0 0 1
Node 1, zone DMA32, type <NULL> 0 0 0 0 0 0 0 0 0 0 0
Node 1, zone Normal, type Unmovable 1 1 21 8 9 4 0 0 0 0 0
Node 1, zone Normal, type Reclaimable 0 0 1 8 7 2 1 0 0 0 0
Node 1, zone Normal, type Movable 834 460 303 202 93 20 13 12 11 6 2
Node 1, zone Normal, type Reserve 19 10 5 5 3 0 0 0 0 0 0
Node 1, zone Normal, type <NULL> 0 0 0 0 0 0 0 0 0 0 0

Number of blocks type Unmovable Reclaimable Movable Reserve <NULL>
Node 1, zone DMA32 439 6 321 2 0
Node 1, zone Normal 73 5 177 1 0

[-- Attachment #7: pagetypeinfo.end.txt --]
[-- Type: text/plain, Size: 3302 bytes --]

Page block order: 9
Pages per block: 512

Free pages count per migrate type at order 0 1 2 3 4 5 6 7 8 9 10
Node 0, zone DMA, type Unmovable 0 0 0 1 1 0 0 1 0 0 0
Node 0, zone DMA, type Reclaimable 0 0 0 0 0 0 0 0 0 1 0
Node 0, zone DMA, type Movable 6 7 4 5 3 2 4 1 1 0 0
Node 0, zone DMA, type Reserve 0 0 0 0 0 0 0 0 0 1 0
Node 0, zone DMA, type <NULL> 0 0 0 0 0 0 0 0 0 0 0
Node 0, zone DMA32, type Unmovable 3 0 0 0 0 0 0 0 0 0 0
Node 0, zone DMA32, type Reclaimable 0 0 0 0 0 0 0 0 0 0 0
Node 0, zone DMA32, type Movable 66649 4 0 0 0 0 0 0 0 0 0
Node 0, zone DMA32, type Reserve 27 19 8 1 0 0 0 0 0 1 0
Node 0, zone DMA32, type <NULL> 0 0 0 0 0 0 0 0 0 0 0

Number of blocks type Unmovable Reclaimable Movable Reserve <NULL>
Node 0, zone DMA 1 1 5 1 0
Node 0, zone DMA32 485 22 507 2 0
Page block order: 9
Pages per block: 512

Free pages count per migrate type at order 0 1 2 3 4 5 6 7 8 9 10
Node 1, zone DMA32, type Unmovable 0 0 0 0 0 0 0 0 0 0 0
Node 1, zone DMA32, type Reclaimable 2 0 0 0 0 0 0 0 0 0 0
Node 1, zone DMA32, type Movable 445 0 0 0 0 0 0 0 0 0 0
Node 1, zone DMA32, type Reserve 1 4 2 3 1 2 0 1 1 1 0
Node 1, zone DMA32, type <NULL> 0 0 0 0 0 0 0 0 0 0 0
Node 1, zone Normal, type Unmovable 68 18 1 0 0 0 0 0 0 0 0
Node 1, zone Normal, type Reclaimable 15 1 0 0 0 0 0 0 0 0 0
Node 1, zone Normal, type Movable 16 5 0 0 0 0 0 0 0 0 0
Node 1, zone Normal, type Reserve 19 10 5 5 3 0 0 0 0 0 0
Node 1, zone Normal, type <NULL> 0 0 0 0 0 0 0 0 0 0 0

Number of blocks type Unmovable Reclaimable Movable Reserve <NULL>
Node 1, zone DMA32 441 6 319 2 0
Node 1, zone Normal 84 6 165 1 0

[-- Attachment #8: slubinfo.txt --]
[-- Type: text/plain, Size: 4797 bytes --]

Name Objects Objsize Space Slabs/Part/Cpu O/S O %Fr %Ef Flg
:0000016 2497 16 61.4K 15/7/4 256 0 46 65 *
:0000024 854 24 28.6K 7/2/4 170 0 28 71 *
:0000032 47932 32 1.5M 375/16/4 128 0 4 99 *
:0000040 528 40 24.5K 6/0/4 102 0 0 85 *
:0000056 142 56 16.3K 4/0/4 73 0 0 48 *
:0000064 30292 64 2.1M 532/168/4 64 0 31 88 *
:0000072 145 72 12.2K 3/0/3 56 0 0 84 *
:0000080 7171 80 577.5K 141/1/4 51 0 0 99 *
:0000096 1968 96 204.8K 50/6/4 42 0 12 92 *
:0000128 855 128 204.8K 50/37/4 32 0 74 53 *
:0000192 14659 192 6.4M 1569/1197/4 21 0 76 43 *
:0000256 417 256 122.8K 30/6/4 16 0 20 86 *
:0000272 525 272 143.3K 35/0/1 15 0 0 99 *
:0000320 12 320 4.0K 1/0/1 12 0 0 93 *A
:0000384 50 376 32.7K 8/3/4 10 0 37 57 *A
:0000448 33 420 20.4K 5/1/4 9 0 20 67 *A
:0000512 447 512 253.9K 62/11/4 8 0 17 90 *
:0000704 136 704 180.2K 22/13/4 11 1 59 53 *A
:0000768 80 760 94.2K 23/12/4 5 0 52 64 *A
:0000832 171 832 212.9K 26/9/4 9 1 34 66 *A
:0000960 15 912 16.3K 4/0/3 4 0 0 83 *A
:0001024 393 1024 442.3K 108/15/4 4 0 13 90 *
:0002048 507 2048 1.1M 137/18/4 4 1 13 92 *
:0004096 42 4096 180.2K 22/0/4 2 1 0 95 *A
anon_vma 622 24 53.2K 13/9/4 128 0 69 28
bdev_cache 33 760 28.6K 7/0/4 5 0 0 87 Aa
blkdev_queue 19 1632 32.7K 4/0/4 5 1 0 94
blkdev_requests 45 288 20.4K 5/1/4 14 0 20 63
buffer_head 84536 104 9.0M 2211/90/4 39 0 4 97 a
cfq_io_context 63 152 16.3K 4/0/4 26 0 0 58
cfq_queue 85 136 16.3K 4/0/4 30 0 0 70
dentry 103088 200 23.6M 5769/1187/4 20 0 20 87 a
file_lock_cache 77 176 16.3K 4/0/4 22 0 0 82
hugetlbfs_inode_cache 7 576 4.0K 1/0/1 7 0 0 98
idr_layer_cache 199 528 131.0K 32/5/4 7 0 15 80
inode_cache 3219 560 1.9M 465/10/4 7 0 2 94 a
kmalloc-8 2698 8 32.7K 8/3/4 512 0 37 65
kmem_cache_node 252 56 20.4K 5/2/3 73 0 40 68 *
mqueue_inode_cache 4 848 4.0K 1/0/1 4 0 0 82 A
nfs_inode_cache 14768 928 15.1M 3692/0/4 4 0 0 90 a
proc_inode_cache 1902 592 1.3M 327/24/4 6 0 7 84 a
radix_tree_node 29226 552 18.8M 4614/962/4 7 0 20 85
revokefs_inode_cache 6 584 4.0K 1/0/1 6 0 0 85 Aa
rpc_inode_cache 18 784 16.3K 2/0/2 9 1 0 86 Aa
shmem_inode_cache 944 768 794.6K 194/6/4 5 0 3 91
sighand_cache 124 2088 417.7K 51/15/4 3 1 29 61 A
sigqueue 74 160 16.3K 4/0/4 25 0 0 72
sock_inode_cache 136 640 172.0K 21/9/4 11 1 42 50 Aa
task_struct 126 1808 352.2K 43/18/4 4 1 41 64
TCP 13 1584 32.7K 4/0/4 5 1 0 62 A
TCPv6 8 1728 16.3K 2/0/2 4 1 0 84 A
vm_area_struct 1028 176 303.1K 74/50/4 23 0 67 59
xfs_acl 52 304 16.3K 4/0/4 13 0 0 96
xfs_buf_item 53 184 16.3K 4/0/4 22 0 0 59
xfs_da_state 29 488 16.3K 4/0/4 8 0 0 86
xfs_efd_item 22 360 16.3K 4/0/4 11 0 0 48
xfs_efi_item 22 352 16.3K 4/0/4 11 0 0 47
xfs_inode 46712 504 23.9M 5846/26/4 8 0 0 98 Aa
xfs_vnode 46717 560 27.3M 6679/19/4 7 0 0 95 Aa

[-- Attachment #9: slubinfo.end.txt --]
[-- Type: text/plain, Size: 4797 bytes --]

Name Objects Objsize Space Slabs/Part/Cpu O/S O %Fr %Ef Flg
:0000016 2356 16 57.3K 14/6/4 256 0 42 65 *
:0000024 564 24 24.5K 6/1/4 170 0 16 55 *
:0000032 1744 32 118.7K 29/20/4 128 0 68 46 *
:0000040 434 40 28.6K 7/1/4 102 0 14 60 *
:0000056 145 56 16.3K 4/0/4 73 0 0 49 *
:0000064 3423 64 266.2K 65/17/4 64 0 26 82 *
:0000072 92 72 12.2K 3/0/3 56 0 0 53 *
:0000080 7171 80 577.5K 141/1/4 51 0 0 99 *
:0000096 1764 96 180.2K 44/4/4 42 0 9 93 *
:0000128 545 128 122.8K 30/18/4 32 0 60 56 *
:0000192 1597 192 442.3K 108/37/4 21 0 34 69 *
:0000256 391 256 110.5K 27/3/4 16 0 11 90 *
:0000272 525 272 143.3K 35/0/1 15 0 0 99 *
:0000320 12 320 4.0K 1/0/1 12 0 0 93 *A
:0000384 59 376 36.8K 9/4/4 10 0 44 60 *A
:0000448 27 420 24.5K 6/2/4 9 0 33 46 *A
:0000512 437 512 245.7K 60/12/4 8 0 20 91 *
:0000704 46 704 81.9K 10/6/4 11 1 60 39 *A
:0000768 62 760 77.8K 19/8/4 5 0 42 60 *A
:0000832 159 832 229.3K 28/12/4 9 1 42 57 *A
:0000960 15 912 16.3K 4/0/3 4 0 0 83 *A
:0001024 389 1024 442.3K 108/18/4 4 0 16 90 *
:0002048 503 2048 1.0M 134/16/4 4 1 11 93 *
:0004096 41 4096 180.2K 22/0/4 2 1 0 93 *A
anon_vma 395 24 45.0K 11/7/4 128 0 63 21
bdev_cache 33 760 28.6K 7/0/4 5 0 0 87 Aa
blkdev_queue 19 1632 32.7K 4/0/4 5 1 0 94
blkdev_requests 52 288 24.5K 6/2/4 14 0 33 60
buffer_head 94 104 16.3K 4/0/4 39 0 0 59 a
cfq_io_context 71 152 20.4K 5/1/4 26 0 20 52
cfq_queue 80 136 20.4K 5/1/4 30 0 20 53
dentry 1552 200 512.0K 125/80/4 20 0 64 60 a
file_lock_cache 72 176 16.3K 4/0/4 22 0 0 77
hugetlbfs_inode_cache 7 576 4.0K 1/0/1 7 0 0 98
idr_layer_cache 197 528 131.0K 32/6/4 7 0 18 79
inode_cache 420 560 262.1K 64/2/4 7 0 3 89 a
kmalloc-8 2692 8 32.7K 8/3/4 512 0 37 65
kmem_cache_node 252 56 20.4K 5/2/3 73 0 40 68 *
mqueue_inode_cache 4 848 4.0K 1/0/1 4 0 0 82 A
nfs_inode_cache 9 928 16.3K 4/0/4 4 0 0 50 a
proc_inode_cache 49 592 57.3K 14/4/4 6 0 28 50 a
radix_tree_node 310 552 389.1K 95/81/4 7 0 85 43
revokefs_inode_cache 6 584 4.0K 1/0/1 6 0 0 85 Aa
rpc_inode_cache 12 784 16.3K 2/0/2 9 1 0 57 Aa
shmem_inode_cache 944 768 794.6K 194/6/4 5 0 3 91
sighand_cache 114 2088 360.4K 44/11/4 3 1 25 66 A
sigqueue 33 160 16.3K 4/0/4 25 0 0 32
sock_inode_cache 39 640 73.7K 9/4/4 11 1 44 33 Aa
task_struct 116 1808 319.4K 39/16/4 4 1 41 65
TCP 7 1584 32.7K 4/0/4 5 1 0 33 A
TCPv6 7 1728 16.3K 2/0/2 4 1 0 73 A
vm_area_struct 491 176 155.6K 38/30/4 23 0 78 55
xfs_acl 52 304 16.3K 4/0/4 13 0 0 96
xfs_buf_item 36 184 16.3K 4/0/4 22 0 0 40
xfs_da_state 25 488 16.3K 4/0/4 8 0 0 74
xfs_efd_item 3 360 16.3K 4/0/4 11 0 0 6
xfs_efi_item 3 352 16.3K 4/0/4 11 0 0 6
xfs_inode 111 504 147.4K 36/30/4 8 0 83 37 Aa
xfs_vnode 117 560 147.4K 36/27/4 7 0 75 44 Aa

[-- Attachment #10: sysrq-m.txt --]
[-- Type: text/plain, Size: 3582 bytes --]

[ 8482.010000] SysRq : Show Memory
[ 8482.020000] Mem-info:
[ 8482.020000] Node 0 DMA per-cpu:
[ 8482.020000] CPU 0: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0
[ 8482.020000] CPU 1: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0
[ 8482.020000] CPU 2: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0
[ 8482.020000] CPU 3: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0
[ 8482.020000] Node 0 DMA32 per-cpu:
[ 8482.020000] CPU 0: Hot: hi: 186, btch: 31 usd: 172 Cold: hi: 62, btch: 15 usd: 3
[ 8482.020000] CPU 1: Hot: hi: 186, btch: 31 usd: 155 Cold: hi: 62, btch: 15 usd: 13
[ 8482.020000] CPU 2: Hot: hi: 186, btch: 31 usd: 0 Cold: hi: 62, btch: 15 usd: 0
[ 8482.020000] CPU 3: Hot: hi: 186, btch: 31 usd: 0 Cold: hi: 62, btch: 15 usd: 0
[ 8482.020000] Node 1 DMA32 per-cpu:
[ 8482.020000] CPU 0: Hot: hi: 186, btch: 31 usd: 61 Cold: hi: 62, btch: 15 usd: 0
[ 8482.020000] CPU 1: Hot: hi: 186, btch: 31 usd: 0 Cold: hi: 62, btch: 15 usd: 0
[ 8482.020000] CPU 2: Hot: hi: 186, btch: 31 usd: 160 Cold: hi: 62, btch: 15 usd: 15
[ 8482.020000] CPU 3: Hot: hi: 186, btch: 31 usd: 165 Cold: hi: 62, btch: 15 usd: 10
[ 8482.020000] Node 1 Normal per-cpu:
[ 8482.020000] CPU 0: Hot: hi: 186, btch: 31 usd: 0 Cold: hi: 62, btch: 15 usd: 0
[ 8482.020000] CPU 1: Hot: hi: 186, btch: 31 usd: 0 Cold: hi: 62, btch: 15 usd: 0
[ 8482.020000] CPU 2: Hot: hi: 186, btch: 31 usd: 142 Cold: hi: 62, btch: 15 usd: 23
[ 8482.020000] CPU 3: Hot: hi: 186, btch: 31 usd: 154 Cold: hi: 62, btch: 15 usd: 9
[ 8482.020000] Active:60448 inactive:150421 dirty:3 writeback:0 unstable:0
[ 8482.020000] free:166065 slab:33652 mapped:1197 pagetables:318 bounce:0
[ 8482.020000] Node 0 DMA free:9416kB min:16kB low:20kB high:24kB active:0kB inactive:0kB present:8876kB pages_scanned:0 all_unreclaimable? no
[ 8482.020000] lowmem_reserve[]: 0 2004 2004 2004
[ 8482.020000] Node 0 DMA32 free:505572kB min:4040kB low:5048kB high:6060kB active:149304kB inactive:347476kB present:2052320kB pages_scanned:0 all_unreclaimable? no
[ 8482.020000] lowmem_reserve[]: 0 0 0 0
[ 8482.020000] Node 1 DMA32 free:78660kB min:3040kB low:3800kB high:4560kB active:37044kB inactive:125728kB present:1544000kB pages_scanned:0 all_unreclaimable? no
[ 8482.020000] lowmem_reserve[]: 0 0 505 505
[ 8482.020000] Node 1 Normal free:70612kB min:1016kB low:1268kB high:1524kB active:55444kB inactive:128480kB present:517120kB pages_scanned:0 all_unreclaimable? no
[ 8482.020000] lowmem_reserve[]: 0 0 0 0
[ 8482.020000] Node 0 DMA: 10*4kB 10*8kB 5*16kB 8*32kB 4*64kB 4*128kB 6*256kB 3*512kB 3*1024kB 1*2048kB 0*4096kB = 9416kB
[ 8482.020000] Node 0 DMA32: 6823*4kB 8419*8kB 9567*16kB 4698*32kB 1296*64kB 100*128kB 24*256kB 7*512kB 0*1024kB 1*2048kB 0*4096kB = 505572kB
[ 8482.020000] Node 1 DMA32: 4441*4kB 1388*8kB 618*16kB 167*32kB 72*64kB 26*128kB 12*256kB 10*512kB 4*1024kB 3*2048kB 2*4096kB = 78660kB
[ 8482.020000] Node 1 Normal: 617*4kB 484*8kB 319*16kB 225*32kB 112*64kB 26*128kB 14*256kB 12*512kB 11*1024kB 6*2048kB 2*4096kB = 70612kB
[ 8482.020000] Swap cache: add 10949, delete 10942, find 3781/4195, race 0+0
[ 8482.020000] Free swap = 9775356kB
[ 8482.020000] Total swap = 9775416kB
[ 8482.020000] Free swap: 9775356kB
[ 8482.080000] 1048576 pages of RAM
[ 8482.080000] 33124 reserved pages
[ 8482.080000] 89800 pages shared
[ 8482.080000] 7 pages swap cached

[-- Attachment #11: sysrq-m.end.txt --]
[-- Type: text/plain, Size: 3525 bytes --]

[11255.020000] SysRq : Show Memory
[11255.030000] Mem-info:
[11255.030000] Node 0 DMA per-cpu:
[11255.030000] CPU 0: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0
[11255.030000] CPU 1: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0
[11255.030000] CPU 2: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0
[11255.030000] CPU 3: Hot: hi: 0, btch: 1 usd: 0 Cold: hi: 0, btch: 1 usd: 0
[11255.030000] Node 0 DMA32 per-cpu:
[11255.030000] CPU 0: Hot: hi: 186, btch: 31 usd: 167 Cold: hi: 62, btch: 15 usd: 50
[11255.030000] CPU 1: Hot: hi: 186, btch: 31 usd: 163 Cold: hi: 62, btch: 15 usd: 56
[11255.030000] CPU 2: Hot: hi: 186, btch: 31 usd: 140 Cold: hi: 62, btch: 15 usd: 0
[11255.030000] CPU 3: Hot: hi: 186, btch: 31 usd: 22 Cold: hi: 62, btch: 15 usd: 0
[11255.030000] Node 1 DMA32 per-cpu:
[11255.030000] CPU 0: Hot: hi: 186, btch: 31 usd: 0 Cold: hi: 62, btch: 15 usd: 0
[11255.030000] CPU 1: Hot: hi: 186, btch: 31 usd: 0 Cold: hi: 62, btch: 15 usd: 0
[11255.030000] CPU 2: Hot: hi: 186, btch: 31 usd: 105 Cold: hi: 62, btch: 15 usd: 50
[11255.030000] CPU 3: Hot: hi: 186, btch: 31 usd: 77 Cold: hi: 62, btch: 15 usd: 16
[11255.030000] Node 1 Normal per-cpu:
[11255.030000] CPU 0: Hot: hi: 186, btch: 31 usd: 0 Cold: hi: 62, btch: 15 usd: 0
[11255.030000] CPU 1: Hot: hi: 186, btch: 31 usd: 0 Cold: hi: 62, btch: 15 usd: 0
[11255.030000] CPU 2: Hot: hi: 186, btch: 31 usd: 159 Cold: hi: 62, btch: 15 usd: 5
[11255.030000] CPU 3: Hot: hi: 186, btch: 31 usd: 99 Cold: hi: 62, btch: 15 usd: 15
[11255.030000] Active:139 inactive:781 dirty:1 writeback:0 unstable:0
[11255.030000] free:70942 slab:1967 mapped:192 pagetables:143 bounce:0
[11255.030000] Node 0 DMA free:8016kB min:16kB low:20kB high:24kB active:0kB inactive:0kB present:8876kB pages_scanned:0 all_unreclaimable? yes
[11255.030000] lowmem_reserve[]: 0 2004 2004 2004
[11255.030000] Node 0 DMA32 free:268808kB min:4040kB low:5048kB high:6060kB active:376kB inactive:2024kB present:2052320kB pages_scanned:0 all_unreclaimable? no
[11255.030000] lowmem_reserve[]: 0 0 0 0
[11255.030000] Node 1 DMA32 free:5816kB min:3040kB low:3800kB high:4560kB active:64kB inactive:308kB present:1544000kB pages_scanned:177 all_unreclaimable? no
[11255.030000] lowmem_reserve[]: 0 0 505 505
[11255.030000] Node 1 Normal free:1128kB min:1016kB low:1268kB high:1524kB active:116kB inactive:792kB present:517120kB pages_scanned:0 all_unreclaimable? no
[11255.030000] lowmem_reserve[]: 0 0 0 0
[11255.030000] Node 0 DMA: 6*4kB 7*8kB 4*16kB 6*32kB 4*64kB 2*128kB 4*256kB 2*512kB 1*1024kB 2*2048kB 0*4096kB = 8016kB
[11255.030000] Node 0 DMA32: 66610*4kB 20*8kB 8*16kB 1*32kB 0*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 1*2048kB 0*4096kB = 268808kB
[11255.030000] Node 1 DMA32: 436*4kB 5*8kB 2*16kB 3*32kB 1*64kB 2*128kB 0*256kB 1*512kB 1*1024kB 1*2048kB 0*4096kB = 5816kB
[11255.030000] Node 1 Normal: 102*4kB 34*8kB 6*16kB 5*32kB 3*64kB 0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1128kB
[11255.030000] Swap cache: add 606569, delete 606407, find 129878/215652, race 2+9
[11255.030000] Free swap = 9770512kB
[11255.030000] Total swap = 9775416kB
[11255.030000] Free swap: 9770512kB
[11255.090000] 1048576 pages of RAM
[11255.090000] 33124 reserved pages
[11255.090000] 297 pages shared
[11255.090000] 163 pages swap cached

[-- Attachment #12: zoneinfo.txt --]
[-- Type: text/plain, Size: 6333 bytes --]

Node 0, zone DMA
pages free 2354
min 4
low 5
high 6
scanned 0 (a: 11 i: 11)
spanned 4096
present 2219
nr_free_pages 2354
nr_inactive 0
nr_active 0
nr_anon_pages 0
nr_mapped 0
nr_file_pages 0
nr_dirty 0
nr_writeback 0
nr_slab_reclaimable 1
nr_slab_unreclaimable 0
nr_page_table_pages 0
nr_unstable 0
nr_bounce 0
nr_vmscan_write 91
numa_hit 2974
numa_miss 5283
numa_foreign 0
numa_interleave 0
numa_local 2974
numa_other 5283
protection: (0, 2004, 2004, 2004)
pagesets
cpu: 0 pcp: 0
count: 0
high: 0
batch: 1
cpu: 0 pcp: 1
count: 0
high: 0
batch: 1
vm stats threshold: 6
cpu: 1 pcp: 0
count: 0
high: 0
batch: 1
cpu: 1 pcp: 1
count: 0
high: 0
batch: 1
vm stats threshold: 6
cpu: 2 pcp: 0
count: 0
high: 0
batch: 1
cpu: 2 pcp: 1
count: 0
high: 0
batch: 1
vm stats threshold: 6
cpu: 3 pcp: 0
count: 0
high: 0
batch: 1
cpu: 3 pcp: 1
count: 0
high: 0
batch: 1
vm stats threshold: 6
all_unreclaimable: 0
prev_priority: 12
start_pfn: 0
Node 0, zone DMA32
pages free 126326
min 1010
low 1262
high 1515
scanned 0 (a: 0 i: 0)
spanned 520192
present 513080
nr_free_pages 126326
nr_inactive 86946
nr_active 37249
nr_anon_pages 1506
nr_mapped 974
nr_file_pages 122689
nr_dirty 0
nr_writeback 0
nr_slab_reclaimable 17106
nr_slab_unreclaimable 5184
nr_page_table_pages 212
nr_unstable 0
nr_bounce 0
nr_vmscan_write 0
numa_hit 150802043
numa_miss 14802772
numa_foreign 335606
numa_interleave 3463
numa_local 150801929
numa_other 14802886
protection: (0, 0, 0, 0)
pagesets
cpu: 0 pcp: 0
count: 172
high: 186
batch: 31
cpu: 0 pcp: 1
count: 3
high: 62
batch: 15
vm stats threshold: 30
cpu: 1 pcp: 0
count: 155
high: 186
batch: 31
cpu: 1 pcp: 1
count: 13
high: 62
batch: 15
vm stats threshold: 30
cpu: 2 pcp: 0
count: 67
high: 186
batch: 31
cpu: 2 pcp: 1
count: 0
high: 62
batch: 15
vm stats threshold: 30
cpu: 3 pcp: 0
count: 0
high: 186
batch: 31
cpu: 3 pcp: 1
count: 0
high: 62
batch: 15
vm stats threshold: 30
all_unreclaimable: 0
prev_priority: 12
start_pfn: 4096
Node 1, zone DMA32
pages free 19665
min 760
low 950
high 1140
scanned 0 (a: 0 i: 22)
spanned 524288
present 386000
nr_free_pages 19665
nr_inactive 31432
nr_active 9261
nr_anon_pages 2
nr_mapped 0
nr_file_pages 40691
nr_dirty 0
nr_writeback 0
nr_slab_reclaimable 4678
nr_slab_unreclaimable 1413
nr_page_table_pages 0
nr_unstable 0
nr_bounce 0
nr_vmscan_write 1999
numa_hit 79992534
numa_miss 192158
numa_foreign 0
numa_interleave 8
numa_local 79992526
numa_other 192166
protection: (0, 0, 505, 505)
pagesets
cpu: 0 pcp: 0
count: 61
high: 186
batch: 31
cpu: 0 pcp: 1
count: 0
high: 62
batch: 15
vm stats threshold: 30
cpu: 1 pcp: 0
count: 0
high: 186
batch: 31
cpu: 1 pcp: 1
count: 0
high: 62
batch: 15
vm stats threshold: 30
cpu: 2 pcp: 0
count: 160
high: 186
batch: 31
cpu: 2 pcp: 1
count: 15
high: 62
batch: 15
vm stats threshold: 30
cpu: 3 pcp: 0
count: 165
high: 186
batch: 31
cpu: 3 pcp: 1
count: 10
high: 62
batch: 15
vm stats threshold: 30
all_unreclaimable: 0
prev_priority: 12
start_pfn: 524288
Node 1, zone Normal
pages free 17894
min 254
low 317
high 381
scanned 0 (a: 0 i: 0)
spanned 131072
present 129280
nr_free_pages 17894
nr_inactive 31983
nr_active 13865
nr_anon_pages 741
nr_mapped 220
nr_file_pages 45135
nr_dirty 1
nr_writeback 0
nr_slab_reclaimable 3258
nr_slab_unreclaimable 2014
nr_page_table_pages 106
nr_unstable 0
nr_bounce 0
nr_vmscan_write 11906
numa_hit 20852363
numa_miss 143448
numa_foreign 14808055
numa_interleave 3453
numa_local 20849005
numa_other 146806
protection: (0, 0, 0, 0)
pagesets
cpu: 0 pcp: 0
count: 0
high: 186
batch: 31
cpu: 0 pcp: 1
count: 0
high: 62
batch: 15
vm stats threshold: 18
cpu: 1 pcp: 0
count: 0
high: 186
batch: 31
cpu: 1 pcp: 1
count: 0
high: 62
batch: 15
vm stats threshold: 18
cpu: 2 pcp: 0
count: 120
high: 186
batch: 31
cpu: 2 pcp: 1
count: 11
high: 62
batch: 15
vm stats threshold: 18
cpu: 3 pcp: 0
count: 99
high: 186
batch: 31
cpu: 3 pcp: 1
count: 3
high: 62
batch: 15
vm stats threshold: 18
all_unreclaimable: 0
prev_priority: 12
start_pfn: 1048576

[-- Attachment #13: zoneinfo.end.txt --]
[-- Type: text/plain, Size: 6292 bytes --]

Node 0, zone DMA
pages free 2004
min 4
low 5
high 6
scanned 0 (a: 5 i: 5)
spanned 4096
present 2219
nr_free_pages 2004
nr_inactive 0
nr_active 0
nr_anon_pages 0
nr_mapped 0
nr_file_pages 0
nr_dirty 0
nr_writeback 0
nr_slab_reclaimable 0
nr_slab_unreclaimable 0
nr_page_table_pages 0
nr_unstable 0
nr_bounce 0
nr_vmscan_write 91
numa_hit 7439
numa_miss 12311
numa_foreign 0
numa_interleave 0
numa_local 7439
numa_other 12311
protection: (0, 2004, 2004, 2004)
pagesets
cpu: 0 pcp: 0
count: 0
high: 0
batch: 1
cpu: 0 pcp: 1
count: 0
high: 0
batch: 1
vm stats threshold: 6
cpu: 1 pcp: 0
count: 0
high: 0
batch: 1
cpu: 1 pcp: 1
count: 0
high: 0
batch: 1
vm stats threshold: 6
cpu: 2 pcp: 0
count: 0
high: 0
batch: 1
cpu: 2 pcp: 1
count: 0
high: 0
batch: 1
vm stats threshold: 6
cpu: 3 pcp: 0
count: 0
high: 0
batch: 1
cpu: 3 pcp: 1
count: 0
high: 0
batch: 1
vm stats threshold: 6
all_unreclaimable: 1
prev_priority: 12
start_pfn: 0
Node 0, zone DMA32
pages free 67771
min 1010
low 1262
high 1515
scanned 579 (a: 13 i: 1)
spanned 520192
present 513080
nr_free_pages 67771
nr_inactive 178
nr_active 2
nr_anon_pages 0
nr_mapped 5
nr_file_pages 151
nr_dirty 0
nr_writeback 0
nr_slab_reclaimable 157
nr_slab_unreclaimable 854
nr_page_table_pages 138
nr_unstable 0
nr_bounce 0
nr_vmscan_write 219556
numa_hit 160297810
numa_miss 21130360
numa_foreign 402418
numa_interleave 3463
numa_local 160297696
numa_other 21130474
protection: (0, 0, 0, 0)
pagesets
cpu: 0 pcp: 0
count: 176
high: 186
batch: 31
cpu: 0 pcp: 1
count: 53
high: 62
batch: 15
vm stats threshold: 30
cpu: 1 pcp: 0
count: 155
high: 186
batch: 31
cpu: 1 pcp: 1
count: 35
high: 62
batch: 15
vm stats threshold: 30
cpu: 2 pcp: 0
count: 58
high: 186
batch: 31
cpu: 2 pcp: 1
count: 13
high: 62
batch: 15
vm stats threshold: 30
cpu: 3 pcp: 0
count: 29
high: 186
batch: 31
cpu: 3 pcp: 1
count: 0
high: 62
batch: 15
vm stats threshold: 30
all_unreclaimable: 1
prev_priority: 12
start_pfn: 4096
Node 1, zone DMA32
pages free 1435
min 760
low 950
high 1140
scanned 60 (a: 0 i: 4)
spanned 524288
present 386000
nr_free_pages 1435
nr_inactive 3
nr_active 30
nr_anon_pages 0
nr_mapped 0
nr_file_pages 4
nr_dirty 1
nr_writeback 0
nr_slab_reclaimable 19
nr_slab_unreclaimable 17
nr_page_table_pages 0
nr_unstable 0
nr_bounce 0
nr_vmscan_write 3476
numa_hit 82078765
numa_miss 249206
numa_foreign 0
numa_interleave 8
numa_local 82078757
numa_other 249214
protection: (0, 0, 505, 505)
pagesets
cpu: 0 pcp: 0
count: 0
high: 186
batch: 31
cpu: 0 pcp: 1
count: 0
high: 62
batch: 15
vm stats threshold: 30
cpu: 1 pcp: 0
count: 0
high: 186
batch: 31
cpu: 1 pcp: 1
count: 0
high: 62
batch: 15
vm stats threshold: 30
cpu: 2 pcp: 0
count: 103
high: 186
batch: 31
cpu: 2 pcp: 1
count: 58
high: 62
batch: 15
vm stats threshold: 30
cpu: 3 pcp: 0
count: 105
high: 186
batch: 31
cpu: 3 pcp: 1
count: 55
high: 62
batch: 15
vm stats threshold: 30
all_unreclaimable: 0
prev_priority: 1
start_pfn: 524288
Node 1, zone Normal
pages free 315
min 254
low 317
high 381
scanned 186 (a: 25 i: 0)
spanned 131072
present 129280
nr_free_pages 315
nr_inactive 224
nr_active 28
nr_anon_pages 11
nr_mapped 72
nr_file_pages 244
nr_dirty 0
nr_writeback 0
nr_slab_reclaimable 99
nr_slab_unreclaimable 824
nr_page_table_pages 16
nr_unstable 0
nr_bounce 0
nr_vmscan_write 18978
numa_hit 23281331
numa_miss 153212
numa_foreign 21142691
numa_interleave 3453
numa_local 23277973
numa_other 156570
protection: (0, 0, 0, 0)
pagesets
cpu: 0 pcp: 0
count: 0
high: 186
batch: 31
cpu: 0 pcp: 1
count: 0
high: 62
batch: 15
vm stats threshold: 18
cpu: 1 pcp: 0
count: 0
high: 186
batch: 31
cpu: 1 pcp: 1
count: 0
high: 62
batch: 15
vm stats threshold: 18
cpu: 2 pcp: 0
count: 98
high: 186
batch: 31
cpu: 2 pcp: 1
count: 14
high: 62
batch: 15
vm stats threshold: 18
cpu: 3 pcp: 0
count: 63
high: 186
batch: 31
cpu: 3 pcp: 1
count: 43
high: 62
batch: 15
vm stats threshold: 18
all_unreclaimable: 1
prev_priority: 1
start_pfn: 1048576