* 2.6.24-rc7: memory leak?
@ 2008-01-17 6:34 CaT
2008-01-17 11:21 ` Pekka Enberg
2008-01-17 11:22 ` Peter Zijlstra
0 siblings, 2 replies; 10+ messages in thread
From: CaT @ 2008-01-17 6:34 UTC (permalink / raw)
To: linux-kernel
[-- Attachment #1: Type: text/plain, Size: 1274 bytes --]
Not sure where to begin so here goes anway. Today I did an rsync backup
of a server with 2million+ files. Before doing so the used memory on the
server this was initiated from was under 200meg (excluding buffers and
cache). During the rsync the memory used grew to just shy of 1.6gig and
now, about 2 hours after the rsync has well and truly finished, the used
memory is at 1.23gig. This is what free reports:
total used free shared buffers cached
Mem: 2058128 1994468 63660 0 688604 11432
-/+ buffers/cache: 1294432 763696
Swap: 1048568 0 1048568
There are 75 processes on the box of which almost 47 are kernel
processes + init. Of the rest, the top 3 have an RSS of 9.4meg, 6.2meg
and 4.8meg, 7 are at around 2meg and the rest are below 2meg with the
majority below 1. So unless I'm misunderstanding something, processess
alone do not account for the amount of used memory.
The destination of the rsync was an ext3 filesystem over raid5 over ahci
sata.
I've included /proc/meminfo, /proc/slabinfo and config.gz. If there's
anything else please shout.
--
"To the extent that we overreact, we proffer the terrorists the
greatest tribute."
- High Court Judge Michael Kirby
[-- Attachment #2: slabinfo --]
[-- Type: text/plain, Size: 14887 bytes --]
slabinfo - version: 2.1
# name <active_objs> <num_objs> <objsize> <objperslab> <pagesperslab> : tunables <limit> <batchcount> <sharedfactor> : slabdata <active_slabs> <num_slabs> <sharedavail>
ip_fib_alias 10 59 64 59 1 : tunables 120 60 8 : slabdata 1 1 0
ip_fib_hash 10 59 64 59 1 : tunables 120 60 8 : slabdata 1 1 0
raid5-md3 256 261 824 9 2 : tunables 54 27 8 : slabdata 29 29 0
UNIX 5 22 704 11 2 : tunables 54 27 8 : slabdata 2 2 0
xt_hashlimit 0 0 88 44 1 : tunables 120 60 8 : slabdata 0 0 0
flow_cache 0 0 128 30 1 : tunables 120 60 8 : slabdata 0 0 0
dm_snap_pending_exception 128 136 112 34 1 : tunables 120 60 8 : slabdata 4 4 0
dm_snap_exception 0 0 32 112 1 : tunables 120 60 8 : slabdata 0 0 0
dm_crypt_io 0 0 56 67 1 : tunables 120 60 8 : slabdata 0 0 0
dm_uevent 0 0 2608 3 2 : tunables 24 12 8 : slabdata 0 0 0
dm_target_io 827 864 24 144 1 : tunables 120 60 8 : slabdata 6 6 0
dm_io 826 828 40 92 1 : tunables 120 60 8 : slabdata 9 9 0
scsi_cmd_cache 50 50 384 10 1 : tunables 54 27 8 : slabdata 5 5 0
cfq_io_context 92 225 152 25 1 : tunables 120 60 8 : slabdata 9 9 0
cfq_queue 98 224 136 28 1 : tunables 120 60 8 : slabdata 8 8 0
bsg_cmd 0 0 312 12 1 : tunables 54 27 8 : slabdata 0 0 0
mqueue_inode_cache 1 4 896 4 1 : tunables 54 27 8 : slabdata 1 1 0
udf_inode_cache 0 0 656 6 1 : tunables 54 27 8 : slabdata 0 0 0
isofs_inode_cache 0 0 632 6 1 : tunables 54 27 8 : slabdata 0 0 0
fat_inode_cache 0 0 664 6 1 : tunables 54 27 8 : slabdata 0 0 0
fat_cache 0 0 32 112 1 : tunables 120 60 8 : slabdata 0 0 0
ext2_inode_cache 0 0 752 5 1 : tunables 54 27 8 : slabdata 0 0 0
journal_handle 32 144 24 144 1 : tunables 120 60 8 : slabdata 1 1 0
journal_head 129 200 96 40 1 : tunables 120 60 8 : slabdata 5 5 0
revoke_table 10 202 16 202 1 : tunables 120 60 8 : slabdata 1 1 0
revoke_record 0 0 32 112 1 : tunables 120 60 8 : slabdata 0 0 0
ext3_inode_cache 1235577 1240565 768 5 1 : tunables 54 27 8 : slabdata 248113 248113 0
dnotify_cache 0 0 40 92 1 : tunables 120 60 8 : slabdata 0 0 0
inotify_event_cache 0 0 40 92 1 : tunables 120 60 8 : slabdata 0 0 0
inotify_watch_cache 0 0 72 53 1 : tunables 120 60 8 : slabdata 0 0 0
kioctx 0 0 320 12 1 : tunables 54 27 8 : slabdata 0 0 0
kiocb 0 0 256 15 1 : tunables 120 60 8 : slabdata 0 0 0
fasync_cache 0 0 24 144 1 : tunables 120 60 8 : slabdata 0 0 0
shmem_inode_cache 6 15 776 5 1 : tunables 54 27 8 : slabdata 3 3 0
nsproxy 0 0 56 67 1 : tunables 120 60 8 : slabdata 0 0 0
posix_timers_cache 0 0 160 24 1 : tunables 120 60 8 : slabdata 0 0 0
uid_cache 1 15 256 15 1 : tunables 120 60 8 : slabdata 1 1 0
UDP-Lite 0 0 768 5 1 : tunables 54 27 8 : slabdata 0 0 0
tcp_bind_bucket 4 112 32 112 1 : tunables 120 60 8 : slabdata 1 1 0
inet_peer_cache 3 59 64 59 1 : tunables 120 60 8 : slabdata 1 1 0
secpath_cache 0 0 64 59 1 : tunables 120 60 8 : slabdata 0 0 0
xfrm_dst_cache 0 0 384 10 1 : tunables 54 27 8 : slabdata 0 0 0
ip_dst_cache 19 20 384 10 1 : tunables 54 27 8 : slabdata 2 2 0
arp_cache 5 15 256 15 1 : tunables 120 60 8 : slabdata 1 1 0
RAW 3 11 704 11 2 : tunables 54 27 8 : slabdata 1 1 0
UDP 3 5 768 5 1 : tunables 54 27 8 : slabdata 1 1 0
tw_sock_TCP 1 20 192 20 1 : tunables 120 60 8 : slabdata 1 1 0
request_sock_TCP 0 0 128 30 1 : tunables 120 60 8 : slabdata 0 0 0
TCP 5 10 1536 5 2 : tunables 24 12 8 : slabdata 2 2 0
eventpoll_pwq 0 0 72 53 1 : tunables 120 60 8 : slabdata 0 0 0
eventpoll_epi 0 0 128 30 1 : tunables 120 60 8 : slabdata 0 0 0
sgpool-128 2 2 4096 1 1 : tunables 24 12 8 : slabdata 2 2 0
sgpool-64 2 2 2048 2 1 : tunables 24 12 8 : slabdata 1 1 0
sgpool-32 2 4 1024 4 1 : tunables 54 27 8 : slabdata 1 1 0
sgpool-16 2 8 512 8 1 : tunables 54 27 8 : slabdata 1 1 0
sgpool-8 45 45 256 15 1 : tunables 120 60 8 : slabdata 3 3 0
scsi_io_context 0 0 112 34 1 : tunables 120 60 8 : slabdata 0 0 0
blkdev_ioc 47 177 64 59 1 : tunables 120 60 8 : slabdata 3 3 0
blkdev_queue 22 25 1624 5 2 : tunables 24 12 8 : slabdata 5 5 0
blkdev_requests 42 78 288 13 1 : tunables 54 27 8 : slabdata 5 6 0
biovec-256 50 50 4096 1 1 : tunables 24 12 8 : slabdata 50 50 0
biovec-128 50 50 2048 2 1 : tunables 24 12 8 : slabdata 25 25 0
biovec-64 50 56 1024 4 1 : tunables 54 27 8 : slabdata 14 14 0
biovec-16 50 75 256 15 1 : tunables 120 60 8 : slabdata 5 5 0
biovec-4 66 118 64 59 1 : tunables 120 60 8 : slabdata 2 2 0
biovec-1 126 202 16 202 1 : tunables 120 60 8 : slabdata 1 1 0
bio 180 180 128 30 1 : tunables 120 60 8 : slabdata 6 6 0
sock_inode_cache 25 40 704 5 1 : tunables 54 27 8 : slabdata 8 8 0
skbuff_fclone_cache 16 16 448 8 1 : tunables 54 27 8 : slabdata 2 2 0
skbuff_head_cache 345 435 256 15 1 : tunables 120 60 8 : slabdata 29 29 0
file_lock_cache 2 22 176 22 1 : tunables 120 60 8 : slabdata 1 1 0
Acpi-Operand 1406 1534 64 59 1 : tunables 120 60 8 : slabdata 26 26 0
Acpi-ParseExt 0 0 64 59 1 : tunables 120 60 8 : slabdata 0 0 0
Acpi-Parse 0 0 40 92 1 : tunables 120 60 8 : slabdata 0 0 0
Acpi-State 0 0 80 48 1 : tunables 120 60 8 : slabdata 0 0 0
Acpi-Namespace 614 672 32 112 1 : tunables 120 60 8 : slabdata 6 6 0
proc_inode_cache 414 414 616 6 1 : tunables 54 27 8 : slabdata 69 69 0
sigqueue 33 48 160 24 1 : tunables 120 60 8 : slabdata 2 2 0
radix_tree_node 20128 24703 552 7 1 : tunables 54 27 8 : slabdata 3529 3529 0
bdev_cache 24 28 832 4 1 : tunables 54 27 8 : slabdata 7 7 0
sysfs_dir_cache 4446 4512 80 48 1 : tunables 120 60 8 : slabdata 94 94 0
mnt_cache 24 45 256 15 1 : tunables 120 60 8 : slabdata 3 3 0
inode_cache 64 77 584 7 1 : tunables 54 27 8 : slabdata 11 11 0
dentry 703661 749797 200 19 1 : tunables 120 60 8 : slabdata 39463 39463 0
filp 330 600 192 20 1 : tunables 120 60 8 : slabdata 30 30 0
names_cache 1 1 4096 1 1 : tunables 24 12 8 : slabdata 1 1 0
idr_layer_cache 121 126 528 7 1 : tunables 54 27 8 : slabdata 18 18 0
buffer_head 174535 209087 104 37 1 : tunables 120 60 8 : slabdata 5651 5651 0
mm_struct 54 54 832 9 2 : tunables 54 27 8 : slabdata 6 6 0
vm_area_struct 913 1081 168 23 1 : tunables 120 60 8 : slabdata 47 47 0
fs_cache 45 177 64 59 1 : tunables 120 60 8 : slabdata 3 3 0
files_cache 46 77 704 11 2 : tunables 54 27 8 : slabdata 7 7 0
signal_cache 93 95 768 5 1 : tunables 54 27 8 : slabdata 19 19 0
sighand_cache 87 87 2112 3 2 : tunables 24 12 8 : slabdata 29 29 0
task_struct 89 92 1824 2 1 : tunables 24 12 8 : slabdata 46 46 0
anon_vma 358 576 24 144 1 : tunables 120 60 8 : slabdata 4 4 0
pid_namespace 0 0 2104 3 2 : tunables 24 12 8 : slabdata 0 0 0
pid_1 95 180 128 30 1 : tunables 120 60 8 : slabdata 6 6 0
size-4194304(DMA) 0 0 4194304 1 1024 : tunables 1 1 0 : slabdata 0 0 0
size-4194304 0 0 4194304 1 1024 : tunables 1 1 0 : slabdata 0 0 0
size-2097152(DMA) 0 0 2097152 1 512 : tunables 1 1 0 : slabdata 0 0 0
size-2097152 0 0 2097152 1 512 : tunables 1 1 0 : slabdata 0 0 0
size-1048576(DMA) 0 0 1048576 1 256 : tunables 1 1 0 : slabdata 0 0 0
size-1048576 0 0 1048576 1 256 : tunables 1 1 0 : slabdata 0 0 0
size-524288(DMA) 0 0 524288 1 128 : tunables 1 1 0 : slabdata 0 0 0
size-524288 0 0 524288 1 128 : tunables 1 1 0 : slabdata 0 0 0
size-262144(DMA) 0 0 262144 1 64 : tunables 1 1 0 : slabdata 0 0 0
size-262144 0 0 262144 1 64 : tunables 1 1 0 : slabdata 0 0 0
size-131072(DMA) 0 0 131072 1 32 : tunables 8 4 0 : slabdata 0 0 0
size-131072 0 0 131072 1 32 : tunables 8 4 0 : slabdata 0 0 0
size-65536(DMA) 0 0 65536 1 16 : tunables 8 4 0 : slabdata 0 0 0
size-65536 0 0 65536 1 16 : tunables 8 4 0 : slabdata 0 0 0
size-32768(DMA) 0 0 32768 1 8 : tunables 8 4 0 : slabdata 0 0 0
size-32768 0 0 32768 1 8 : tunables 8 4 0 : slabdata 0 0 0
size-16384(DMA) 0 0 16384 1 4 : tunables 8 4 0 : slabdata 0 0 0
size-16384 9 9 16384 1 4 : tunables 8 4 0 : slabdata 9 9 0
size-8192(DMA) 0 0 8192 1 2 : tunables 8 4 0 : slabdata 0 0 0
size-8192 5 5 8192 1 2 : tunables 8 4 0 : slabdata 5 5 0
size-4096(DMA) 0 0 4096 1 1 : tunables 24 12 8 : slabdata 0 0 0
size-4096 121 121 4096 1 1 : tunables 24 12 8 : slabdata 121 121 0
size-2048(DMA) 0 0 2048 2 1 : tunables 24 12 8 : slabdata 0 0 0
size-2048 482 488 2048 2 1 : tunables 24 12 8 : slabdata 244 244 6
size-1024(DMA) 0 0 1024 4 1 : tunables 54 27 8 : slabdata 0 0 0
size-1024 364 384 1024 4 1 : tunables 54 27 8 : slabdata 96 96 0
size-512(DMA) 0 0 512 8 1 : tunables 54 27 8 : slabdata 0 0 0
size-512 213 224 512 8 1 : tunables 54 27 8 : slabdata 28 28 0
size-256(DMA) 0 0 256 15 1 : tunables 120 60 8 : slabdata 0 0 0
size-256 75 75 256 15 1 : tunables 120 60 8 : slabdata 5 5 0
size-192(DMA) 0 0 192 20 1 : tunables 120 60 8 : slabdata 0 0 0
size-192 279 320 192 20 1 : tunables 120 60 8 : slabdata 16 16 0
size-128(DMA) 0 0 128 30 1 : tunables 120 60 8 : slabdata 0 0 0
size-64(DMA) 0 0 64 59 1 : tunables 120 60 8 : slabdata 0 0 0
size-64 537590 850249 64 59 1 : tunables 120 60 8 : slabdata 14411 14411 0
size-32(DMA) 0 0 32 112 1 : tunables 120 60 8 : slabdata 0 0 0
size-128 46918 52140 128 30 1 : tunables 120 60 8 : slabdata 1738 1738 0
size-32 4756 4816 32 112 1 : tunables 120 60 8 : slabdata 43 43 0
kmem_cache 136 150 128 30 1 : tunables 120 60 8 : slabdata 5 5 0
[-- Attachment #3: meminfo --]
[-- Type: text/plain, Size: 630 bytes --]
MemTotal: 2058128 kB
MemFree: 56080 kB
Buffers: 689768 kB
Cached: 14240 kB
SwapCached: 0 kB
Active: 464176 kB
Inactive: 265096 kB
SwapTotal: 1048568 kB
SwapFree: 1048568 kB
Dirty: 48 kB
Writeback: 0 kB
AnonPages: 25432 kB
Mapped: 6820 kB
Slab: 1257136 kB
SReclaimable: 1173316 kB
SUnreclaim: 83820 kB
PageTables: 924 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
CommitLimit: 2077632 kB
Committed_AS: 50976 kB
VmallocTotal: 34359738367 kB
VmallocUsed: 66284 kB
VmallocChunk: 34359671799 kB
[-- Attachment #4: config.gz --]
[-- Type: application/octet-stream, Size: 9057 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: 2.6.24-rc7: memory leak?
2008-01-17 6:34 2.6.24-rc7: memory leak? CaT
@ 2008-01-17 11:21 ` Pekka Enberg
2008-01-17 11:22 ` Peter Zijlstra
1 sibling, 0 replies; 10+ messages in thread
From: Pekka Enberg @ 2008-01-17 11:21 UTC (permalink / raw)
To: CaT; +Cc: linux-kernel
Hi,
On Jan 17, 2008 8:34 AM, CaT <cat@zip.com.au> wrote:
> There are 75 processes on the box of which almost 47 are kernel
> processes + init. Of the rest, the top 3 have an RSS of 9.4meg, 6.2meg
> and 4.8meg, 7 are at around 2meg and the rest are below 2meg with the
> majority below 1. So unless I'm misunderstanding something, processess
> alone do not account for the amount of used memory.
Looking at /proc/slabinfo, 965 MB is used by the ext3 inode cache and
144 MB by the dentry cache.
Pekka
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: 2.6.24-rc7: memory leak?
2008-01-17 6:34 2.6.24-rc7: memory leak? CaT
2008-01-17 11:21 ` Pekka Enberg
@ 2008-01-17 11:22 ` Peter Zijlstra
2008-01-17 11:40 ` CaT
1 sibling, 1 reply; 10+ messages in thread
From: Peter Zijlstra @ 2008-01-17 11:22 UTC (permalink / raw)
To: CaT; +Cc: linux-kernel
On Thu, 2008-01-17 at 17:34 +1100, CaT wrote:
> Not sure where to begin so here goes anway. Today I did an rsync backup
> of a server with 2million+ files. Before doing so the used memory on the
> server this was initiated from was under 200meg (excluding buffers and
> cache). During the rsync the memory used grew to just shy of 1.6gig and
> now, about 2 hours after the rsync has well and truly finished, the used
> memory is at 1.23gig. This is what free reports:
>
> total used free shared buffers cached
> Mem: 2058128 1994468 63660 0 688604 11432
> -/+ buffers/cache: 1294432 763696
> Swap: 1048568 0 1048568
>
> There are 75 processes on the box of which almost 47 are kernel
> processes + init. Of the rest, the top 3 have an RSS of 9.4meg, 6.2meg
> and 4.8meg, 7 are at around 2meg and the rest are below 2meg with the
> majority below 1. So unless I'm misunderstanding something, processess
> alone do not account for the amount of used memory.
>
> The destination of the rsync was an ext3 filesystem over raid5 over ahci
> sata.
>
> I've included /proc/meminfo, /proc/slabinfo and config.gz. If there's
> anything else please shout.
How much memory does:
echo 3 > /proc/sys/vm/drop_caches
gain you?
> ext3_inode_cache 1235577 1240565 768 5 1 : tunables 54 27 8 : slabdata 248113 248113 0
> dentry 703661 749797 200 19 1 : tunables 120 60 8 : slabdata 39463 39463 0
> buffer_head 174535 209087 104 37 1 : tunables 120 60 8 : slabdata 5651 5651 0
would get freed by doing that.
this one:
> size-64 537590 850249 64 59 1 : tunables 120 60 8 : slabdata 14411 14411 0
I'm unsure about, if that one sticks around that'd be something to worry
about. See if you can monitor this value and try to determine:
- if it ever drops
- what makes it grow (fastest)
I guess we could stick some instrumentation in there to track that
bucket.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: 2.6.24-rc7: memory leak?
2008-01-17 11:22 ` Peter Zijlstra
@ 2008-01-17 11:40 ` CaT
2008-01-17 12:07 ` Peter Zijlstra
0 siblings, 1 reply; 10+ messages in thread
From: CaT @ 2008-01-17 11:40 UTC (permalink / raw)
To: Peter Zijlstra; +Cc: linux-kernel
On Thu, Jan 17, 2008 at 12:22:03PM +0100, Peter Zijlstra wrote:
> On Thu, 2008-01-17 at 17:34 +1100, CaT wrote:
> > cache). During the rsync the memory used grew to just shy of 1.6gig and
> > now, about 2 hours after the rsync has well and truly finished, the used
> > memory is at 1.23gig. This is what free reports:
> >
> > total used free shared buffers cached
> > Mem: 2058128 1994468 63660 0 688604 11432
> > -/+ buffers/cache: 1294432 763696
> > Swap: 1048568 0 1048568
>
> How much memory does:
>
> echo 3 > /proc/sys/vm/drop_caches
>
> gain you?
56M used now. Should all this cache usage not be counted towards the
'Cached' entry in meminfo rather then getting counted as part of used
ram. I assume that this would not cause an oom situation and would be
freed up if all that memory really did need to be used.
> > ext3_inode_cache 1235577 1240565 768 5 1 : tunables 54 27 8 : slabdata 248113 248113 0
> > dentry 703661 749797 200 19 1 : tunables 120 60 8 : slabdata 39463 39463 0
> > buffer_head 174535 209087 104 37 1 : tunables 120 60 8 : slabdata 5651 5651 0
>
> would get freed by doing that.
They were indeed.
> this one:
>
> > size-64 537590 850249 64 59 1 : tunables 120 60 8 : slabdata 14411 14411 0
>
> I'm unsure about, if that one sticks around that'd be something to worry
> about. See if you can monitor this value and try to determine:
This went away also.
> - if it ever drops
> - what makes it grow (fastest)
>
> I guess we could stick some instrumentation in there to track that
> bucket.
Might help prevent upraised eyebrows or worse. :)
--
"To the extent that we overreact, we proffer the terrorists the
greatest tribute."
- High Court Judge Michael Kirby
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: 2.6.24-rc7: memory leak?
2008-01-17 11:40 ` CaT
@ 2008-01-17 12:07 ` Peter Zijlstra
2008-01-17 12:14 ` CaT
0 siblings, 1 reply; 10+ messages in thread
From: Peter Zijlstra @ 2008-01-17 12:07 UTC (permalink / raw)
To: CaT; +Cc: linux-kernel
On Thu, 2008-01-17 at 22:40 +1100, CaT wrote:
> On Thu, Jan 17, 2008 at 12:22:03PM +0100, Peter Zijlstra wrote:
> > On Thu, 2008-01-17 at 17:34 +1100, CaT wrote:
> > > cache). During the rsync the memory used grew to just shy of 1.6gig and
> > > now, about 2 hours after the rsync has well and truly finished, the used
> > > memory is at 1.23gig. This is what free reports:
> > >
> > > total used free shared buffers cached
> > > Mem: 2058128 1994468 63660 0 688604 11432
> > > -/+ buffers/cache: 1294432 763696
> > > Swap: 1048568 0 1048568
> >
> > How much memory does:
> >
> > echo 3 > /proc/sys/vm/drop_caches
> >
> > gain you?
>
> 56M used now.
Good :-)
> Should all this cache usage not be counted towards the
> 'Cached' entry in meminfo rather then getting counted as part of used
> ram.
Cached is only the page-cache, not all the other caches we have..
This someones confuses people, but one gets used to it. slabinfo allows
one to easily view others.
> I assume that this would not cause an oom situation and would be
> freed up if all that memory really did need to be used.
Correct.
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: 2.6.24-rc7: memory leak?
2008-01-17 12:07 ` Peter Zijlstra
@ 2008-01-17 12:14 ` CaT
2008-01-17 12:21 ` Peter Zijlstra
2008-01-17 12:22 ` KOSAKI Motohiro
0 siblings, 2 replies; 10+ messages in thread
From: CaT @ 2008-01-17 12:14 UTC (permalink / raw)
To: Peter Zijlstra; +Cc: linux-kernel
On Thu, Jan 17, 2008 at 01:07:12PM +0100, Peter Zijlstra wrote:
> On Thu, 2008-01-17 at 22:40 +1100, CaT wrote:
> > > How much memory does:
> > >
> > > echo 3 > /proc/sys/vm/drop_caches
> > >
> > > gain you?
> >
> > 56M used now.
>
> Good :-)
Indeed. :)
> > Should all this cache usage not be counted towards the
> > 'Cached' entry in meminfo rather then getting counted as part of used
> > ram.
>
> Cached is only the page-cache, not all the other caches we have..
> This someones confuses people, but one gets used to it. slabinfo allows
> one to easily view others.
So what would I have to do to find out the real amount of memory free?
Would that be the MemFree field in /proc/meminfo plus, from
/proc/slabinfo, *_cache and size-*?
If that's the case it would seem to be somewhat of a pain to get and
kind of out of left field as I'd say most people would expect MemFree to
indicate the amount of memory that's no longer freely available
(ignoring swapping it out for simplicities sake).
I'm wondering as I'm trying to monitor this stuff and I've just found
out that the easily accessible values are not as useful as I previously
thought.
--
"To the extent that we overreact, we proffer the terrorists the
greatest tribute."
- High Court Judge Michael Kirby
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: 2.6.24-rc7: memory leak?
2008-01-17 12:14 ` CaT
@ 2008-01-17 12:21 ` Peter Zijlstra
2008-01-17 12:28 ` CaT
2008-01-17 12:22 ` KOSAKI Motohiro
1 sibling, 1 reply; 10+ messages in thread
From: Peter Zijlstra @ 2008-01-17 12:21 UTC (permalink / raw)
To: CaT; +Cc: linux-kernel
On Thu, 2008-01-17 at 23:14 +1100, CaT wrote:
> On Thu, Jan 17, 2008 at 01:07:12PM +0100, Peter Zijlstra wrote:
> > On Thu, 2008-01-17 at 22:40 +1100, CaT wrote:
> > > > How much memory does:
> > > >
> > > > echo 3 > /proc/sys/vm/drop_caches
> > > >
> > > > gain you?
> > >
> > > 56M used now.
> >
> > Good :-)
>
> Indeed. :)
>
> > > Should all this cache usage not be counted towards the
> > > 'Cached' entry in meminfo rather then getting counted as part of used
> > > ram.
> >
> > Cached is only the page-cache, not all the other caches we have..
> > This someones confuses people, but one gets used to it. slabinfo allows
> > one to easily view others.
>
> So what would I have to do to find out the real amount of memory free?
> Would that be the MemFree field in /proc/meminfo plus, from
> /proc/slabinfo, *_cache and size-*?
MemFree is the actual amount of free memory. Page reclaim and such
might, on memory pressure, free memory from cached, buffered and some
slabs that have shrinkers.
> If that's the case it would seem to be somewhat of a pain to get and
> kind of out of left field as I'd say most people would expect MemFree to
> indicate the amount of memory that's no longer freely available
> (ignoring swapping it out for simplicities sake).
I'm somewhat confused as to what you're saying. How would anyone expect
MemFree to be memory that is _not_ freely available?
Or are you asking how to compute the amount of freeable memory, that is
memory that isn't currently free, but could be freed up under pressure?
If that is indeed your question, then yes, thats rather hard as slabinfo
and the like don't indicate which buckets have shrinkers (but even if
they would have shrinkers there is no guarantee they'd be able to shrink
100%)
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: 2.6.24-rc7: memory leak?
2008-01-17 12:21 ` Peter Zijlstra
@ 2008-01-17 12:28 ` CaT
0 siblings, 0 replies; 10+ messages in thread
From: CaT @ 2008-01-17 12:28 UTC (permalink / raw)
To: Peter Zijlstra; +Cc: linux-kernel
On Thu, Jan 17, 2008 at 01:21:51PM +0100, Peter Zijlstra wrote:
> > If that's the case it would seem to be somewhat of a pain to get and
> > kind of out of left field as I'd say most people would expect MemFree to
> > indicate the amount of memory that's no longer freely available
> > (ignoring swapping it out for simplicities sake).
>
> I'm somewhat confused as to what you're saying. How would anyone expect
> MemFree to be memory that is _not_ freely available?
I think we have conflicting ideas of what 'freely available' means here
which is causing the confusion. :)
> Or are you asking how to compute the amount of freeable memory, that is
> memory that isn't currently free, but could be freed up under pressure?
Yes. I think I am. (heh)
> If that is indeed your question, then yes, thats rather hard as slabinfo
> and the like don't indicate which buckets have shrinkers (but even if
Ah. Foo.
> they would have shrinkers there is no guarantee they'd be able to shrink
> 100%)
Whacky.
:/
--
"To the extent that we overreact, we proffer the terrorists the
greatest tribute."
- High Court Judge Michael Kirby
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: 2.6.24-rc7: memory leak?
2008-01-17 12:14 ` CaT
2008-01-17 12:21 ` Peter Zijlstra
@ 2008-01-17 12:22 ` KOSAKI Motohiro
2008-01-17 12:35 ` CaT
1 sibling, 1 reply; 10+ messages in thread
From: KOSAKI Motohiro @ 2008-01-17 12:22 UTC (permalink / raw)
To: CaT; +Cc: kosaki.motohiro, Peter Zijlstra, linux-kernel
Hi
> > > Should all this cache usage not be counted towards the
> > > 'Cached' entry in meminfo rather then getting counted as part of used
> > > ram.
> >
> > Cached is only the page-cache, not all the other caches we have..
> > This someones confuses people, but one gets used to it. slabinfo allows
> > one to easily view others.
>
> So what would I have to do to find out the real amount of memory free?
> Would that be the MemFree field in /proc/meminfo plus, from
> /proc/slabinfo, *_cache and size-*?
SReclaimable field can't fill requilrement? :)
-kosaki
^ permalink raw reply [flat|nested] 10+ messages in thread
* Re: 2.6.24-rc7: memory leak?
2008-01-17 12:22 ` KOSAKI Motohiro
@ 2008-01-17 12:35 ` CaT
0 siblings, 0 replies; 10+ messages in thread
From: CaT @ 2008-01-17 12:35 UTC (permalink / raw)
To: KOSAKI Motohiro; +Cc: Peter Zijlstra, linux-kernel
On Thu, Jan 17, 2008 at 09:22:50PM +0900, KOSAKI Motohiro wrote:
> Hi
>
> > > > Should all this cache usage not be counted towards the
> > > > 'Cached' entry in meminfo rather then getting counted as part of used
> > > > ram.
> > >
> > > Cached is only the page-cache, not all the other caches we have..
> > > This someones confuses people, but one gets used to it. slabinfo allows
> > > one to easily view others.
> >
> > So what would I have to do to find out the real amount of memory free?
> > Would that be the MemFree field in /proc/meminfo plus, from
> > /proc/slabinfo, *_cache and size-*?
>
> SReclaimable field can't fill requilrement? :)
Ah. Hmm. *pokes about* That does seem to correspond pretty closely to
what happens with the echo 3 >/proc/.... Though it never seems to get to
0 (~4 meg is as close as it gets on this box) which seems to jive with
what Peter said in the other email.
Hmm. I think tomorrow I'll be writing free.pl and modifying my graphing
and monitoring stuff to take this into account somehow. :)
Thanks!
--
"To the extent that we overreact, we proffer the terrorists the
greatest tribute."
- High Court Judge Michael Kirby
^ permalink raw reply [flat|nested] 10+ messages in thread
end of thread, other threads:[~2008-01-17 12:38 UTC | newest]
Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2008-01-17 6:34 2.6.24-rc7: memory leak? CaT
2008-01-17 11:21 ` Pekka Enberg
2008-01-17 11:22 ` Peter Zijlstra
2008-01-17 11:40 ` CaT
2008-01-17 12:07 ` Peter Zijlstra
2008-01-17 12:14 ` CaT
2008-01-17 12:21 ` Peter Zijlstra
2008-01-17 12:28 ` CaT
2008-01-17 12:22 ` KOSAKI Motohiro
2008-01-17 12:35 ` CaT
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox