* Re: Unfreeable buffer/cache problem in 2.4.17-rc1 still there
@ 2001-12-16 19:39 Dmitry Volkoff
2001-12-17 17:01 ` vda
0 siblings, 1 reply; 11+ messages in thread
From: Dmitry Volkoff @ 2001-12-16 19:39 UTC (permalink / raw)
To: linux-kernel
Hello!
Below is simple test case which I think is related to "memory disappear"
problem.
My real program is doing something like this:
// test.c
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <stdio.h>
#include <unistd.h>
int main(void)
{
int fd;
int r;
char data[10] = "0123456789";
int i;
int end = 30;
for (i=0;i<end;i++) {
fd = open("testfile", O_WRONLY | O_NDELAY | O_TRUNC | O_CREAT, 0644);
if (fd == -1) {
printf("unable to open\n");
return;
}
r = write(fd,data,sizeof data);
if (r == -1) {
printf("unable to write\n");
close(fd);
return;
}
close(fd);
sleep(1);
}
}
// end test.c
Each time I run `free; ./test; free` I see evergrowing memory usage.
I mean used memory + buffers. At some point system just starts swapping
even if no other processes are running. Tested on 2.4.13 and
2.4.17-pre4aa1. I think something is wrong here because the very same
test program does not show such behaviour on 2.2.19. It does not lose
any single byte of memory. I've even tested this on freebsd-4.4 with
the same result as on 2.2.19.
Example output on 2.4.17-pre4aa1:
bash-2.03$ free; ./test; free
total used free shared buffers cached
Mem: 514528 212508 302020 0 7952 150664
-/+ buffers/cache: 53892 460636
Swap: 1028120 0 1028120
total used free shared buffers cached
Mem: 514528 212616 301912 0 8000 150664
-/+ buffers/cache: 53952 460576
Swap: 1028120 0 1028120
bash-2.03$ free; ./test; free
total used free shared buffers cached
Mem: 514528 212616 301912 0 8008 150664
-/+ buffers/cache: 53944 460584
Swap: 1028120 0 1028120
total used free shared buffers cached
Mem: 514528 212700 301828 0 8056 150664
-/+ buffers/cache: 53980 460548
Swap: 1028120 0 1028120
The results are very consistent. I lose 30-40 byte per run.
--
DV
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: Unfreeable buffer/cache problem in 2.4.17-rc1 still there
2001-12-16 19:39 Unfreeable buffer/cache problem in 2.4.17-rc1 still there Dmitry Volkoff
@ 2001-12-17 17:01 ` vda
2001-12-18 4:26 ` Dmitry Volkoff
0 siblings, 1 reply; 11+ messages in thread
From: vda @ 2001-12-17 17:01 UTC (permalink / raw)
To: Dmitry Volkoff, linux-kernel
[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="PT 154", Size: 980 bytes --]
On Sunday 16 December 2001 17:39, Dmitry Volkoff wrote:
> Hello!
>
> Below is simple test case which I think is related to "memory disappear"
> problem.
>
> My real program is doing something like this:
>
> // test.c
> #include <sys/types.h>
> #include <sys/stat.h>
> #include <fcntl.h>
> #include <stdio.h>
> #include <unistd.h>
>
> int main(void)
> {
> int fd;
> int r;
> char data[10] = "0123456789";
> int i;
> int end = 30;
> for (i=0;i<end;i++) {
> fd = open("testfile", O_WRONLY | O_NDELAY | O_TRUNC | O_CREAT, 0644);
> if (fd == -1) {
> printf("unable to open\n");
> return;
> }
> r = write(fd,data,sizeof data);
> if (r == -1) {
> printf("unable to write\n");
> close(fd);
> return;
> }
> close(fd);
> sleep(1);
> }
> }
> // end test.c
I removed sleep(1). Is it needed?
After 10000+ runs of this proggy swap usage isn't changed on 2.4.17-pre7.
top reports constant 2304K of swap usage.
--
vda
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: Unfreeable buffer/cache problem in 2.4.17-rc1 still there
2001-12-17 17:01 ` vda
@ 2001-12-18 4:26 ` Dmitry Volkoff
0 siblings, 0 replies; 11+ messages in thread
From: Dmitry Volkoff @ 2001-12-18 4:26 UTC (permalink / raw)
To: linux-kernel
On Mon, Dec 17, 2001 at 03:01:12PM -0200, vda wrote:
> > if (r == -1) {
> > printf("unable to write\n");
> > close(fd);
> > return;
> > }
> > close(fd);
> > sleep(1);
> > }
> > }
> > // end test.c
>
> I removed sleep(1). Is it needed?
>
Yes, you need it in order to see the memory leakage.
> After 10000+ runs of this proggy swap usage isn't changed on 2.4.17-pre7.
> top reports constant 2304K of swap usage.
I know. You'll notice this effect only after 1000000+ runs.
Try it again with sleep(1).
> --
> vda
>
--
DV
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Unfreeable buffer/cache problem in 2.4.17-rc1 still there
@ 2001-12-16 20:41 Willy Tarreau
0 siblings, 0 replies; 11+ messages in thread
From: Willy Tarreau @ 2001-12-16 20:41 UTC (permalink / raw)
To: vdb; +Cc: linux-kernel
Hi Dmitry !
> Below is simple test case which I think is related to "memory disappear"
> problem.
--snip--
> The results are very consistent. I lose 30-40 byte per run.
Nearly the same here on 2.4.10-ac12, but I loose only 4-8 kB each time.
So I think this is not related to the new VM, and perhaps it's a very
old thing.
Regards,
Willy
^ permalink raw reply [flat|nested] 11+ messages in thread
[parent not found: <Pine.LNX.4.33.0112151141280.19022-100000@coffee.psychology.mcmaster.ca>]
* Re: Unfreeable buffer/cache problem in 2.4.17-rc1 still there
[not found] <Pine.LNX.4.33.0112151141280.19022-100000@coffee.psychology.mcmaster.ca>
@ 2001-12-15 17:09 ` Chris Chabot
0 siblings, 0 replies; 11+ messages in thread
From: Chris Chabot @ 2001-12-15 17:09 UTC (permalink / raw)
To: linux-kernel; +Cc: hahn, andrea, marcelo, brownfld
Mark Han wrote,
> first, forget silly crap like top; look at the /proc files.
> inode/dentry caches are just slab caches, which afaik
> are not considered part of the 'cached' that top is talking about.
> (since top preceeds slab and is referring to buffer+page caches.)
Man somebody got out of the wrong side of the bed this morning ;-) The
problem is that those prety user end lights are the only things the user
see's ;-) So it might be worth considering exporting this 'secret'
information to the user end (count it as cache in /proc/memusage?)
> what makes you think there's anything wrong with this? you have tons of
> memory, and aren't using it hard, so the kernel uses it to cache files,
> inodes, dentries, etc.
I know, cache is good. However first of all, 400 to 600 mb of cache used
for dentries/inodes seems a little steep to me (as not kernel hacker),
and when i do fire up memory hogging applications (mysql,apache,java
etc) the 'evaporated' memory is not returned for those applications.
Resulting in heavy swapping and a non-responsive system. For a dual p3
with 1 gig of ram, this feels like a problem, yes ;-)
do note then when i do a simple find /, it do see the memory being used
in cached and buffers. This is not the case for the 'missing memory'
Ken Brownfield wrote,
> I think "updatedb" at 4am is what you're looking for... How much disk
> space do you have on this system?
The system has 2 x 18Gb scsi disks (/ and /home) and a single raid0 volume (4x 80 gig ide) as archival storage. Doing a find | wc -l on the archives alone tells me i have more then 340000 files there.. (ranging between a few bytes to > 1 gig)
But indeed, when i run updatedb, the problem of non-visable memory (for
me using top / free anyways) does apear.. good catch!
Andrea wrote,
> this is an icache/dcache problem, can you reproduce on 2.4.17rc1aa1,
> it will shrink more aggressively.
doing that (updatedb), i dont have to wait a day or so to see what
happens, mem free (+buffers + cache from /proc/meminfo) is around 550Mb,
'used memory' (counting ps aux res usage) is < 100Mb. So quite a couple
of megabytes have disapeard again ;/
-- Chris
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Unfreeable buffer/cache problem in 2.4.17-rc1 still there
@ 2001-12-15 16:05 Ed Tomlinson
0 siblings, 0 replies; 11+ messages in thread
From: Ed Tomlinson @ 2001-12-15 16:05 UTC (permalink / raw)
To: linux-kernel; +Cc: Andrea Arcangeli
Andrea Arcangeli wrote:
> On Sat, Dec 15, 2001 at 01:36:11PM +0100, Chris Chabot wrote:
>> inode_cache 686896 686896 480 85862 85862 1 : 124 62
>> dentry_cache 696810 696810 128 23227 23227 1 : 252 126
>
> this is an icache/dcache problem, can you reproduce on 2.4.17rc1aa1, it
> will shrink more aggressively.
>
> really to get an even better balance we should add the icache/dcache
> slab pages into the lru as well... that would trigger the icache/dcache
> flushes more easily when too much ram is in those caches.
Interesting idea. Is this what you are thinking? We find a slab page at the tail of the
lru so we call the related shrink function. If the page is still active after shrinking, we
requeue it at the head of the lru. The slab page freeing logic would have to how to
unlink from the lru.
The priority arguement of the shink functions would now allow us to keep the ratio of
lru size vs icache/dcache/dqcache under control. This might be a knob that would
be interesting to have in proc...
Comments?
Ed Tomlinson
^ permalink raw reply [flat|nested] 11+ messages in thread
* Unfreeable buffer/cache problem in 2.4.17-rc1 still there
@ 2001-12-15 12:36 Chris Chabot
[not found] ` <004601c18567$a8be1600$0801a8c0@Stev.org>
2001-12-15 14:24 ` Andrea Arcangeli
0 siblings, 2 replies; 11+ messages in thread
From: Chris Chabot @ 2001-12-15 12:36 UTC (permalink / raw)
To: linux-kernel; +Cc: marcelo
[-- Attachment #1: Type: text/plain, Size: 1724 bytes --]
Marcelo Tosatti wrote,
>Well, I want people with the "unfreeable" buffer/cache problem to
>confirm with me that 2.4.17-rc1 is working ok.
I'm afraid the problem still seems to exist on my box. It takes a while
to kick in, but after 1 day and 8 hours, i have just lost about 400 mb
of usable memory (!)
I have attached the output from free, slabinfo and a ps aux for 2.4.16
after 1 day and 20 hours, and for 2.4.17-rc1 after 1 day and 8 hours.
When the system comes fresh out of a reboot, there's about 900 and some
megabytes of memory 'free' in the 'free' output.. this number has
remained steady for about a day or so (it was still ok last night), then
i checked this morning, and 400 megs of memory seems to evaporated on me
;-)
The box is a dual p3-600, 1 gig of (ecc) ram, AIC7xxx/u2w, 2 scsi disks
(/ and /home), scsi cdrom, scsi DLT tape, 4 ide disks (internal PIIX4
ide controller) in a single raid0 volume (320 gigs), using ext3 on all
mounts, and 3 network cards (1 3com 3c905tx and 2 intel etherexpresspro
10/100's).
the box runs a basic firewall setup (with a ip route to enable it to
handle both cable modem and adsl), and some other basic services (smb,
named, nfs, dhcp, xinetd, pppd). Nothing fancy realy that could explain
this behaviour.
Ofcource i did check ps aux to see if anything was _using_ this memory,
but unfortunatly this is not the case.
Hope the outputs help trace this problem. If any more input is required,
dont hesitate to bomb my inbox ;-)
-- Chris Chabot
ps, i don't know if its related, but i also found the folowing messages
in my dmesg, dont know where there from.. but i figured i'd mention it
invalidate: busy buffer
invalidate: busy buffer
...
<repeats about 20 times>
[-- Attachment #2: slabinfo-2.4.16 --]
[-- Type: text/plain, Size: 8103 bytes --]
[uptime]
2:42pm up 1 day, 22:38, 1 user, load average: 0.00, 0.00, 0.00
[free]
total used free shared buffers cached
Mem: 1029752 1018424 11328 0 130904 404380
-/+ buffers/cache: 483140 546612
Swap: 2104464 1328 2103136
[slabinfo]
slabinfo - version: 1.1 (SMP)
kmem_cache 80 80 244 5 5 1 : 252 126
ip_conntrack 66 66 352 6 6 1 : 124 62
ip_fib_hash 22 226 32 2 2 1 : 252 126
ip_mrt_cache 0 0 96 0 0 1 : 252 126
tcp_tw_bucket 40 40 96 1 1 1 : 252 126
tcp_bind_bucket 226 226 32 2 2 1 : 252 126
tcp_open_request 59 59 64 1 1 1 : 252 126
inet_peer_cache 2 118 64 2 2 1 : 252 126
ip_dst_cache 197 240 160 10 10 1 : 252 126
arp_cache 7 60 128 2 2 1 : 252 126
blkdev_requests 1024 1040 96 26 26 1 : 252 126
journal_head 424 1560 48 19 20 1 : 252 126
revoke_table 4 253 12 1 1 1 : 252 126
revoke_record 113 113 32 1 1 1 : 252 126
dnotify cache 0 0 20 0 0 1 : 252 126
file lock cache 168 168 92 4 4 1 : 252 126
fasync cache 0 0 16 0 0 1 : 252 126
uid_cache 3 226 32 2 2 1 : 252 126
skbuff_head_cache 351 528 160 22 22 1 : 252 126
sock 126 126 832 14 14 2 : 124 62
sigqueue 58 58 132 2 2 1 : 252 126
cdev_cache 2320 2360 64 40 40 1 : 252 126
bdev_cache 14 118 64 2 2 1 : 252 126
mnt_cache 14 118 64 2 2 1 : 252 126
inode_cache 686896 686896 480 85862 85862 1 : 124 62
dentry_cache 696810 696810 128 23227 23227 1 : 252 126
dquot 0 0 128 0 0 1 : 252 126
filp 450 480 128 16 16 1 : 252 126
names_cache 4 4 4096 4 4 1 : 60 30
buffer_head 134892 218120 96 5453 5453 1 : 252 126
mm_struct 150 168 160 7 7 1 : 252 126
vm_area_struct 832 1040 96 26 26 1 : 252 126
fs_cache 152 177 64 3 3 1 : 252 126
files_cache 86 90 416 10 10 1 : 124 62
signal_act 68 69 1312 23 23 1 : 60 30
size-131072(DMA) 0 0 131072 0 0 32 : 0 0
size-131072 0 0 131072 0 0 32 : 0 0
size-65536(DMA) 0 0 65536 0 0 16 : 0 0
size-65536 0 0 65536 0 0 16 : 0 0
size-32768(DMA) 0 0 32768 0 0 8 : 0 0
size-32768 0 0 32768 0 0 8 : 0 0
size-16384(DMA) 0 0 16384 0 0 4 : 0 0
size-16384 10 10 16384 10 10 4 : 0 0
size-8192(DMA) 0 0 8192 0 0 2 : 0 0
size-8192 2 2 8192 2 2 2 : 0 0
size-4096(DMA) 0 0 4096 0 0 1 : 60 30
size-4096 43 43 4096 43 43 1 : 60 30
size-2048(DMA) 0 0 2048 0 0 1 : 60 30
size-2048 202 202 2048 101 101 1 : 60 30
size-1024(DMA) 0 0 1024 0 0 1 : 124 62
size-1024 114 124 1024 30 31 1 : 124 62
size-512(DMA) 0 0 512 0 0 1 : 124 62
size-512 656 656 512 82 82 1 : 124 62
size-256(DMA) 0 0 256 0 0 1 : 252 126
size-256 224 255 256 16 17 1 : 252 126
size-128(DMA) 0 0 128 0 0 1 : 252 126
size-128 2910 2910 128 97 97 1 : 252 126
size-64(DMA) 0 0 64 0 0 1 : 252 126
size-64 28261 28261 64 479 479 1 : 252 126
size-32(DMA) 0 0 32 0 0 1 : 252 126
size-32 121924 121927 32 1079 1079 1 : 252 126
[ps aux]
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 1416 472 ? S Nov30 0:03 init [3]
root 2 0.0 0.0 0 0 ? SW Nov30 0:00 [keventd]
root 3 0.0 0.0 0 0 ? SWN Nov30 0:00 [ksoftirqd_CPU0]
root 4 0.0 0.0 0 0 ? SWN Nov30 0:00 [ksoftirqd_CPU1]
root 5 0.0 0.0 0 0 ? SW Nov30 0:11 [kswapd]
root 6 0.0 0.0 0 0 ? SW Nov30 0:00 [bdflush]
root 7 0.0 0.0 0 0 ? SW Nov30 0:00 [kupdated]
root 8 0.0 0.0 0 0 ? SW Nov30 0:00 [scsi_eh_0]
root 9 0.0 0.0 0 0 ? SW< Nov30 0:00 [mdrecoveryd]
root 10 0.0 0.0 0 0 ? SW Nov30 0:01 [kjournald]
root 145 0.0 0.0 0 0 ? SW Nov30 0:00 [kjournald]
root 146 0.0 0.0 0 0 ? SW Nov30 0:01 [kjournald]
root 147 0.0 0.0 0 0 ? SW Nov30 0:08 [kjournald]
root 717 0.0 0.0 1756 848 ? S Nov30 0:00 syslogd -m 0 -r
root 722 0.0 0.0 1404 476 ? S Nov30 0:00 klogd -2 -x
bin 742 0.0 0.0 1660 764 ? S Nov30 0:00 portmap
root 799 0.0 0.0 1792 568 ? S Nov30 0:00 rpc.rquotad
root 804 0.0 0.0 1620 708 ? S Nov30 0:00 rpc.mountd
root 810 0.0 0.0 0 0 ? SW Nov30 0:07 [nfsd]
root 811 0.0 0.0 0 0 ? SW Nov30 0:07 [nfsd]
root 812 0.0 0.0 0 0 ? SW Nov30 0:07 [nfsd]
root 813 0.0 0.0 0 0 ? SW Nov30 0:07 [nfsd]
root 814 0.0 0.0 0 0 ? SW Nov30 0:07 [nfsd]
root 815 0.0 0.0 0 0 ? SW Nov30 0:06 [nfsd]
root 816 0.0 0.0 0 0 ? SW Nov30 0:06 [nfsd]
root 817 0.0 0.0 0 0 ? SW Nov30 0:07 [nfsd]
root 818 0.0 0.0 0 0 ? SW Nov30 0:00 [lockd]
root 819 0.0 0.0 0 0 ? SW Nov30 0:00 [rpciod]
root 984 0.0 0.0 3260 628 ? S Nov30 0:00 smbd -D
root 989 0.0 0.0 2448 820 ? S Nov30 0:00 nmbd -D
root 1038 0.0 0.1 5684 1248 ? S Nov30 0:00 sendmail: accepti
root 1057 0.0 0.0 1648 676 ? S Nov30 0:00 crond
root 1536 0.0 0.0 1388 376 tty1 S Nov30 0:00 /sbin/mingetty tt
root 1537 0.0 0.0 1388 372 tty2 S Nov30 0:00 /sbin/mingetty tt
root 1996 0.0 0.0 1920 896 ? S Nov30 0:00 /usr/sbin/pppd ca
root 1997 0.2 0.0 1456 528 ? S Nov30 7:07 /usr/sbin/pptp pp
root 1999 0.0 0.0 1448 560 ? S Nov30 0:00 /usr/sbin/pptp pp
named 15213 0.0 0.4 15348 4540 ? S Dec01 0:00 named -u named
named 15215 0.0 0.4 15348 4540 ? S Dec01 0:00 named -u named
named 15216 0.0 0.4 15348 4540 ? S Dec01 0:02 named -u named
named 15217 0.0 0.4 15348 4540 ? S Dec01 0:02 named -u named
named 15218 0.0 0.4 15348 4540 ? S Dec01 0:00 named -u named
named 15219 0.0 0.4 15348 4540 ? S Dec01 0:00 named -u named
root 16544 0.0 0.1 2312 1036 ? S Dec01 0:00 xinetd -stayalive
root 18557 0.0 0.1 2676 1276 ? S 04:04 0:00 /usr/sbin/sshd
root 19251 0.0 0.1 3600 2048 ? S 14:29 0:00 /usr/sbin/sshd
root 19252 0.0 0.1 2588 1424 pts/0 S 14:29 0:00 -bash
root 19348 0.0 0.0 2664 760 pts/0 R 14:43 0:00 ps aux
[-- Attachment #3: slabinfo-2.4.17-rc1 --]
[-- Type: text/plain, Size: 8193 bytes --]
[uptime]
1:16pm up 1 day, 8:50, 1 user, load average: 0.00, 0.00, 0.00
[free]
total used free shared buffers cached
Mem: 1029744 1018420 11324 0 119856 413224
-/+ buffers/cache: 485340 544404
Swap: 2104464 776 2103688
[/proc/slabinfo]
slabinfo - version: 1.1 (SMP)
kmem_cache 80 80 244 5 5 1 : 252 126
ip_conntrack 110 110 352 10 10 1 : 124 62
ip_fib_hash 21 226 32 2 2 1 : 252 126
ip_mrt_cache 0 0 96 0 0 1 : 252 126
tcp_tw_bucket 120 120 96 3 3 1 : 252 126
tcp_bind_bucket 226 226 32 2 2 1 : 252 126
tcp_open_request 59 59 64 1 1 1 : 252 126
inet_peer_cache 3 59 64 1 1 1 : 252 126
ip_dst_cache 367 408 160 17 17 1 : 252 126
arp_cache 7 60 128 2 2 1 : 252 126
blkdev_requests 1024 1040 96 26 26 1 : 252 126
journal_head 342 702 48 9 9 1 : 252 126
revoke_table 4 253 12 1 1 1 : 252 126
revoke_record 113 113 32 1 1 1 : 252 126
dnotify cache 0 0 20 0 0 1 : 252 126
file lock cache 159 168 92 4 4 1 : 252 126
fasync cache 0 0 16 0 0 1 : 252 126
uid_cache 226 226 32 2 2 1 : 252 126
skbuff_head_cache 402 528 160 22 22 1 : 252 126
sock 117 117 832 13 13 2 : 124 62
sigqueue 116 116 132 4 4 1 : 252 126
cdev_cache 2319 2360 64 40 40 1 : 252 126
bdev_cache 14 118 64 2 2 1 : 252 126
mnt_cache 14 118 64 2 2 1 : 252 126
inode_cache 688304 688304 480 86038 86038 1 : 124 62
dentry_cache 696690 696690 128 23223 23223 1 : 252 126
dquot 0 0 128 0 0 1 : 252 126
filp 456 480 128 16 16 1 : 252 126
names_cache 5 5 4096 5 5 1 : 60 30
buffer_head 132808 231880 96 5797 5797 1 : 252 126
mm_struct 144 144 160 6 6 1 : 252 126
vm_area_struct 834 960 96 24 24 1 : 252 126
fs_cache 177 177 64 3 3 1 : 252 126
files_cache 72 72 416 8 8 1 : 124 62
signal_act 72 72 1312 24 24 1 : 60 30
size-131072(DMA) 0 0 131072 0 0 32 : 0 0
size-131072 0 0 131072 0 0 32 : 0 0
size-65536(DMA) 0 0 65536 0 0 16 : 0 0
size-65536 0 0 65536 0 0 16 : 0 0
size-32768(DMA) 0 0 32768 0 0 8 : 0 0
size-32768 0 0 32768 0 0 8 : 0 0
size-16384(DMA) 0 0 16384 0 0 4 : 0 0
size-16384 10 11 16384 10 11 4 : 0 0
size-8192(DMA) 0 0 8192 0 0 2 : 0 0
size-8192 2 3 8192 2 3 2 : 0 0
size-4096(DMA) 0 0 4096 0 0 1 : 60 30
size-4096 81 81 4096 81 81 1 : 60 30
size-2048(DMA) 0 0 2048 0 0 1 : 60 30
size-2048 220 232 2048 110 116 1 : 60 30
size-1024(DMA) 0 0 1024 0 0 1 : 124 62
size-1024 144 144 1024 36 36 1 : 124 62
size-512(DMA) 0 0 512 0 0 1 : 124 62
size-512 672 672 512 84 84 1 : 124 62
size-256(DMA) 0 0 256 0 0 1 : 252 126
size-256 255 255 256 17 17 1 : 252 126
size-128(DMA) 0 0 128 0 0 1 : 252 126
size-128 2983 3090 128 103 103 1 : 252 126
size-64(DMA) 0 0 64 0 0 1 : 252 126
size-64 28143 28143 64 477 477 1 : 252 126
size-32(DMA) 0 0 32 0 0 1 : 252 126
size-32 122217 122266 32 1082 1082 1 : 252 126
[ps aux]
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 1416 528 ? S Dec15 0:04 init [3]
root 2 0.0 0.0 0 0 ? SW Dec15 0:10 [keventd]
root 3 0.0 0.0 0 0 ? SWN Dec15 0:00 [ksoftirqd_CPU0]
root 4 0.0 0.0 0 0 ? SWN Dec15 0:00 [ksoftirqd_CPU1]
root 5 0.0 0.0 0 0 ? SW Dec15 0:04 [kswapd]
root 6 0.0 0.0 0 0 ? SW Dec15 0:00 [bdflush]
root 7 0.0 0.0 0 0 ? SW Dec15 0:01 [kupdated]
root 8 0.0 0.0 0 0 ? SW Dec15 0:00 [scsi_eh_0]
root 9 0.0 0.0 0 0 ? SW< Dec15 0:00 [mdrecoveryd]
root 10 0.0 0.0 0 0 ? SW Dec15 0:00 [kjournald]
root 142 0.0 0.0 0 0 ? SW Dec15 0:00 [kjournald]
root 143 0.0 0.0 0 0 ? SW Dec15 0:01 [kjournald]
root 144 0.0 0.0 0 0 ? SW Dec15 0:07 [kjournald]
root 716 0.0 0.0 1476 604 ? S Dec15 0:00 syslogd -m 0 -r
root 721 0.0 0.0 1404 480 ? S Dec15 0:00 klogd -2 -x
bin 741 0.0 0.0 1660 764 ? S Dec15 0:00 portmap
root 798 0.0 0.0 1792 632 ? S Dec15 0:00 rpc.rquotad
root 803 0.0 0.0 1620 744 ? S Dec15 0:00 rpc.mountd
root 808 0.0 0.0 0 0 ? SW Dec15 0:10 [nfsd]
root 809 0.0 0.0 0 0 ? SW Dec15 0:10 [nfsd]
root 810 0.0 0.0 0 0 ? SW Dec15 0:10 [nfsd]
root 811 0.0 0.0 0 0 ? SW Dec15 0:10 [nfsd]
root 812 0.0 0.0 0 0 ? SW Dec15 0:10 [nfsd]
root 813 0.0 0.0 0 0 ? SW Dec15 0:10 [nfsd]
root 814 0.0 0.0 0 0 ? SW Dec15 0:10 [nfsd]
root 815 0.0 0.0 0 0 ? SW Dec15 0:10 [nfsd]
root 816 0.0 0.0 0 0 ? SW Dec15 0:00 [lockd]
root 817 0.0 0.0 0 0 ? SW Dec15 0:00 [rpciod]
root 889 0.0 0.0 1920 912 ? S Dec15 0:00 /usr/sbin/pppd ca
root 929 0.0 0.1 2312 1056 ? S Dec15 0:00 xinetd -stayalive
root 950 0.0 0.0 1796 688 ? S Dec15 0:00 /usr/sbin/dhcpd
root 986 0.0 0.0 3260 796 ? S Dec15 0:00 smbd -D
root 991 0.0 0.0 2448 936 ? S Dec15 0:01 nmbd -D
root 1033 0.3 0.0 1456 528 ? S Dec15 6:59 /usr/sbin/pptp pp
root 1035 0.0 0.0 1448 572 ? S Dec15 0:00 /usr/sbin/pptp pp
root 1043 0.0 0.1 5684 1672 ? S Dec15 0:05 sendmail: accepti
root 1062 0.0 0.0 1648 748 ? S Dec15 0:00 crond
root 1541 0.0 0.0 1388 452 tty1 S Dec15 0:00 /sbin/mingetty tt
root 1542 0.0 0.0 1388 452 tty2 S Dec15 0:00 /sbin/mingetty tt
root 1722 0.0 0.1 2680 1280 ? S Dec15 0:03 /usr/sbin/sshd
root 5133 0.2 0.2 3652 2080 ? S 13:15 0:00 /usr/sbin/sshd
root 5135 0.0 0.1 2656 1492 pts/1 S 13:15 0:00 -bash
named 5221 0.0 0.3 14068 4064 ? S 13:15 0:00 named -u named
named 5223 0.0 0.3 14068 4064 ? S 13:15 0:00 named -u named
named 5224 0.1 0.3 14068 4064 ? S 13:15 0:00 named -u named
named 5225 0.0 0.3 14068 4064 ? S 13:15 0:00 named -u named
named 5226 0.0 0.3 14068 4064 ? S 13:15 0:00 named -u named
named 5227 0.0 0.3 14068 4064 ? S 13:15 0:00 named -u named
root 5342 0.0 0.0 2660 756 pts/1 R 13:17 0:00 ps aux
^ permalink raw reply [flat|nested] 11+ messages in thread[parent not found: <004601c18567$a8be1600$0801a8c0@Stev.org>]
* Re: Unfreeable buffer/cache problem in 2.4.17-rc1 still there
[not found] ` <004601c18567$a8be1600$0801a8c0@Stev.org>
@ 2001-12-15 13:42 ` Chris Chabot
0 siblings, 0 replies; 11+ messages in thread
From: Chris Chabot @ 2001-12-15 13:42 UTC (permalink / raw)
To: James Stevenson; +Cc: linux-kernel
James Stevenson wrote,
> does the same thing happen when you do
> find / -type f -print0 |xargs -0 cat > /dev/null
<snip>
> what cronjobs are running at night ?
> this could be normal because free ram is really a waste
> so it might as well be used for somthing and whats
> better than speeding up disk access
> if things do start to use memory they take it from the free section then
> the disk cache gets droped and refills the free section.
I apreciate the effort and sentiment. However if you would look at the
output of 'free' (originaly attached files), you would notice i am
talking about free = (available + cache + buffer) and not just
'available' ;-)
This is a problem i only have on one of the 40 or so servers i manage,
however the one that has it is my personal gateway & firewall machine,
so it feels prety sore ;-)
There are one or two other people on the list who also had the same
problem, is it fixed for you guys, or still seeing the same problem? I'm
having the sneaking suspission this behaviour isnt gone yet.
Anyways, thanks for trying to educate james, however i only wish it was
that simple ;-)
-- Chris
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Unfreeable buffer/cache problem in 2.4.17-rc1 still there
2001-12-15 12:36 Chris Chabot
[not found] ` <004601c18567$a8be1600$0801a8c0@Stev.org>
@ 2001-12-15 14:24 ` Andrea Arcangeli
2001-12-15 16:05 ` Chris Chabot
1 sibling, 1 reply; 11+ messages in thread
From: Andrea Arcangeli @ 2001-12-15 14:24 UTC (permalink / raw)
To: Chris Chabot; +Cc: linux-kernel, marcelo
On Sat, Dec 15, 2001 at 01:36:11PM +0100, Chris Chabot wrote:
> inode_cache 686896 686896 480 85862 85862 1 : 124 62
> dentry_cache 696810 696810 128 23227 23227 1 : 252 126
this is an icache/dcache problem, can you reproduce on 2.4.17rc1aa1, it
will shrink more aggressively.
really to get an even better balance we should add the icache/dcache
slab pages into the lru as well... that would trigger the icache/dcache
flushes more easily when too much ram is in those caches.
Andrea
^ permalink raw reply [flat|nested] 11+ messages in thread* Re: Unfreeable buffer/cache problem in 2.4.17-rc1 still there
2001-12-15 14:24 ` Andrea Arcangeli
@ 2001-12-15 16:05 ` Chris Chabot
2001-12-15 16:09 ` Ken Brownfield
0 siblings, 1 reply; 11+ messages in thread
From: Chris Chabot @ 2001-12-15 16:05 UTC (permalink / raw)
To: Andrea Arcangeli; +Cc: linux-kernel, marcelo
Ok, rc1-aa1 is installed, up and running. Since it took a little more
then a day last time for this behavour to be displayed, i'll have to
wait before i can report back anything usefull.
Ps, a side question. if this is 'inode' and 'dentry' cache, why isnt it
reported as 'used by cache' in free, top, gtop, etc?
Also, why is this such a progressive problem? Even if i do a find on the
full +/- 400 gigs, create and remove 2 gig files, create 1000 temp
files, etc, in the first day the memory isnt 'disapearing' yet.
(ofcource it uses a lot of mem for cache/buffers, but no 'unaccounted
for in free or top' memory.
Will report back soon,
-- Chris
On Sat, 2001-12-15 at 15:24, Andrea Arcangeli wrote:
> On Sat, Dec 15, 2001 at 01:36:11PM +0100, Chris Chabot wrote:
> > inode_cache 686896 686896 480 85862 85862 1 : 124 62
> > dentry_cache 696810 696810 128 23227 23227 1 : 252 126
>
> this is an icache/dcache problem, can you reproduce on 2.4.17rc1aa1, it
> will shrink more aggressively.
>
> really to get an even better balance we should add the icache/dcache
> slab pages into the lru as well... that would trigger the icache/dcache
> flushes more easily when too much ram is in those caches.
>
> Andrea
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: Unfreeable buffer/cache problem in 2.4.17-rc1 still there
2001-12-15 16:05 ` Chris Chabot
@ 2001-12-15 16:09 ` Ken Brownfield
0 siblings, 0 replies; 11+ messages in thread
From: Ken Brownfield @ 2001-12-15 16:09 UTC (permalink / raw)
To: Chris Chabot; +Cc: Andrea Arcangeli, linux-kernel, marcelo
I think "updatedb" at 4am is what you're looking for... How much disk
space do you have on this system?
--
Ken.
brownfld@irridia.com
On Sat, Dec 15, 2001 at 05:05:20PM +0100, Chris Chabot wrote:
| Ok, rc1-aa1 is installed, up and running. Since it took a little more
| then a day last time for this behavour to be displayed, i'll have to
| wait before i can report back anything usefull.
|
| Ps, a side question. if this is 'inode' and 'dentry' cache, why isnt it
| reported as 'used by cache' in free, top, gtop, etc?
|
| Also, why is this such a progressive problem? Even if i do a find on the
| full +/- 400 gigs, create and remove 2 gig files, create 1000 temp
| files, etc, in the first day the memory isnt 'disapearing' yet.
| (ofcource it uses a lot of mem for cache/buffers, but no 'unaccounted
| for in free or top' memory.
|
| Will report back soon,
|
| -- Chris
|
|
| On Sat, 2001-12-15 at 15:24, Andrea Arcangeli wrote:
| > On Sat, Dec 15, 2001 at 01:36:11PM +0100, Chris Chabot wrote:
| > > inode_cache 686896 686896 480 85862 85862 1 : 124 62
| > > dentry_cache 696810 696810 128 23227 23227 1 : 252 126
| >
| > this is an icache/dcache problem, can you reproduce on 2.4.17rc1aa1, it
| > will shrink more aggressively.
| >
| > really to get an even better balance we should add the icache/dcache
| > slab pages into the lru as well... that would trigger the icache/dcache
| > flushes more easily when too much ram is in those caches.
| >
| > Andrea
|
|
| -
| To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
| the body of a message to majordomo@vger.kernel.org
| More majordomo info at http://vger.kernel.org/majordomo-info.html
| Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2001-12-18 4:28 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2001-12-16 19:39 Unfreeable buffer/cache problem in 2.4.17-rc1 still there Dmitry Volkoff
2001-12-17 17:01 ` vda
2001-12-18 4:26 ` Dmitry Volkoff
-- strict thread matches above, loose matches on Subject: below --
2001-12-16 20:41 Willy Tarreau
[not found] <Pine.LNX.4.33.0112151141280.19022-100000@coffee.psychology.mcmaster.ca>
2001-12-15 17:09 ` Chris Chabot
2001-12-15 16:05 Ed Tomlinson
2001-12-15 12:36 Chris Chabot
[not found] ` <004601c18567$a8be1600$0801a8c0@Stev.org>
2001-12-15 13:42 ` Chris Chabot
2001-12-15 14:24 ` Andrea Arcangeli
2001-12-15 16:05 ` Chris Chabot
2001-12-15 16:09 ` Ken Brownfield
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox