Re: Memory Measurements and Lots of Files and Inodes

["Patrick R. McManus" <mcmanus@ducksong.com>]

[Andrew Morton: Dec 11 16:09]
> "Patrick R. McManus" wrote:
> > 
> > ...
> > Just sitting back and watching vmstat while this runs my 'free' memory
> > drops from ~600MB to about ~16MB.. the buffers and cache remain roughly
> > constant.. at 16MB some sort of garbage collection kicks in - there is
> > a notable system pause and ~70MB moves from the used to the 'free'
> > column... this process repeats more or less in a steady state.
> 
> Probably `negative dentries' - cached directory entries which say
> "this file isn't there" so we don't need to go into the fs to
> find that out.
> 
> If you could share your test apps that would help a lot.

sure! this is the "lots of files" program.

#include <stdio.h>
#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>

#define BSZ 2000
unsigned char buf[BSZ+256];

main ()
{
    int fd,i,j;
    unsigned int ctr=0;
    unsigned char r=0;
    char f1[32],f2[32],f3[32],f4[32];
    
    while (1)
    {
        sprintf (f1,"fxa_%06d",ctr++);  sprintf (f2,"fxa_%06d",ctr++);
        sprintf (f3,"fxa_%06d",ctr++);  sprintf (f4,"fxa_%06d",ctr++);

        fd = open (f1,O_CREAT | O_RDWR);
        memset (buf,r++,BSZ+256);  write (fd,buf,BSZ+r);    close (fd);
        
        fd = open (f2,O_CREAT | O_RDWR);
        memset (buf,r++,BSZ+256);   write (fd,buf,BSZ+r);    close (fd);
        
        fd = open (f3,O_CREAT | O_RDWR);
        memset (buf,r++,BSZ+256);   write (fd,buf,BSZ+r);     close (fd);
        
        fd = open (f4,O_CREAT | O_RDWR);
        memset (buf,r++,BSZ+256);  write (fd,buf,BSZ+r);      close (fd);
        
        unlink (f1);   unlink (f2);    unlink (f3);   unlink (f4);
        if (!r)
          fprintf (stdout,".");fflush (stdout);
    }
}

and this is the "gimme 300MB even though free sez I hardly have
anything" program:

#define BS (30*1024*1024)
main()
{
    char *x;
    x = malloc (BS);
    if (!x)
        printf ("heh - x is NULL\n");
    else
    {
        printf ("allocd\n");
        memset (x,0x31,BS);
        printf ("set\n");
        free (x);
        printf ("freed\n");
    }
}

> On your machine it'll be "all of swap plus all of physical memory
> minus whatever malloc'ed memory you're using now minus 8-12 megabytes".
> There isn't much memory which cannot be reclaimed unless you have a
> huge machine or you're doing odd things.

this is useful advice, thanks. Basically what the new procps does?