xfs and Linux Linux 2.6.22 and Memory

xfs and Linux Linux 2.6.22 and Memory

[Mariella Petrini]

Hi All,

I have compiled xfs that is available with Linux
2.6.22 on a Debian 4.0
The system is a 2 cpus (Intel Xeon 3 GHz) with 4 cores
each
The system has 8 GB of RAM + 2GB of swap

I have 4 Hard Drives on the system and each is
formatted using xfs
Each filesystem is mounted with noatime option
I am including the output generated during one of the
format:

mkfs.xfs -f /dev/sdb1

meta-data=/dev/sdb1              isize=256   
agcount=16, agsize=4881374 blks
         =                       sectsz=512   attr=0
data     =                       bsize=4096  
blocks=78101984, imaxpct=25
         =                       sunit=0      swidth=0
blks, unwritten=1
naming   =version 2              bsize=4096
log      =internal log           bsize=4096  
blocks=32768, version=1
         =                       sectsz=512   sunit=0
blks
realtime =none                   extsz=65536 
blocks=0, rtextents=0


I have been using an sql server to populate the 4 xfs
filesystems.
So all the files represents part of a relational
database.

In each filesystem I have 5 top directory and each
directory contains 100,000 files, so in total each
filesystem has 500,000 regular files + 5 regular
directories.
The sql server is the only program that writes the
regular files into the xfs filesystems.

During the time taken to populate the 4 filesystems
(2,000,000 files)
I have noticed that approximately 5 GB of RAM were
taken.

Once all the filesystems were populated I have
shutdown the sql server (the only process that
accesses/reads/writes the filesystems) and the 5 GB of
RAM would not be released.

At that point I have unmounted the 4 xfs filesystems
and the 5 GB of RAM would be released (be available
again).


QUESTION:

Is there any way to release that amount of memory
without unmounting the file systems ?
Is that caused to some caching mechanism ?
Or could that be caused by something else ?

Could you please help ?
Thanks a lot in advance for your help,

Mariella





       
____________________________________________________________________________________
Moody friends. Drama queens. Your life? Nope! - their life, your story. Play Sims Stories at Yahoo! Games.
http://sims.yahoo.com/  

Re: xfs and Linux Linux 2.6.22 and Memory

[Eric Sandeen]

Mariella Petrini wrote:

> QUESTION:
> 
> Is there any way to release that amount of memory
> without unmounting the file systems ?
> Is that caused to some caching mechanism ?
> Or could that be caused by something else ?

It's most likely the linux VFS caching the dentries & inodes, and
therefore caching the xfs inodes as well.

cat /proc/sys/fs/dentry-state and /proc/sys/fs/inode-state to see how
many are in memory, how many are active, and the age limit.

slabtop would fairly easily show you how much is in the dentry_cache &
the inode_cache too.

Are you actually seeing a problem with this (caching is generally good)
or just curious?

-Eric

Re: xfs and Linux Linux 2.6.22 and Memory

[David Chinner]

On Thu, Sep 20, 2007 at 04:02:57PM -0500, Eric Sandeen wrote:
> Mariella Petrini wrote:
> 
> > QUESTION:
> > 
> > Is there any way to release that amount of memory
> > without unmounting the file systems ?
> > Is that caused to some caching mechanism ?
> > Or could that be caused by something else ?
> 
> It's most likely the linux VFS caching the dentries & inodes, and
> therefore caching the xfs inodes as well.

Not just inodes, but file data in the page cache as well.

If you want to blow it all away without unmountin filesystems, then
you should read up on /proc/sys/vm sysctls (i.e.
/proc/sys/vm/drop_caches)....

Cheers,

Dave.

-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group

Re: xfs and Linux Linux 2.6.22 and Memory

[Mariella Petrini]

> 
> Are you actually seeing a problem with this (caching
> is generally good)
> or just curious?
>
I was mostly curious, especially because In some cases
after having created all the files I may not need to
have all of them cached and I may need to availability
of the RAM for other purposes.


Thank you all for all the info,

Mariella


--- Eric Sandeen <sandeen@sandeen.net> wrote:

> Mariella Petrini wrote:
> 
> > QUESTION:
> > 
> > Is there any way to release that amount of memory
> > without unmounting the file systems ?
> > Is that caused to some caching mechanism ?
> > Or could that be caused by something else ?
> 
> It's most likely the linux VFS caching the dentries
> & inodes, and
> therefore caching the xfs inodes as well.
> 
> cat /proc/sys/fs/dentry-state and
> /proc/sys/fs/inode-state to see how
> many are in memory, how many are active, and the age
> limit.
> 
> slabtop would fairly easily show you how much is in
> the dentry_cache &
> the inode_cache too.
> 
> Are you actually seeing a problem with this (caching
> is generally good)
> or just curious?
> 
> -Eric
> 
> 
> 



       
____________________________________________________________________________________
Be a better Globetrotter. Get better travel answers from someone who knows. Yahoo! Answers - Check it out.
http://answers.yahoo.com/dir/?link=list&sid=396545469

Re: xfs and Linux Linux 2.6.22 and Memory

[Jason White]

On Thu, Sep 20, 2007 at 07:14:23PM -0700, Mariella Petrini wrote:
> I was mostly curious, especially because In some cases
> after having created all the files I may not need to
> have all of them cached and I may need to availability
> of the RAM for other purposes.

I may be wrong, but as I understand it, if the RAM is needed for other
purposes, cache pages are made available automatically and the size of the
cache is reduced as the RAM is allocated by processes.