Disabling file system caching

Disabling file system caching

[Jean-Eric Cuendet]

Hi,
Is it possible to disable file caching for a given partition or mount?
Or at least to limit it at a certain quantity of memory?
I think of a BIG compilation (several GB of files) and would tell gcc to 
not cache the compiled files.
Thanks
-jec

Re: Disabling file system caching

[Rik van Riel]

On Sun, 19 Jan 2003, Jean-Eric Cuendet wrote:

> Is it possible to disable file caching for a given partition or mount?

No, if you do that mmap(), read(), write() etc. would be impossible.

> Or at least to limit it at a certain quantity of memory?

Not yet.  I'm thinking of implementing something like this
for the next version of -rmap (reclaim only from the cache
if the cache occupies more than a certain fraction of ram).

> I think of a BIG compilation (several GB of files) and would tell gcc to
> not cache the compiled files.

But what about the .h files ?  You will want to cache those.

regards,

Rik
-- 
Bravely reimplemented by the knights who say "NIH".
http://www.surriel.com/		http://guru.conectiva.com/
Current spamtrap:  <a href=mailto:"october@surriel.com">october@surriel.com</a>

RE: Disabling file system caching

[Folkert van Heusden]

> > Is it possible to disable file caching for a given partition or mount?
> No, if you do that mmap(), read(), write() etc. would be impossible.

Hmmm, maybe there's some way to explicitly flush the read/write-cache?
Ok, sync will do nice for the write-cache, but for the read-one?

Re: Disabling file system caching

[Nuno Silva]

Folkert van Heusden wrote:
>>>Is it possible to disable file caching for a given partition or mount?
>>
>>No, if you do that mmap(), read(), write() etc. would be impossible.
> 
> 
> Hmmm, maybe there's some way to explicitly flush the read/write-cache?
> Ok, sync will do nice for the write-cache, but for the read-one?

AFAIK, you simply can't... I'm trying to do this for several months and 
no luck. Linux simply caches everything it can in the read-cache. For 
99% of all cases this is very good but, for some situations, this is not 
desireable.

For the write cache, you can minimize memory usage playing with 
/proc/sys/vm (see Documentation/filesystems/proc.txt).

Regards,
Nuno Silva

> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: Disabling file system caching

[Balbir]

Not sure if posting to the newsgroup linux.kernel sends
it to the mailing list too.

"Balbir Singh" <balbir_soni@yahoo.com> wrote in message
news:b0g6q2$lfq$1@tilde.itg.ti.com...
> "Rik van Riel" <riel@conectiva.com.br> wrote in message
> news:20030120011009$2d98@gated-at.bofh.it...
> > On Sun, 19 Jan 2003, Jean-Eric Cuendet wrote:
> >
> > > Is it possible to disable file caching for a given partition or mount?
> >
> > No, if you do that mmap(), read(), write() etc. would be impossible.
> >
> > > Or at least to limit it at a certain quantity of memory?
> >
> > Not yet.  I'm thinking of implementing something like this
> > for the next version of -rmap (reclaim only from the cache
> > if the cache occupies more than a certain fraction of ram).
>

I think that this feature is very important. In an embedded system
using an NFS root filesystem, we found that the file cache
would take a lot of memory and all insmods would fail. This is
especially true when the system boots up and looks for /lib/modules.

I think it should be possible to modify the slab allocator to
implement a memory pool. We could add a flag which would prevent
the slab from growing beyond its initial size.

This approach would work only if the cache is allocated by
using the slab allocator.

Balbir

Re: Disabling file system caching

[Jan Hudec]

On Mon, Jan 20, 2003 at 01:17:42PM +0530, Balbir wrote:
> Not sure if posting to the newsgroup linux.kernel sends
> it to the mailing list too.
> 
> "Balbir Singh" <balbir_soni@yahoo.com> wrote in message
> news:b0g6q2$lfq$1@tilde.itg.ti.com...
> > "Rik van Riel" <riel@conectiva.com.br> wrote in message
> > news:20030120011009$2d98@gated-at.bofh.it...
> > > On Sun, 19 Jan 2003, Jean-Eric Cuendet wrote:
> > >
> > > > Is it possible to disable file caching for a given partition or mount?
> > >
> > > No, if you do that mmap(), read(), write() etc. would be impossible.
> > >
> > > > Or at least to limit it at a certain quantity of memory?
> > >
> > > Not yet.  I'm thinking of implementing something like this
> > > for the next version of -rmap (reclaim only from the cache
> > > if the cache occupies more than a certain fraction of ram).
> >
> 
> I think that this feature is very important. In an embedded system
> using an NFS root filesystem, we found that the file cache
> would take a lot of memory and all insmods would fail. This is
> especially true when the system boots up and looks for /lib/modules.
> 
> I think it should be possible to modify the slab allocator to
> implement a memory pool. We could add a flag which would prevent
> the slab from growing beyond its initial size.
> 
> This approach would work only if the cache is allocated by
> using the slab allocator.

No, it's not! And the slab alocator should mercilessly rip the cache
when it needs some pages. In the NFS root case, I would guess it's
a problem of NFS implementation allocating too much kernel memory, since
both cache and user-land pages are riped in favor of kernel allocations.

-------------------------------------------------------------------------------
						 Jan 'Bulb' Hudec <bulb@ucw.cz>

Re: Disabling file system caching

[Martin Knoblauch]

> Re: Disabling file system caching
> 
> From: Rik van Riel (riel@conectiva.com.br)
> Date: Sun Jan 19 2003 - 20:05:00 EST
> 
> On Sun, 19 Jan 2003, Jean-Eric Cuendet wrote:
> 
> > Is it possible to disable file caching for a given partition or mount?
> 
> No, if you do that mmap(), read(), write() etc. would be impossible.
> 
> > Or at least to limit it at a certain quantity of memory?
> 
> Not yet. I'm thinking of implementing something like this
> for the next version of -rmap (reclaim only from the cache
> if the cache occupies more than a certain fraction of ram).
> 

 Yes please !!!
   Yes please !!!
     Yes please !!!

 :-)

 Having a maximum [and maybe minimum] knob for controlling the cache
would be an extremely useful feature in some situations.

Martin