Re: large files and low memory

[Enrico Weigelt <weigelt@metux.de>]

* Nicolas Pitre <nico@fluxnic.net> wrote:

> > > The mmap() isn't the problem.  Its the allocation of a buffer that is
> > > larger than the file in order to hold the result of deflating the file
> > > before it gets written to disk.  
> > > When the file is bigger than physical memory, the kernel has to
> > > page in parts of the file as well as swap in and out parts of
> > > that allocated buffer to hold the deflated file.
> > 
> > What are the access pattern of these memory areas ?
> 
> Perfectly linear.

In this case, I wonder why my machine goes into thrashing so easily
(P3 w/ 256MB ram). Seems the mmu/paging code doesnt recognize that
the previously-used pages can be kicked-off quickly ;-o
Perhaps I should talk to the kernel folks.

> > Perhaps madvise() could help ?
> 
> Perhaps.

hmm, so we should try it ;-p
where'd be the right place to add it ?


cu
-- 
----------------------------------------------------------------------
 Enrico Weigelt, metux IT service -- http://www.metux.de/

 phone:  +49 36207 519931  email: weigelt@metux.de
 mobile: +49 151 27565287  icq:   210169427         skype: nekrad666
----------------------------------------------------------------------
 Embedded-Linux / Portierung / Opensource-QM / Verteilte Systeme
----------------------------------------------------------------------