Re: [RFC] Parallelize IO for e2fsck

Bryan Henderson
2008-01-22T17:43:12Z

>I think there is a clear need for applications to be able to
>register a callback from the kernel to indicate that the machine as
>a whole is running out of memory and that the application should
>trim it's caches to reduce memory utilisation.
>
>Perhaps instead of swapping immediately, a SIGLOWMEM could be sent ...

The problem with that approach is that the Fsck process doesn't know how
its need for memory compares with other process' need for memory. How
much memory should it give up? Maybe it should just quit altogether if
other processes are in danger of deadlocking. Or maybe it's best for it
to keep all its memory and let some other frivolous process give up its
memory instead.

It's the OS's job to have a view of the entire system and make resource
allocation decisions.

If it's just a matter of the application choosing a better page frame to
vacate than what the kernel would have taken, (which is more a matter of
self-interest than resource allocation), then Fsck can do that more
directly by just monitoring its own page fault rate. If it's high, then
it's using more real memory than the kernel thinks it's entitled to and it
can reduce its memory footprint to improve its speed. It can even check
whether an access to readahead data caused a page fault; if so, it knows
reading ahead is actually making things worse and therefore reduce
readahead until the page faults stop happening.

--
Bryan Henderson IBM Almaden Research Center
San Jose CA Filesystems