Re: [PATCH] O_STREAMING - flag for optimal streaming I/O

[Robert Love <rml@tech9.net>]

On Wed, 2002-10-09 at 23:29, Erik Andersen wrote:

> I don't think grep is a very good candidate for O_STREAMING.  I
> usually want the stuff I grep to stay in cache.  O_STREAMING is
> much better suited to applications like ogle, vlc, xine, xmovie,
> xmms etc since there is little reason for the OS to cache things
> like songs and movies you aren't likely to hear/see again any
> time soon.

Yes.  Good point.  People are taking this too far.  There is a big
difference between being just sequential and use-once.  Grep(1) is a
great example of something that _should_ use the pagecache.  Subsequent
file accesses, which will occur, should hit.

Look, the pagecache is already smart.  New stuff will replace unusued
old stuff.  On VM pressure, the pagecache will be pruned.  Streaming I/O
is a fundamentally different problem in that the data is so large it
_continually_ thrashes the pagecache.  Such I/O is sequential and
use-once.  You end up with a permanent waste of memory (the cached
I/O).  

Let's prove we have a solution to this problem before going after
tangent ones.

	Robert Love