Re: [PATCH] O_STREAMING - flag for optimal streaming I/O

[Robert Love <rml@tech9.net>]

On Tue, 2002-10-08 at 15:05, Chris Wedgwood wrote:

> > In other words, this flag pretty much disables the pagecache for
> > this mapping, although we happily keep it around for write-behind
> > and read-ahead.  But once the data is behind us and safe to kill, we
> > do.  It is manual drop-behind.
> 
> OK.  What might use this though?  What applications might want to
> disable the page-cache but still use write-behind?

Streaming I/O wants read-ahead.  Filesystems themselves implement the
write-behind and we do not want to circumvent so much of the kernel.

The point of O_STREAMING is one change: drop pages in the pagecache
behind our current position, that are free-able, because we know we will
never want them.  Its a hint from the application saying "I will never
revisit this so dump it".

O_DIRECT is a much bigger can of worms.  You lose a lot of what the
kernel provides.  You have to do things in block-sized chunks.  Etc.
etc.

> > O_DIRECT has a lot of semantics, one of which is to attempt to
> > minimize cache effects.
> 
> It depends on the OS.  Some OS are broken and treat O_DIRECT as a
> hint, Linux and IRIX know it's a *requirement*.

Yep.  Linux treats most "hints" (e.g. madvise) as a requirement - it
fails if it cannot do it.  That is against the spec most of the time,
but oh well...

> > O_STREAMING would be for your TiVo or network audio streamer.  Any
> > file I/O that is inherently sequential and access-once.  No point
> > trashing the pagecache with its data - but otherwise the behavior is
> > normal.
> 
> Actually, this sounds perfect for O_DIRECT.  But I don't know much
> about streaming video.
>
> Since you only want the data once, why use the page-cache at all and
> needlessly copy?  Certainly, the requirements for O_DIRECT are not
> that hard to meet or implement.
>
> Don't get me wrong, I'm not saying this is a bad thing at all.  The
> patch is small and elegant so it's hard to object; I'm just trying to
> understand where in practice I would use this over O_DIRECT.

Shrug.  I do not have much experience with O_DIRECT.  I suspect the
synchronous nature and the requirement of aligned buffers is not ideal.

With O_STREAMING you can simply set the flag and use your normal I/O and
normal interfaces and have a field day.

Andrew, any experience on one vs. the other?

	Robert Love