benchmarks of O_STREAMING in 2.5

benchmarks of O_STREAMING in 2.5

[Robert Love]

I gave the O_STREAMING in Andrew's 2.5-mm tree the treatment..

Short summary: It works.

The streaming read test in the following benchmarks is simply a read()
in 64KB byte chunks of an 800MB file.

First test.  Show the cache effects are indeed as we intend and the
overhead is negligible.  Here, mem=2G.

	O_STREAMING?	Wall time	Cache Delta
	Yes		21.827s		0
	No		21.734s		+800MB

Second test.  Same deal, but mem=8M.  There was slight swapping, so I
suspect the reduced VM pressure is why the O_STREAMING run is faster.

	O_STREAMING?	Wall time	Cache Delta
	Yes		22.303s		0
	No		28.812s		+1MB

Third and final test. Kernel compile (make -j2) with a couple streaming
reads in the background.  Again, mem=2G.  This shows that actually
saving the pagecache from the horrid waste is useful.

	O_STREAMING	Wall time to complete Kernel compile
	Yes		5m30.494s
	No		4m59.661s

So, uh, Andrew's 2.5 code works ;-)

Someone buy me a dual Xeon,

	Robert Love

RE: benchmarks of O_STREAMING in 2.5

[Giuliano Pochini]

> I gave the O_STREAMING in Andrew's 2.5-mm tree the treatment..
>       O_STREAMING     Wall time to complete Kernel compile
>       Yes             5m30.494s
>       No              4m59.661s
>
> So, uh, Andrew's 2.5 code works ;-)

?? O_STREAMING makes it slower or is it a typo ?


Bye.

RE: benchmarks of O_STREAMING in 2.5

[Robert Love]

On Thu, 2002-10-17 at 03:33, Giuliano Pochini wrote:
> 
> > I gave the O_STREAMING in Andrew's 2.5-mm tree the treatment..
> >       O_STREAMING     Wall time to complete Kernel compile
> >       Yes             5m30.494s
> >       No              4m59.661s
> >
> > So, uh, Andrew's 2.5 code works ;-)
> 
> ?? O_STREAMING makes it slower or is it a typo ?

Yes, typo, those should be switched.

	Robert Love

RE: benchmarks of O_STREAMING in 2.5

[Dieter Nützel]

> Third and final test. Kernel compile (make -j2) with a couple streaming
> reads in the background.  Again, mem=2G.  This shows that actually
> saving the pagecache from the horrid waste is useful.
>
>        O_STREAMING     Wall time to complete Kernel compile
>        Yes             4m59.661s
>        No              5m30.494s
>
> So, uh, Andrew's 2.5 code works ;-)

Corrected version.

> Someone buy me a dual Xeon,
>
>        Robert Love

It's "Hammer time"...;-)

Greetings,
	Dieter
-- 
Dieter Nützel
Graduate Student, Computer Science

University of Hamburg
Department of Computer Science
@home: Dieter.Nuetzel at hamburg.de (replace at with @)

Re: benchmarks of O_STREAMING in 2.5

[Padraig Brady]

Alan Cox wrote:
> On Thu, 2002-10-17 at 03:53, Robert Love wrote:
> 
>>I gave the O_STREAMING in Andrew's 2.5-mm tree the treatment..
>>
>>Short summary: It works.
>>
>>The streaming read test in the following benchmarks is simply a read()
>>in 64KB byte chunks of an 800MB file.
> 
> All you now need to do is make it work with an API thats usable by the
> other 99% of real world apps, is extensible and sensible ways and
> therefore can be used.

I'm confused. Isn't this just an O_STREAM flag on open/fcntl ?
How could it be simpler? I suppose the VM could do better than
it currently does automatically but there is no harm in the app
giving a hint like this thus allowing stuff to be dropped from
the cache more aggresively?

Pádraig.

Re: benchmarks of O_STREAMING in 2.5

[Andi Kleen]

Alan Cox <alan@lxorguk.ukuu.org.uk> writes:

> All you now need to do is make it work with an API thats usable by the
> other 99% of real world apps, is extensible and sensible ways and
> therefore can be used.

An streaming hint as chattr would be quite nice.

-Andi

Re: benchmarks of O_STREAMING in 2.5

[Alan Cox]

On Thu, 2002-10-17 at 03:53, Robert Love wrote:
> I gave the O_STREAMING in Andrew's 2.5-mm tree the treatment..
> 
> Short summary: It works.
> 
> The streaming read test in the following benchmarks is simply a read()
> in 64KB byte chunks of an 800MB file.

All you now need to do is make it work with an API thats usable by the
other 99% of real world apps, is extensible and sensible ways and
therefore can be used.

Re: benchmarks of O_STREAMING in 2.5

[Alan Cox]

On Mon, 2002-10-21 at 15:32, Padraig Brady wrote:
> I'm confused. Isn't this just an O_STREAM flag on open/fcntl ?
> How could it be simpler? I suppose the VM could do better than
> it currently does automatically but there is no harm in the app
> giving a hint like this thus allowing stuff to be dropped from
> the cache more aggresively?

Which bit are you streaming, what kind of things are you doing with the
data ?

Take a database table are you - accessing it sequentially with caching,
accessing it randomly without readahead or streaming it and discarding
data. All three are valid answers, for the same data, and potentially on
a per table basis in the same file.

See the fadvise proposal. O_STREAMING is basically a subset of what is
needed