Re: [BENCHMARK] 2.4.{18,19{-ck9},20rc1{-aa1}} with contest

[Andrew Morton <akpm@digeo.com>]

Andrea Arcangeli wrote:
> 
> On Sun, Nov 10, 2002 at 08:03:01PM -0800, Andrew Morton wrote:
> > Andrea Arcangeli wrote:
> > >
> > > the slowdown happens in this case:
> > >
> > >         queue 5 6 7 8 9
> > >
> > > insert read 3
> > >
> > >         queue 3 5 6 7 8 9
> >
> > read-latency will not do that.
> 
> So what will it do? Must do something very much like what I described or
> it is a noop period. Please elaborate.

If a read was not merged with another read on the tail->head walk
the read will be inserted near the head.  The head->tail walk bypasses
all reads, six (default) writes and then inserts the new read.

It has the shortcoming that earlier reads may be walked past in the
tail->head phase.  It's a three-liner to prevent that but I was never
able to demonstrate any difference.

> >
> > > However I think even read-latency is more a workarond to a problem in
> > > the I/O queue dimensions.
> >
> > The problem is the 2.4 algorithm.  If a read is not mergeable or
> > insertable it is placed at the tail of the queue.  Which is the
> > worst possible place it can be put because applications wait on
> > reads, not on writes.
> 
> O_SYNC/-osync waits on writes too, so are you saying writes must go to
> the head because of that?

It has been discussed: boost a request to head-of-queue when a thread
starts to wait on a buffer/page which is inside that request.

But we don't care about synchronous writes.  As long as we don't
starve them out completely, optimise the (vastly more) common case.

> reads should be not too bad at the end too if
> only the queue wasn't that oversized when the merging is at its maximum.
> Fix the oversizing of the queue, then read-latency will matter much
> less.

Think about two threads.  One is generating a stream of writes and
the other is trying to read a file.  The reader needs to read the 
directory, the inode, the first data blocks, the first indirect and
then some more data blocks.  That's at least three synchronous reads.
Even if those reads are placed just three requests from head-of-queue,
the reader will make one tenth of the progress of the writer.

And the current code places those reads 64 requests from head-of-queue.

When the various things which were congesting write queueing were fixed
in the 2.5 VM a streaming write was slowing such read operations down by
a factor of 4000.