Re: [QUESTION] Long read latencies on mixed rw buffered IO

[Dave Chinner <david@fromorbit.com>]

On Tue, Mar 26, 2019 at 05:44:34AM +0200, Amir Goldstein wrote:
> On Tue, Mar 26, 2019 at 1:48 AM Dave Chinner <david@fromorbit.com> wrote:
> > On Mon, Mar 25, 2019 at 09:57:46PM +0200, Amir Goldstein wrote:
> > > Not as bad as v1. Only a little bit worse than master...
> > > The whole deal with the read/write balance and on SSD, I imagine
> > > the balance really changes. That's why I was skeptical about
> > > one-size-fits all read/write balance.
> >
> > You're not testing your SSD. You're testing writes into cache vs
> > reads from disk. There is a massive latency difference in the two
> > operations, so unless you use O_DSYNC for the writes, you are going
> > to see this cache-vs-uncached performance unbalance. i.e. unless the
> > rwsem is truly fair, there is always going to be more writer
> > access to the lock because they spend less time holding it and so
> > can put much more pressure on it.
> >
> 
> Yeh, I know. SSD makes the balance better because of faster reads
> from disk. Was pointing out that the worse case I am interested in is
> on spindles. That said, O_DSYNC certainly does improve the balance
> and gives shorter worse case latencies. However, it does not make the
> problem go away. i_rwsem taken (even for 4K reads) takes its toll
> on write latencies (compared to ext4).

Sure, that's because it's serialising access across the entire file,
not just the page that is being read/written to.

The solution here is to move to concurrent buffered writes
similar to how we do direct IO. We need to track IO ranges for that,
though, so we can serialise unaligned and/or overlapping IOs. We
need fast, efficient range locks for this, though.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com