Re: Inconsistent Performance

[Kent Overstreet <kmo-PEzghdH756F8UrSeD/g0lQ@public.gmane.org>]

On Mon, Jul 29, 2013 at 03:32:49PM +1200, Andrew Thrift wrote:
> Hi,
> 
> We are noticing some odd performance characteristics with bcache on
> 3.10 Mainline kernel.
> 
> We are using Intel DC S3700 SSD and Seagate Constellation ES.2 HDD
> 
> Using FIO to perform a 4K random write test with QD=32 and 4
> instances we get:
> 
> RAW SSD:    55.5K iops
> 
> RAW HDD:   540 iops
> 
> Fresh bcache on top of SSD/HDD [writeback]:    36.8K iops
> 
> Which is not great, but OK.
> 
> However if I run the FIO test a number of times, letting it write
> out the data to the backing device after each run, after a few runs
> I will notice the IOPS drop.  When the benchmark starts it will sit
> around 14K IOPS, then as soon as it starts writing to the backing
> device this will drop to around 5K IOPS. On the subsequent runs it
> will never go above 5K IOPS.

Hmm... that sounds like something an SSD would do, but it sounds like
you've checked that performance to the bare SSD isn't degrading, and
that is a rather large regression.

> Has there been a performance regression on 3.10 ?

Certainly not impossible.

Could you try doing some profiling with perf and seeing if anything
shows up? Also, try flipping off discards if you have them on.

It's just write performance that's degrading, not read?