Re: Poor performance with bcache write-back mode

[Kent Overstreet <koverstreet-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>]

On Thu, May 16, 2013 at 02:51:46PM -0700, Patrick Ng wrote:
> Hi all,
> 
> I have installed Linux kernel 3.9.2 in my Fedora 18.  When I ran FIO in bcache write-through mode, the performance was good, but running a similar test in bcache write-back mode, I got very poor performance.
> Did I miss any options when setting the write-back mode?
> 
> Environment
> ===========
> 500G backing store at /dev/sdb1
> 150G SSD at /dev/sdc1
> 
> Write-through mode
> ==================
> make-bcache -B /dev/sdb1 -C /dev/sdc1
> echo /dev/sdb1 > /sys/fs/bcache/register
> echo /dev/sdc1 > /sys/fs/bcache/register
> mkfs.ext4 /dev/bcache0
> mount /dev/bcache0 /mnt/bcache
> 
> 
> Running FIO with random, 100% read, iosize=4096, queue depth=16
> the result is approx. 65k IOPS
> 
> 
> Write-backe mode (setup is almost the same as write-through except the extra step at the end "echo writeback ...")
> ================
> make-bcache -B /dev/sdb1 -C /dev/sdc1
> echo /dev/sdb1 > /sys/fs/bcache/register
> echo /dev/sdc1 > /sys/fs/bcache/register
> mkfs.ext4 /dev/bcache0
> mount /dev/bcache0 /mnt/bcache
> echo writeback > /sys/block/bcache0/bcache/cache_mode
> 
> Running FIO with random, 100% read, iosize=4096, queue depth=16
> the result is approx. 12k IOPS
> 
> Running FIO with random, 80% read 20% write, iosize=4096, queue depth=16
> the result is approx. 2k IOPS

You're probably getting bit by the congested bypass - try setting
congested_read_threshold_us and congested_write_threshold_us both to 0.

I wish there was a better solution to the congestion bypass stuff, it's
a hard problem...