Re: bcache gets stuck flushing writeback cache when used in combination with LUKS/dm-crypt and non-default bucket size

[Tim Small <tim@buttersideup.com>]

Hello,

I've just hit the same bug in production, as it happens on a similar config:

. 4.5.1 (Debian backports kernel)
. bcache with 2M bucket (Intel DC S3500), with dm-crypt layered on top
of the backing device (4x 8TB RAID5).

I'm reducing the cc list, as I think this is bcache specific.

On 11/05/16 02:38, Eric Wheeler wrote:

> You might try LUKS atop of bcache instead of under it.  This might be 
> better for privacy too, otherwise your cached data is unencrypted.

I chose the same config as James, because the SSD has hardware
encryption (whereas the hard drives don't), and it'd be nice if cache
reads didn't get the extra CPU overhead/latency since the workload is
read-heavy, and cache hit should be pretty high (and the CPU doesn't
have AES-NI).

>> # Make cache set on second drive
>> # IMPORTANT:  Problem does not occur if I omit --bucket parameter.
>> make-bcache --bucket 2M -C /dev/sdb
> 
> 2MB is quite large, maybe it exceeds the 256-bvec limit.

In my case I followed the instructions in the make-bcache manual page
which say:

"The bucket size is intended to be equal to the size of your SSD's erase
blocks"

A bit of research suggested an erase block size of either 2M or 4M for
the SSD I was using.  Is this manual page incorrect?

> Ming Lei's patch got in to 4.6 yet, but try this:
>   https://lkml.org/lkml/2016/4/5/1046
> 
> and maybe Shaohua Li's patch too:
>   http://www.spinics.net/lists/raid/msg51830.html

I'll give them both a go...

Perhaps there should probably be a bcache wiki hosted on kernel.org to
cover this sort of stuff, as the bcache docs seem to be a bit lacking
currently?

Tim.