dm-cache with a very slow device

dm-cache with a very slow device

[Lauri Niskanen]

Hi,

In my understanding dm-cache is usually used with fairly fast spinning 
hard drives as the "slow" device. Because of this the cache is not used 
very aggressively and sequential accesses skip the cache. Is it possible 
to configure dm-cache for a scenario where the fast disk is a 
state-of-the-art SSD, but the slow disk is very slow especially so that 
sequential reads and writes are significantly slower than on the fast 
device?

I'd like it to work as a very aggressive writeback buffer, so that all 
or at least close to all writes are done to the fast device if there is 
at least some free non-dirty space left. And sequential reads should 
also be promoted fairly easily. In addition there should be aggressive 
demotions to make room for future promotions always keeping a sizable 
buffer available.

So the goal would be to achieve the performance characteristics of the 
fast device for all kinds of use cases even when the backing slow device 
is at least a magnitude slower.

-- 
Lauri Niskanen

Re: dm-cache with a very slow device

[Mike Snitzer]

On Fri, Aug 07 2015 at  1:39am -0400,
Lauri Niskanen <ape@ape3000.com> wrote:

> Hi,
> 
> In my understanding dm-cache is usually used with fairly fast
> spinning hard drives as the "slow" device. Because of this the cache
> is not used very aggressively and sequential accesses skip the
> cache. Is it possible to configure dm-cache for a scenario where the
> fast disk is a state-of-the-art SSD, but the slow disk is very slow
> especially so that sequential reads and writes are significantly
> slower than on the fast device?
> 
> I'd like it to work as a very aggressive writeback buffer, so that
> all or at least close to all writes are done to the fast device if
> there is at least some free non-dirty space left. And sequential
> reads should also be promoted fairly easily. In addition there
> should be aggressive demotions to make room for future promotions
> always keeping a sizable buffer available.
> 
> So the goal would be to achieve the performance characteristics of
> the fast device for all kinds of use cases even when the backing
> slow device is at least a magnitude slower.

You can disable the sequential IO bypass of the MQ policy using the
following (with a recent lvm2):
  lvchange --cachesettings 'sequential_threshold=0' <vg/lv>

This will cause all sequential IO to be considered for promotion to the
cache.  But even in "writeback" mode dm-cache isn't a purist writeback
cache like you're hoping for.  dm-cache is a hotspot cache; so by
default it will wait for evidence that promoting particular blocks is
meaningful.

With the MQ policy you can alter the default for the
write_promote_adjustment (e.g. setting it to 0) to try to get writes
promoted more aggressively.

But even with these steps the fast device likely won't be able to buffer
all IO written to the device in the hope of later writeback to the
really slow device.  You'll potentially thrash the cache if new writes
demote older blocks aggressively (not to mention partial cache block
writes will require reads from the really slow device).