Cache everything

Cache everything

[Jure Erznožnik]

I have a large array sitting in my server containing a bit of data
that is used ofter and a lot of data that is used rarely.
My goal is to make the platters spin up as rarely as possible while
bcache handles most of the requests to the array.

I have set the disks spin-down time to 15 minutes so that during
high-activity periods they don't cycle power states too much.

I have set bcache parameters such:
cache_mode - writeback
readahead - 0
sequential_cutoff - 0
writeback_delay - 3600 (one hour)
writeback_percent - 40 (won't go higher)

I was hoping that this would make bcache cache everything leaving the
array practically untouched for an hour. However, stats_hour/bypassed
still shows megabytes of bypassed data and a cache_hit ratio of ~50%
even when I'm working ONLY with the frequently accessed data portion
(some 20% of total cache capacity). Also tiny writes are being made to
the array despite writeback_delay not expiring.

What other parameter am I forgetting to achieve what I want?

Thanks,
Jure

Re: Cache everything

[Killian De Volder]

Also check  congested_write_threshold_us, if the SSD is "congested" it bypasses it to speedup writes.

Killian De Volder


On 30-12-16 09:38, Jure Erznožnik wrote:
> I have a large array sitting in my server containing a bit of data
> that is used ofter and a lot of data that is used rarely.
> My goal is to make the platters spin up as rarely as possible while
> bcache handles most of the requests to the array.
> 
> I have set the disks spin-down time to 15 minutes so that during
> high-activity periods they don't cycle power states too much.
> 
> I have set bcache parameters such:
> cache_mode - writeback
> readahead - 0
> sequential_cutoff - 0
> writeback_delay - 3600 (one hour)
> writeback_percent - 40 (won't go higher)
> 
> I was hoping that this would make bcache cache everything leaving the
> array practically untouched for an hour. However, stats_hour/bypassed
> still shows megabytes of bypassed data and a cache_hit ratio of ~50%
> even when I'm working ONLY with the frequently accessed data portion
> (some 20% of total cache capacity). Also tiny writes are being made to
> the array despite writeback_delay not expiring.
> 
> What other parameter am I forgetting to achieve what I want?
> 
> Thanks,
> Jure
> --
> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: Cache everything

[Jure Erznožnik]

This seems to have fixed that, thanks. I keep forgetting there's two
places I have to check for configuration.

LP,
Jure

On Fri, Dec 30, 2016 at 9:58 AM, Killian De Volder
<killian.de.volder@megasoft.be> wrote:
> Also check  congested_write_threshold_us, if the SSD is "congested" it bypasses it to speedup writes.
>
> Killian De Volder
>
>
> On 30-12-16 09:38, Jure Erznožnik wrote:
>> I have a large array sitting in my server containing a bit of data
>> that is used ofter and a lot of data that is used rarely.
>> My goal is to make the platters spin up as rarely as possible while
>> bcache handles most of the requests to the array.
>>
>> I have set the disks spin-down time to 15 minutes so that during
>> high-activity periods they don't cycle power states too much.
>>
>> I have set bcache parameters such:
>> cache_mode - writeback
>> readahead - 0
>> sequential_cutoff - 0
>> writeback_delay - 3600 (one hour)
>> writeback_percent - 40 (won't go higher)
>>
>> I was hoping that this would make bcache cache everything leaving the
>> array practically untouched for an hour. However, stats_hour/bypassed
>> still shows megabytes of bypassed data and a cache_hit ratio of ~50%
>> even when I'm working ONLY with the frequently accessed data portion
>> (some 20% of total cache capacity). Also tiny writes are being made to
>> the array despite writeback_delay not expiring.
>>
>> What other parameter am I forgetting to achieve what I want?
>>
>> Thanks,
>> Jure
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Cache everything

[Jure Erznožnik]

OK, having fixed that issue, I'm faced with another one I don't know
how to solve:

I have my bcache set such that platters would be spun down most of the
time if possible. writeback_delay is set to 6 hours, there's enough
cache size, everything is set to hold data on the SSD as much as
possible.

However, my platters never spin down. iostat shows that there are some
**zero-sized** writes to the backing device every 12 or so seconds on
average (I ran iostat 600 to obtain those numbers). If I start dumping
traffic, it's generated by bcache's kworker thread (identified by
observing its activity while flushing dirty data). I have tried to
modify parameters, but nothing I tried seemed to have any effect.
Those writes are happening even if I reduce writeback_delay or any
other parameter for that matter. They are not related to any actual
writes to the bcache device (it doesn't matter whether I'm writing
gigabytes or just doing minor iSCSI touches)

Any suggestions highly appreciated,
Jure

On Sat, Dec 31, 2016 at 6:20 AM, Jure Erznožnik
<jure.erznoznik@gmail.com> wrote:
> This seems to have fixed that, thanks. I keep forgetting there's two
> places I have to check for configuration.
>
> LP,
> Jure
>
> On Fri, Dec 30, 2016 at 9:58 AM, Killian De Volder
> <killian.de.volder@megasoft.be> wrote:
>> Also check  congested_write_threshold_us, if the SSD is "congested" it bypasses it to speedup writes.
>>
>> Killian De Volder
>>
>>