bcache compared to more RAM

bcache compared to more RAM

[C Sights]

Hi all,
   Has anyone tried creating a bcache device in RAM and benchmarking 
performance of that versus just allowing that RAM to be used as file system 
buffers?

(Not on the mailing list, please reply to me directly as well...) 
Chad.

Re: bcache compared to more RAM

[John McCabe-Dansted]

On Wed, Aug 15, 2012 at 7:30 PM, C Sights <csights-97jfqw80gc6171pxa8y+qA@public.gmane.org> wrote:
> Hi all,
>    Has anyone tried creating a bcache device in RAM and benchmarking
> performance of that versus just allowing that RAM to be used as file system
> buffers?

I doubt it; bcache is designed to use non-volatile flash memory rather
than ram.

If the filesystem is readonly, I imagine that using RAM for bcache
would lead to worse performance as it would just add overhead. If the
filesystem is read-write then it might lead to better performance but
would also lead to the file-system being trashed on if unexpected
power loss occurs.

Under what circumstance would you intend to actually use ram backed bcache?

-- 
John C. McCabe-Dansted

Re: bcache compared to more RAM

[Dave Jiang]

On 08/15/2012 04:51 AM, John McCabe-Dansted wrote:
> On Wed, Aug 15, 2012 at 7:30 PM, C Sights <csights-97jfqw80gc6171pxa8y+qA@public.gmane.org> wrote:
>> Hi all,
>>    Has anyone tried creating a bcache device in RAM and benchmarking
>> performance of that versus just allowing that RAM to be used as file system
>> buffers?
> I doubt it; bcache is designed to use non-volatile flash memory rather
> than ram.
>
> If the filesystem is readonly, I imagine that using RAM for bcache
> would lead to worse performance as it would just add overhead. If the
> filesystem is read-write then it might lead to better performance but
> would also lead to the file-system being trashed on if unexpected
> power loss occurs.
>
> Under what circumstance would you intend to actually use ram backed bcache?
>

So the Intel Xeon E5-2600 series platforms has this feature called
asynchronous DRAM refresh (ADR). Essentially it's battery backed DRAM on
per DIMM socket basis. I have written a RAM disk driver on top of that
experimentally and played a little bit with it used by bcache instead of
SSD. Writing to the memory region isn't going to be as fast as normal
cached RAM because you have to ensure the data is flushed to the DRAM.
However, read should be the same as cached DRAM. But the writing should
still be faster than SSD. Anyhow, I'm more curious as to how I can
benchmark this to demonstrate if this is a benefit with bcache. i.e.
what benchmarking tool(s) I should use and with what parameters etc.

Re: bcache compared to more RAM

[Joseph Glanville]

On 16 August 2012 02:23, Dave Jiang <dave.jiang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> wrote:
> On 08/15/2012 04:51 AM, John McCabe-Dansted wrote:
>> On Wed, Aug 15, 2012 at 7:30 PM, C Sights <csights-97jfqw80gc6171pxa8y+qA@public.gmane.org> wrote:
>>> Hi all,
>>>    Has anyone tried creating a bcache device in RAM and benchmarking
>>> performance of that versus just allowing that RAM to be used as file system
>>> buffers?
>> I doubt it; bcache is designed to use non-volatile flash memory rather
>> than ram.
>>
>> If the filesystem is readonly, I imagine that using RAM for bcache
>> would lead to worse performance as it would just add overhead. If the
>> filesystem is read-write then it might lead to better performance but
>> would also lead to the file-system being trashed on if unexpected
>> power loss occurs.
>>
>> Under what circumstance would you intend to actually use ram backed bcache?
>>
>
> So the Intel Xeon E5-2600 series platforms has this feature called
> asynchronous DRAM refresh (ADR). Essentially it's battery backed DRAM on
> per DIMM socket basis. I have written a RAM disk driver on top of that
> experimentally and played a little bit with it used by bcache instead of
> SSD. Writing to the memory region isn't going to be as fast as normal
> cached RAM because you have to ensure the data is flushed to the DRAM.
> However, read should be the same as cached DRAM. But the writing should
> still be faster than SSD. Anyhow, I'm more curious as to how I can
> benchmark this to demonstrate if this is a benefit with bcache. i.e.
> what benchmarking tool(s) I should use and with what parameters etc.

fio - the flexible I/O tester is by far the best tool.
It's usage is demonstrated on the bcache wiki frontpage and there are
numerous tutorials.
I suggest using the linux AIO engine and a high queue depth to see the benefit.

>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Joseph.

-- 
CTO | Orion Virtualisation Solutions | www.orionvm.com.au
Phone: 1300 56 99 52 | Mobile: 0428 754 846

Re: bcache compared to more RAM

[Jack Wang]

2012/8/16 Dave Jiang <dave.jiang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>:
> On 08/15/2012 04:51 AM, John McCabe-Dansted wrote:
>> On Wed, Aug 15, 2012 at 7:30 PM, C Sights <csights-97jfqw80gc6171pxa8y+qA@public.gmane.org> wrote:
>>> Hi all,
>>>    Has anyone tried creating a bcache device in RAM and benchmarking
>>> performance of that versus just allowing that RAM to be used as file system
>>> buffers?
>> I doubt it; bcache is designed to use non-volatile flash memory rather
>> than ram.
>>
>> If the filesystem is readonly, I imagine that using RAM for bcache
>> would lead to worse performance as it would just add overhead. If the
>> filesystem is read-write then it might lead to better performance but
>> would also lead to the file-system being trashed on if unexpected
>> power loss occurs.
>>
>> Under what circumstance would you intend to actually use ram backed bcache?
>>
>
> So the Intel Xeon E5-2600 series platforms has this feature called
> asynchronous DRAM refresh (ADR). Essentially it's battery backed DRAM on
> per DIMM socket basis. I have written a RAM disk driver on top of that
> experimentally and played a little bit with it used by bcache instead of
> SSD. Writing to the memory region isn't going to be as fast as normal
> cached RAM because you have to ensure the data is flushed to the DRAM.
> However, read should be the same as cached DRAM. But the writing should
> still be faster than SSD. Anyhow, I'm more curious as to how I can
> benchmark this to demonstrate if this is a benefit with bcache. i.e.
> what benchmarking tool(s) I should use and with what parameters etc.
>
Hi Dave,

Intesting, wish to see the result. what cache mode is intended to use with ADR.

Jack

Re: bcache compared to more RAM

[Dave Jiang]

On Wed 15 Aug 2012 08:29:38 PM MST, Jack Wang wrote:
> 2012/8/16 Dave Jiang <dave.jiang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>:
>> On 08/15/2012 04:51 AM, John McCabe-Dansted wrote:
>>> On Wed, Aug 15, 2012 at 7:30 PM, C Sights <csights-97jfqw80gc6171pxa8y+qA@public.gmane.org> wrote:
>>>> Hi all,
>>>>    Has anyone tried creating a bcache device in RAM and benchmarking
>>>> performance of that versus just allowing that RAM to be used as file system
>>>> buffers?
>>> I doubt it; bcache is designed to use non-volatile flash memory rather
>>> than ram.
>>>
>>> If the filesystem is readonly, I imagine that using RAM for bcache
>>> would lead to worse performance as it would just add overhead. If the
>>> filesystem is read-write then it might lead to better performance but
>>> would also lead to the file-system being trashed on if unexpected
>>> power loss occurs.
>>>
>>> Under what circumstance would you intend to actually use ram backed bcache?
>>>
>>
>> So the Intel Xeon E5-2600 series platforms has this feature called
>> asynchronous DRAM refresh (ADR). Essentially it's battery backed DRAM on
>> per DIMM socket basis. I have written a RAM disk driver on top of that
>> experimentally and played a little bit with it used by bcache instead of
>> SSD. Writing to the memory region isn't going to be as fast as normal
>> cached RAM because you have to ensure the data is flushed to the DRAM.
>> However, read should be the same as cached DRAM. But the writing should
>> still be faster than SSD. Anyhow, I'm more curious as to how I can
>> benchmark this to demonstrate if this is a benefit with bcache. i.e.
>> what benchmarking tool(s) I should use and with what parameters etc.
>>
> Hi Dave,
>
> Intesting, wish to see the result. what cache mode is intended to use with ADR.
>
> Jack
> --

Jack,
Right now I'm using wb mode for cache. However when I write I call 
clflush_cache_range() and then read the last dword to make really sure 
that everything has made it to the DRAM. I do wish it's possible on IA 
to just flush without invalidate the cache though.

Re: bcache compared to more RAM

[C Sights]

> >>> Under what circumstance would you intend to actually use ram backed
> >>> bcache?

A laptop with a fully charged battery?  :)

Thanks for the flexible I/O tester idea.  Would it be possible to mount bcache 
to a file stored in /dev/shm? 

Mostly I am curious as to whether bcache would do better than Linux's other 
RAM based caching of filesystem data.

Thanks for the ideas and the code!
Chad.