comparison with rival projects

comparison with rival projects

[luvar-2S8i3W4CjuIzjkFyG6Beag]

Hi,
Iam new in problematic of caching persistent data. I want to use caching (on ssd disk) on my desktop with a lot of ram. I hope, it will provide me better latency for starting programs (developer/scientific desktop). My questions are:

Is it possible to use single ssd disk for making two caches? One for /dev/md2 (root partition, raid10) and second for /dev/md3 (raid5,home partition, etc) (I hope this is possible by partitioning that ssd disk and using partitions for caching)

Is it dangerous (in terms of data loss) to use writethrough caching to single ssd disk of raid5 block device? (I have 5 disks, using lvm on top of raid).

For caching device is most critical number for random reads or operations per second. Is it right?

Trim is not need feature for cache ssd device. It will have same cache performance without it. Is it right?

What benefits/disadvantages have bcache [1] project over dm-cache [2] and flashcache [3]? My current knowledge is that writethrough is possible only in bcache, but dm-cache is implemented in more "standard" way (thdough devicemaper). Also bcache has some more information on web.

[1] - http://bcache.evilpiepirate.org/
[2] - http://users.cis.fiu.edu/~zhaom/dmcache/index.html
[3] - https://github.com/facebook/flashcache

PS: Is there any tutorial for gentoo users?

Thanks for any explanations / answers,
--
LuVar
--
To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: comparison with rival projects

[Kent Overstreet]

On 12/15/2010 03:26 PM, luvar-2S8i3W4CjuIzjkFyG6Beag@public.gmane.org wrote:
> Hi,
> Iam new in problematic of caching persistent data. I want to use caching (on ssd disk) on my desktop with a lot of ram. I hope, it will provide me better latency for starting programs (developer/scientific desktop). My questions are:
>
> Is it possible to use single ssd disk for making two caches? One for /dev/md2 (root partition, raid10) and second for /dev/md3 (raid5,home partition, etc) (I hope this is possible by partitioning that ssd disk and using partitions for caching)

Yep. One cache device can be used with (currently) up to 256 backing 
devices.

>
> Is it dangerous (in terms of data loss) to use writethrough caching to single ssd disk of raid5 block device? (I have 5 disks, using lvm on top of raid).

Shouldn't be, recovery from unclean shutdown is quite well tested, both 
writethrough and writeback should be perfectly safe.

>
> For caching device is most critical number for random reads or operations per second. Is it right?

For writethrough caching random reads, for writeback both reads and writes.

>
> Trim is not need feature for cache ssd device. It will have same cache performance without it. Is it right?

Really just depends on the ssd; some (cheaper, earlier) drives are known 
for performance significantly degrading over time, with trim helps with 
(but doesn't necessarily eliminate). Bcache uses trim if it's available.

>
> What benefits/disadvantages have bcache [1] project over dm-cache [2] and flashcache [3]? My current knowledge is that writethrough is possible only in bcache, but dm-cache is implemented in more "standard" way (thdough devicemaper). Also bcache has some more information on web.
>
> [1] - http://bcache.evilpiepirate.org/
> [2] - http://users.cis.fiu.edu/~zhaom/dmcache/index.html
> [3] - https://github.com/facebook/flashcache

Flashcache is based off of dm-cache.

Flashcache has been used in production awhile, bcache is still a little 
rough around the edges - but bcache has better performance, more 
features, and it always orders writes correctly so as to be crash safe 
(flashcache has a "torn write" problem).

>
> PS: Is there any tutorial for gentoo users?

Nah, I'm an ubuntu/debian user. For caching / the important thing is to 
hook into your initramfs and get everything loaded before you mount your 
root filesystem.

>
> Thanks for any explanations / answers,
> --
> LuVar
> --
> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: comparison with rival projects

[Nauman Rafique]

On Wed, Dec 15, 2010 at 4:05 PM, Kent Overstreet
<kent.overstreet-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> On 12/15/2010 03:26 PM, luvar-2S8i3W4CjuIzjkFyG6Beag@public.gmane.org wrote:
>>
>> Hi,
>> Iam new in problematic of caching persistent data. I want to use caching
>> (on ssd disk) on my desktop with a lot of ram. I hope, it will provide me
>> better latency for starting programs (developer/scientific desktop). My
>> questions are:
>>
>> Is it possible to use single ssd disk for making two caches? One for
>> /dev/md2 (root partition, raid10) and second for /dev/md3 (raid5,home
>> partition, etc) (I hope this is possible by partitioning that ssd disk and
>> using partitions for caching)
>
> Yep. One cache device can be used with (currently) up to 256 backing
> devices.
>
>>
>> Is it dangerous (in terms of data loss) to use writethrough caching to
>> single ssd disk of raid5 block device? (I have 5 disks, using lvm on top of
>> raid).
>
> Shouldn't be, recovery from unclean shutdown is quite well tested, both
> writethrough and writeback should be perfectly safe.
>
>>
>> For caching device is most critical number for random reads or operations
>> per second. Is it right?
>
> For writethrough caching random reads, for writeback both reads and writes.
>
>>
>> Trim is not need feature for cache ssd device. It will have same cache
>> performance without it. Is it right?
>
> Really just depends on the ssd; some (cheaper, earlier) drives are known for
> performance significantly degrading over time, with trim helps with (but
> doesn't necessarily eliminate). Bcache uses trim if it's available.
>
>>
>> What benefits/disadvantages have bcache [1] project over dm-cache [2] and
>> flashcache [3]? My current knowledge is that writethrough is possible only
>> in bcache, but dm-cache is implemented in more "standard" way (thdough
>> devicemaper). Also bcache has some more information on web.
>>
>> [1] - http://bcache.evilpiepirate.org/
>> [2] - http://users.cis.fiu.edu/~zhaom/dmcache/index.html
>> [3] - https://github.com/facebook/flashcache
>
> Flashcache is based off of dm-cache.
>
> Flashcache has been used in production awhile, bcache is still a little
> rough around the edges - but bcache has better performance, more features,
> and it always orders writes correctly so as to be crash safe (flashcache has
> a "torn write" problem).

Kent, now that we are on the topic, I wonder if there has been some
benchmarking comparing bcache to flashcache performance?

Also I wonder if you plan to target for getting bcache included in
mainline kernel. I wonder if such attempts were made for flashcache
but that's probably off-topic.

Thanks.
--
Nauman

>
>>
>> PS: Is there any tutorial for gentoo users?
>
> Nah, I'm an ubuntu/debian user. For caching / the important thing is to hook
> into your initramfs and get everything loaded before you mount your root
> filesystem.
>
>>
>> Thanks for any explanations / answers,
>> --
>> LuVar
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
--
To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: comparison with rival projects

[Kent Overstreet]

On 12/15/2010 04:18 PM, Nauman Rafique wrote:
> Kent, now that we are on the topic, I wonder if there has been some
> benchmarking comparing bcache to flashcache performance?

The only comparison I know of has been on MySQL performance, measured 
with sysbench - but flashcache was written specifically for mysql so I 
think that's fair.

With an X25-E, Flashcache was getting - if memory serves - around 160 
transactions per second. After enough runs bcache stabilizes at 220-230 
tps (starts out a bit higher, 250-260). Running off just the SSD gets 
around 290.

I got to test a bit on an ioDrive (I'll have my own soon and plan on 
testing more and posting benchmarks when I have it) - bcache was getting 
a pretty consistent 600 tps, with just the ioDrive it started out at 750 
tps and looked like it was stabilizing at ~680 tps. I haven't tried 
flashcache on that hardware, though.

>
> Also I wonder if you plan to target for getting bcache included in
> mainline kernel. I wonder if such attempts were made for flashcache
> but that's probably off-topic.

I am, I've just been chipping away at the todo list to get it ready to 
submit. I was just reworking my bio splitting code to not depend on some 
changes to the generic block layer - that was a big one, but I'm almost 
finished debugging that code.

Besides that, the next thing is adding and testing with fault injection, 
and making memory allocation deadlock proof; that should keep me busy 
for a bit. The main thing though is going to be figuring out what to do 
about the hooks in the block layer bcache is using; I don't expect nor 
want that code to be merged but I'm loathe to give up that functionality.

When thin provisioning/volume management is added those hooks won't be 
necessary, if you're not transparently caching an existing device - so 
what could conceivably happen is the transparent caching functionality 
never gets merged and the mainline version just has bcache managed 
backing devices. The other alternative could be using device mapper for 
transparently hooking in to existing block devices... which would still 
mean a loss of functionality, but it'd probably be less code to write 
and a shorter path to mainline.

I've been planning on pinging various other devs and asking what they 
think once I've got the code synced up to mainline - I'm still working 
off 2.6.35, I plan on resyncing with the latest stable once I've got 
some of the fault injection work done, hopefully not too much longer. 
Timeframe will be dependent on how the funding situation works out :)
--
To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: comparison with rival projects

[luvar-2S8i3W4CjuIzjkFyG6Beag]

First of all, I would like to thanks for answers.

One more question and some reply in text.

After making ssd disc a cache for some partition, there is possibility to detach cache. What should I do to "erase" (trim) whole disk for another benchmarking? I think that "dd if=/dev/zero of=/dev/sdx" wouldnt be right command.

----- "Nauman Rafique" <nauman-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org> wrote:

> On Wed, Dec 15, 2010 at 4:05 PM, Kent Overstreet
> <kent.overstreet-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> wrote:
> > On 12/15/2010 03:26 PM, luvar-2S8i3W4CjuIzjkFyG6Beag@public.gmane.org wrote:
> >>
> >> Hi,
> >> Iam new in problematic of caching persistent data. I want to use
> caching
> >> (on ssd disk) on my desktop with a lot of ram. I hope, it will
> provide me
> >> better latency for starting programs (developer/scientific
> desktop). My
> >> questions are:
> >>
> >> Is it possible to use single ssd disk for making two caches? One
> for
> >> /dev/md2 (root partition, raid10) and second for /dev/md3
> (raid5,home
> >> partition, etc) (I hope this is possible by partitioning that ssd
> disk and
> >> using partitions for caching)
> >
> > Yep. One cache device can be used with (currently) up to 256
> backing
> > devices.
> >
> >>
> >> Is it dangerous (in terms of data loss) to use writethrough caching
> to
> >> single ssd disk of raid5 block device? (I have 5 disks, using lvm
> on top of
> >> raid).
> >
> > Shouldn't be, recovery from unclean shutdown is quite well tested,
> both
> > writethrough and writeback should be perfectly safe.
> >
> >>
> >> For caching device is most critical number for random reads or
> operations
> >> per second. Is it right?
> >
> > For writethrough caching random reads, for writeback both reads and
> writes.
> >
> >>
> >> Trim is not need feature for cache ssd device. It will have same
> cache
> >> performance without it. Is it right?
> >
> > Really just depends on the ssd; some (cheaper, earlier) drives are
> known for
> > performance significantly degrading over time, with trim helps with
> (but
> > doesn't necessarily eliminate). Bcache uses trim if it's available.
> >
> >>
> >> What benefits/disadvantages have bcache [1] project over dm-cache
> [2] and
> >> flashcache [3]? My current knowledge is that writethrough is
> possible only
> >> in bcache, but dm-cache is implemented in more "standard" way
> (thdough
> >> devicemaper). Also bcache has some more information on web.
> >>
> >> [1] - http://bcache.evilpiepirate.org/
> >> [2] - http://users.cis.fiu.edu/~zhaom/dmcache/index.html
> >> [3] - https://github.com/facebook/flashcache
> >
> > Flashcache is based off of dm-cache.
> >
> > Flashcache has been used in production awhile, bcache is still a
> little
> > rough around the edges - but bcache has better performance, more
> features,
> > and it always orders writes correctly so as to be crash safe
> (flashcache has
> > a "torn write" problem).
> 
> Kent, now that we are on the topic, I wonder if there has been some
> benchmarking comparing bcache to flashcache performance?
> 
> Also I wonder if you plan to target for getting bcache included in
> mainline kernel. I wonder if such attempts were made for flashcache
> but that's probably off-topic.

I hope, next two links will help in making comparison.
http://www.mysqlperformanceblog.com/2010/05/10/flashcache-first-experiments/
http://www.mysqlperformanceblog.com/2010/05/18/flashcache-more-benchmarks/
There is simple methodology of cache testing by using it with mysql database. I hope, someone will catch this discussion and try some comparable test on bcache.

For me will be next test sufficient:
Activate caching and make for example 20 reboots. Measure time from computer bios beep till starting gimp, which will be in gnome/kde/fluxbox autostart script. Of course before starting gimp, there will be need for starting some other programs like firefox, eclipse, mplayer with some film...

I would like to see these times of booting in graph. I promise that after installation of bcache I will do this test with bcache and post it here.

> 
> Thanks.
> --
> Nauman
> 
> >
> >>
> >> PS: Is there any tutorial for gentoo users?
> >
> > Nah, I'm an ubuntu/debian user. For caching / the important thing is
> to hook
> > into your initramfs and get everything loaded before you mount your
> root
> > filesystem.
> >
> >>
> >> Thanks for any explanations / answers,
> >> --
> >> LuVar
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe
> linux-bcache" in
> >> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> >> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >
> > --
> > To unsubscribe from this list: send the line "unsubscribe
> linux-bcache" in
> > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >

PS: I am now decided to use bcache (after buying some ssd).
--
To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: comparison with rival projects

[Kent Overstreet]

On 12/15/2010 04:46 PM, luvar-2S8i3W4CjuIzjkFyG6Beag@public.gmane.org wrote:
> First of all, I would like to thanks for answers.
>
> One more question and some reply in text.
>
> After making ssd disc a cache for some partition, there is possibility to detach cache. What should I do to "erase" (trim) whole disk for another benchmarking? I think that "dd if=/dev/zero of=/dev/sdx" wouldnt be right command.

Yeah, one nifty thing is you can disable caching on a mounted, in use 
filesystem (and reenable it provided you invalidate the cache somehow).

Hdparm can send a trim, but it can't do the whole disk at once, you've 
got to script it. I wrote one awhile back, this is what I use:

#!/bin/bash

DEV=$1

if [ ! -b "$DEV" ]; then
         echo "Can't open $DEV"
         exit 1
fi

SECTORS=`hdparm -I "$DEV"|awk '/LBA48/ { print $5 }'`
OFFSET=0

echo "$DEV has $SECTORS sectors:"

while (($SECTORS)); do
         i=$(( $SECTORS > 65535 ? 65535 : $SECTORS ))

         echo "wiping $i sectors at offset $OFFSET"
         hdparm --please-destroy-my-drive \
		--trim-sector-ranges $OFFSET:$i $DEV

         SECTORS=$(($SECTORS - $i))
         OFFSET=$(($OFFSET + $i))
done
--
To unsubscribe from this list: send the line "unsubscribe linux-bcache" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html