Thoughts on using SSD

Thoughts on using SSD

[Bill Davidsen]

I'm building a fairly aggressive machine for both a backup host for 
virtual machines and spare time development platform, compile engine and 
testbed both. I want to get cost effective use from an SSD unit, and I 
propose to use a 32GB unit as follows: for the root filesystem, 12GB, 
which should hold all the usual root things, and 16GB for swap (12GB 
RAM, and I want boot and/or hibernate to happen NOW). The remaining 
space I think might be used for various high impact things, and one of 
those with speeding raid.

If I were to create a small raid device, raid1, made of the 4GB Ssd and 
4GB of SATA space, if I made the SATA write-mostly and write-behind, and 
put the journal for my raid arrays (and bitmaps?) that seems likely to 
provide a significant performance gain in small storage.

Am I missing anything here? Is there an obvious drawback I'm missing?

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc

"You are disgraced professional losers. And by the way, give us our money back."
    - Representative Earl Pomeroy,  Democrat of North Dakota
on the A.I.G. executives who were paid bonuses  after a federal bailout.

Re: Thoughts on using SSD

[David Rees]

On Thu, Mar 26, 2009 at 4:09 PM, Bill Davidsen <davidsen@tmr.com> wrote:
> If I were to create a small raid device, raid1, made of the 4GB Ssd and 4GB
> of SATA space, if I made the SATA write-mostly and write-behind, and put the
> journal for my raid arrays (and bitmaps?) that seems likely to provide a
> significant performance gain in small storage.
>
> Am I missing anything here? Is there an obvious drawback I'm missing?

Yeah, the fact that it doesn't seem to be possible to take advantage
of the write-behind feature? :-p

Besides fsync, I think it may have something to do with barriers.  As
in, when you flush a barrier, that causes entire cache to get flushed
to disk, including your write-behind disk.

Hopefully Neil can chime in here.  I found this thread from a while
back which seemed relevant, but I haven't been able to digest the
whole thing yet: http://lkml.org/lkml/2007/5/25/71

-Dave

Re: Thoughts on using SSD

[Neil Brown]

On Thursday March 26, drees76@gmail.com wrote:
> On Thu, Mar 26, 2009 at 4:09 PM, Bill Davidsen <davidsen@tmr.com> wrote:
> > If I were to create a small raid device, raid1, made of the 4GB Ssd and 4GB
> > of SATA space, if I made the SATA write-mostly and write-behind, and put the
> > journal for my raid arrays (and bitmaps?) that seems likely to provide a
> > significant performance gain in small storage.
> >
> > Am I missing anything here? Is there an obvious drawback I'm missing?
> 
> Yeah, the fact that it doesn't seem to be possible to take advantage
> of the write-behind feature? :-p
> 
> Besides fsync, I think it may have something to do with barriers.  As
> in, when you flush a barrier, that causes entire cache to get flushed
> to disk, including your write-behind disk.

Nope.  The barriers are sent separately to both devices.  Yes, they
both get flushed, but we don't wait for the write-behind flush to
complete.

NeilBrown


> 
> Hopefully Neil can chime in here.  I found this thread from a while
> back which seemed relevant, but I haven't been able to digest the
> whole thing yet: http://lkml.org/lkml/2007/5/25/71
> 
> -Dave
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Thoughts on using SSD

[Neil Brown]

On Thursday March 26, davidsen@tmr.com wrote:
> I'm building a fairly aggressive machine for both a backup host for 
> virtual machines and spare time development platform, compile engine and 
> testbed both. I want to get cost effective use from an SSD unit, and I 
> propose to use a 32GB unit as follows: for the root filesystem, 12GB, 
> which should hold all the usual root things, and 16GB for swap (12GB 
> RAM, and I want boot and/or hibernate to happen NOW). The remaining 
> space I think might be used for various high impact things, and one of 
> those with speeding raid.
> 
> If I were to create a small raid device, raid1, made of the 4GB Ssd and 
> 4GB of SATA space, if I made the SATA write-mostly and write-behind, and 
> put the journal for my raid arrays (and bitmaps?) that seems likely to 
> provide a significant performance gain in small storage.
> 
> Am I missing anything here? Is there an obvious drawback I'm missing?


I'd probably just put the journal on the SSD and mount my ext3
filesystem data=journal

That has a similar effect to raid1/write-behind in that data is
written to both but we only wait for the write to the SSD to
complete.  But as it is done at the filesystem level - and the
filesystem has a much better idea what it is doing - you would expect
to get much more efficient results.  e.g. less wasted memory, much
larger amount of data that is safe of SSD but still trickling out to
the HD.

I guess there might be a small lost in data safety as is the journal
device fails you lose your journal.  But that is only 5 seconds of
data, and I suspect SSDs don't suffer from many of the failure modes
of HDs.

.... I wonder if it would make sense to mirror two partitions of the
one SSD?? It would save you from media errors and only expose you to
total-drive-death errors.


NeilBrown

Re: Thoughts on using SSD

[Lars Marowsky-Bree]

On 2009-03-27T15:37:23, Neil Brown <neilb@suse.de> wrote:

> .... I wonder if it would make sense to mirror two partitions of the
> one SSD?? It would save you from media errors and only expose you to
> total-drive-death errors.

Depends; SSDs have different failure modes. One of them is that for
performance, memory is often interleaved and heavily remapped. So if one
of the cells dies, you may in fact lose random sectors in _both_ of your
partitions, so they wouldn't be independent and not necessarily make for
good RAID targets ;-)


Regards,
    Lars

-- 
Teamlead Kernel, SuSE Labs, Research and Development
SUSE LINUX Products GmbH, GF: Markus Rex, HRB 16746 (AG Nürnberg)
"Experience is the name everyone gives to their mistakes." -- Oscar Wilde

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Thoughts on using SSD

[Bill Davidsen]

Neil Brown wrote:
> On Thursday March 26, davidsen@tmr.com wrote:
>   
>> I'm building a fairly aggressive machine for both a backup host for 
>> virtual machines and spare time development platform, compile engine and 
>> testbed both. I want to get cost effective use from an SSD unit, and I 
>> propose to use a 32GB unit as follows: for the root filesystem, 12GB, 
>> which should hold all the usual root things, and 16GB for swap (12GB 
>> RAM, and I want boot and/or hibernate to happen NOW). The remaining 
>> space I think might be used for various high impact things, and one of 
>> those with speeding raid.
>>
>> If I were to create a small raid device, raid1, made of the 4GB Ssd and 
>> 4GB of SATA space, if I made the SATA write-mostly and write-behind, and 
>> put the journal for my raid arrays (and bitmaps?) that seems likely to 
>> provide a significant performance gain in small storage.
>>
>> Am I missing anything here? Is there an obvious drawback I'm missing?
>>     
>
>
> I'd probably just put the journal on the SSD and mount my ext3
> filesystem data=journal
>
> That has a similar effect to raid1/write-behind in that data is
> written to both but we only wait for the write to the SSD to
> complete.  But as it is done at the filesystem level - and the
> filesystem has a much better idea what it is doing - you would expect
> to get much more efficient results.  e.g. less wasted memory, much
> larger amount of data that is safe of SSD but still trickling out to
> the HD.
>   

But I think for raid in general you would benefit from having the bitmap 
on SSD as well. In my dreams I also put the inodes on that SSD, and 
everything runs 10x faster. Unfortunately no f/s seems to offer this.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc

"You are disgraced professional losers. And by the way, give us our money back."
    - Representative Earl Pomeroy,  Democrat of North Dakota
on the A.I.G. executives who were paid bonuses  after a federal bailout.

Re: Thoughts on using SSD

[Neil Brown]

On Thursday April 9, davidsen@tmr.com wrote:
> Neil Brown wrote:
> > On Thursday March 26, davidsen@tmr.com wrote:
> >   
> >> I'm building a fairly aggressive machine for both a backup host for 
> >> virtual machines and spare time development platform, compile engine and 
> >> testbed both. I want to get cost effective use from an SSD unit, and I 
> >> propose to use a 32GB unit as follows: for the root filesystem, 12GB, 
> >> which should hold all the usual root things, and 16GB for swap (12GB 
> >> RAM, and I want boot and/or hibernate to happen NOW). The remaining 
> >> space I think might be used for various high impact things, and one of 
> >> those with speeding raid.
> >>
> >> If I were to create a small raid device, raid1, made of the 4GB Ssd and 
> >> 4GB of SATA space, if I made the SATA write-mostly and write-behind, and 
> >> put the journal for my raid arrays (and bitmaps?) that seems likely to 
> >> provide a significant performance gain in small storage.
> >>
> >> Am I missing anything here? Is there an obvious drawback I'm missing?
> >>     
> >
> >
> > I'd probably just put the journal on the SSD and mount my ext3
> > filesystem data=journal
> >
> > That has a similar effect to raid1/write-behind in that data is
> > written to both but we only wait for the write to the SSD to
> > complete.  But as it is done at the filesystem level - and the
> > filesystem has a much better idea what it is doing - you would expect
> > to get much more efficient results.  e.g. less wasted memory, much
> > larger amount of data that is safe of SSD but still trickling out to
> > the HD.
> >   
> 
> But I think for raid in general you would benefit from having the bitmap 
> on SSD as well. In my dreams I also put the inodes on that SSD, and 
> everything runs 10x faster. Unfortunately no f/s seems to offer this.

Sounds like hierarchical storage management at a different level.
Keep the metadata on a fast device and allow the data to migrate on to
slow devices.
I'm sure we'll get there one day.... if only there were more hours in
the day :-)

NeilBrown