Sharing disks amoung multiple software RAIDs

Sharing disks amoung multiple software RAIDs

[Alex Davis]

Is this a bad thing? I'm guessing that it is, but I want independent
confirmation before I spoke to someone I know who's doing this.

Thanks


      ____________________________________________________________________________________
Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

Re: Sharing disks amoung multiple software RAIDs

[Justin Piszcz]

On Thu, 1 May 2008, Alex Davis wrote:

> Is this a bad thing? I'm guessing that it is, but I want independent
> confirmation before I spoke to someone I know who's doing this.
>
> Thanks
>
>
>      ____________________________________________________________________________________
> Be a better friend, newshound, and
> know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

What is the use case, why would you want to do that?
I have seen people on the list do it before, for example are you going to 
be utilizing both raids at the same time?  If so, I would advise against 
it.

What is the reasoning?

Re: Sharing disks amoung multiple software RAIDs

[Alex Davis]

--- On Thu, 5/1/08, Justin Piszcz <jpiszcz@lucidpixels.com> wrote:

> From: Justin Piszcz <jpiszcz@lucidpixels.com>
> Subject: Re: Sharing disks amoung multiple software RAIDs
> To: "Alex Davis" <alex14641@yahoo.com>
> Cc: linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org
> Date: Thursday, May 1, 2008, 8:50 AM
> On Thu, 1 May 2008, Alex Davis wrote:
> 
> > Is this a bad thing? I'm guessing that it is, but
> I want independent
> > confirmation before I spoke to someone I know
> who's doing this.
> >
> > Thanks
> >
> >
> >     
> What is the use case, why would you want to do that?
> I have seen people on the list do it before, for example
> are you going to be utilizing both raids at the same time? 
Yes.

> If so, I would advise against it.
> 
> What is the reasoning?

No, I don't want to do this. I know someone who is, and I wanted to get
more input before I advised them to get more disks. The RAIDs are running
in degraded mode, so they'll need more disks anyway. Since they are (or
hopefully soon will be) buying more disks, I'll advise them to get
dedicated disks for each RAID.



      ____________________________________________________________________________________
Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

Re: Sharing disks amoung multiple software RAIDs

[Bill Davidsen]

Alex Davis wrote:
> --- On Thu, 5/1/08, Justin Piszcz <jpiszcz@lucidpixels.com> wrote:
>
>   
>> From: Justin Piszcz <jpiszcz@lucidpixels.com>
>> Subject: Re: Sharing disks amoung multiple software RAIDs
>> To: "Alex Davis" <alex14641@yahoo.com>
>> Cc: linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org
>> Date: Thursday, May 1, 2008, 8:50 AM
>> On Thu, 1 May 2008, Alex Davis wrote:
>>
>>     
>>> Is this a bad thing? I'm guessing that it is, but
>>>       
>> I want independent
>>     
>>> confirmation before I spoke to someone I know
>>>       
>> who's doing this.
>>     
>>> Thanks
>>>
>>>
>>>     
>>>       
>> What is the use case, why would you want to do that?
>> I have seen people on the list do it before, for example
>> are you going to be utilizing both raids at the same time? 
>>     
> Yes.
>
>   
>> If so, I would advise against it.
>>
>> What is the reasoning?
>>     
>
> No, I don't want to do this. I know someone who is, and I wanted to get
> more input before I advised them to get more disks. The RAIDs are running
> in degraded mode, so they'll need more disks anyway. Since they are (or
> hopefully soon will be) buying more disks, I'll advise them to get
> dedicated disks for each RAID.
>   

Depending on the use, dedicated disk may not be better, unless the 
budget is large. I ran an application which had a heavily read database 
and a large collection of files thich were read based on offsets read 
from the database. I have a limited number of drives available 
(rackspace limit, not $). I partitioned the drives with a small 
partition for the heavily read database, using three copies raid1, and 
raid5 for the more lightly used data, across the same disks.

I tried almost every layout possible with six drives, and spreading the 
required head motion to all drives was a big win on the heavily read 
database, while spreading the storage over all drives was required 
because of capacity. And split and shared the performance was optimized. 
And it did stay up with a drive fail, although "up" means "didn't lose 
data" rather than "usefully fast."



-- 
Bill Davidsen <davidsen@tmr.com>
  "Woe unto the statesman who makes war without a reason that will still
  be valid when the war is over..." Otto von Bismark 

Re: Sharing disks amoung multiple software RAIDs

[Kasper Sandberg]

On Thu, 2008-05-01 at 08:50 -0400, Justin Piszcz wrote:
> 
> On Thu, 1 May 2008, Alex Davis wrote:
> 
> > Is this a bad thing? I'm guessing that it is, but I want independent
> > confirmation before I spoke to someone I know who's doing this.
> >
> > Thanks
> >
> >
> >      ____________________________________________________________________________________
> > Be a better friend, newshound, and
> > know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> > the body of a message to majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> >
> 
> What is the use case, why would you want to do that?
> I have seen people on the list do it before, for example are you going to 
> be utilizing both raids at the same time?  If so, I would advise against 
> it.
> 
> What is the reasoning?

I do this!

is this really bad? i would surely like a list of reasons why..

I do it because.. well.. first off, it allows me to have /boot on
different raidlevel than / or /home without extra disks.
secondly, it allows me to with the same disks use different
filesystems.. for instance, it allows me to have /home encrypted with
dm-crypt, while still raided.. Not that i would mind encrypting /
and /home as 1 partition, but it creates a whole slew of issues with
having to create initrd and stuff..

I realize that performance probably suffers abit from this, but well..
is there any stability or security wise risk? i mostly use raid1 and
raid5 only..

> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Re: Sharing disks amoung multiple software RAIDs

[Alex Davis]

--- On Thu, 5/1/08, Kasper Sandberg <lkml@metanurb.dk> wrote:

> From: Kasper Sandberg <lkml@metanurb.dk>
> Subject: Re: Sharing disks amoung multiple software RAIDs
> To: "Justin Piszcz" <jpiszcz@lucidpixels.com>
> Cc: "Alex Davis" <alex14641@yahoo.com>, linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org
> Date: Thursday, May 1, 2008, 9:39 PM
> On Thu, 2008-05-01 at 08:50 -0400, Justin Piszcz wrote:
> > 
> > On Thu, 1 May 2008, Alex Davis wrote:
> > 
> > > Is this a bad thing? I'm guessing that it is,
> but I want independent
> > > confirmation before I spoke to someone I know
> who's doing this.
> > >
> > > Thanks
> > >
> > >
> > >     
> ____________________________________________________________________________________
> > > Be a better friend, newshound, and
> > > know-it-all with Yahoo! Mobile.  Try it now. 
> http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
> > > --
> > > To unsubscribe from this list: send the line
> "unsubscribe linux-raid" in
> > > the body of a message to
> majordomo@vger.kernel.org
> > > More majordomo info at 
> http://vger.kernel.org/majordomo-info.html
> > >
> > 
> > What is the use case, why would you want to do that?
> > I have seen people on the list do it before, for
> example are you going to 
> > be utilizing both raids at the same time?  If so, I
> would advise against 
> > it.
> > 
> > What is the reasoning?
> 
> I do this!
> 
> is this really bad? i would surely like a list of reasons
> why..
> 
> I do it because.. well.. first off, it allows me to have
> /boot on
> different raidlevel than / or /home without extra disks.
> secondly, it allows me to with the same disks use different
> filesystems.. for instance, it allows me to have /home
> encrypted with
> dm-crypt, while still raided.. Not that i would mind
> encrypting /
> and /home as 1 partition, but it creates a whole slew of
> issues with
> having to create initrd and stuff..
> 
> I realize that performance probably suffers abit from this,
> but well..
> is there any stability or security wise risk? i mostly use
> raid1 and
> raid5 only..
> 
I would guess if the RAIDs are heavily used simultaneously it could
cause the disk head actuators to jump around more, causing more wear
and tear.


      ____________________________________________________________________________________
Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

RE: Sharing disks amoung multiple software RAIDs

[David Lethe]

In a perfect world, my expectation is that sharing a disk is safe and
even a reasonable thing to do. The shared disk is effectively a range of
addressable blocks in the same way an individual disk or even a hardware
RAID LUN is a range of blocks. From this limited perspective, the only
penalty is performance related, assuming you don't mind having to
explain yourself to an IT supervisor somewhere in your company.

However, consider what COULD happen in event of a drive failure, either
with the shared or an unshared disk. What are the odds all of the
failure scenarios have been tested for all ATA and SCSI command sets;
with threaded I/Os pending; with dirty cache; bus resets when hardware
dies; etc... ?

The odds are zero.  Look how many problems people post to the thread on
a weekly basis where people lose their data when md rebuilds go bad with
non-shared disks.  Why make the problem worse?

-----Original Message-----
From: linux-raid-owner@vger.kernel.org
[mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Alex Davis
Sent: Thursday, May 01, 2008 8:52 PM
To: Justin Piszcz; Kasper Sandberg
Cc: linux-raid@vger.kernel.org; linux-kernel@vger.kernel.org
Subject: Re: Sharing disks amoung multiple software RAIDs

--- On Thu, 5/1/08, Kasper Sandberg <lkml@metanurb.dk> wrote:

> From: Kasper Sandberg <lkml@metanurb.dk>
> Subject: Re: Sharing disks amoung multiple software RAIDs
> To: "Justin Piszcz" <jpiszcz@lucidpixels.com>
> Cc: "Alex Davis" <alex14641@yahoo.com>, linux-raid@vger.kernel.org,
linux-kernel@vger.kernel.org
> Date: Thursday, May 1, 2008, 9:39 PM
> On Thu, 2008-05-01 at 08:50 -0400, Justin Piszcz wrote:
> > 
> > On Thu, 1 May 2008, Alex Davis wrote:
> > 
> > > Is this a bad thing? I'm guessing that it is,
> but I want independent
> > > confirmation before I spoke to someone I know
> who's doing this.
> > >
> > > Thanks
> > >
> > >
> > >     
>
________________________________________________________________________
____________
> > > Be a better friend, newshound, and
> > > know-it-all with Yahoo! Mobile.  Try it now. 
> http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
> > > --
> > > To unsubscribe from this list: send the line
> "unsubscribe linux-raid" in
> > > the body of a message to
> majordomo@vger.kernel.org
> > > More majordomo info at 
> http://vger.kernel.org/majordomo-info.html
> > >
> > 
> > What is the use case, why would you want to do that?
> > I have seen people on the list do it before, for
> example are you going to 
> > be utilizing both raids at the same time?  If so, I
> would advise against 
> > it.
> > 
> > What is the reasoning?
> 
> I do this!
> 
> is this really bad? i would surely like a list of reasons
> why..
> 
> I do it because.. well.. first off, it allows me to have
> /boot on
> different raidlevel than / or /home without extra disks.
> secondly, it allows me to with the same disks use different
> filesystems.. for instance, it allows me to have /home
> encrypted with
> dm-crypt, while still raided.. Not that i would mind
> encrypting /
> and /home as 1 partition, but it creates a whole slew of
> issues with
> having to create initrd and stuff..
> 
> I realize that performance probably suffers abit from this,
> but well..
> is there any stability or security wise risk? i mostly use
> raid1 and
> raid5 only..
> 
I would guess if the RAIDs are heavily used simultaneously it could
cause the disk head actuators to jump around more, causing more wear
and tear.


 
________________________________________________________________________
____________
Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.
http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Sharing disks amoung multiple software RAIDs

[Kasper Sandberg]

On Thu, 2008-05-01 at 21:31 -0500, David Lethe wrote:
> In a perfect world, my expectation is that sharing a disk is safe and
> even a reasonable thing to do. The shared disk is effectively a range of
> addressable blocks in the same way an individual disk or even a hardware
> RAID LUN is a range of blocks. From this limited perspective, the only
> penalty is performance related, assuming you don't mind having to
> explain yourself to an IT supervisor somewhere in your company.
> 
> However, consider what COULD happen in event of a drive failure, either
> with the shared or an unshared disk. What are the odds all of the
> failure scenarios have been tested for all ATA and SCSI command sets;
> with threaded I/Os pending; with dirty cache; bus resets when hardware
> dies; etc... ?
> 
> The odds are zero.  Look how many problems people post to the thread on
> a weekly basis where people lose their data when md rebuilds go bad with
> non-shared disks.  Why make the problem worse?

What? are you saying linux md is unreliable?! in other words, would i be
safer to run rsync every day to my other disk, and run in non-raid
mode?!

> 
> -----Original Message-----
> From: linux-raid-owner@vger.kernel.org
> [mailto:linux-raid-owner@vger.kernel.org] On Behalf Of Alex Davis
> Sent: Thursday, May 01, 2008 8:52 PM
> To: Justin Piszcz; Kasper Sandberg
> Cc: linux-raid@vger.kernel.org; linux-kernel@vger.kernel.org
> Subject: Re: Sharing disks amoung multiple software RAIDs
> 
> --- On Thu, 5/1/08, Kasper Sandberg <lkml@metanurb.dk> wrote:
> 
> > From: Kasper Sandberg <lkml@metanurb.dk>
> > Subject: Re: Sharing disks amoung multiple software RAIDs
> > To: "Justin Piszcz" <jpiszcz@lucidpixels.com>
> > Cc: "Alex Davis" <alex14641@yahoo.com>, linux-raid@vger.kernel.org,
> linux-kernel@vger.kernel.org
> > Date: Thursday, May 1, 2008, 9:39 PM
> > On Thu, 2008-05-01 at 08:50 -0400, Justin Piszcz wrote:
> > > 
> > > On Thu, 1 May 2008, Alex Davis wrote:
> > > 
> > > > Is this a bad thing? I'm guessing that it is,
> > but I want independent
> > > > confirmation before I spoke to someone I know
> > who's doing this.
> > > >
> > > > Thanks
> > > >
> > > >
> > > >     
> >
> ________________________________________________________________________
> ____________
> > > > Be a better friend, newshound, and
> > > > know-it-all with Yahoo! Mobile.  Try it now. 
> > http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
> > > > --
> > > > To unsubscribe from this list: send the line
> > "unsubscribe linux-raid" in
> > > > the body of a message to
> > majordomo@vger.kernel.org
> > > > More majordomo info at 
> > http://vger.kernel.org/majordomo-info.html
> > > >
> > > 
> > > What is the use case, why would you want to do that?
> > > I have seen people on the list do it before, for
> > example are you going to 
> > > be utilizing both raids at the same time?  If so, I
> > would advise against 
> > > it.
> > > 
> > > What is the reasoning?
> > 
> > I do this!
> > 
> > is this really bad? i would surely like a list of reasons
> > why..
> > 
> > I do it because.. well.. first off, it allows me to have
> > /boot on
> > different raidlevel than / or /home without extra disks.
> > secondly, it allows me to with the same disks use different
> > filesystems.. for instance, it allows me to have /home
> > encrypted with
> > dm-crypt, while still raided.. Not that i would mind
> > encrypting /
> > and /home as 1 partition, but it creates a whole slew of
> > issues with
> > having to create initrd and stuff..
> > 
> > I realize that performance probably suffers abit from this,
> > but well..
> > is there any stability or security wise risk? i mostly use
> > raid1 and
> > raid5 only..
> > 
> I would guess if the RAIDs are heavily used simultaneously it could
> cause the disk head actuators to jump around more, causing more wear
> and tear.
> 
> 
>  
> ________________________________________________________________________
> ____________
> Be a better friend, newshound, and 
> know-it-all with Yahoo! Mobile.  Try it now.
> http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

Re: Sharing disks amoung multiple software RAIDs

[David Rees]

On Thu, May 1, 2008 at 7:42 PM, Kasper Sandberg <lkml@metanurb.dk> wrote:
>  in other words, would i be
>  safer to run rsync every day to my other disk, and run in non-raid
>  mode?!

What would be safer is to run rsync every day from one redundant array
to another array - preferably on another machine that is located as
far away as possible from the one that you are backing up.

RAID is not the same as a backup, though unfortunately, too many
people treat it as such.

-Dave

Re: Sharing disks amoung multiple software RAIDs

[Kasper Sandberg]

On Fri, 2008-05-02 at 00:06 -0700, David Rees wrote:
> On Thu, May 1, 2008 at 7:42 PM, Kasper Sandberg <lkml@metanurb.dk> wrote:
> >  in other words, would i be
> >  safer to run rsync every day to my other disk, and run in non-raid
> >  mode?!
> 
> What would be safer is to run rsync every day from one redundant array
> to another array - preferably on another machine that is located as
> far away as possible from the one that you are backing up.
> 
> RAID is not the same as a backup, though unfortunately, too many
> people treat it as such.
Im not treating it as a backup, what i want, is to make sure that if 1
disk dies, the data is still intact and ill hopefully be able to run
with 1 disk till the newly ordered one arrives - and while i naturally
will keep offsite backups of most important data, it is certainly far
easier to simply rip out a faulty disk, and put in another, instead of
restoring from backup from scratch..

So my question remains.. Is md raid1 not suited for this need? would it
be safer to run in non-raid1 mode and daily(maybe hourly) rsync
everything over to the second disk?

> 
> -Dave

Re: Sharing disks amoung multiple software RAIDs

[David Greaves]

Kasper Sandberg wrote:
> Im not treating it as a backup, what i want, is to make sure that if 1
> disk dies, the data is still intact and ill hopefully be able to run
> with 1 disk till the newly ordered one arrives
Probably one of the main design objectives behind RAID/md

> So my question remains.. Is md raid1 not suited for this need? would it
> be safer to run in non-raid1 mode and daily(maybe hourly) rsync
> everything over to the second disk?

md is 100% guaranteed perfect or your money back...
rsync is 100% guaranteed perfect or your money back...
your backups are 100% guaranteed perfect or your money back...
your hard drives are 100% guaranteed perfect or your money back...
your CPU and RAM are 100% guaranteed perfect or your money back...
your CPU and PSU fans are 100% guaranteed perfect or your money back...

Clearly if you want to panic over reliability you have lots of choices :)

David
PS, FWIW md has saved my data* countless times over the past 'n' years in
exactly the scenario you describe.

*(or more accurately has saved me from having to restore my data)

Re: Sharing disks amoung multiple software RAIDs

[Kasper Sandberg]

On Fri, 2008-05-02 at 09:25 +0100, David Greaves wrote:
> Kasper Sandberg wrote:
> > Im not treating it as a backup, what i want, is to make sure that if 1
> > disk dies, the data is still intact and ill hopefully be able to run
> > with 1 disk till the newly ordered one arrives
> Probably one of the main design objectives behind RAID/md

Exactly, but once people start saying: "Look how many problems people
post to the thread on
a weekly basis where people lose their data when md rebuilds go bad with
non-shared disks" i begin to worry..

> 
> > So my question remains.. Is md raid1 not suited for this need? would it
> > be safer to run in non-raid1 mode and daily(maybe hourly) rsync
> > everything over to the second disk?
> 
> md is 100% guaranteed perfect or your money back...
> rsync is 100% guaranteed perfect or your money back...
> your backups are 100% guaranteed perfect or your money back...
> your hard drives are 100% guaranteed perfect or your money back...
> your CPU and RAM are 100% guaranteed perfect or your money back...
> your CPU and PSU fans are 100% guaranteed perfect or your money back...
> 
> Clearly if you want to panic over reliability you have lots of choices :)

I do not wish to panic, i merely wished to know if linux MD is believed
to work in most cases, or believed to do all sorts of weird stuff when
resyncing :)

> 
> David
> PS, FWIW md has saved my data* countless times over the past 'n' years in
> exactly the scenario you describe.

It has also been useful to people i know, i just wished to be sure :)
and as Keld Jørn Simonsen and Helge Hafting's comments seems to confirm,
linux md IS nice and stable :)

and as said, what im looking for isnt an in-box backup solution, merely
safety in case one disk burns :)

> 
> *(or more accurately has saved me from having to restore my data)

--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Sharing disks amoung multiple software RAIDs

[Alex Davis]

--- On Fri, 5/2/08, Kasper Sandberg <lkml@metanurb.dk> wrote:

> From: Kasper Sandberg <lkml@metanurb.dk>
> Subject: Re: Sharing disks amoung multiple software RAIDs
> To: "David Greaves" <david@dgreaves.com>
> Cc: "David Rees" <drees76@gmail.com>, "David Lethe" <david@santools.com>, alex14641@yahoo.com, "Justin Piszcz" <jpiszcz@lucidpixels.com>, linux-raid@vger.kernel.org, linux-kernel@vger.kernel.org
> Date: Friday, May 2, 2008, 5:43 PM
> On Fri, 2008-05-02 at 09:25 +0100, David Greaves wrote:
> > Kasper Sandberg wrote:
> > > Im not treating it as a backup, what i want, is
> to make sure that if 1
> > > disk dies, the data is still intact and ill
> hopefully be able to run
> > > with 1 disk till the newly ordered one arrives
> > Probably one of the main design objectives behind
> RAID/md
> 
> Exactly, but once people start saying: "Look how many
> problems people
> post to the thread on
> a weekly basis where people lose their data when md
> rebuilds go bad with
> non-shared disks" i begin to worry..
> 
> > 
> > > So my question remains.. Is md raid1 not suited
> for this need? would it
> > > be safer to run in non-raid1 mode and daily(maybe
> hourly) rsync
> > > everything over to the second disk?
> > 
> > md is 100% guaranteed perfect or your money back...
> > rsync is 100% guaranteed perfect or your money back...
> > your backups are 100% guaranteed perfect or your money
> back...
> > your hard drives are 100% guaranteed perfect or your
> money back...
> > your CPU and RAM are 100% guaranteed perfect or your
> money back...
> > your CPU and PSU fans are 100% guaranteed perfect or
> your money back...
> > 
> > Clearly if you want to panic over reliability you have
> lots of choices :)
> 
> I do not wish to panic, i merely wished to know if linux MD
> is believed
> to work in most cases, or believed to do all sorts of weird
> stuff when
> resyncing :)
> 
> > 
> > David
> > PS, FWIW md has saved my data* countless times over
> the past 'n' years in
> > exactly the scenario you describe.
> 
> It has also been useful to people i know, i just wished to
> be sure :)
> and as Keld Jørn Simonsen and Helge Hafting's comments
> seems to confirm,
> linux md IS nice and stable :)
> 
> and as said, what im looking for isnt an in-box backup
> solution, merely
> safety in case one disk burns :)
> 
> > 
> > *(or more accurately has saved me from having to
> restore my data)

Just to add another data point, I've been using md in RAID 5 configuration for ~3 years with dedicated USB and SATA disks 
(not mixed) and have had disks go bad, and have yet to lose 
any data. Given the 'quality' of high-capacity disks nowadays,
RAIDing them is the right thing to do.


      ____________________________________________________________________________________
Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

RE: Sharing disks amoung multiple software RAIDs

[David Lethe]

-----Original Message-----
From: Kasper Sandberg [mailto:lkml@metanurb.dk] 
Sent: Friday, May 02, 2008 4:44 PM
To: David Greaves
Cc: David Rees; David Lethe; alex14641@yahoo.com; Justin Piszcz; linux-raid@vger.kernel.org; linux-kernel@vger.kernel.org
Subject: Re: Sharing disks amoung multiple software RAIDs

On Fri, 2008-05-02 at 09:25 +0100, David Greaves wrote:
> Kasper Sandberg wrote:
> > Im not treating it as a backup, what i want, is to make sure that if 1
> > disk dies, the data is still intact and ill hopefully be able to run
> > with 1 disk till the newly ordered one arrives
> Probably one of the main design objectives behind RAID/md

Exactly, but once people start saying: "Look how many problems people
post to the thread on
a weekly basis where people lose their data when md rebuilds go bad with
non-shared disks" i begin to worry..

> 
> > So my question remains.. Is md raid1 not suited for this need? would it
> > be safer to run in non-raid1 mode and daily(maybe hourly) rsync
> > everything over to the second disk?
> 
> md is 100% guaranteed perfect or your money back...
> rsync is 100% guaranteed perfect or your money back...
> your backups are 100% guaranteed perfect or your money back...
> your hard drives are 100% guaranteed perfect or your money back...
> your CPU and RAM are 100% guaranteed perfect or your money back...
> your CPU and PSU fans are 100% guaranteed perfect or your money back...
> 
> Clearly if you want to panic over reliability you have lots of choices :)

I do not wish to panic, i merely wished to know if linux MD is believed
to work in most cases, or believed to do all sorts of weird stuff when
resyncing :)

> 
> David
> PS, FWIW md has saved my data* countless times over the past 'n' years in
> exactly the scenario you describe.

It has also been useful to people i know, i just wished to be sure :)
and as Keld Jørn Simonsen and Helge Hafting's comments seems to confirm,
linux md IS nice and stable :)

and as said, what im looking for isnt an in-box backup solution, merely
safety in case one disk burns :)

> 
> *(or more accurately has saved me from having to restore my data)
=======================
Since I am the person that wrote that "Look how many problems people
post to the thread on a weekly basis where people lose their data when md rebuilds go bad with
non-shared disks", I think need to clarify.
 
I was being too literal. md is a good solution (I use it myself).  However, the OP wanted to "Make sure that if one disk dies, the data is still in tact".  

Only way to do that is have a good offsite backup.

Nothing can prevent opportunity for data loss.  I don't care what RAID level (with single redundancy) you use or whether you have software or hardware-based RAID .. rebuilds can not reconstruct data if you are in degraded mode and have an unrecoverable read error on surviving disks.  It is not md's job or design point to insure data consistency check/repairs are run 24x7.  As such, by definition, there is no way to insure that a md rebuild will always be successful.

David
---------





--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Sharing disks amoung multiple software RAIDs

[Alex Davis]

I code, therefore I am


--- On Fri, 5/2/08, David Lethe <david@santools.com> wrote:
[snipped]
> I was being too literal. md is a good solution (I use it
> myself).  However, the OP wanted to "Make sure that if
> one disk dies, the data is still in tact".  
I was also concerned about performance. It seems that if the RAIDs will 
be used heavily and independent of each other, and performance is a 
concern, then dedicated disks (and possibly dedicated controllers)
are the better option.


      ____________________________________________________________________________________
Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

Re: Sharing disks amoung multiple software RAIDs

[Keld Jørn Simonsen]

On Fri, May 02, 2008 at 05:44:18PM -0700, Alex Davis wrote:
> 
> 
> I code, therefore I am
> 
> 
> --- On Fri, 5/2/08, David Lethe <david@santools.com> wrote:
> [snipped]
> > I was being too literal. md is a good solution (I use it
> > myself).  However, the OP wanted to "Make sure that if
> > one disk dies, the data is still in tact".  
> I was also concerned about performance. It seems that if the RAIDs will 
> be used heavily and independent of each other, and performance is a 
> concern, then dedicated disks (and possibly dedicated controllers)
> are the better option.

My understanding of having multiple, possibly differnet types of RAID
partitions say on two drives, are that performance-wise this are the
same considerations as for a single drive. You can worry about head
movement and seek times and transfer rates, but the problems are the
same for raid and non-raid. IMHO architecturally multiple RAIDs on same disks
do not introduce additional performance issues compared with single disk
solutions, nor with one-raid-per-set-of-drives solutions.

The above statement on having separate disks per raid is more or less the
same as saying that the more disks you have, the better.

If performance is an issue, then I am recommending my pet
raid10,f2 layout, which has a 30 - 80 % increased performance over
traditional RAID1, because it uses only the outer faster tracks on the
disk, and it has reduced seek time only searching on half of the disk
(and this the again the outer part, so that the head movement is reduced
to about 1/3). 

Best regards
keld

RE: Sharing disks amoung multiple software RAIDs

[Kasper Sandberg]

On Fri, 2008-05-02 at 17:24 -0500, David Lethe wrote:
> 
> -----Original Message-----
<snip>
> > *(or more accurately has saved me from having to restore my data)
> =======================
> Since I am the person that wrote that "Look how many problems people
> post to the thread on a weekly basis where people lose their data when md rebuilds go bad with
> non-shared disks", I think need to clarify.
>  
> I was being too literal. md is a good solution (I use it myself).  However, the OP wanted to "Make sure that if one disk dies, the data is still in tact".  
> 
> Only way to do that is have a good offsite backup.
> 
> Nothing can prevent opportunity for data loss.  I don't care what RAID level (with single redundancy) you use or whether you have software or hardware-based RAID .. rebuilds can not reconstruct data if you are in degraded mode and have an unrecoverable read error on surviving disks.  It is not md's job or design point to insure data consistency check/repairs are run 24x7.  As such, by definition, there is no way to insure that a md rebuild will always be successful.
I realize this, but the original wording really sounded like more like
"well, there is a CHANCE your reconstruct will work good", rather than
"well, there is a CHANCE your reconstruct will fail", if you get what i
mean :)

Naturally there can always be errors or bugs, but the whole point (for
me) in doing the raid1, is that i will have much lower chance of loosing
my data, since a disk can suddenly die, and hopefully in this case, the
other will not. and well, an extra disk doesent really cost so much.

Thanks for the clarification :)
> 
> David
> ---------
> 
> 
> 
> 
> 

Re: Sharing disks amoung multiple software RAIDs

[Keld Jørn Simonsen]

On Fri, May 02, 2008 at 10:09:23AM +0200, Kasper Sandberg wrote:
> On Fri, 2008-05-02 at 00:06 -0700, David Rees wrote:
> > On Thu, May 1, 2008 at 7:42 PM, Kasper Sandberg <lkml@metanurb.dk> wrote:
> > >  in other words, would i be
> > >  safer to run rsync every day to my other disk, and run in non-raid
> > >  mode?!
> > 
> > What would be safer is to run rsync every day from one redundant array
> > to another array - preferably on another machine that is located as
> > far away as possible from the one that you are backing up.
> > 
> > RAID is not the same as a backup, though unfortunately, too many
> > people treat it as such.
> Im not treating it as a backup, what i want, is to make sure that if 1
> disk dies, the data is still intact and ill hopefully be able to run
> with 1 disk till the newly ordered one arrives - and while i naturally
> will keep offsite backups of most important data, it is certainly far
> easier to simply rip out a faulty disk, and put in another, instead of
> restoring from backup from scratch..
> 
> So my question remains.. Is md raid1 not suited for this need? would it
> be safer to run in non-raid1 mode and daily(maybe hourly) rsync
> everything over to the second disk?

I join in on this question. I have written a howto on our linux raid
page, that exactly advocates a way to with 2 drives use these 2 drives
for multiple partitions, of different types of RAID. The partitions
include /boot / /home and swap - which are actually all allocated 
as RAID partitions on the same 2 drives. See
http://linux-raid.osdl.org/index.php/Preventing_against_a_failing_disk

I am trying to advocate that this be a recommended setup for normal
workstations, to for the price of one extra harddisk get additional
safety and performance.

My take is that Linux RAID code is so stable, including in the degarded
mode, that this is robust and stable. However, there are still bugs to
be found, so you can never be 100 % sure. 

Best regards
keld

Re: Sharing disks amoung multiple software RAIDs

[Keld Jørn Simonsen]

On Sun, May 04, 2008 at 04:06:44PM -0600, Maurice Hilarius wrote:
> Keld Jørn Simonsen wrote:
> >..
> >Hmm, would raid10,n2, or raid10,f2 then have these flaws too?
> >  
> I do not know.
> I am, however, about to do some testing to find out.
> Our company routinely was using RAID1 on systems, until we discovered this.
> With no ability to boot from RAID5 and similar this leaves us in a bit 
> of a bind.

As raid10,n2 is data wise equivalent to RAID1, then grub/lilo should be
able to boot from raid10,n2. 

raid10,n2 should have better performance than RAID10, but raid10,f2
actually should have better performance than both of the others. 
See http://linux-raid.osdl.org/index.php/Performance (which I wrote).
That is why I advocated raid10,f2 for the system partitions, as far as
possible, only having /boot as a RAID1. 
I have a setup which implements this and it runs fine.

I look forward to hear about your tests on this.

Best regards
keld
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Sharing disks amoung multiple software RAIDs

[Keld Jørn Simonsen]

On Mon, May 05, 2008 at 01:10:16AM +0200, Keld Jørn Simonsen wrote:
> On Sun, May 04, 2008 at 04:06:44PM -0600, Maurice Hilarius wrote:
> > Keld Jørn Simonsen wrote:
> > >..
> > >Hmm, would raid10,n2, or raid10,f2 then have these flaws too?
> > >  
> > I do not know.
> > I am, however, about to do some testing to find out.
> > Our company routinely was using RAID1 on systems, until we discovered this.
> > With no ability to boot from RAID5 and similar this leaves us in a bit 
> > of a bind.
> 
> As raid10,n2 is data wise equivalent to RAID1, then grub/lilo should be
> able to boot from raid10,n2. 
> 
> raid10,n2 should have better performance than RAID10, but raid10,f2

I meant: raid10,n2 should have better performance than RAID1

best regards
keld
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Sharing disks amoung multiple software RAIDs

[Helge Hafting]

Kasper Sandberg wrote:
> On Fri, 2008-05-02 at 00:06 -0700, David Rees wrote:
>   
>> On Thu, May 1, 2008 at 7:42 PM, Kasper Sandberg <lkml@metanurb.dk> wrote:
>>     
>>>  in other words, would i be
>>>  safer to run rsync every day to my other disk, and run in non-raid
>>>  mode?!
>>>       
>> What would be safer is to run rsync every day from one redundant array
>> to another array - preferably on another machine that is located as
>> far away as possible from the one that you are backing up.
>>
>> RAID is not the same as a backup, though unfortunately, too many
>> people treat it as such.
>>     
> Im not treating it as a backup, what i want, is to make sure that if 1
> disk dies, the data is still intact and ill hopefully be able to run
> with 1 disk till the newly ordered one arrives - and while i naturally
> will keep offsite backups of most important data, it is certainly far
> easier to simply rip out a faulty disk, and put in another, instead of
> restoring from backup from scratch..
>
> So my question remains.. Is md raid1 not suited for this need? would it
> be safer to run in non-raid1 mode and daily(maybe hourly) rsync
> everything over to the second disk?
>   
Raid protects against disk failure. So your use  is fine.

Backup also protects against user error and virus/worm/cracker damage.
If you accidentally overwrite a file, or a virus/worm/cracker messes up 
the filesystem,
then RAID loose because the overwiting/messing happens on both disks.
In that case you'll still be able to get stuff from a backup.


Helge Hafting

Re: Sharing disks amoung multiple software RAIDs

[Alex Davis]

> > > Can you explain what you mean, exactly?
> > As an example, you have disks /dev/sd[abc]. /dev/md0 would be made 
> > from /dev/sda1, /dev/sdb1, and /dev/sdc1; /dev/md1 would be made
> > from /dev/sda2, /dev/sdb2, and /dev/sdc2.

> I've been watching this thread with interest, because I wanted
> clarification on what the OP meant.

> If by "sharing disks", the OP meant the above, I'm surprised (and
>  concerned) there's any question at all.  How is this anything other    > than the definition of "mirror"? 
md0 and md1 are both RAID 5 arrays, not mirrors.



      ____________________________________________________________________________________
Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

Re: Sharing disks amoung multiple software RAIDs

[David Greaves]

Alex Davis wrote:
> Is this a bad thing? I'm guessing that it is, but I want independent
> confirmation before I spoke to someone I know who's doing this.
> 
> Thanks
Depends what you're trying to do.

It has the potential to increase i/o contention so it's not going to be a
performance win. Many systems really don't need to worry about performance. But
if you have 2x250 Gb drives and 1x500 Gb drive then your data is at less risk if
you partition the 500Gb into 2x250Gb and then mirror each partition with a full
250Gb disk. That would be a perfectly rational setup in many instances.

Equally, md has a mechanism to share hot spares amongst multiple RAIDs - but I
doubt that's what you meant.

David

Re: Sharing disks amoung multiple software RAIDs

[Nick Andrew]

On Thu, May 01, 2008 at 04:35:07AM -0700, Alex Davis wrote:
> Is this a bad thing? I'm guessing that it is, but I want independent
> confirmation before I spoke to someone I know who's doing this.

I use it to ensure that multiple partitions are RAIDed. I have a /boot
partition which is ext3 and a large LVM partition. Both of them are
RAID-1 onto a second drive (md0 and md1) for redundancy. The second
drive has an identical partition table to the first drive.

Nick.
-- 
PGP Key ID = 0x418487E7                      http://www.nick-andrew.net/
PGP Key fingerprint = B3ED 6894 8E49 1770 C24A  67E3 6266 6EB9 4184 87E7

Re: Sharing disks amoung multiple software RAIDs

[George Spelvin]

Can you explain what you mean, exactly?

Do you just mean having two partitions on the same drive used as part
of other RAID arrays?  That works fine.

Having two heavilty used partitions on the same drive causes some
performance issues, but no correctness ones.  And there's nothing
special about RAID for this consideration; it would apply with
non-RAID partitions as well.

But I have mirrored swap striped across all drives; I don't
use swap a lot and it's not worth getting dedicated drives.

Likewise, /boot is a 6-way RAID-1 emergency rescue partition.
I can boot off any drive, and I have a basic text-mode install
with all the disaster recovery tools.  Again, not heavily used.

If you're doing serious database work, it's common to split the
system, log, and database across different spindles.  But that's
independent of whether RAID is used for any of them.


But there are other possible interpretations of "sharing among multiple
RAIDs", like hot spares and the like.  Could you be more specific?
Obviously, having the same partition active in multiple different
arrays would be an unmitigated disaster, but I don't think you mean that.
(And I don't think mdadm lets you do it, either.)

One thing that's very nice about Linux software RAID is that you *don't*
have to RAID whole drives.  It took me a while to understand Intel's
"Matrix RAID" feature because it had never occurred to me that a RAID
array *couldn't* be set up that way.

Re: Sharing disks amoung multiple software RAIDs

[Alex Davis]

--- On Fri, 5/2/08, George Spelvin <linux@horizon.com> wrote:

> From: George Spelvin <linux@horizon.com>
> Subject: Re: Sharing disks amoung multiple software RAIDs
> To: alex14641@yahoo.com
> Cc: linux@horizon.com, linux-raid@vger.kernel.org
> Date: Friday, May 2, 2008, 4:36 AM
> Can you explain what you mean, exactly?
As an example, you have disks /dev/sd[abc]. /dev/md0 would be made 
from /dev/sda1, /dev/sdb1, and /dev/sdc1; /dev/md1 would be made  from /dev/sda2, /dev/sdb2, and /dev/sdc2.

> Do you just mean having two partitions on the same drive
> used as part
> of other RAID arrays?  That works fine.
> Having two heavilty used partitions on the same drive
> causes some
> performance issues, 
In this case they are heavily used.

>but no correctness ones.  And
> there's nothing
> special about RAID for this consideration; it would apply
> with
> non-RAID partitions as well.
> 
> But I have mirrored swap striped across all drives; I
> don't
> use swap a lot and it's not worth getting dedicated
> drives.
> 
> Likewise, /boot is a 6-way RAID-1 emergency rescue
> partition.
> I can boot off any drive, and I have a basic text-mode
> install
> with all the disaster recovery tools.  Again, not heavily
> used.
> 
> If you're doing serious database work, it's common
> to split the
> system, log, and database across different spindles.  But
> that's
> independent of whether RAID is used for any of them.
> 
> 
> But there are other possible interpretations of
> "sharing among multiple
> RAIDs", like hot spares and the like.  Could you be
> more specific?
> Obviously, having the same partition active in multiple
> different
> arrays would be an unmitigated disaster, but I don't
> think you mean that.
> (And I don't think mdadm lets you do it, either.)
> 
> One thing that's very nice about Linux software RAID is
> that you *don't*
> have to RAID whole drives.  It took me a while to
> understand Intel's
> "Matrix RAID" feature because it had never
> occurred to me that a RAID
> array *couldn't* be set up that way.



      ____________________________________________________________________________________
Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

Re: Sharing disks amoung multiple software RAIDs

[Richard Michael]

> > Can you explain what you mean, exactly?
> As an example, you have disks /dev/sd[abc]. /dev/md0 would be made 
> from /dev/sda1, /dev/sdb1, and /dev/sdc1; /dev/md1 would be made  from /dev/sda2, /dev/sdb2, and /dev/sdc2.

I've been watching this thread with interest, because I wanted
clarification on what the OP meant.

If by "sharing disks", the OP meant the above, I'm surprised (and
concerned) there's any question at all.  How is this anything other than
the definition of "mirror"?  What would a mirror be, if not this;
perhaps the entire disk in one partition (which is still
partition-based, albeit one), or an entire raw disk?

I've been doing this for years (with hardware RAID, and with other
operating system software RAID implementations as well, e.g. on
Solaris).  (I even just split the mirror on my own system and ran off
half, then wiped the second disk for emergency scratch space for a few
hours, then put the second disk back in and resynced.  No problems.)

In fact, I currently have a system with a three-way mirror.  The third
disk is removed for offsite storage and returned to the pool on a
schedule for resync (to implement a crude offsite policy where other
means of data transfer are not effective, and the entire system can be
recovered with a slight data loss window).

I'm now wondering, what is OP's concern?  Just performance?

Regards,
Richard