LINEAR RAID, little help

LINEAR RAID, little help

[Rich]

Hello,

I currently have a linear RAID setup via mdadm with is made up of 3 
drives. I just have a few questions that I can't seem to find searching 
around on Google, etc.

First question, what happens if one drive fails (I know I will loose the 
data on that drive) but how, if at all can I recover the data on the 
other drives can I plug them in (on their own) as though they were in 
fact an individual drives? Do I need to execute a rebuild command in 
mdadm at all to rebuild the array?

Second question, how can I go about adding a drive to my linear RAID, I 
wish to add two new 500GB drives but I'm unsure how. I have found 
howto's for RAID 5 but I just wanted to check it was a similar process 
for Linear? Also this won't effect any data currently on the drive will it?


Thanks,

Richard

Re: LINEAR RAID, little help

[Gavin McCullagh]

Hi,

The main reason I'm posting (given others can answer these questions
better) is to ask a further question: 

Why would anyone use RAID-linear?  If RAID-0 gives better performance for
the same (reduced) reliability, what's the point of using Linear?  Do you
get slightly more space out of it?  I presume it's for stuff where you
don't place much value no your data.

On Sat, 07 Apr 2007, Rich wrote:

> I currently have a linear RAID setup via mdadm with is made up of 3 
> drives. I just have a few questions that I can't seem to find searching 
> around on Google, etc.

I must admit I've never used linear raid.  May I ask what made you choose
it over say raid-0?

> First question, what happens if one drive fails (I know I will loose the 
> data on that drive) but how, if at all can I recover the data on the 
> other drives can I plug them in (on their own) as though they were in 
> fact an individual drives? Do I need to execute a rebuild command in 
> mdadm at all to rebuild the array?

http://tldp.org/LDP/EVMSUG/html/characraidlvls.html

I'm sure someone else will give you a better answer but it would appear
from these docs that your situation is similar to that of a person using
RAID-0 who has a drive failure.  Basically, you end up with a big portion
of your filesystem data missing so you may lose everything.  I guess you
might be able to recover some data using some filesystem recovery tools but
it's likely to be a pretty unreliable process and likely to vary from one
filesystem to the next.  It's probably a little more reliable than RAID-0
in that what's left is probably contiguous but I would certainly not want
to rely on such a process for my data.

The faq says similar "RAID-linear is a simple concatenation of partitions
to create a larger virtual partition. It is handy if you have a number
small drives, and wish to create a single, large partition. This
concatenation offers no redundancy, and in fact decreases the overall
reliability: if any one disk fails, the combined partition will fail."

http://tldp.org/HOWTO/Software-RAID-0.4x-HOWTO-2.html

> Second question, how can I go about adding a drive to my linear RAID, I
> wish to add two new 500GB drives but I'm unsure how. I have found howto's
> for RAID 5 but I just wanted to check it was a similar process for
> Linear? Also this won't effect any data currently on the drive will it?

You don't say what you're doing on this array, but before modifying it, I'd
be seriously inclined to question whether RAID-linear is really the right
thing to be using at all.  Anyway, the mdadm manpage says 

"Currently the only support available is to
 · change the "size" attribute for RAID1, RAID5 and RAID6.
 · increase the "raid-disks" attribute of RAID1 and RAID5.
 · add a write-intent bitmap to any array which support these bitmaps, or
   remove  a  write-intent bitmap from such an array."

which suggests you can't.  That might be wrong though as it sounds (to me
anyway) like linear would be one of the easier ones to implement grow for.
I'm not sure.

Gavin

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: LINEAR RAID, little help

[Rich]

Gavin McCullagh wrote:
> Hi,
>
> The main reason I'm posting (given others can answer these questions
> better) is to ask a further question: 
>
> Why would anyone use RAID-linear?  If RAID-0 gives better performance for
> the same (reduced) reliability, what's the point of using Linear?  Do you
> get slightly more space out of it?  I presume it's for stuff where you
> don't place much value no your data.
>
> On Sat, 07 Apr 2007, Rich wrote:
>
>   
>> I currently have a linear RAID setup via mdadm with is made up of 3 
>> drives. I just have a few questions that I can't seem to find searching 
>> around on Google, etc.
>>     
>
> I must admit I've never used linear raid.  May I ask what made you choose
> it over say raid-0?
>   
Er, I went with Linear as reading around people seemed to recommend this 
for odd sized drives (my old drives are 80's, 120 and 320's) also a read 
somewhere that data on the other drives is more recoverable that most of 
the other RAID's.
>   
>> First question, what happens if one drive fails (I know I will loose the 
>> data on that drive) but how, if at all can I recover the data on the 
>> other drives can I plug them in (on their own) as though they were in 
>> fact an individual drives? Do I need to execute a rebuild command in 
>> mdadm at all to rebuild the array?
>>     
>
> http://tldp.org/LDP/EVMSUG/html/characraidlvls.html
>
> I'm sure someone else will give you a better answer but it would appear
> from these docs that your situation is similar to that of a person using
> RAID-0 who has a drive failure.  Basically, you end up with a big portion
> of your filesystem data missing so you may lose everything.  I guess you
> might be able to recover some data using some filesystem recovery tools but
> it's likely to be a pretty unreliable process and likely to vary from one
> filesystem to the next.  It's probably a little more reliable than RAID-0
> in that what's left is probably contiguous but I would certainly not want
> to rely on such a process for my data.
>
> The faq says similar "RAID-linear is a simple concatenation of partitions
> to create a larger virtual partition. It is handy if you have a number
> small drives, and wish to create a single, large partition. This
> concatenation offers no redundancy, and in fact decreases the overall
> reliability: if any one disk fails, the combined partition will fail."
>
> http://tldp.org/HOWTO/Software-RAID-0.4x-HOWTO-2.html
>   
>> Second question, how can I go about adding a drive to my linear RAID, I
>> wish to add two new 500GB drives but I'm unsure how. I have found howto's
>> for RAID 5 but I just wanted to check it was a similar process for
>> Linear? Also this won't effect any data currently on the drive will it?
>>     
>
> You don't say what you're doing on this array, but before modifying it, I'd
> be seriously inclined to question whether RAID-linear is really the right
> thing to be using at all.  Anyway, the mdadm manpage says 
>   
Well it contains stuff like TV and films a lot of which my friends have, 
so loosing it is not that big of a deal just inconvenient. As I said 
before I read linear was a better choice for odd sized drives and had 
better data recovery that the other RAID's
> "Currently the only support available is to
>  · change the "size" attribute for RAID1, RAID5 and RAID6.
>  · increase the "raid-disks" attribute of RAID1 and RAID5.
>  · add a write-intent bitmap to any array which support these bitmaps, or
>    remove  a  write-intent bitmap from such an array."
>
> which suggests you can't.  That might be wrong though as it sounds (to me
> anyway) like linear would be one of the easier ones to implement grow for.
> I'm not sure.
>
> Gavin
>
>   

I'm sure I read somewhere on the mdadm page in one of the posts about 
adding support for linear growing, and that was dated 2004 so it might 
be in there? Just wasn't to sure how to do it...


Thanks,

Rich :)
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: LINEAR RAID, little help

[Gavin McCullagh]

Hi,

On Sat, 07 Apr 2007, Rich wrote:

> Er, I went with Linear as reading around people seemed to recommend this 
> for odd sized drives (my old drives are 80's, 120 and 320's) also a read 
> somewhere that data on the other drives is more recoverable that most of 
> the other RAID's.

You just want to make a large filesystem out of odd sized disks.  I guess
that's fair enough.  The only reservation I'd have is that with N disks
your likelihood of failure is multiplied by N as any disk failure takes the
array down.  Personally, I'd be more inclined to try and put a separate
filesystem on each disk and use symlinks to pull them together into one
tree.

But now I know why linear raid can be more useful than raid-0, thanks.

> Well it contains stuff like TV and films a lot of which my friends have, 
> so loosing it is not that big of a deal just inconvenient. As I said 
> before I read linear was a better choice for odd sized drives and had 
> better data recovery that the other RAID's

I guess you're using it similarly to LVM then. 

http://en.wikipedia.org/wiki/Logical_Volume_Manager_(Linux)
http://tldp.org/HOWTO/LVM-HOWTO/

It sounds reasonable that recovery would be more feasible than RAID-0 but
neither would be nice to have to do.  RAID1, 5, etc. would be less likely
to require recovery as they have redundancy so a single disk failure
shouldn't take down the array at all.

> I'm sure I read somewhere on the mdadm page in one of the posts about 
> adding support for linear growing, and that was dated 2004 so it might 
> be in there? Just wasn't to sure how to do it...

This looks like your page.

http://cgi.cse.unsw.edu.au/~neilb/SoftRaid

and the mdadm changelog has it:

Changes Prior to 1.7.0 release
   -   Support "--grow --add" to add a device to a linear array, if the
       kernel supports it.  Not documented yet.

I've only grown raid5 arrays myself.  I imagine you must do --add to add
the extra disk partition to the array and then do "mdadm --grow
--size=max", same as with raid5.  

Gavin

Re: LINEAR RAID, little help

[Henrik Holst]

On Sat, 2007-04-07 at 21:34 +0100, Gavin McCullagh wrote:
> Hi,
> 
> On Sat, 07 Apr 2007, Rich wrote:
> 
> > Er, I went with Linear as reading around people seemed to recommend this 
> > for odd sized drives (my old drives are 80's, 120 and 320's) also a read 
> > somewhere that data on the other drives is more recoverable that most of 
> > the other RAID's.
> 
> You just want to make a large filesystem out of odd sized disks.  I guess
> that's fair enough.  The only reservation I'd have is that with N disks
> your likelihood of failure is multiplied by N as any disk failure takes the
> array down.  Personally, I'd be more inclined to try and put a separate
> filesystem on each disk and use symlinks to pull them together into one
> tree.
> 
> But now I know why linear raid can be more useful than raid-0, thanks.
> 

(This is maybe sort of off topic since this list about raid and not
filesystems and virtual such either.)

Maybe you just want the appearance of things being as one volume? I
might recommend you looking into unionfs:

http://www.filesystems.org/project-unionfs.html

It's actually quite useful. You might have some logical division of the
data. Put TV stuff on one disk, Movies on another. The unionfs overlay
maps everything together into a virtual bigdisk. It's also very simple
and no superblocks or headers needed either.

-- 
Henrik Holst <holst@matmech.com>

Re: LINEAR RAID, little help

[Neil Brown]

On Saturday April 7, rich@pcfusion.co.uk wrote:
> Gavin McCullagh wrote:
> >
> > I must admit I've never used linear raid.  May I ask what made you choose
> > it over say raid-0?
> >   
> Er, I went with Linear as reading around people seemed to recommend this 
> for odd sized drives (my old drives are 80's, 120 and 320's) also a read 
> somewhere that data on the other drives is more recoverable that most of 
> the other RAID's.

Linux raid0 works with varying sized drives without problem.

> >   
> >> First question, what happens if one drive fails (I know I will loose the 
> >> data on that drive) but how, if at all can I recover the data on the 
> >> other drives can I plug them in (on their own) as though they were in 
> >> fact an individual drives? Do I need to execute a rebuild command in 
> >> mdadm at all to rebuild the array?

If you lose one drive, you should consider that you have lost the
whole array.  You might be able to recover some data, if you are lucky
and spend a lot of time hunting for it.  But it is quite unlikely that
you will get much that is useful.

> >   
> >> Second question, how can I go about adding a drive to my linear RAID, I
> >> wish to add two new 500GB drives but I'm unsure how. I have found howto's
> >> for RAID 5 but I just wanted to check it was a similar process for
> >> Linear? Also this won't effect any data currently on the drive will it?
> >>     
> >
> > You don't say what you're doing on this array, but before modifying it, I'd
> > be seriously inclined to question whether RAID-linear is really the right
> > thing to be using at all.  Anyway, the mdadm manpage says 
> >   
> > "Currently the only support available is to
> >  · change the "size" attribute for RAID1, RAID5 and RAID6.
> >  · increase the "raid-disks" attribute of RAID1 and RAID5.
> >  · add a write-intent bitmap to any array which support these bitmaps, or
> >    remove  a  write-intent bitmap from such an array."
> >
> > which suggests you can't.  That might be wrong though as it sounds (to me
> > anyway) like linear would be one of the easier ones to implement grow for.
> > I'm not sure.

The man page is out of date.  With reasonably recent kernel/mdadm you
can
  mdadm --grow /dev/mdX --add /dev/sdY

To add a drive to a linear array.  You then need to grow the
filesystem of course.

But yes:  A big linear array is good for scratch space, but I wouldn't
want to store data that I couldn't afford to lose.

NeilBrown
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: LINEAR RAID, little help

[Rich]

I forgot to ask something, sorry. With RAID0 and all the odd size drives 
I have I'd get a lot of left over unused space. Is there anyway to make 
use of this slack?

Thanks,

Rich

Rich wrote:
> Neil Brown wrote:
>> (This is maybe sort of off topic since this list about raid and not
>> filesystems and virtual such either.)
>>
>> Maybe you just want the appearance of things being as one volume? I
>> might recommend you looking into unionfs:
>>
>> http://www.filesystems.org/project-unionfs.html
>>
>> It's actually quite useful. You might have some logical division of the
>> data. Put TV stuff on one disk, Movies on another. The unionfs overlay
>> maps everything together into a virtual bigdisk. It's also very simple
>> and no superblocks or headers needed either.
> Thanks I shall take a look into it :)
>>
>> Linux raid0 works with varying sized drives without problem.   
> Ah, kewl.. I might do, depends how much drive space I loose. Speed 
> wise I don't find linear to bad I can read and write over gigabit 
> network at around 30mb/sec.
>>
>> If you lose one drive, you should consider that you have lost the
>> whole array.  You might be able to recover some data, if you are lucky
>> and spend a lot of time hunting for it.  But it is quite unlikely that
>> you will get much that is useful.
>>
>>   
> Ah right, well for the time being A lot of my friends have the same 
> data so loosing it is just inconvenient rather than a major no-go.
>>
>> The man page is out of date.  With reasonably recent kernel/mdadm you
>> can
>>   mdadm --grow /dev/mdX --add /dev/sdY
>>
>> To add a drive to a linear array.  You then need to grow the
>> filesystem of course.
>>
>> But yes:  A big linear array is good for scratch space, but I wouldn't
>> want to store data that I couldn't afford to lose.
>>
>> NeilBrown
>>   
> Cheers Neil, Once I can find enough hdd space else where to backup I 
> might play around with RAID0 etc I do plan to replace my odd size 
> drives with 500GB ones and RAID5 but I can't afford that at the moment.
>
>
> Thanks,
>
> Rich
>
>
>
>
>

Re: LINEAR RAID, little help

[Neil Brown]

On Thursday April 12, rich@pcfusion.co.uk wrote:
> I forgot to ask something, sorry. With RAID0 and all the odd size drives 
> I have I'd get a lot of left over unused space. Is there anyway to make 
> use of this slack?

md/raid0 makes use of all available space (modulo chunk size).
To quote from "man md"

   RAID0
       A  RAID0  array  (which has zero redundancy) is also known as a striped
       array.  A RAID0 array is configured at creation with a Chunk Size which
       must be a power of two, and at least 4 kibibytes.

       The  RAID0  driver  assigns  the  first chunk of the array to the first
       device, the second chunk to the second device,  and  so  on  until  all
       drives have been assigned one chunk.  This collection of chunks forms a
       stripe.  Further chunks are gathered into stripes in the same way which
       are assigned to the remaining space in the drives.

       If devices in the array are not all the same size, then once the small‐
       est device has been  exhausted,  the  RAID0  driver  starts  collecting
       chunks  into smaller stripes that only span the drives which still have
       remaining space.


(If you think that isn't suitable clear, you are probably correct.
Improvements always welcome).

NeilBrown

Re: LINEAR RAID, little help

[matt s.]

Neil Brown wrote:
> 
> md/raid0 makes use of all available space (modulo chunk size).
> To quote from "man md"
> 
> 
>        If devices in the array are not all the same size, then once the small‐
>        est device has been  exhausted,  the  RAID0  driver  starts  collecting
>        chunks  into smaller stripes that only span the drives which still have
>        remaining space.
> 

And at the point where the space is exhausted on all drives but one, I 
assume it will keep writing to that last one, non-striped?

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: LINEAR RAID, little help

[Bill Davidsen]

Neil Brown wrote:
> On Thursday April 12, rich@pcfusion.co.uk wrote:
>   
>> I forgot to ask something, sorry. With RAID0 and all the odd size drives 
>> I have I'd get a lot of left over unused space. Is there anyway to make 
>> use of this slack?
>>     
>
> md/raid0 makes use of all available space (modulo chunk size).
> To quote from "man md"
>
>    RAID0
>        A  RAID0  array  (which has zero redundancy) is also known as a striped
>        array.  A RAID0 array is configured at creation with a Chunk Size which
>        must be a power of two, and at least 4 kibibytes.
>
>        The  RAID0  driver  assigns  the  first chunk of the array to the first
>        device, the second chunk to the second device,  and  so  on  until  all
>        drives have been assigned one chunk.  This collection of chunks forms a
>        stripe.  Further chunks are gathered into stripes in the same way which
>        are assigned to the remaining space in the drives.
>
>        If devices in the array are not all the same size, then once the small‐
>        est device has been  exhausted,  the  RAID0  driver  starts  collecting
>        chunks  into smaller stripes that only span the drives which still have
>        remaining space.

That has to result in some bizarre transfer rate changes! I saved this
explanation in my RAID stuff folder, clearly you could use techniques
from raster graphics to make transfer rate more uniform, but since the
most used will also be the fastest currently, that's probably not an
optimization.

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979