Docs and operation of RAID10, size limitations on 0.90

Docs and operation of RAID10, size limitations on 0.90

[Richard Michael]

Hello list,

I'm building a new system with 8 disks, in two RAID5 arrays of 4 disks
each, and a RAID1 array across them.  (Then LVM2 on top of the RAID1
array.)

I approached it this way for two reasons:

1/ It feels "cleanest"; in the sense that it's not a special case, it's
just RAID1 with two devices, which happen to be RAID5.

2/ I have a particular usage in mind: I need to be able to split the
mirror and remove half; return it later and resync.  Therefore, I want
to know which disks comprise which halves.

However, I'm rethinking toward RAID10.  However, I can't find much
documentation about layout, etc.  I believe this was recently discussed
on the list.


Can someone with RAID10 experience enlighten me?  Can it be cleaned
split, half moved to another host, or rejoined with the first half, etc.
How about performance?  (Not a huge issue, but it is better than nested
arrays as I first considered?)


Aside, I'd like it to be bootable.  I was told grub only supports
booting from 0.90 superblocks, but that 0.90 has a 2TB limitation?
Meaning, I need /boot with 0.90 and another partition (LVM) with a 1.0
superblocks?



Thanks,
Richard

Re: Docs and operation of RAID10, size limitations on 0.90

[Bill Davidsen]

Richard Michael wrote:
> Hello list,
>
> I'm building a new system with 8 disks, in two RAID5 arrays of 4 disks
> each, and a RAID1 array across them.  (Then LVM2 on top of the RAID1
> array.)
>
> I approached it this way for two reasons:
>
> 1/ It feels "cleanest"; in the sense that it's not a special case, it's
> just RAID1 with two devices, which happen to be RAID5.
>
> 2/ I have a particular usage in mind: I need to be able to split the
> mirror and remove half; return it later and resync.  Therefore, I want
> to know which disks comprise which halves.
>
> However, I'm rethinking toward RAID10.  However, I can't find much
> documentation about layout, etc.  I believe this was recently discussed
> on the list.
>
>
> Can someone with RAID10 experience enlighten me?  Can it be cleaned
> split, half moved to another host, or rejoined with the first half, etc.
> How about performance?  (Not a huge issue, but it is better than nested
> arrays as I first considered?)
>   

10,f2 read performance is very good, write performance is okay. AFAIK 
you can't do a split, the object is to spread head motion and improve 
performance.
>
> Aside, I'd like it to be bootable.  I was told grub only supports
> booting from 0.90 superblocks, but that 0.90 has a 2TB limitation?
> Meaning, I need /boot with 0.90 and another partition (LVM) with a 1.0
> superblocks?
>   

I usually create a 100MB or so partition, raid1, on my drives and boot 
off that using 0.90 superblock. The rest of the storage can be anything 
you want.

-- 
Bill Davidsen <davidsen@tmr.com>
  "Woe unto the statesman who makes war without a reason that will still
  be valid when the war is over..." Otto von Bismark 

Re: Docs and operation of RAID10, size limitations on 0.90

[Richard Michael]

> 10,f2 read performance is very good, write performance is okay. AFAIK you 
> can't do a split, the object is to spread head motion and improve 
> performance.

Thanks Bill.  "Split-a-bility" is a requirement unfortunately, so maybe
RAID10 is not viable.  

> I usually create a 100MB or so partition, raid1, on my drives and boot off 
> that using 0.90 superblock. The rest of the storage can be anything you 
> want.

Ok; as do I.  I guess I'll do so again to make sure everything works.

Regards,
Richard

Re: Docs and operation of RAID10, size limitations on 0.90

[Keld Jørn Simonsen]

On Wed, Jul 30, 2008 at 12:18:06PM -0400, Bill Davidsen wrote:
> Richard Michael wrote:
> >Hello list,
> >
> >I'm building a new system with 8 disks, in two RAID5 arrays of 4 disks
> >each, and a RAID1 array across them.  (Then LVM2 on top of the RAID1
> >array.)
> >
> >I approached it this way for two reasons:
> >
> >1/ It feels "cleanest"; in the sense that it's not a special case, it's
> >just RAID1 with two devices, which happen to be RAID5.

To me it seems muddy. Just having one big raid5 or raid10 would be cleaner and
give better performance.

> >2/ I have a particular usage in mind: I need to be able to split the
> >mirror and remove half; return it later and resync.  Therefore, I want
> >to know which disks comprise which halves.


I am not sure what you mean with this. You want to remove half of the
disks? I think you can do that with raid10, you just remove every other
disk. I have not tried that out, however. You could try it out before
actually emplouing it.

> >However, I'm rethinking toward RAID10.  However, I can't find much
> >documentation about layout, etc.  I believe this was recently discussed
> >on the list.

There is documentation on http://linux-raid.osdl.org/
and on http://std.dkuug.dk/keld/stripemirror.html (for raid10,f2), and 
on http://en.wikipedia.org/wiki/Non-standard_RAID_levels


Best regards
keld

Re: Docs and operation of RAID10, size limitations on 0.90

[Richard Michael]

On Wed, Jul 30, 2008 at 08:29:07PM +0200, Keld Jørn Simonsen wrote:
> On Wed, Jul 30, 2008 at 12:18:06PM -0400, Bill Davidsen wrote:
> > Richard Michael wrote:
> > >Hello list,
> > >
> > >I'm building a new system with 8 disks, in two RAID5 arrays of 4 disks
> > >each, and a RAID1 array across them.  (Then LVM2 on top of the RAID1
> > >array.)
> > >
> > >I approached it this way for two reasons:
> > >
> > >1/ It feels "cleanest"; in the sense that it's not a special case, it's
> > >just RAID1 with two devices, which happen to be RAID5.
> 
> To me it seems muddy. Just having one big raid5 or raid10 would be cleaner and
> give better performance.

I can't have one big RAID5 because I want to split it in half
regularly.  I refer to it as "cleaner", because it treats the devices
separately, which is what they are; one is RAID1 device, the other is a
RAID5 device.  Each is agnostic, and independent, with regard to the
other.  "Muddy" in my opinion is a mdadm hybrid RAID10, of which I need
special knowledge.  YMMV.
 
> > >2/ I have a particular usage in mind: I need to be able to split
> > >the mirror and remove half; return it later and resync.  Therefore,
> > >I want to know which disks comprise which halves.
> 
> 
> I am not sure what you mean with this. You want to remove half of the
> disks? I think you can do that with raid10, you just remove every
> other disk. I have not tried that out, however. You could try it out
> before actually emplouing it.

Re-reading what I wrote, it's a mess of typos. :)  But, yes, I want to
remove half the disks on a regular basis, as off-site storage, and return
them to the array for periodic resync.

Regards,
Richard
--
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Docs and operation of RAID10, size limitations on 0.90

[Jon Nelson]

On Wed, Jul 30, 2008 at 4:49 PM, Richard Michael
<rmichael-raid@edgeofthenet.org> wrote:
> Re-reading what I wrote, it's a mess of typos. :)  But, yes, I want to
> remove half the disks on a regular basis, as off-site storage, and return
> them to the array for periodic resync.

Basically what you want is a raid 51.
2x RAID5 (4 disks each) mirrored to form a RAID1.
Using bitmaps in the RAID1 would let you remove one of the RAID5, do
with it what you will, and add it back in again and only the changed
parts will get syncd.
At least in theory.

Look also into using drbd.

-- 
Jon

Re: Docs and operation of RAID10, size limitations on 0.90

[Richard Michael]

On Wed, Jul 30, 2008 at 04:56:38PM -0500, Jon Nelson wrote:
> On Wed, Jul 30, 2008 at 4:49 PM, Richard Michael
> <rmichael-raid@edgeofthenet.org> wrote:
> > Re-reading what I wrote, it's a mess of typos. :)  But, yes, I want to
> > remove half the disks on a regular basis, as off-site storage, and return
> > them to the array for periodic resync.
> 
> Basically what you want is a raid 51.
> 2x RAID5 (4 disks each) mirrored to form a RAID1.

Exactly.  Have you built such a config and managed to get the system
booting from the nested RAID1 (perhaps even with LVM inside)?  This is
where I am having difficulty.  Else, I'll just create an 8-way RAID1 of
100MB or so for /boot (as usual, as I mentioned elsewhere).

> Look also into using drbd.

Will do; thanks.

Richard

Re: Docs and operation of RAID10, size limitations on 0.90

[Jon Nelson]

On Wed, Jul 30, 2008 at 5:47 PM, Richard Michael
<rmichael-raid@edgeofthenet.org> wrote:
> On Wed, Jul 30, 2008 at 04:56:38PM -0500, Jon Nelson wrote:
>> On Wed, Jul 30, 2008 at 4:49 PM, Richard Michael
>> <rmichael-raid@edgeofthenet.org> wrote:
>> > Re-reading what I wrote, it's a mess of typos. :)  But, yes, I want to
>> > remove half the disks on a regular basis, as off-site storage, and return
>> > them to the array for periodic resync.
>>
>> Basically what you want is a raid 51.
>> 2x RAID5 (4 disks each) mirrored to form a RAID1.
>
> Exactly.  Have you built such a config and managed to get the system
> booting from the nested RAID1 (perhaps even with LVM inside)?  This is
> where I am having difficulty.  Else, I'll just create an 8-way RAID1 of
> 100MB or so for /boot (as usual, as I mentioned elsewhere).

Built? Yes. Managed to get it to boot? No. Not bothered.
Just partition the drives with a bit to spare (a few gig) and use
that. No great loss and it saves a great deal of hassle.
I'd use 4 drives for storage in raid5 and the other 4 also in raid5 for swap.


-- 
Jon