Fwd: Raid 5 --grow to fewer, larger drives

Fwd: Raid 5 --grow to fewer, larger drives

[Jeff Rippy]

I too am looking at this.  My current setup is 4x 500GB drives and I
want to move to 3x  >=1TB drives.
My usable space should increase from 1.5TB to >= 2TB with this move if possible.

I'm fairly new to linux-raid in general but I know you can replace
drives with bigger ones.  You just can't use their space until all
drives in the array are of that size.  So I'm thinking of replacing 3
of the 500's with the 1 TBs.  Then fail and remove the 4th drive
(assuming enough free space) and somehow forcing a reshape on the
remaining 3 drives.  Not sure if that is possible but I can't imagine
why it wouldn't be.  After the resync, finally grow to the new size
and be done.

Can someone tell me if this is even possible?  I saw the one reply in
Nov. to Alex's email but I don't have another machine to swap the
disks too.

Re: Fwd: Raid 5 --grow to fewer, larger drives

[Robin Hill]

[-- Attachment #1: Type: text/plain, Size: 1354 bytes --]

On Sat Jan 03, 2009 at 05:27:49PM -0600, Jeff Rippy wrote:

> I too am looking at this.  My current setup is 4x 500GB drives and I
> want to move to 3x  >=1TB drives.
> My usable space should increase from 1.5TB to >= 2TB with this move if possible.
> 
> I'm fairly new to linux-raid in general but I know you can replace
> drives with bigger ones.  You just can't use their space until all
> drives in the array are of that size.  So I'm thinking of replacing 3
> of the 500's with the 1 TBs.  Then fail and remove the 4th drive
> (assuming enough free space) and somehow forcing a reshape on the
> remaining 3 drives.  Not sure if that is possible but I can't imagine
> why it wouldn't be.  After the resync, finally grow to the new size
> and be done.
> 
> Can someone tell me if this is even possible?  I saw the one reply in
> Nov. to Alex's email but I don't have another machine to swap the
> disks too.
> 
This is currently not possible (though it is on the roadmap for future
work).  For now, the only solution is to set up the new disks as a
separate array and copy the data.

Cheers,
    Robin
-- 
     ___        
    ( ' }     |       Robin Hill        <robin@robinhill.me.uk> |
   / / )      | Little Jim says ....                            |
  // !!       |      "He fallen in de water !!"                 |

[-- Attachment #2: Type: application/pgp-signature, Size: 197 bytes --]

Re: Raid 5 --grow to fewer, larger drives

[Jon Nelson]

If you are using LVM on top of the raid (the raid is the only PV -
physical volume - backing the volume group) then it's much easier.

Create a *new* array of your 3x 1TB drives (let's call it md99)
Identify the raid is a new physical volume:

pvcreate /dev/md99

Add the array to the volume group:

vgextend YourVolumeGroup /dev/md99

Now, disable allocation on the old array:

pvchange --allocatable n /dev/oldMD

Use pvmove to move all extents from the old pv to the new one:

pvmove /dev/oldMD

When that is done, you can vgreduce the volume group which removes the
old array, and then you can do whatever you want with the old array.

This is how I have upgraded arrays for the past few years or more.


On Sat, Jan 3, 2009 at 5:27 PM, Jeff Rippy <jrippy@gmail.com> wrote:
> I too am looking at this.  My current setup is 4x 500GB drives and I
> want to move to 3x  >=1TB drives.
> My usable space should increase from 1.5TB to >= 2TB with this move if possible.
>
> I'm fairly new to linux-raid in general but I know you can replace
> drives with bigger ones.  You just can't use their space until all
> drives in the array are of that size.  So I'm thinking of replacing 3
> of the 500's with the 1 TBs.  Then fail and remove the 4th drive
> (assuming enough free space) and somehow forcing a reshape on the
> remaining 3 drives.  Not sure if that is possible but I can't imagine
> why it wouldn't be.  After the resync, finally grow to the new size
> and be done.
>
> Can someone tell me if this is even possible?  I saw the one reply in
> Nov. to Alex's email but I don't have another machine to swap the
> disks too.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>



-- 
Jon

Re: Fwd: Raid 5 --grow to fewer, larger drives

[John Robinson]

On 03/01/2009 23:27, Jeff Rippy wrote:
> I too am looking at this.  My current setup is 4x 500GB drives and I
> want to move to 3x  >=1TB drives.
> My usable space should increase from 1.5TB to >= 2TB with this move if possible.
> 
> I'm fairly new to linux-raid in general but I know you can replace
> drives with bigger ones.  You just can't use their space until all
> drives in the array are of that size.  So I'm thinking of replacing 3
> of the 500's with the 1 TBs.  Then fail and remove the 4th drive
> (assuming enough free space) and somehow forcing a reshape on the
> remaining 3 drives.  Not sure if that is possible but I can't imagine
> why it wouldn't be.  After the resync, finally grow to the new size
> and be done.
> 
> Can someone tell me if this is even possible?  I saw the one reply in
> Nov. to Alex's email but I don't have another machine to swap the
> disks too.

No, it'll only work if you move to 4x1TB, but even then it would be 
risky; you wouldn't have any redundancy during 3 rebuilds and a reshape, 
and if one drive goes bad you'll lose your data.

If you can physically fit the 3 new drives, just build the new array and 
copy over instead. Jon Nelson's suggestion sounds to me like a good one 
to make the copying easy if you're using LVM.

If you can physically fit 2 more drives, you can either:
a) build the new array with a drive missing, copy the data, then pull 
the old array and resync with the third new drive, or
b) go read-only, yank one of the old drives, build the new array and 
copy over

If you can physically fit 1 more drive, you can do both of the above 
simultaneously, i.e. go read-only, pull one of the old drives, build the 
new array with a disc missing, copy the data, then resync with the third 
new disc.

If you can't fit any more drives, err, open the case and hang the drives 
outside the case?

Actually there is a funky thing you can do if you can't fit any more 
drives, as long as your data will fit on just one of the new drives: go 
read-only, pull one of the old drives, install one of the new ones, 
create a 2-disc RAID-5 with a disc missing (! - yes, this works), copy 
the data, pull the rest of the old array, add the second new disc and 
resync the new array onto it, add the third new disc and reshape the 
array onto it.

None of the above methods where I say "go read-only" will work with the 
LVM copy method, but your old array will remain intact so you won't lose 
data, as long as you're being careful about what you're doing.

If you're going to do any of this b*ggering about, you'd be well advised 
to make a backup first, save a copy of your mdadm.conf and various 
outputs from mdadm --examine, perhaps even on paper(!), and try to keep 
track of what drives came from where, in case it all goes horribly wrong.

Cheers,

John.

Re: Fwd: Raid 5 --grow to fewer, larger drives

[Jeff Rippy]

Thanks for the suggestions.  I am running LVM on top of the raid.  I
can't fit any more drives in the case but I do have room to hang them
out, the problem though is SATA ports.  I'm out.  The cards are cheap
so maybe I'll pick one up for this move.  Any suggestions on brands to
stay away from or is pretty everything supported in the kernel by now?

On Sat, Jan 3, 2009 at 7:34 PM, John Robinson
<john.robinson@anonymous.org.uk> wrote:
> On 03/01/2009 23:27, Jeff Rippy wrote:
>>
>> I too am looking at this.  My current setup is 4x 500GB drives and I
>> want to move to 3x  >=1TB drives.
>> My usable space should increase from 1.5TB to >= 2TB with this move if
>> possible.
>>
>> I'm fairly new to linux-raid in general but I know you can replace
>> drives with bigger ones.  You just can't use their space until all
>> drives in the array are of that size.  So I'm thinking of replacing 3
>> of the 500's with the 1 TBs.  Then fail and remove the 4th drive
>> (assuming enough free space) and somehow forcing a reshape on the
>> remaining 3 drives.  Not sure if that is possible but I can't imagine
>> why it wouldn't be.  After the resync, finally grow to the new size
>> and be done.
>>
>> Can someone tell me if this is even possible?  I saw the one reply in
>> Nov. to Alex's email but I don't have another machine to swap the
>> disks too.
>
> No, it'll only work if you move to 4x1TB, but even then it would be risky;
> you wouldn't have any redundancy during 3 rebuilds and a reshape, and if one
> drive goes bad you'll lose your data.
>
> If you can physically fit the 3 new drives, just build the new array and
> copy over instead. Jon Nelson's suggestion sounds to me like a good one to
> make the copying easy if you're using LVM.
>
> If you can physically fit 2 more drives, you can either:
> a) build the new array with a drive missing, copy the data, then pull the
> old array and resync with the third new drive, or
> b) go read-only, yank one of the old drives, build the new array and copy
> over
>
> If you can physically fit 1 more drive, you can do both of the above
> simultaneously, i.e. go read-only, pull one of the old drives, build the new
> array with a disc missing, copy the data, then resync with the third new
> disc.
>
> If you can't fit any more drives, err, open the case and hang the drives
> outside the case?
>
> Actually there is a funky thing you can do if you can't fit any more drives,
> as long as your data will fit on just one of the new drives: go read-only,
> pull one of the old drives, install one of the new ones, create a 2-disc
> RAID-5 with a disc missing (! - yes, this works), copy the data, pull the
> rest of the old array, add the second new disc and resync the new array onto
> it, add the third new disc and reshape the array onto it.
>
> None of the above methods where I say "go read-only" will work with the LVM
> copy method, but your old array will remain intact so you won't lose data,
> as long as you're being careful about what you're doing.
>
> If you're going to do any of this b*ggering about, you'd be well advised to
> make a backup first, save a copy of your mdadm.conf and various outputs from
> mdadm --examine, perhaps even on paper(!), and try to keep track of what
> drives came from where, in case it all goes horribly wrong.
>
> Cheers,
>
> John.
>

Re: Fwd: Raid 5 --grow to fewer, larger drives

[Justin Piszcz]

On Sat, 3 Jan 2009, Jeff Rippy wrote:

> Thanks for the suggestions.  I am running LVM on top of the raid.  I
> can't fit any more drives in the case but I do have room to hang them
> out, the problem though is SATA ports.  I'm out.  The cards are cheap
> so maybe I'll pick one up for this move.  Any suggestions on brands to
> stay away from or is pretty everything supported in the kernel by now?

PCI or PCI-e?

Justin.