looking for advice on raid drives upgrade

looking for advice on raid drives upgrade

[Vincent Schut]

Hi all,

one of my raid's is currently a raid5 with 4 1TB drives, which I'm going 
to upgrade by replacing all drives with 2TB drives.
Main question: what is recommended:

- offline the raid, dd(rescue) each 1TB drive onto a new 2TB one, 
replace drives, put raid online again

or:
- manually replace one drive at a time, and let the raid recover and 
thus write the data onto the new drive?

As far as I know all current drives are healthy; regularly SMART 
monitoring and raid checks have not ringed any bells sofar.

I guess offlining and dd-ing would be more secure, but I might miss 
something and would like to hear other considerations from people more 
experienced in this area.

Regards,
Vincent Schut.

Re: looking for advice on raid drives upgrade

[Mikael Abrahamsson]

On Wed, 30 Mar 2011, Vincent Schut wrote:

> - offline the raid, dd(rescue) each 1TB drive onto a new 2TB one, 
> replace drives, put raid online again

This works if you have a superblock version that stores its superblock in 
the beginning, not the end (or if you use a partition). This is the 
safest.

> or:
> - manually replace one drive at a time, and let the raid recover and thus 
> write the data onto the new drive?

If you have a bad block anywhere on the degraded drives, you're going to 
lose data.

-- 
Mikael Abrahamsson    email: swmike@swm.pp.se

Re: looking for advice on raid drives upgrade

[Vincent Schut]

On 03/30/2011 01:10 PM, Mikael Abrahamsson wrote:
> On Wed, 30 Mar 2011, Vincent Schut wrote:
>
>> - offline the raid, dd(rescue) each 1TB drive onto a new 2TB one,
>> replace drives, put raid online again
>
> This works if you have a superblock version that stores its superblock
> in the beginning, not the end (or if you use a partition). This is the
> safest.

Thanks. That is something I would not have thought about myself.

according to mdadm --detail, my superblock version is 1.1:

  mdadm --detail /dev/md0
/dev/md0:
         Version : 1.1
   Creation Time : Fri Apr  2 11:59:08 2010
      Raid Level : raid5
      Array Size : 2930287104 (2794.54 GiB 3000.61 GB)
   Used Dev Size : 976762368 (931.51 GiB 1000.20 GB)
    Raid Devices : 4
   Total Devices : 4
     Persistence : Superblock is persistent

     Update Time : Wed Mar 30 13:17:09 2011
           State : clean
  Active Devices : 4
Working Devices : 4
  Failed Devices : 0
   Spare Devices : 0

          Layout : left-symmetric
      Chunk Size : 512K

            Name : fransje:0  (local to host fransje)
            UUID : 0b2e8a3c:a84a6f4e:31ed60f8:d33de960
          Events : 319880

     Number   Major   Minor   RaidDevice State
        0       8       32        0      active sync   /dev/sdc
        1       8       16        1      active sync   /dev/sdb
        3       8        0        2      active sync   /dev/sda
        4       8       48        3      active sync   /dev/sdd

Version 1.1 stores the superblock at the beginning of the device, right? 
So that would allow me to use the dd method?

>
>> or:
>> - manually replace one drive at a time, and let the raid recover and
>> thus write the data onto the new drive?
>
> If you have a bad block anywhere on the degraded drives, you're going to
> lose data.
>
Exactly. A risk I'd like to avoid, of course.

Thanks for the info!
Vincent.

Re: looking for advice on raid drives upgrade

[David Brown]

On 30/03/2011 11:46, Vincent Schut wrote:
> Hi all,
>
> one of my raid's is currently a raid5 with 4 1TB drives, which I'm going
> to upgrade by replacing all drives with 2TB drives.
> Main question: what is recommended:
>
> - offline the raid, dd(rescue) each 1TB drive onto a new 2TB one,
> replace drives, put raid online again
>
> or:
> - manually replace one drive at a time, and let the raid recover and
> thus write the data onto the new drive?
>
> As far as I know all current drives are healthy; regularly SMART
> monitoring and raid checks have not ringed any bells sofar.
>
> I guess offlining and dd-ing would be more secure, but I might miss
> something and would like to hear other considerations from people more
> experienced in this area.
>
> Regards,
> Vincent Schut.
>


If you've got a spare drive slot, and plenty of time, there is another 
option that lets you keep online and redundant during the process. 
Re-shape the array to raid 6, while adding one of the new drives.  Then 
you can replace each drive, one at a time, letting the raid recover in 
between.  For the last drive, you just have to remove it then reshape 
back to raid 5 (although ideally you could buy an extra drive and keep 
the raid 6 layout - it's worth the cost, assuming you have the drive slot).

Once everything is swapped over, you grow the array to fit the new disks.

Re: looking for advice on raid drives upgrade

[Steven Haigh]

On 31/03/2011 12:14 AM, David Brown wrote:
> If you've got a spare drive slot, and plenty of time, there is another
> option that lets you keep online and redundant during the process.
> Re-shape the array to raid 6, while adding one of the new drives. Then
> you can replace each drive, one at a time, letting the raid recover in
> between. For the last drive, you just have to remove it then reshape
> back to raid 5 (although ideally you could buy an extra drive and keep
> the raid 6 layout - it's worth the cost, assuming you have the drive slot).
>
> Once everything is swapped over, you grow the array to fit the new disks.

This makes me wonder... What is recommended as the best practice... Use 
the entire disk or 1 partition per device? Assuming of course that you 
won't be booting off the RAID5/6....

-- 
Steven Haigh

Email: netwiz@crc.id.au
Web: http://www.crc.id.au
Phone: (03) 9001 6090 - 0412 935 897
Fax: (03) 8338 0299

Re: looking for advice on raid drives upgrade

[David Brown]

On 30/03/2011 15:31, Steven Haigh wrote:
> On 31/03/2011 12:14 AM, David Brown wrote:
>> If you've got a spare drive slot, and plenty of time, there is another
>> option that lets you keep online and redundant during the process.
>> Re-shape the array to raid 6, while adding one of the new drives. Then
>> you can replace each drive, one at a time, letting the raid recover in
>> between. For the last drive, you just have to remove it then reshape
>> back to raid 5 (although ideally you could buy an extra drive and keep
>> the raid 6 layout - it's worth the cost, assuming you have the drive
>> slot).
>>
>> Once everything is swapped over, you grow the array to fit the new disks.
>
> This makes me wonder... What is recommended as the best practice... Use
> the entire disk or 1 partition per device? Assuming of course that you
> won't be booting off the RAID5/6....
>

I think the answer depends on what you are trying to do.  If you are 
just building a single big array, then whole devices make most sense. 
But sometimes you want to try something more complicated (see my recent 
thread - though no one has yet said if it is a good idea...).  You may 
also want more than one array with different characteristics, such as 
the balance between capacity, speed (for different uses) and redundancy. 
  For example, you might think raid 5 is appropriate for some of your 
data, and raid10 for other parts.  By dividing each disk into two and 
using partitions from one half for one array, and the other half for the 
other array, you get both areas.  Clearly you'll get a performance hit 
if both arrays are used at the same time, but it's your choice.

Re: looking for advice on raid drives upgrade

[hansbkk]

On Wed, Mar 30, 2011 at 6:22 PM, Vincent Schut <schut@sarvision.nl> wrote:
>> If you have a bad block anywhere on the degraded drives, you're going to
>> lose data.
>>
> Exactly. A risk I'd like to avoid, of course.

IMO you should definitely go to RAID6 with such large drives, IMO with
a hot spare as well. The recovery process takes *so* long, it's very
likely that a second drive will fail during that vulnerable window and
poof. Use the money you're saving on per-GB costs these days to buy
yourself a bit more redundancy, otherwise what's the point of using
RAID at all eh?

Of course if the data is getting frequently archived to well-tested
backups, or is otherwise easily replace or of little value (e.g. video
collection) then the extra cost might not be worth it to you. Just
don't underestimate the geometric increase in risk by going to bigger
and bigger size drives.

Re: looking for advice on raid drives upgrade

[Vincent Schut]

On 03/30/2011 03:14 PM, David Brown wrote:
> On 30/03/2011 11:46, Vincent Schut wrote:
>> Hi all,
>>
>> one of my raid's is currently a raid5 with 4 1TB drives, which I'm going
>> to upgrade by replacing all drives with 2TB drives.
>> Main question: what is recommended:
>>
>> - offline the raid, dd(rescue) each 1TB drive onto a new 2TB one,
>> replace drives, put raid online again
>>
>> or:
>> - manually replace one drive at a time, and let the raid recover and
>> thus write the data onto the new drive?
>>
>> As far as I know all current drives are healthy; regularly SMART
>> monitoring and raid checks have not ringed any bells sofar.
>>
>> I guess offlining and dd-ing would be more secure, but I might miss
>> something and would like to hear other considerations from people more
>> experienced in this area.
>>
>> Regards,
>> Vincent Schut.
>>
>
>
> If you've got a spare drive slot, and plenty of time, there is another
> option that lets you keep online and redundant during the process.
> Re-shape the array to raid 6, while adding one of the new drives. Then
> you can replace each drive, one at a time, letting the raid recover in
> between. For the last drive, you just have to remove it then reshape
> back to raid 5 (although ideally you could buy an extra drive and keep
> the raid 6 layout - it's worth the cost, assuming you have the drive slot).
>
> Once everything is swapped over, you grow the array to fit the new disks.

Good thinking. I've asked for budget for one other drive and think I'll 
go the raid6 way.

Thanks for all the advice, everyone.

Vincent.

>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-raid" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
>