Swapping out for larger disks

Swapping out for larger disks

[Brad Campbell]

G'day all,

I've got 3 arrays here. A 3 drive raid-5, a 10 drive raid-5 and a 15 drive raid-6. They are all 
currently 250GB SATA drives.

I'm contemplating an upgrade to 500GB drives on one or more of the arrays and wondering the best way 
to do the physical swap.

The slow and steady way would be to degrade the array, remove a disk, add the new disk, lather, 
rinse, repeat. After which I could use mdadm --grow. There is the concern of a degraded array here 
though (and one of the reasons I'm looking to swap is some of the disks have about 30,000 hours on 
the clock and are growing the odd defect).

I was more wondering about the feasibility of using dd to copy the drive contents to the larger 
drives (then I could do 5 at a time) and working it from there.

It occurs though that the superblocks would be in the wrong place for the new drives and I'm 
wondering if the kernel or mdadm might not find them.

Ideas? Suggestions ?

Brad
-- 
"Human beings, who are almost unique in having the ability
to learn from the experience of others, are also remarkable
for their apparent disinclination to do so." -- Douglas Adams

Re: Swapping out for larger disks

[Michael Tokarev]

Brad Campbell wrote:
[]
> It occurs though that the superblocks would be in the wrong place for
> the new drives and I'm wondering if the kernel or mdadm might not find
> them.

I once had a similar issue.  And wrote a tiny program (a hack, sort of),
to read or write md superblock from/to a component device.  The only
thing it really does is to calculate the superblock location - exactly
as it is done in mdadm.  Here it is:

  http://www.corpit.ru/mjt/mdsuper.c

Usage is like:

 mdsuper read /dev/old-device | mdsuper write /dev/new-device

(or using an intermediate file).

So you're doing like this:

 shutdown array
 for i in all-devices-in-array
   dd if=old-device[i] of=new-device[i] iflag=direct oflag=direct
   mdsuper read old-device | mdsuper write new-device
 done
 assemble-array-on-new-devices
 mdadm -G --size=max /dev/mdx

or something like that.

Note that the program does not work for anything but 0.90
superblocks (i haven't used 1.0 superblocks yet - 0.90 works
for me just fine).  However, it should be trivial to extend
it to handle v1 superblocks too.

Note also that it's trivial to do something like that in shell
too, with blockdev --getsz to get the device size, some shell-
style $((math)), and dd magic.

And 3rd note: using direct as above speeds up the copying *alot*,
while keeping system load at zero.  Without direct, one pair of
disks and the system is doing nothing but the copying...

/mjt

Re: Swapping out for larger disks

[David Greaves]

Brad Campbell wrote:
> G'day all,
> 
> I've got 3 arrays here. A 3 drive raid-5, a 10 drive raid-5 and a 15
> drive raid-6. They are all currently 250GB SATA drives.
> 
> I'm contemplating an upgrade to 500GB drives on one or more of the
> arrays and wondering the best way to do the physical swap.
> 
> The slow and steady way would be to degrade the array, remove a disk,
> add the new disk, lather, rinse, repeat. After which I could use mdadm
> --grow. There is the concern of a degraded array here though (and one of
> the reasons I'm looking to swap is some of the disks have about 30,000
> hours on the clock and are growing the odd defect).


Assuming hotswap and for maximum uptime/minimal exposure to risk... a while back
there was a discussion of a fiddly way that involved adding a disk, making a
mirror, removing the old disk, breaking the mirror. ( See archive for details)

> 
> I was more wondering about the feasibility of using dd to copy the drive
> contents to the larger drives (then I could do 5 at a time) and working
> it from there.
Err, if you can dd the drives, why can't you create a new array and use xfsdump
or equivalent? Is downtime due to copying that bad?

David

Re: Swapping out for larger disks

[Brad Campbell]

David Greaves wrote:

>> I was more wondering about the feasibility of using dd to copy the drive
>> contents to the larger drives (then I could do 5 at a time) and working
>> it from there.

> Err, if you can dd the drives, why can't you create a new array and use xfsdump
> or equivalent? Is downtime due to copying that bad?

I can only do 5 at a time. (10 slots, 5 source - 5 destination).

I'm not worried about the downtime so much as the constant swapping of disks. This way I can do it 
in 2 or 3 blocks at most.


Brad
-- 
"Human beings, who are almost unique in having the ability
to learn from the experience of others, are also remarkable
for their apparent disinclination to do so." -- Douglas Adams