how to which to bigger disk (md can do it?)

how to which to bigger disk (md can do it?)

[Farkas Levente]

hi,
we've got a server with 3ware 7500-8 and 8 pieces of 120 GB maxtor hd 
and software raid-5. the 1TB disk is running out of the space so we 
decided to switch to 200 GB hard disks. unfortunately there is only one 
day (saturday) when we can turn off the machine. how we can do it in the 
fastest way?:
- put the 8 disk to another server and run rsync through two gigabit 
ethernet card. but it can takes more than a day to sync about 800GB.
- switch the disks one-by-one in the current server and leave linux's 
software raid to sync the data. in this case we only have to stop the 
system a few mintues per day and takes 8 days, but continous operation. 
does it possible? after I put all new hard disk the totel capacity will 
be (8-1)*200 GB or it will remain (8-1)*120 GB? since during the disk 
switch the total space is the smallest hd's*(n-1), but what happens when 
I put the last 200GB disk?
- dd (or somehow imageing) the one 120GB disk content to one 200 GB 
disk. if they are on the same ide bus (or better on the same 3ware 
controller) the cloning is much faster. can I do a
dd if=/dev/sda1 of=/dev/sdb1
if they are different size and after the dd the second will be 200 GB 
raid component?
- rsync between two local disk?
any other or better solution?
thanks in advance.
yours.

-- 
   Levente                               "Si vis pacem para bellum!"

Re: how to which to bigger disk (md can do it?)

[Hendrik Visage]

On Sun, Nov 16, 2003 at 12:03:46PM +0100, Farkas Levente wrote:
> hi,
> we've got a server with 3ware 7500-8 and 8 pieces of 120 GB maxtor hd 
> and software raid-5. the 1TB disk is running out of the space so we 

Are you sure about the Software RAID on this 3ware adapater?? I was
under the impression it's a HW Raid adapter?
(I have a preference to HW RAID if available...)

> - put the 8 disk to another server and run rsync through two gigabit 
> ethernet card. but it can takes more than a day to sync about 800GB.

You could do this initially during the week before the swapout, and then
just update it  before the swap out. Something te note, if it's lots of files,
I'd advise to run multiple rsyncs over smaller directories, as it speeds
up the rsync processing alot.

> - switch the disks one-by-one in the current server and leave linux's 
> software raid to sync the data. in this case we only have to stop the 
> system a few mintues per day and takes 8 days, but continous operation. 
> does it possible? after I put all new hard disk the totel capacity will 
> be (8-1)*200 GB or it will remain (8-1)*120 GB? since during the disk 
> switch the total space is the smallest hd's*(n-1), but what happens when 
> I put the last 200GB disk?

If it's SW raid, it'll be based on the partitions, so you'll have to create the
partitions bigger than the current 120GB partitions. Then after the last disk,
the size would be the bigger size, however you'll then need to run a growfs
(or similar) for your filesystem to expand it to the bigger size.

Fr HW Raid, it'll depend on the controller.

Hendrik

Re: how to which to bigger disk (md can do it?)

[Farkas Levente]

Hendrik Visage wrote:
> On Sun, Nov 16, 2003 at 12:03:46PM +0100, Farkas Levente wrote:
> 
>>hi,
>>we've got a server with 3ware 7500-8 and 8 pieces of 120 GB maxtor hd 
>>and software raid-5. the 1TB disk is running out of the space so we 
> 
> 
> Are you sure about the Software RAID on this 3ware adapater?? I was
> under the impression it's a HW Raid adapter?
> (I have a preference to HW RAID if available...)

we've got a kernel creash with hardware raid 5 what's more software raid 
5 seems to be faster.

>>- put the 8 disk to another server and run rsync through two gigabit 
>>ethernet card. but it can takes more than a day to sync about 800GB.
> 
> 
> You could do this initially during the week before the swapout, and then
> just update it  before the swap out. Something te note, if it's lots of files,
> I'd advise to run multiple rsyncs over smaller directories, as it speeds
> up the rsync processing alot.

can't help since this is the backup server and almost everything 
changing on it.

>>- switch the disks one-by-one in the current server and leave linux's 
>>software raid to sync the data. in this case we only have to stop the 
>>system a few mintues per day and takes 8 days, but continous operation. 
>>does it possible? after I put all new hard disk the totel capacity will 
>>be (8-1)*200 GB or it will remain (8-1)*120 GB? since during the disk 
>>switch the total space is the smallest hd's*(n-1), but what happens when 
>>I put the last 200GB disk?
> 
> 
> If it's SW raid, it'll be based on the partitions, so you'll have to create the
> partitions bigger than the current 120GB partitions. Then after the last disk,
> the size would be the bigger size, however you'll then need to run a growfs
> (or similar) for your filesystem to expand it to the bigger size.

which partitions? it's not clear to me. we've got :
- 8 pieces of 120 GB Linux raid autodetect (/dev/sda1, /dev/sdb1 etc)
- 1 pieces of 840 GB ext3 (/dev/md2)
so which partition should I have to create and grow?
but what I'd like to prefere the cloning. is there any way to 'rsync' 
between a 120 GB and 200 GB disk which is Linux raid autodetect (and not 
eg. ext2)?

another good question anybody use 200 GB maxtor hd with 3ware 7500-8 
controller?

thanks.

-- 
   Levente                               "Si vis pacem para bellum!"

Re: how to which to bigger disk (md can do it?)

[Maurice Hilarius]

With regards to your message at 06:31 AM 11/16/03, Hendrik Visage. Where 
you stated:
>On Sun, Nov 16, 2003 at 12:03:46PM +0100, Farkas Levente wrote:
> > hi,
> > we've got a server with 3ware 7500-8 and 8 pieces of 120 GB maxtor hd
> > and software raid-5. the 1TB disk is running out of the space so we
>
>Are you sure about the Software RAID on this 3ware adapater?? I was
>under the impression it's a HW Raid adapter?
>(I have a preference to HW RAID if available...)
The 3Ware is a very good hardware RAID device.
That is irrelevant in this context.
One of the many advantages of  software RAID is the ability to use multiple 
controllers in separate PCI slots ( and in many cases on multiple PCI 
buses) to accomplish much better throughput and I/O rates.
I could perhaps agree with your preference for HWare RAID IF:
1) It could be spanned over multiple controllers
2) It could allow one to create non-RAID boot partitions on the disks used.
3) It could compete for RAID5 performance with software RAID on a modern 
machine.



With our best regards,

Maurice W. Hilarius       Telephone: 01-780-456-9771
Hard Data Ltd.               FAX:       01-780-456-9772
11060 - 166 Avenue        mailto:maurice@harddata.com
Edmonton, AB, Canada      http://www.harddata.com/
    T5X 1Y3

Re: how to which to bigger disk (md can do it?)

[Luca Berra]

On Sun, Nov 16, 2003 at 12:03:46PM +0100, Farkas Levente wrote:
>hi,
>- put the 8 disk to another server and run rsync through two gigabit 
>ethernet card. but it can takes more than a day to sync about 800GB.
well rsync is incremental, you could do an initial rsync during the
working week and do another to upgrade deltas during the weekend.

>- switch the disks one-by-one in the current server and leave linux's 
>software raid to sync the data. in this case we only have to stop the 
in this case you will have disks used up to 120Gb and an 80 gb of unused
space at the end. you could create another raid5 there, but it does not
seem appealing to me.

Really weird idea: please do not do it on real data.
You will probably shoot yourself doing this
- Switch the disks one by one.
- When you are over create a new raid5 over the existing one (keeping the
same settings for stripe size and parity algo)
- Resize the filesystem.

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: how to which to bigger disk (md can do it?)

[Neil Brown]

On Monday November 17, bluca@comedia.it wrote:
> 
> Really weird idea: please do not do it on real data.
> You will probably shoot yourself doing this
> - Switch the disks one by one.
> - When you are over create a new raid5 over the existing one (keeping the
> same settings for stripe size and parity algo)
> - Resize the filesystem.
> 

Not so weird - it is exactly what I was going to suggest.
Providing you get the chunk size, parity algoithm and device order
right, and use "--force" to make sure mdadm doesn't re-arrange the
devices on on it should work perfectly.

Somewhere low-down on my todo list for raid is allowing this to be
done on-line (mdadm /dev/md0 --size=max).  It is right along side
allowing the number of live devices in a raid1 to be changed 
(mdadm /dev/md1 --devices=3) and adding devices to a linear array 
(mdadm /dev/md2 --add /dev/sdd).

NeilBrown

Re: how to which to bigger disk (md can do it?)

[Farkas Levente]

Neil Brown wrote:
> On Monday November 17, bluca@comedia.it wrote:
> 
>>Really weird idea: please do not do it on real data.
>>You will probably shoot yourself doing this
>>- Switch the disks one by one.
>>- When you are over create a new raid5 over the existing one (keeping the
>>same settings for stripe size and parity algo)
>>- Resize the filesystem.
>>
> 
> 
> Not so weird - it is exactly what I was going to suggest.
> Providing you get the chunk size, parity algoithm and device order
> right, and use "--force" to make sure mdadm doesn't re-arrange the
> devices on on it should work perfectly.

this's one of our production server, so I'd like to be sure. so:
- switch the disk one by one (do these steps 8 times):
   - put out one 120 hd (mdadm /dev/md0 -f /dev/sda1 -r /dev/sda1)
   - put the new 200 GB one and create a 200 GB partition (Linux raid
     autodetect)
   - add to md0 (mdadm -a /dev/hda1) this step do not change the device
     order???
- resize2fs

when should I use --force??

> Somewhere low-down on my todo list for raid is allowing this to be
> done on-line (mdadm /dev/md0 --size=max).  It is right along side

that's be nice:-)

> allowing the number of live devices in a raid1 to be changed 
> (mdadm /dev/md1 --devices=3) and adding devices to a linear array 
> (mdadm /dev/md2 --add /dev/sdd).

that would be useful for raid5 too?
yes I know the reason why this is not possible, BUT it can do (rebuild 
the whole raid5) the same as the create for raid5 and it can be run in 
the backgroup. that would be a very useful feature for many people.
yours.

-- 
   Levente                               "Si vis pacem para bellum!"

Re: how to which to bigger disk (md can do it?)

[Luca Berra]

On Mon, Nov 17, 2003 at 03:05:26PM +0100, Farkas Levente wrote:
>>Not so weird - it is exactly what I was going to suggest.
>>Providing you get the chunk size, parity algoithm and device order
>>right, and use "--force" to make sure mdadm doesn't re-arrange the
>>devices on on it should work perfectly.
>
>this's one of our production server, so I'd like to be sure. so:
this is the reason i said it is a weird idea, it is not easy to
understand and if you fuck up you fuck up badly!

>- switch the disk one by one (do these steps 8 times):
>   - put out one 120 hd (mdadm /dev/md0 -f /dev/sda1 -r /dev/sda1)
>   - put the new 200 GB one and create a 200 GB partition (Linux raid
>     autodetect)
>   - add to md0 (mdadm -a /dev/hda1) this step do not change the device
>     order???
at this point you will find yourself with a 120GBx8 raid 5 array
now you would have to (but i would be very careful in doing this)

check the exact device order (/proc/mdstat ???)
write it down
check the chunk size
write it down
check the parity algorithm
write it down

umount the fs
stop the damned array

mdadm -C /dev/md<whatever> -c <your chink size> -p <your parity> -n 8 -f <list of all
devices in the same exact order>

THIS WILL REWRITE THE PARITY ON ALL DRIVES, SO IF YOU GET IT WRONG YOU
WILL DESTROY YOUR DATA

(would using missing as the last drive prevent the parity rebuild and
allow to check the expected result ?????)

cross you fingers,
mount your fs back and see what happened
umount it again
>- resize2fs
mount it for the last time


>when should I use --force??
only when you are really sure of what you are doing.

please test it on a test system before doing it on real data

you might well find that a slower way might be better for you

(i don't want to sound catastrophic, but i am not going to be held
responsible if someone reads these instruction and fucks-up badly)

L.

-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: how to which to bigger disk (md can do it?)

[Neil Brown]

On Monday November 17, lfarkas@bnap.hu wrote:
> this's one of our production server, so I'd like to be sure. so:
> - switch the disk one by one (do these steps 8 times):
>    - put out one 120 hd (mdadm /dev/md0 -f /dev/sda1 -r /dev/sda1)

I would do this as two separate steps
   mdadm /dev/md0 -f /dev/sda1
   mdadm /dev/md0 -r /dev/sda1

as there are some possibly race problems when removing a device the
instant that it failes.

>    - put the new 200 GB one and create a 200 GB partition (Linux raid
>      autodetect)
>    - add to md0 (mdadm -a /dev/hda1) this step do not change the device
>      order???

No, it shouldnt' but it shouldn't matter.

> - resize2fs
> 
> when should I use --force??

When you have the array full of 200GB drives, use
  mdadm -D /dev/md0
to find out what it looks like.  This will include algorithm, chunk
size, etc.  Keep a copy of the output.
Then 
   mdadm -S /dev/md0

It probably would be to use the "missing" directive as Luca
suggested.  This will create a degraded array and so will not
automatically regenerate any data.  

Something like:
mdadm -C /dev/md0 --level 5 -n 8 --chunk=64 /dev/sda1 /dev/sdb1 /dev/sdc1
    .... /dev/sdf1 /dev/sdg1 missing

If you do this, it will only over-write the raid superblocks and will
not touch the data.

You can then
   fsck -n /dev/md0
just to make sure it looks right (you don't have to let this run to
completion. Just let it run for a new minutes to make sure nothing
obvious is wrong) Then mount the filesystem and have a look around.
If everything is OK, unmount and try resize2fs.

When you are happy the the array looks good, add the last drive back
in with
   mdadm /dev/md0 -a /dev/sdh1

NeilBrown