RE: Growing an Array?

RE: Growing an Array?

[Huntress Gary B NPRI]

Actually I meant an array of six 73GB drives switching to six 180GB drives, but thats conceptually the same.


Regards,

Gary Huntress
Code 4113
Naval Undersea Warfare Center
Newport, RI  02841
1-800-669-6892 x28990


-----Original Message-----
From: Derek Vadala [mailto:derek@cynicism.com]
Sent: Wednesday, November 06, 2002 12:18 PM
To: Huntress Gary B NPRI
Cc: 'linux-raid@vger.kernel.org'
Subject: RE: Growing an Array?


On Wed, 6 Nov 2002, Huntress Gary B NPRI wrote:

> One point of clarification that probably makes no difference.  I am not
> trying to go
>
> From:   6 X 73GB    To:   7 x 73GB
>
> but rather
>
> From:	  6 X 73GB 	  To:   6 x 180GB

So you have a RAID-5 of six 15GB disks and you want to switch to a RAID-5
of six 36GB disks?

-- 
Derek Vadala, derek@cynicism.com, http://www.cynicism.com/~derek

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Growing an Array?

[Derek Vadala]

On Wed, 6 Nov 2002, Huntress Gary B NPRI wrote:

> Actually I meant an array of six 73GB drives switching to six 180GB
> drives, but thats conceptually the same.

I think that your best bet is to bring the new array online and copy the
data over using traditional methods. That's going to be the fastest way to
get the new array online. The other methods (backup and restore,
raidreconf) are either slower or don't apply specifically to this case.

--
Derek Vadala, derek@cynicism.com, http://www.cynicism.com/~derek

Re: Growing an Array?

[H. Peter Anvin]

Followup to:  <20021106173553.163A03B807@jaded.cynicism.com>
By author:    Derek Vadala <derek@cynicism.com>
In newsgroup: linux.dev.raid
>
> On Wed, 6 Nov 2002, Huntress Gary B NPRI wrote:
> 
> > Actually I meant an array of six 73GB drives switching to six 180GB
> > drives, but thats conceptually the same.
> 
> I think that your best bet is to bring the new array online and copy the
> data over using traditional methods. That's going to be the fastest way to
> get the new array online. The other methods (backup and restore,
> raidreconf) are either slower or don't apply specifically to this case.
> 

Note that in large-scale data moves, like this, where you're reluctant
to take the system offline: if you can put the new storage online
(connected either to the same or a different server) you run the copy
process while the data is still online, then when you close for the
migration you can do an incremental update (rsync -aHS --delete is
good for this; if you want to be extra paranoid at the expense of
significantly slower execution rsync -acHS --delete.)  This is
typically a lot quicker than copying all the data once offline.

	-hpa

-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt	<amsp@zytor.com>

Re: Growing an Array?

[Derek Vadala]

On 11 Nov 2002, H. Peter Anvin wrote:

> Note that in large-scale data moves, like this, where you're reluctant
> to take the system offline: if you can put the new storage online
> (connected either to the same or a different server) you run the copy
> process while the data is still online, then when you close for the
> migration you can do an incremental update (rsync -aHS --delete is
> good for this; if you want to be extra paranoid at the expense of
> significantly slower execution rsync -acHS --delete.)  This is
> typically a lot quicker than copying all the data once offline.

This is exactly what I meant, sorry if I implied otherwise.

--
Derek Vadala, derek@cynicism.com, http://www.cynicism.com/~derek

RE: Growing an Array?

[Huntress Gary B NPRI]

One point of clarification that probably makes no difference.  I am not trying to go 

From:   6 X 73GB    To:   7 x 73GB  

but rather

From:	  6 X 73GB 	  To:   6 x 180GB  



-----Original Message-----
From: Derek Vadala [mailto:derek@cynicism.com]
Sent: Wednesday, November 06, 2002 11:56 AM
To: Huntress Gary B NPRI
Cc: 'Alistair Riddell'; 'linux-raid@vger.kernel.org'
Subject: RE: Growing an Array?


On Wed, 6 Nov 2002, Huntress Gary B NPRI wrote:

> My array is currently 6 X 73GB.
>
> I think you are saying that I can do this now using resize2fs?  I think
> I pessimistically assumed that tool was destructive.

Sort of...

Alistair's point was that growing an array takes a long time, so the
backup and restore process might not be as slow as you think. I suspect it
will take the better part of a day to do what you have proposed (add a
disk to a 6x73GB RAID-5).

The first step is to use raidreconf to grow the array (add a disk). This
is the slow part of the process.

Get raidreconf here http://unthought.net/raidreconf/

It also ships with RedHat 8.0 (raidtools-1.00.2, anyone know why this
version of raidtools still hasn't shown up in the usual spots???)

Once the array is resized, you then need to resize the filesystem. This
typically takes very little time and is done using the utility
appropriate for your FS. So for ext2 you us resize2fs.

--
Derek Vadala, derek@cynicism.com, http://www.cynicism.com/~derek

RE: Growing an Array?

[Derek Vadala]

On Wed, 6 Nov 2002, Huntress Gary B NPRI wrote:

> One point of clarification that probably makes no difference.  I am not
> trying to go
>
> From:   6 X 73GB    To:   7 x 73GB
>
> but rather
>
> From:	  6 X 73GB 	  To:   6 x 180GB

So you have a RAID-5 of six 15GB disks and you want to switch to a RAID-5
of six 36GB disks?

-- 
Derek Vadala, derek@cynicism.com, http://www.cynicism.com/~derek

RE: Growing an Array?

[Huntress Gary B NPRI]

My array is currently 6 X 73GB.

I think you are saying that I can do this now using resize2fs?   I think I pessimistically assumed that tool was destructive.


Regards,

Gary Huntress
Code 4113
Naval Undersea Warfare Center
Newport, RI  02841
1-800-669-6892 x28990

-----Original Message-----
From: Alistair Riddell [mailto:ali@gwc.org.uk]
Sent: Wednesday, November 06, 2002 10:13 AM
Cc: 'linux-raid@vger.kernel.org'
Subject: Re: Growing an Array?


On Wed, 6 Nov 2002, Huntress Gary B NPRI wrote:

> 1)  pop out drive 1
> 2)  Insert new drive
> 3)  "initialize" the new drive
> 4)  Some magic happens and the larger drive becomes part of the array
>
> lather, rinse, repeat for the remaining replacement drives and *poof* I have a larger array.

In theory you could do this now...

> This capability would save me an incredible amount of labor and hand wringing.  Our "technology refresh" upgrades happen quite regularly because our data never stops growing.   Is this a pipe dream or an eventual reality?

How much data are you talking about? It took me 16 hours to resize to a
10x73GB array. If you were to do that in stages then it could end up
taking days or weeks in which case backing up to DLT might become more
attractive.

Note that the raid array will be offline while the resize takes place.

-- 
Alistair Riddell - BOFH
IT Manager, George Watson's College, Edinburgh
Tel: +44 131 446 6070    Fax: +44 131 452 8594
Microsoft - because god hates us

-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: Growing an Array?

[Derek Vadala]

On Wed, 6 Nov 2002, Huntress Gary B NPRI wrote:

> My array is currently 6 X 73GB.
>
> I think you are saying that I can do this now using resize2fs?  I think
> I pessimistically assumed that tool was destructive.

Sort of...

Alistair's point was that growing an array takes a long time, so the
backup and restore process might not be as slow as you think. I suspect it
will take the better part of a day to do what you have proposed (add a
disk to a 6x73GB RAID-5).

The first step is to use raidreconf to grow the array (add a disk). This
is the slow part of the process.

Get raidreconf here http://unthought.net/raidreconf/

It also ships with RedHat 8.0 (raidtools-1.00.2, anyone know why this
version of raidtools still hasn't shown up in the usual spots???)

Once the array is resized, you then need to resize the filesystem. This
typically takes very little time and is done using the utility
appropriate for your FS. So for ext2 you us resize2fs.

--
Derek Vadala, derek@cynicism.com, http://www.cynicism.com/~derek

Growing an Array?

[Huntress Gary B NPRI]

I know exactly what I would like to do, but I'm not RAID-Smart enough to know if this is an existing capability or completely impossible.

Speaking in very generic terms, I have a RAID 5 array that is reaching it's capacity.  Drives are cheap and I want to grow the array.  Backing up the array to DLT is "hard" and I don't want to take my production system offline.   I want to buy a stack of new (hot swappable) drives and do the following to my array:

1)  pop out drive 1
2)  Insert new drive
3)  "initialize" the new drive
4)  Some magic happens and the larger drive becomes part of the array

lather, rinse, repeat for the remaining replacement drives and *poof* I have a larger array.

This capability would save me an incredible amount of labor and hand wringing.  Our "technology refresh" upgrades happen quite regularly because our data never stops growing.   Is this a pipe dream or an eventual reality?

TIA

Regards,

Gary Huntress
Code 4113
Naval Undersea Warfare Center
Newport, RI  02841
1-800-669-6892 x28990

Re: Growing an Array?

[Alistair Riddell]

On Wed, 6 Nov 2002, Huntress Gary B NPRI wrote:

> 1)  pop out drive 1
> 2)  Insert new drive
> 3)  "initialize" the new drive
> 4)  Some magic happens and the larger drive becomes part of the array
>
> lather, rinse, repeat for the remaining replacement drives and *poof* I have a larger array.

In theory you could do this now...

> This capability would save me an incredible amount of labor and hand wringing.  Our "technology refresh" upgrades happen quite regularly because our data never stops growing.   Is this a pipe dream or an eventual reality?

How much data are you talking about? It took me 16 hours to resize to a
10x73GB array. If you were to do that in stages then it could end up
taking days or weeks in which case backing up to DLT might become more
attractive.

Note that the raid array will be offline while the resize takes place.

-- 
Alistair Riddell - BOFH
IT Manager, George Watson's College, Edinburgh
Tel: +44 131 446 6070    Fax: +44 131 452 8594
Microsoft - because god hates us