MD devices renaming or re-ordering question

MD devices renaming or re-ordering question

[Maurice Hilarius]

Hi to all.

I wonder if somebody would care to help me to solve a problem?

I have some servers.
They are running CentOS5
This OS has a limitation where the maximum filesystem size is 8TB.

Each server curr3ently has a  AMCC/3WARE 16 port SATA controllers. Total
of 16 ports / drives
I am using 750GB drives.

I am exporting the drives as single, NOT as hardware RAID
That is due to the filesystem and controller limitations, among other
reasons.

Each server currently has 16 disks attached to the one controller

I want to add a 2nd controller, and, for now, 4 more disks on it.

I want to have the boot disk as a plain disk, as presently configured as
sda1,2,3

The remaining 15 disks are configured as :
sdb1 through sde1 as md0 ( 4 devices/partitions)
sdf1 through sdp1 as md1 (10 devices/partitions)
I want to add a 2nd controller, and 4 more drives, to the md0 device.

But, I do not want md0 to be "split" across the 2 controllers this way.
I prefer to do the split on md1

Other than starting from scratch, the best solution would be to add the
disks to md0, then to "magically" turn md0 into md1, and md1 into md0

So, the question:
How does one make md1 into md0, and vice versa?
Without losing the data on these md's ?

Thanks in advance for any suggestions.



-- 
Regards, Maurice


/09 F9 11 02 9D 74 E3 5B D8 41 56 C5 63 56 88 C0/

/1001 11111001 00010001 00000010 10011101 01110100 11100011 01011011
11011000 01000001 01010110 11000101 01100011 01010110 10001000 11000000/

/10 base 13,256,278,887,989,457,651,018,865,901,401,704,640/

Re: MD devices renaming or re-ordering question

[Goswin von Brederlow]

Maurice Hilarius <maurice@harddata.com> writes:

> Hi to all.
>
> I wonder if somebody would care to help me to solve a problem?
>
> I have some servers.
> They are running CentOS5
> This OS has a limitation where the maximum filesystem size is 8TB.
>
> Each server curr3ently has a  AMCC/3WARE 16 port SATA controllers. Total
> of 16 ports / drives
> I am using 750GB drives.
>
> I am exporting the drives as single, NOT as hardware RAID
> That is due to the filesystem and controller limitations, among other
> reasons.
>
> Each server currently has 16 disks attached to the one controller
>
> I want to add a 2nd controller, and, for now, 4 more disks on it.
>
> I want to have the boot disk as a plain disk, as presently configured as
> sda1,2,3
>
> The remaining 15 disks are configured as :
> sdb1 through sde1 as md0 ( 4 devices/partitions)
> sdf1 through sdp1 as md1 (10 devices/partitions)
> I want to add a 2nd controller, and 4 more drives, to the md0 device.
>
> But, I do not want md0 to be "split" across the 2 controllers this way.
> I prefer to do the split on md1
>
> Other than starting from scratch, the best solution would be to add the
> disks to md0, then to "magically" turn md0 into md1, and md1 into md0
>
> So, the question:
> How does one make md1 into md0, and vice versa?
> Without losing the data on these md's ?
>
> Thanks in advance for any suggestions.

The simplest is to pull the disks from md1 from the first controler
and put them into the 2nd controler and then add the new disks to the
first controler.

Actually I would pull the 4 disks from md0 and put them on the second
controler. Have no split at all.

Or alternatively split both raids evenly to balance loads between the
controlers. Might be faster.

MfG
        Goswin

Re: MD devices renaming or re-ordering question

[Goswin von Brederlow]

Goswin von Brederlow <brederlo@informatik.uni-tuebingen.de> writes:

> The simplest is to pull the disks from md1 from the first controler
> and put them into the 2nd controler and then add the new disks to the
> first controler.

That is of cause whith the raid stoped. You didn't say what kind of
raid you have but while it is online it would need resyncing or would
just plain break if you hot-plug a disk.

MfG
        Goswin

Re: MD devices renaming or re-ordering question

[Bill Davidsen]

Maurice Hilarius wrote:
> Hi to all.
>
> I wonder if somebody would care to help me to solve a problem?
>
> I have some servers.
> They are running CentOS5
> This OS has a limitation where the maximum filesystem size is 8TB.
>
> Each server curr3ently has a  AMCC/3WARE 16 port SATA controllers. Total
> of 16 ports / drives
> I am using 750GB drives.
>
> I am exporting the drives as single, NOT as hardware RAID
> That is due to the filesystem and controller limitations, among other
> reasons.
>
> Each server currently has 16 disks attached to the one controller
>
> I want to add a 2nd controller, and, for now, 4 more disks on it.
>
> I want to have the boot disk as a plain disk, as presently configured as
> sda1,2,3
>   

I'm not clear on what you mean by a "plain disk" followed by a list of 
partitions. If that means putting all your initial data on a single disk 
without RAID protection, that's a far worse idea in my experience than 
splitting arrays across controllers.
> The remaining 15 disks are configured as :
> sdb1 through sde1 as md0 ( 4 devices/partitions)
> sdf1 through sdp1 as md1 (10 devices/partitions)
> I want to add a 2nd controller, and 4 more drives, to the md0 device.
>
> But, I do not want md0 to be "split" across the 2 controllers this way.
> I prefer to do the split on md1
>   

Move the md0 drives to the 2nd controller, add more.
> Other than starting from scratch, the best solution would be to add the
> disks to md0, then to "magically" turn md0 into md1, and md1 into md0
>   

Unless you want to practice doing critical config changes, why? Moving 
the drives won't effect their name, at least not unless you have done 
something like configure by physical partition name instead of UUID. 
Doing that for more than a few drives is a learning experience waiting 
to happen. If that's the case, backup your mdadm.conf file and 
reconfigure using UUID, then start moving things around.
> So, the question:
> How does one make md1 into md0, and vice versa?
> Without losing the data on these md's ?
>
> Thanks in advance for any suggestions.

I would start by being sure the computer is doing the work, not the 
administrator, use UUID for assembly. Then move the drives for md0 and 
grow it.

Then consider the performance vs. reliability issues of having all 
drives on a single controller. Multiple controllers give you more points 
of failure unless you are mirroring across them, but better peak 
performance. Note, I'm suggesting evaluating what you are doing only, it 
may be fine, just avoids "didn't think about that" events.

Well, you asked for suggestions...  ;-)

-- 
bill davidsen <davidsen@tmr.com>
  CTO TMR Associates, Inc
  Doing interesting things with small computers since 1979

Re: MD devices renaming or re-ordering question

[Maurice Hilarius]

Bill Davidsen wrote:
> ..
> I'm not clear on what you mean by a "plain disk" followed by a list of
> partitions. If that means putting all your initial data on a single
> disk without RAID protection, that's a far worse idea in my experience
> than splitting arrays across controllers.
It is easiest enough for now to mirror the boot drive.
I worded it this way to make it clear the migration is not a booting issue.
>> The remaining 15 disks are configured as :
>> sdb1 through sde1 as md0 ( 4 devices/partitions)
>> sdf1 through sdp1 as md1 (10 devices/partitions)
>> I want to add a 2nd controller, and 4 more drives, to the md0 device.
>>
>> But, I do not want md0 to be "split" across the 2 controllers this way.
>> I prefer to do the split on md1
>>   
> Move the md0 drives to the 2nd controller, add more.
Yes, that is one way, involving some hardware swapping and more downtime.
>> Other than starting from scratch, the best solution would be to add the
>> disks to md0, then to "magically" turn md0 into md1, and md1 into md0
>>   
>
> Unless you want to practice doing critical config changes, why? Moving
> the drives won't effect their name, at least not unless you have done
> something like configure by physical partition name instead of UUID.
> Doing that for more than a few drives is a learning experience waiting
> to happen. If that's the case, backup your mdadm.conf file and
> reconfigure using UUID, then start moving things around.
OK, where may I learn more on using UUID for drive identification?
I have always assembled a RAID using the syntax /dev/sdxx ( sd drive
letter and partition number)
I take it there is a way to identify the UUID of a drive and partition
and assemble and maintain using syntax that way?

I hope that this will also get me past the problem sometimes of running
out  of letters in the 26 char alphabet!
I never thought the day where I would have a problem with more than 24
drives.. OK, so I show my age there!
> ..
> Then consider the performance vs. reliability issues of having all
> drives on a single controller.
> Multiple controllers give you more points of failure unless you are
> mirroring across them, but better peak performance.
Controller reliability seems to be not an issue. I have rarely seen a
3Ware card fail.
Drives, OTOH, well..
Hence the desire to have duplicated arrays, so we can clone across on MD
to another.

> Note, I'm suggesting evaluating what you are doing only, it may be
> fine, just avoids "didn't think about that" events.
>
Agreed. All good points.
> Well, you asked for suggestions...  ;-)
These are appreciated.
I am still looking , however, for a way to rename and md device.

Another case where it comes up is when I take a set of drives from one
machine and move them to another.
Having conflicting md devices comes to mind..

Thanks Bill




-- 

With our best regards,


Maurice W. Hilarius        Telephone: 01-780-456-9771
Hard Data Ltd.  FAX:       01-780-456-9772
11060 - 166 Avenue         email:maurice@harddata.com
Edmonton, AB, Canada       http://www.harddata.com/
   T5X 1Y3