[linux-lvm] About Extending LVM size with Hardware Raid

[linux-lvm] About Extending LVM size with Hardware Raid

[Vahric MUHTARYAN]

Hi Everybody ?! 

	Maybe everybody know Raid5 can extend easily. I'm using hardware
raid and I wonder What extend utulity I have to use for system . resize or
LVM .. 
Because with hardware raid system looks all disks like a one logical disk at
this moment How can I intorduce extended array to system ... 


Thanks  

Re: [linux-lvm] About Extending LVM size with Hardware Raid

[Chris Cox]

Vahric MUHTARYAN wrote:
> Hi Everybody ?! 
> 
> 	Maybe everybody know Raid5 can extend easily. I'm using hardware
> raid and I wonder What extend utulity I have to use for system . resize or
> LVM .. 
> Because with hardware raid system looks all disks like a one logical disk at
> this moment How can I intorduce extended array to system ... 

Uh... you don't generally extend RAID 5.  But if your hardware
raid solution has unutilized disks (or disks you can add), you can
create a new logical hw raid disk and easily add that disk into existing
volume groups.

If you have left over space on your existing logical disk, that
too can be added quite easily.  You need to provide more info.

RE: [linux-lvm] About Extending LVM size with Hardware Raid

[Vahric MUHTARYAN]

	Okey you mean create a new raid5 with new disks and add it to volume
group okey you right . 

	But My question is , Think that I have raid5 array with 3 disk ( I'm
using ICP Raid controller card )  then I add 4.th disk to my raid array ( I
know clearly that I can extend raid5 ) at this I have more disk space but I
have to intorduce it to system , and I can't imagine How ?! it's easy if I
have free partiton but I can't show this extended array like a free
partition  :( 

Thanks For your answer 
Vahric 

-----Original Message-----
From: linux-lvm-bounces@redhat.com [mailto:linux-lvm-bounces@redhat.com] On
Behalf Of Chris Cox
Sent: Wednesday, April 21, 2004 9:18 PM
To: LVM general discussion and development
Subject: Re: [linux-lvm] About Extending LVM size with Hardware Raid

Vahric MUHTARYAN wrote:
> Hi Everybody ?! 
> 
> 	Maybe everybody know Raid5 can extend easily. I'm using hardware
> raid and I wonder What extend utulity I have to use for system . resize or
> LVM .. 
> Because with hardware raid system looks all disks like a one logical disk
at
> this moment How can I intorduce extended array to system ... 

Uh... you don't generally extend RAID 5.  But if your hardware
raid solution has unutilized disks (or disks you can add), you can
create a new logical hw raid disk and easily add that disk into existing
volume groups.

If you have left over space on your existing logical disk, that
too can be added quite easily.  You need to provide more info.



_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

RE: [linux-lvm] About Extending LVM size with Hardware Raid

[Jeffrey Layton]

On Wed, 2004-04-21 at 15:05, Vahric MUHTARYAN wrote:
> 	Okey you mean create a new raid5 with new disks and add it to volume
> group okey you right . 
> 
> 	But My question is , Think that I have raid5 array with 3 disk ( I'm
> using ICP Raid controller card )  then I add 4.th disk to my raid array ( I
> know clearly that I can extend raid5 ) at this I have more disk space but I
> have to intorduce it to system , and I can't imagine How ?! it's easy if I
> have free partiton but I can't show this extended array like a free
> partition  :( 
> 

This is not so much a LVM thing, as a question for your controller card
manufacturer. Linux software RAID5 can extend a software RAID5 array, by
shuffling data around on the existing and new disks, and recalculating
the checksums (see the raidreconf command).

With hardware RAID5, however, unless your hardware vendor has provided a
tool to do this, you're stuck with saving off the data to another
location, rebuilding the array, and copying the data back.

Maybe the ICP folks can help you,
Jeff

Re: [linux-lvm] About Extending LVM size with Hardware Raid

[Chris Cox]

Vahric MUHTARYAN wrote:
> 	Okey you mean create a new raid5 with new disks and add it to volume
> group okey you right . 
> 
> 	But My question is , Think that I have raid5 array with 3 disk ( I'm
> using ICP Raid controller card )  then I add 4.th disk to my raid array ( I
> know clearly that I can extend raid5 ) at this I have more disk space but I
> have to intorduce it to system , and I can't imagine How ?! it's easy if I
> have free partiton but I can't show this extended array like a free
> partition  :( 

If you truly can "extend raid5", you have a totally different disk
essentially.  It's nice that you have a HW raid that allows you to
extend RAID 5 (an icky mess for sure.. probably takes a while to
do the reshuffling).  I don't know how to tell LVM that the disk itself
has actually gotten bigger.  Interesting problem, I doubt there's
an answer for this though (but I could be wrong).

I think if I wanted an extensible array setup, I would consider
just aggregating a set of mirror'd drives into a volume group.
That way, I could just add two more disks, create a new hw logical
raid 1 drive, and add that to my volume group.  Something I will
consider doing.

Re: [linux-lvm] About Extending LVM size with Hardware Raid

[Greg Freemyer]

On Wed, 2004-04-21 at 13:38, Vahric MUHTARYAN wrote:
> Hi Everybody ?! 
> 
> 	Maybe everybody know Raid5 can extend easily. I'm using hardware
> raid and I wonder What extend utulity I have to use for system . resize or
> LVM .. 
> Because with hardware raid system looks all disks like a one logical disk at
> this moment How can I intorduce extended array to system ... 
> 
> 
> Thanks  
In that case you may not want to be using LVM.

Most of the Linux filesystems have an extend capability, so that is not
a problem.

I don't know how to get the kernel itself to rescan the hardware for
capacity changes.  Obviously you can reboot, but there must be a better
way.

I would look for SAN documentation that describes the resize process. 
The commands will likely be exactly the same from the Linux perspective.

Greg
-- 
Greg Freemyer

Re: [linux-lvm] About Extending LVM size with Hardware Raid

[Glen Harris]

I have a similar system - a fibre channel RAID box. I added a disk to it
and expanded the RAID5 device, then used the scsiadd command to rescan
the device. If your card emulates a SCSI disk, this may work. Otherwise,
try unloading/loading the kernel driver module(if possible), and failing
that a reboot should work.

Anyway, the new size was reported correctly as 1.25Tb, so I then used
pvresize to grow the pv to the new size of the device.

At this point, pvresize reported "Command not implemented yet".

/sigh/

Vahric MUHTARYAN wrote:
> Hi Everybody ?! 
> 
> 	Maybe everybody know Raid5 can extend easily. I'm using hardware
> raid and I wonder What extend utulity I have to use for system . resize or
> LVM .. 
> Because with hardware raid system looks all disks like a one logical disk at
> this moment How can I intorduce extended array to system ... 
> 
> 
> Thanks  
> 
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

Re: [linux-lvm] About Extending LVM size with Hardware Raid

[Clint Byrum]

On Wednesday, April 21, 2004, at 02:09 PM, Glen Harris wrote:

> I have a similar system - a fibre channel RAID box. I added a disk to 
> it
> and expanded the RAID5 device, then used the scsiadd command to rescan
> the device. If your card emulates a SCSI disk, this may work. 
> Otherwise,
> try unloading/loading the kernel driver module(if possible), and 
> failing
> that a reboot should work.
>
> Anyway, the new size was reported correctly as 1.25Tb, so I then used
> pvresize to grow the pv to the new size of the device.
>

Just curious, but what kind of rebuild time do you see on that 1.25TB 
RAID5 volume?

I know you have a hardware controller, but I've found that with Linux's 
software RAID5, anything over 400GB or so will take over 3 hours on a 
decent disk system to rebuild. Thats why I use LVM to aggregate several 
RAID5 volumes, rather than one giant RAID5. I generally expect that it 
will take a couple of hours to see the failure, find a replacement, and 
get it installed in the machine (thats if I'm onsite, and have spares 
on hand ) With a 2 hour rebuild time, I'm already looking at 4 hours of 
exposure where another disk failure means data loss. Thats already 
pretty high to me, especially if there's something systemic causing 
drives to fail.

Even with hot spares, you still have to be careful during the array 
rebuild. :-P

Re: [linux-lvm] About Extending LVM size with Hardware Raid

[Glen Harris]

Clint Byrum wrote:
> Just curious, but what kind of rebuild time do you see on that 1.25TB 
> RAID5 volume?

Adding a single 250Gb 5400RPM Maxtor disk to the 5-disk RAID5 took just
over 30 hours. For future disks, we'll be adding them in multiples - if
we can get pvresize working, of course. The box can take 12 disks.

We're willing to take the time penalty, and in return get more space, as
this box is a rsync server. The servers rsync to this box quickly overnight,
then it backs up to the tape library in the early morning/during the day.
This is a (relatively)cheap backup solution, and as such, doesn't need the
on-line resiliancy of the file or database servers.

For the file/IMAP/Oracle servers with their Sun SCSI FC raid box, we're
running several raids as needs grow and aggregating them with the metadisk
software.

Re: [linux-lvm] About Extending LVM size with Hardware Raid

[Clint Byrum]

On Wed, 2004-04-21 at 22:08, Glen Harris wrote:
> Clint Byrum wrote:
> > Just curious, but what kind of rebuild time do you see on that 1.25TB 
> > RAID5 volume?
> 
> Adding a single 250Gb 5400RPM Maxtor disk to the 5-disk RAID5 took just
> over 30 hours. For future disks, we'll be adding them in multiples - if
> we can get pvresize working, of course. The box can take 12 disks.
> 

I've taken the route of only ever adding 3 or more disks. My system is
similar, a backup server that does rsyncing and archiving of older data.
Whenever we need more space (seems about every 8 - 12 months), we add
disks, pvmove (dunno how well pvmove works in lvm2.. which we just went
to) the data off of the old PV, and then take the old disks out for use
as offsite backup disks (tapes suck! ;) 

I recently migrated about 300GB of data off our old really slow 4x100GB
PATA drives onto some nice new 250GB SATA drives on an Escalade
controller. Unfortunately, those 100GB drives were the last of the
non-LVM data we had on the box, so I don't know how long it would have
taken to pvmove.

So I guess what I'm getting at is, why grow a RAID5 slowly, 1 disk at a
time, when disks are *so* cheap? Get 3 or 4 more giant disks, toss 'em
in, RAID5 'em up, and add their PV to the VG of your choice. Thats what
makes LVM so nice! :)

> We're willing to take the time penalty, and in return get more space, as
> this box is a rsync server. The servers rsync to this box quickly overnight,
> then it backs up to the tape library in the early morning/during the day.
> This is a (relatively)cheap backup solution, and as such, doesn't need the
> on-line resiliancy of the file or database servers.
> 
> For the file/IMAP/Oracle servers with their Sun SCSI FC raid box, we're
> running several raids as needs grow and aggregating them with the metadisk
> software.
> 

Sounds like the expensive version of my solution. ;-)


Side Note: if anyone knows of a cheap way to hotswap PATA disks, I'd
like to hear about it. Right now I stick 'em in a crappy old box with
those $2 removable IDE tray things, and have to reboot that box whenever
I'm done copying stuff to the disks. No big deal, but it seems kinda
cheesy. :-P

Re: [linux-lvm] About Extending LVM size with Hardware Raid

[Kevin P. Fleming]

Clint Byrum wrote:

> Side Note: if anyone knows of a cheap way to hotswap PATA disks, I'd
> like to hear about it. Right now I stick 'em in a crappy old box with
> those $2 removable IDE tray things, and have to reboot that box whenever
> I'm done copying stuff to the disks. No big deal, but it seems kinda
> cheesy. :-P

I don't think there's a reliable, safe way to hotswap PATA, and even if 
the hardware exists Linux doesn't support it well. Your best bet may be 
to get some cheap USB2 enclosures for them (one per drive :-)) and then 
you can hotswap to your heart's content.

Re: [linux-lvm] About Extending LVM size with Hardware Raid

[Piete Brooks]

> I don't think there's a reliable, safe way to hotswap PATA,

I thought it was safe (i.e. would not damage the IDE bus drivers in the 
computer or disc) so long as the caddy takes care with the order in which the 
pins are (dis)connected [e.g. by having different pin lengths] or less subtly, 
by having the caddy lock disconnect the power on unlocking.

[ of course, you only have one device per IDE bus ]

> and even if the hardware exists Linux doesn't support it well.

Best bet is to look at the various "select bay" commands for laptops.

Re: [linux-lvm] About Extending LVM size with Hardware Raid

[Greg Freemyer]

> Sounds like the expensive version of my solution. ;-)
> 
> 
> Side Note: if anyone knows of a cheap way to hotswap PATA disks, I'd
> like to hear about it. Right now I stick 'em in a crappy old box with
> those $2 removable IDE tray things, and have to reboot that box whenever
> I'm done copying stuff to the disks. No big deal, but it seems kinda
> cheesy. :-P
> 
It may be a little on the expensive side, but I use 3ware controllers in
combination with ICYdock trays.

Especially if you are doing Raid 1 or 5, it works great.  The linux
kernel doesn't even know you pulled a drive.

If your just doing JBOD, then it may or may not work.  Never tried it.

Greg
-- 
Greg Freemyer

Re: [linux-lvm] About Extending LVM size with Hardware Raid

[Glen Harris]

Clint Byrum wrote:
> Side Note: if anyone knows of a cheap way to hotswap PATA disks, I'd
> like to hear about it. Right now I stick 'em in a crappy old box with
> those $2 removable IDE tray things, and have to reboot that box whenever
> I'm done copying stuff to the disks. No big deal, but it seems kinda
> cheesy. :-P

PATA disks are difficult. There is a very slight chance of damaging the
interface electronics, and motor driver electronics. There is also a
chance of data loss, as it is difficult to guarantee that no write-
behind cache is in operation. Finally, the operating system has to
be able to disable an IDE device and redetect it, including partition
tables, etc - somewhat of a grey area in the Linux kernel from my
reading of the kernel sources. If that's changed, please let me know!

There are cheap($30) hotswap caddies with a power switch which removes
power from the drive and places the interface bus in Hi-Z state, but
this still leaves the caching and operating system disable/redetect
issues.

Cheers, glen.

Re: [linux-lvm] About Extending LVM size with Hardware Raid

[Piete Brooks]

> PATA disks are difficult.

Similar problems for SATA and SCSI ...

> There is a very slight chance of damaging the interface electronics,
> and motor driver electronics.

As below, that can be fixed ...

> There is also a chance of data loss, as it is difficult to guarantee that
> no write-behind cache is in operation.

If under RAID, use "raidsetfaulty" first, or otherwise make it quiescent.

> Finally, the operating system has to be able to disable an IDE device
> and redetect it,

There is code to do that, e.g. for laptop selectbays.

> including partition tables, etc

That works just fine, so long as no partition is in use, which it should not 
be.

> - somewhat of a grey area in the Linux kernel from my reading
> of the kernel sources. If that's changed, please let me know!

I re-partition disks "on the fly" fairly frequently.
I use RAID1 for all data, so I turn swapping off on the required disc, the 
"raidsetfaulty" and "raidhotremove" all the data partitions, then write the 
new partition tables which the kernels sees (check with /proc/partitions), 
then raidhotadd the partitions, mkswap and swapon, then repeat on the other 
disk, and bingo -- repartitioned without needing a reboot.
> There are cheap($30) hotswap caddies with a power switch which removes
> power from the drive and places the interface bus in Hi-Z state, but
> this still leaves the caching and operating system disable/redetect
> issues.
> 
> Cheers, glen.
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/