[linux-lvm] SAN setup with host mirrored disks

[linux-lvm] SAN setup with host mirrored disks

[Fiederling, Daniel]

[-- Attachment #1: Type: text/plain, Size: 1073 bytes --]

Hi,

I'm relatively new to SAN setup with linux boxes, so maybe this is a
very easy question. Today I set up a linux box with two qlogic hbas,
each one is connected to a EMC Clarion. So I got two disks (/dev/sda and
/dev/sdb) from my storages that I have to mirror on the host system.
There is on mirror set up in the san environment!

How can I set up mirroring of the two disks if I want to use lvm for the
ease of later disk expansion? If I use raidtools (md-stuff) will it be
possible to expand the two disks and the logical disk without data loss?
What are the needed steps if I have to increase the disk space? First
expand the md devices through raidreconf and then expand the volume
group and file system? As far as I read the man pages raidreconf only
works with raid 0 and 5. It seems to me as if I'm not on the right
lane...

Is it possible to use lvm for mirroring physical volumes? If this is
possible I expect that this will be the better solution.

Hope someone can tell me about his experiences with such an scenario.

Ciao
Daniel


[-- Attachment #2: Type: text/html, Size: 1686 bytes --]

Re: [linux-lvm] SAN setup with host mirrored disks

[Luca Berra]

On Thu, Jul 13, 2006 at 12:04:01AM +0200, Fiederling, Daniel wrote:
>Hi,
>
>I'm relatively new to SAN setup with linux boxes, so maybe this is a
>very easy question. Today I set up a linux box with two qlogic hbas,
>each one is connected to a EMC Clarion. So I got two disks (/dev/sda and
>/dev/sdb) from my storages that I have to mirror on the host system.
>There is on mirror set up in the san environment!
>
>How can I set up mirroring of the two disks if I want to use lvm for the
>ease of later disk expansion? If I use raidtools (md-stuff) will it be
do not use raidtools, it is an obsolete tool. use mdadm if you want md.

>possible to expand the two disks and the logical disk without data loss?
yes

>What are the needed steps if I have to increase the disk space? First
>expand the md devices through raidreconf and then expand the volume

you can use a recent mdadm with the --grow option instead of raidreconf

another option is to create and additional lun instead of enlarging the
current one, lvm will handle a vg on multiple pvs just fine.

>Is it possible to use lvm for mirroring physical volumes? If this is
>possible I expect that this will be the better solution.

recent lvm2 support mirroring of logical volumes, so you could use this
instead.
it is a relatively new feature and i did not test it personally, so
maybe someone else on the list can give you more feedback.

Regards,
L.


-- 
Luca Berra -- bluca@comedia.it
        Communication Media & Services S.r.l.
 /"\
 \ /     ASCII RIBBON CAMPAIGN
  X        AGAINST HTML MAIL
 / \

Re: [linux-lvm] SAN setup with host mirrored disks

[Michael Loftis]

--On July 13, 2006 12:04:01 AM +0200 "Fiederling, Daniel" 
<daniel.fiederling@warema.de> wrote:

>
> Hi,
>
> I'm relatively new to SAN setup with linux boxes, so maybe this is a very
> easy question. Today I set up a linux box with two qlogic hbas, each one
> is connected to a EMC Clarion. So I got two disks (/dev/sda and /dev/sdb)
> from my storages that I have to mirror on the host system. There is on
> mirror set up in the san environment!

Uhhhmm...Clariion Can and does do RAID 0,1,5, as well as any combination of 
the three.  Depending on the model of Clariion and your specific setup (esp 
depending on if you're using powerpath or not) you'll see one or two 
'copies' of each LUN.

In fact I'd highly recomment using the Clariion to do all of your real RAID 
stuff, and if you can't use powerpath, then use md *just* for failover. 
You do not need MD for actual RAID with a Clariion.  Navisphere will allow 
you to configure mirrors of sets of drives.

>
> How can I set up mirroring of the two disks if I want to use lvm for the
> ease of later disk expansion? If I use raidtools (md-stuff) will it be
> possible to expand the two disks and the logical disk without data loss?
> What are the needed steps if I have to increase the disk space? First
> expand the md devices through raidreconf and then expand the volume group
> and file system? As far as I read the man pages raidreconf only works
> with raid 0 and 5. It seems to me as if I'm not on the right lane...
>
> Is it possible to use lvm for mirroring physical volumes? If this is
> possible I expect that this will be the better solution.
>
> Hope someone can tell me about his experiences with such an scenario.
>
> Ciao
> Daniel



--
"Genius might be described as a supreme capacity for getting its possessors
into trouble of all kinds."
-- Samuel Butler

Re: [linux-lvm] SAN setup with host mirrored disks

[Kelly Sauke]

Michael Loftis wrote:
>
>
> --On July 13, 2006 12:04:01 AM +0200 "Fiederling, Daniel" 
> <daniel.fiederling@warema.de> wrote:
>
>>
>> Hi,
>>
>> I'm relatively new to SAN setup with linux boxes, so maybe this is a 
>> very
>> easy question. Today I set up a linux box with two qlogic hbas, each one
>> is connected to a EMC Clarion. So I got two disks (/dev/sda and 
>> /dev/sdb)
>> from my storages that I have to mirror on the host system. There is on
>> mirror set up in the san environment!
>
> Uhhhmm...Clariion Can and does do RAID 0,1,5, as well as any 
> combination of the three.  Depending on the model of Clariion and your 
> specific setup (esp depending on if you're using powerpath or not) 
> you'll see one or two 'copies' of each LUN.
>
> In fact I'd highly recomment using the Clariion to do all of your real 
> RAID stuff, and if you can't use powerpath, then use md *just* for 
> failover. You do not need MD for actual RAID with a Clariion.  
> Navisphere will allow you to configure mirrors of sets of drives.
>
>
Do the Raiding on the Clariion.  Then use either PowerPath or mpio (on 
RHEL 4 U3 or newer) to do your failover.  If you use md for your 
failover layer you may get IO errors on the non-active paths.  You need 
something that is aware of the clariion setup and that would be 
PowerPath or the shipped mpio.


>
>
> -- 
> "Genius might be described as a supreme capacity for getting its 
> possessors
> into trouble of all kinds."
> -- Samuel Butler
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

AW: [linux-lvm] SAN setup with host mirrored disks

[Fiederling, Daniel]

Hi,

thank you all for your answers. I currently test a setup with md for failover.
To clarify: we have two Clariions in two different locations that are used for failover. Of course on each Clariion the disks are set up as raid 5. My question was about setting up a mirror that "copies" the data on each Clariion.

Currently I use OpenSuSE 10.1 as distribution so unfortunately I can't use PowerPath because I only get rpms for sles or rhel that can't be installed. I know it's the wrong place for this question but: had someone success on installing PowerPath on non supported systems?

Kelly, I installed a package called multipath-tools version 0.4.6 which contains multipath and multipathd by Christophe Varoqui. Is that the mpio package you mentioned? Why do I have to expect io errors when using md? What is special about the Clariion setup? I would expect that the san disks are only "normal" scsi disks for linux...

Ciao
Daniel



-----Urspr�ngliche Nachricht-----
Von: linux-lvm-bounces@redhat.com [mailto:linux-lvm-bounces@redhat.com] Im Auftrag von Kelly Sauke
Gesendet: Donnerstag, 13. Juli 2006 20:04
An: LVM general discussion and development
Betreff: Re: [linux-lvm] SAN setup with host mirrored disks

Do the Raiding on the Clariion.  Then use either PowerPath or mpio (on 
RHEL 4 U3 or newer) to do your failover.  If you use md for your 
failover layer you may get IO errors on the non-active paths.  You need 
something that is aware of the clariion setup and that would be 
PowerPath or the shipped mpio.

Re: AW: [linux-lvm] SAN setup with host mirrored disks

[Kelly Sauke]

Fiederling, Daniel wrote:
> Hi,
>
> thank you all for your answers. I currently test a setup with md for failover.
> To clarify: we have two Clariions in two different locations that are used for failover. Of course on each Clariion the disks are set up as raid 5. My question was about setting up a mirror that "copies" the data on each Clariion.
>   
We use either Veritas Volume Manager or EMC MirrorView to accomplish 
this.  We've had no luck in our lab getting mirrored LVM volumes to work 
properly.

> Currently I use OpenSuSE 10.1 as distribution so unfortunately I can't use PowerPath because I only get rpms for sles or rhel that can't be installed. I know it's the wrong place for this question but: had someone success on installing PowerPath on non supported systems?
>   
There used to be a tarball install for PowerPath.  I haven't used it for 
about 3 years now since all we have are RHEL systems and the RPMs work 
fine but they might still be shipping the tarball.
> Kelly, I installed a package called multipath-tools version 0.4.6 which contains multipath and multipathd by Christophe Varoqui. Is that the mpio package you mentioned? 
Yes, that is the package that I was talking about.

> Why do I have to expect io errors when using md? What is special about the Clariion setup? I would expect that the san disks are only "normal" scsi disks for linux...
>   
The difference comes in how the multiple paths to the same disk are 
handled under linux.  If you only have one path to each disk, then this 
doesn't apply to you. I guess I forgot to clarify if you had more than 
one path.   Anyway, if you have multiple paths mpio knows which is the 
Active path and which is the Passive path.  Sending IO down a passive 
path will result in IO errors.

> Ciao
> Daniel
>
>
>
> -----Urspr�ngliche Nachricht-----
> Von: linux-lvm-bounces@redhat.com [mailto:linux-lvm-bounces@redhat.com] Im Auftrag von Kelly Sauke
> Gesendet: Donnerstag, 13. Juli 2006 20:04
> An: LVM general discussion and development
> Betreff: Re: [linux-lvm] SAN setup with host mirrored disks
>
> Do the Raiding on the Clariion.  Then use either PowerPath or mpio (on 
> RHEL 4 U3 or newer) to do your failover.  If you use md for your 
> failover layer you may get IO errors on the non-active paths.  You need 
> something that is aware of the clariion setup and that would be 
> PowerPath or the shipped mpio.
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>   

AW: AW: [linux-lvm] SAN setup with host mirrored disks

[Fiederling, Daniel]

Currently I have only one path to each disk but further on I'd like to test a setup with multiple paths. But am I right that I have to use md on top of multipathd for getting the two disks mirrored?
I tested different qla2xxx drivers (SuSE shipped, origanal download packages from qlogic, packages from hp) and the non suse offered much more module parameters - one of them was about en-/disabling failover. I would guess that the qla2xxx driver or the card firmware can handle multiple paths. But I'm not sure about that...


-----Urspr�ngliche Nachricht-----
Von: linux-lvm-bounces@redhat.com [mailto:linux-lvm-bounces@redhat.com] Im Auftrag von Kelly Sauke
Gesendet: Montag, 17. Juli 2006 15:36
An: LVM general discussion and development
Betreff: Re: AW: [linux-lvm] SAN setup with host mirrored disks

> Why do I have to expect io errors when using md? What is special about the Clariion setup? I would expect that the san disks are only "normal" scsi disks for linux...
>   
The difference comes in how the multiple paths to the same disk are 
handled under linux.  If you only have one path to each disk, then this 
doesn't apply to you. I guess I forgot to clarify if you had more than 
one path.   Anyway, if you have multiple paths mpio knows which is the 
Active path and which is the Passive path.  Sending IO down a passive 
path will result in IO errors.

Re: AW: AW: [linux-lvm] SAN setup with host mirrored disks

[Kelly Sauke]

Fiederling, Daniel wrote:
> Currently I have only one path to each disk but further on I'd like to test a setup with multiple paths. But am I right that I have to use md on top of multipathd for getting the two disks mirrored?
>   
Yes, multipathd only handles the multiple paths on a per disk basis.  If 
you need them mirrored you have to use something on top of the multipath 
layer.  Using md is one option that should work just fine. 

> I tested different qla2xxx drivers (SuSE shipped, origanal download packages from qlogic, packages from hp) and the non suse offered much more module parameters - one of them was about en-/disabling failover. I would guess that the qla2xxx driver or the card firmware can handle multiple paths. But I'm not sure about that...
>
>
>   
I've yet to meet an HBA that doesn't work with multiple paths.   I would 
check EMC's support matrix for your particular card.  I know that we 
have to run specific EMC firmware on our HBAs to be fully supported.  I 
think by running the non EMC firmware you lose some of the functionality 
and may run into some stability issues.  I can't comment on that further 
since we've never done any testing with non-EMC firmware.

> -----Urspr�ngliche Nachricht-----
> Von: linux-lvm-bounces@redhat.com [mailto:linux-lvm-bounces@redhat.com] Im Auftrag von Kelly Sauke
> Gesendet: Montag, 17. Juli 2006 15:36
> An: LVM general discussion and development
> Betreff: Re: AW: [linux-lvm] SAN setup with host mirrored disks
>
>   
>> Why do I have to expect io errors when using md? What is special about the Clariion setup? I would expect that the san disks are only "normal" scsi disks for linux...
>>   
>>     
> The difference comes in how the multiple paths to the same disk are 
> handled under linux.  If you only have one path to each disk, then this 
> doesn't apply to you. I guess I forgot to clarify if you had more than 
> one path.   Anyway, if you have multiple paths mpio knows which is the 
> Active path and which is the Passive path.  Sending IO down a passive 
> path will result in IO errors.
>
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>   

Re: AW: [linux-lvm] SAN setup with host mirrored disks

[Michael Loftis]

--On July 17, 2006 8:11:17 AM +0200 "Fiederling, Daniel" 
<daniel.fiederling@warema.de> wrote:

> Hi,
>
> thank you all for your answers. I currently test a setup with md for
> failover. To clarify: we have two Clariions in two different locations
> that are used for failover. Of course on each Clariion the disks are set
> up as raid 5. My question was about setting up a mirror that "copies" the
> data on each Clariion.
>
> Currently I use OpenSuSE 10.1 as distribution so unfortunately I can't
> use PowerPath because I only get rpms for sles or rhel that can't be
> installed. I know it's the wrong place for this question but: had someone
> success on installing PowerPath on non supported systems?

Yeah, don't bother.  It's a kernel module.  No source.  I've tried to get 
them to do the sane thing (wrapper .c with binary lib, like nvidia) so they 
could support other than the specific binary kernels in RedHat but they 
either refused or didn't get it.  When they say Linux they don't mean 
Linux, they mean RedHat.  YMMV but for us RedHat has been nothing but weird 
problems.  Unless you're willing to install and run a RedHat kernel you're 
SOL.

>
> Kelly, I installed a package called multipath-tools version 0.4.6 which
> contains multipath and multipathd by Christophe Varoqui. Is that the mpio
> package you mentioned? Why do I have to expect io errors when using md?
> What is special about the Clariion setup? I would expect that the san
> disks are only "normal" scsi disks for linux...
>
> Ciao
> Daniel
>
>
>
> -----Ursprüngliche Nachricht-----
> Von: linux-lvm-bounces@redhat.com [mailto:linux-lvm-bounces@redhat.com]
> Im Auftrag von Kelly Sauke Gesendet: Donnerstag, 13. Juli 2006 20:04
> An: LVM general discussion and development
> Betreff: Re: [linux-lvm] SAN setup with host mirrored disks
>
> Do the Raiding on the Clariion.  Then use either PowerPath or mpio (on
> RHEL 4 U3 or newer) to do your failover.  If you use md for your
> failover layer you may get IO errors on the non-active paths.  You need
> something that is aware of the clariion setup and that would be
> PowerPath or the shipped mpio.
>
> _______________________________________________
> linux-lvm mailing list
> linux-lvm@redhat.com
> https://www.redhat.com/mailman/listinfo/linux-lvm
> read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/
>



--
"Genius might be described as a supreme capacity for getting its possessors
into trouble of all kinds."
-- Samuel Butler