All of lore.kernel.org
 help / color / mirror / Atom feed
* [linux-lvm] How to associate two paths for each EMC device?
@ 2004-03-19 17:21 Shan Jing
  2004-03-19 17:44 ` Theo Van Dinter
  0 siblings, 1 reply; 7+ messages in thread
From: Shan Jing @ 2004-03-19 17:21 UTC (permalink / raw)
  To: LVM general discussion and development; +Cc: linux-lvm-bounces


I have 22  EMC symmetrix disks on two HBA links.  At this moment, there is no working copy of EMC powerpath or HP Securepath available for my kernel (2.4.21-112-itanium2-smp); therefore, I have to use the luns directly with lvm without any DMP software in the middle.

Without DMP software, there are two set of 22 luns, each representing one HBA path.  See blow.

Question,
What is the best way to associate two paths for each real disk?

eg, I have the following 44 luns, what's the best way to identiy the pairs that represent the two paths of the same disk?  Thanks!

lvmdiskscan -- /dev/sdag   [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdah   [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdai   [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdaj   [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdak   [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdal1  [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdam1  [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdan   [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdao   [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdap   [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdaq   [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdar   [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdas   [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdat   [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdau1  [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdq    [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdr    [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sds    [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdt    [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdu    [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdv    [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdw    [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdx1   [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdz    [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdaa   [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdab   [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdac   [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdad   [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdae   [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdaf   [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdc    [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdd    [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sde    [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdf    [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdg    [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdh    [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdi    [      42.15 GB] free whole disk
lvmdiskscan -- /dev/sdj1   [      42.15 GB] free whole disk


Regards,
Shan Jing

Unix Group - Americas
Deutsche Bank
2 Gatehall Drive, 3rd Floor
Parsippany, New Jersey, USA
shan.jing@db.com
973.682.3463




--

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

^ permalink raw reply	[flat|nested] 7+ messages in thread
* Re: [linux-lvm] How to associate two paths for each EMC device?
@ 2004-03-19 18:50 Shan Jing
  0 siblings, 0 replies; 7+ messages in thread
From: Shan Jing @ 2004-03-19 18:50 UTC (permalink / raw)
  To: LVM general discussion and development; +Cc: linux-lvm-bounces


I remove the driver,reloaded manaully like this:

insmod qla2300 ql2xfailover=1 ql2xmaxqdepth=64

I still have 44 luns (each disk presents twice on the system).

this is my qla driver
-------------------------------------
QLogic PCI to Fibre Channel Host Adapter for QLA2340:
        Firmware version:  3.02.13, Driver version 6.06.00-fo
Entry address = a000000000793770
HBA: QLA2312 , Serial# L08747
Request Queue = 0x1af3c000, Response Queue = 0x1ac50000
Request Queue count= 128, Response Queue count= 512
Total number of active commands = 0
Total number of interrupts = 257
Total number of IOCBs (used/max) = (0/600)
Total number of queued commands = 0
    Device queue depth = 0x40
Number of free request entries = 121
Number of mailbox timeouts = 0
Number of ISP aborts = 0
Number of loop resyncs = 0
Number of retries for empty slots = 0
Number of reqs in pending_q= 0, retry_q= 0, done_q= 0, scsi_retry_q= 0
Number of reqs in failover_q= 0
Host adapter:loop state= <READY>, flags= 0xc008e0833
Dpc flags = 0x40
MBX flags = 0x0
SRB Free Count = 4096
Link down Timeout = 000
Port down retry = 045
Login retry count = 045
Commands retried with dropped frame(s) = 0


SCSI Device Information:
scsi-qla0-adapter-node=200000e08b100beb;
scsi-qla0-adapter-port=210000e08b100beb;
scsi-qla0-port-0=50060482c4648c2c:50060482c4648c2c;

SCSI LUN Information:
(Id:Lun)  * - indicates lun is not registered with the OS.
( 0: 0): Total reqs 13, Pending reqs 0, flags 0x0, 0:0:81,
( 0: 1): Total reqs 12, Pending reqs 0, flags 0x0, 0:0:81,
( 0: 2): Total reqs 11, Pending reqs 0, flags 0x0, 0:0:81,
( 0: 3): Total reqs 11, Pending reqs 0, flags 0x0, 0:0:81,
( 0: 4): Total reqs 11, Pending reqs 0, flags 0x0, 0:0:81,
( 0: 5): Total reqs 11, Pending reqs 0, flags 0x0, 0:0:81,
( 0: 6): Total reqs 11, Pending reqs 0, flags 0x0, 0:0:81,
( 0: 7): Total reqs 11, Pending reqs 0, flags 0x0, 0:0:81,
( 0: 8): Total reqs 11, Pending reqs 0, flags 0x0, 0:0:81,
( 0: 9): Total reqs 11, Pending reqs 0, flags 0x0, 0:0:81,
( 0:10): Total reqs 11, Pending reqs 0, flags 0x0, 0:0:81,
( 0:11): Total reqs 11, Pending reqs 0, flags 0x0, 0:0:81,
( 0:12): Total reqs 11, Pending reqs 0, flags 0x0, 0:0:81,
( 0:13): Total reqs 11, Pending reqs 0, flags 0x0, 0:0:81,
( 0:14): Total reqs 11, Pending reqs 0, flags 0x0, 0:0:81,
( 0:15): Total reqs 11, Pending reqs 0, flags 0x0, 0:0:81,
( 0:16): Total reqs 11, Pending reqs 0, flags 0x0, 0:0:81,
( 0:17): Total reqs 11, Pending reqs 0, flags 0x0, 0:0:81,
( 0:18): Total reqs 11, Pending reqs 0, flags 0x0, 0:0:81,
( 0:19): Total reqs 11, Pending reqs 0, flags 0x0, 0:0:81,
( 0:20): Total reqs 11, Pending reqs 0, flags 0x0, 0:0:81,
( 0:21): Total reqs 11, Pending reqs 0, flags 0x0, 0:0:81,
( 0:22): Total reqs 11, Pending reqs 0, flags 0x0, 0:0:81,


Regards,
Shan Jing

Unix Group - Americas
Deutsche Bank
2 Gatehall Drive, 3rd Floor
Parsippany, New Jersey, USA
shan.jing@db.com
973.682.3463




                                                                                                                                       
                      Theo Van Dinter                                                                                                  
                      <felicity@kluge.n        To:       LVM general discussion and development <linux-lvm@redhat.com>                 
                      et>                      cc:                                                                                     
                      Sent by:                 Subject:  Re: [linux-lvm] How to associate two paths for each EMC device?               
                      linux-lvm-bounces                                                                                                
                      @redhat.com                                                                                                      
                                                                                                                                       
                                                                                                                                       
                      03/19/2004 12:44                                                                                                 
                      PM                                                                                                               
                      Please respond to                                                                                                
                      LVM general                                                                                                      
                      discussion and                                                                                                   
                      development                                                                                                      
                                                                                                                                       
                                                                                                                                       




On Fri, Mar 19, 2004 at 12:21:51PM -0500, Shan Jing wrote:
> I have 22  EMC symmetrix disks on two HBA links.  At this moment, there is no working copy of EMC powerpath or HP Securepath available for my kernel (2.4.21-112-itanium2-smp); therefore, I have to use the luns directly with lvm without any DMP software in the middle.
>
> Without DMP software, there are two set of 22 luns, each representing one HBA path.  See blow.

Can you set your HBA driver to do failover?   ie: something like this
in modules.conf:

options qla2300 ql2xfailover=1 ql2xmaxqdepth=64


This works well for my servers connected to a CX-series array. :)   The
driver handles failing between HBAs (active-passive), and only has the
LUN presented once to the system.

--
Randomly Generated Tagline:
"I didn't know Allman was a stand-up comedian ..."
         - Theo about laughter from sendmail tutorial

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

The following file(s) have been deleted by: Shan Jing on 3/19/2004 1:45:07 PM

C.DTF
atthx9wo.dat





--

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

^ permalink raw reply	[flat|nested] 7+ messages in thread
* Re: [linux-lvm] How to associate two paths for each EMC device?
@ 2004-03-19 19:01 Shan Jing
  2004-03-23 15:09 ` Thomas Meller
  0 siblings, 1 reply; 7+ messages in thread
From: Shan Jing @ 2004-03-19 19:01 UTC (permalink / raw)
  To: LVM general discussion and development; +Cc: linux-lvm-bounces


The very basic way to identify the pair is to lvm each of the first 22 luns in a sequence and see which lun in the other 22 set is also lvm'ed.

However, there must be a more technical way to resolve this and eventually make the disk represent only once.  Any guru out there can help?

Regards,
Shan Jing

Unix Group - Americas
Deutsche Bank
2 Gatehall Drive, 3rd Floor
Parsippany, New Jersey, USA
shan.jing@db.com
973.682.3463




                                                                                                                                       
                      Theo Van Dinter                                                                                                  
                      <felicity@kluge.n        To:       LVM general discussion and development <linux-lvm@redhat.com>                 
                      et>                      cc:                                                                                     
                      Sent by:                 Subject:  Re: [linux-lvm] How to associate two paths for each EMC device?               
                      linux-lvm-bounces                                                                                                
                      @redhat.com                                                                                                      
                                                                                                                                       
                                                                                                                                       
                      03/19/2004 12:44                                                                                                 
                      PM                                                                                                               
                      Please respond to                                                                                                
                      LVM general                                                                                                      
                      discussion and                                                                                                   
                      development                                                                                                      
                                                                                                                                       
                                                                                                                                       




On Fri, Mar 19, 2004 at 12:21:51PM -0500, Shan Jing wrote:
> I have 22  EMC symmetrix disks on two HBA links.  At this moment, there is no working copy of EMC powerpath or HP Securepath available for my kernel (2.4.21-112-itanium2-smp); therefore, I have to use the luns directly with lvm without any DMP software in the middle.
>
> Without DMP software, there are two set of 22 luns, each representing one HBA path.  See blow.

Can you set your HBA driver to do failover?   ie: something like this
in modules.conf:

options qla2300 ql2xfailover=1 ql2xmaxqdepth=64


This works well for my servers connected to a CX-series array. :)   The
driver handles failing between HBAs (active-passive), and only has the
LUN presented once to the system.

--
Randomly Generated Tagline:
"I didn't know Allman was a stand-up comedian ..."
         - Theo about laughter from sendmail tutorial

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

The following file(s) have been deleted by: Shan Jing on 3/19/2004 1:58:20 PM

C.DTF
attq5vht.dat





--

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

^ permalink raw reply	[flat|nested] 7+ messages in thread
* RE: [linux-lvm] How to associate two paths for each EMC device?
@ 2004-03-19 20:43 Dan_Caulfield
  0 siblings, 0 replies; 7+ messages in thread
From: Dan_Caulfield @ 2004-03-19 20:43 UTC (permalink / raw)
  To: linux-lvm; +Cc: linux-lvm-bounces

This is normal behavior for multipath devices.  The OS will "see" the 22
luns on all available paths.  The is the way EMC PowerPath, Veritas DMP
and LVM work.  Each of these products rely on the OS detecting all
available paths to the luns.  EMC PowerPath uses the device the
Symmetrix pdev id of each lun to identify multiple paths to a device.
LVM will use the PV UUIDs to identify the multipath devices.  

For EMC arrays, there is a version of inq that will run on Linux.  You
will just need to make enough sg devices for it to recognize all 44
devices.  The inq utility will help you to determine which pdev is
associated to which sd device. See EMC Powerlink Doc emc59180 for all of
the details.


Regards,
Dan

-----Original Message-----
From: linux-lvm-bounces@redhat.com [mailto:linux-lvm-bounces@redhat.com]
On Behalf Of Shan Jing
Sent: Friday, March 19, 2004 1:01 PM
To: LVM general discussion and development
Cc: LVM general discussion and development; linux-lvm-bounces@redhat.com
Subject: Re: [linux-lvm] How to associate two paths for each EMC device?


The very basic way to identify the pair is to lvm each of the first 22
luns in a sequence and see which lun in the other 22 set is also lvm'ed.

However, there must be a more technical way to resolve this and
eventually make the disk represent only once.  Any guru out there can
help?

Regards,
Shan Jing

Unix Group - Americas
Deutsche Bank
2 Gatehall Drive, 3rd Floor
Parsippany, New Jersey, USA
shan.jing@db.com
973.682.3463




 

                      Theo Van Dinter

                      <felicity@kluge.n        To:       LVM general
discussion and development <linux-lvm@redhat.com>                 
                      et>                      cc:

                      Sent by:                 Subject:  Re: [linux-lvm]
How to associate two paths for each EMC device?               
                      linux-lvm-bounces

                      @redhat.com

 

 

                      03/19/2004 12:44

                      PM

                      Please respond to

                      LVM general

                      discussion and

                      development

 

 





On Fri, Mar 19, 2004 at 12:21:51PM -0500, Shan Jing wrote:
> I have 22  EMC symmetrix disks on two HBA links.  At this moment,
there is no working copy of EMC powerpath or HP Securepath available for
my kernel (2.4.21-112-itanium2-smp); therefore, I have to use the luns
directly with lvm without any DMP software in the middle.
>
> Without DMP software, there are two set of 22 luns, each representing
one HBA path.  See blow.

Can you set your HBA driver to do failover?   ie: something like this
in modules.conf:

options qla2300 ql2xfailover=1 ql2xmaxqdepth=64


This works well for my servers connected to a CX-series array. :)   The
driver handles failing between HBAs (active-passive), and only has the
LUN presented once to the system.

--
Randomly Generated Tagline:
"I didn't know Allman was a stand-up comedian ..."
         - Theo about laughter from sendmail tutorial

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

The following file(s) have been deleted by: Shan Jing on 3/19/2004
1:58:20 PM

C.DTF
attq5vht.dat





--

This e-mail may contain confidential and/or privileged information. If
you are not the intended recipient (or have received this e-mail in
error) please notify the sender immediately and destroy this e-mail. Any
unauthorized copying, disclosure or distribution of the material in this
e-mail is strictly forbidden.


_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

^ permalink raw reply	[flat|nested] 7+ messages in thread
* RE: [linux-lvm] How to associate two paths for each EMC device?
@ 2004-03-19 20:54 Shan Jing
  0 siblings, 0 replies; 7+ messages in thread
From: Shan Jing @ 2004-03-19 20:54 UTC (permalink / raw)
  To: LVM general discussion and development; +Cc: linux-lvm-bounces


Theo and Dan,

Thanks for the info.

I've associated the pairs now.


Regards,
Shan Jing

Unix Group - Americas
Deutsche Bank
2 Gatehall Drive, 3rd Floor
Parsippany, New Jersey, USA
shan.jing@db.com
973.682.3463




                                                                                                                                       
                      <Dan_Caulfield@De                                                                                                
                      ll.com>                  To:       <linux-lvm@redhat.com>                                                        
                      Sent by:                 cc:       <linux-lvm-bounces@redhat.com>                                                
                      linux-lvm-bounces        Subject:  RE: [linux-lvm] How to associate two paths for each EMC device?               
                      @redhat.com                                                                                                      
                                                                                                                                       
                                                                                                                                       
                      03/19/2004 03:43                                                                                                 
                      PM                                                                                                               
                      Please respond to                                                                                                
                      LVM general                                                                                                      
                      discussion and                                                                                                   
                      development                                                                                                      
                                                                                                                                       
                                                                                                                                       




This is normal behavior for multipath devices.  The OS will "see" the 22
luns on all available paths.  The is the way EMC PowerPath, Veritas DMP
and LVM work.  Each of these products rely on the OS detecting all
available paths to the luns.  EMC PowerPath uses the device the
Symmetrix pdev id of each lun to identify multiple paths to a device.
LVM will use the PV UUIDs to identify the multipath devices.

For EMC arrays, there is a version of inq that will run on Linux.  You
will just need to make enough sg devices for it to recognize all 44
devices.  The inq utility will help you to determine which pdev is
associated to which sd device. See EMC Powerlink Doc emc59180 for all of
the details.


Regards,
Dan

-----Original Message-----
From: linux-lvm-bounces@redhat.com [mailto:linux-lvm-bounces@redhat.com]
On Behalf Of Shan Jing
Sent: Friday, March 19, 2004 1:01 PM
To: LVM general discussion and development
Cc: LVM general discussion and development; linux-lvm-bounces@redhat.com
Subject: Re: [linux-lvm] How to associate two paths for each EMC device?


The very basic way to identify the pair is to lvm each of the first 22
luns in a sequence and see which lun in the other 22 set is also lvm'ed.

However, there must be a more technical way to resolve this and
eventually make the disk represent only once.  Any guru out there can
help?

Regards,
Shan Jing

Unix Group - Americas
Deutsche Bank
2 Gatehall Drive, 3rd Floor
Parsippany, New Jersey, USA
shan.jing@db.com
973.682.3463






                      Theo Van Dinter

                      <felicity@kluge.n        To:       LVM general
discussion and development <linux-lvm@redhat.com>
                      et>                      cc:

                      Sent by:                 Subject:  Re: [linux-lvm]
How to associate two paths for each EMC device?
                      linux-lvm-bounces

                      @redhat.com





                      03/19/2004 12:44

                      PM

                      Please respond to

                      LVM general

                      discussion and

                      development









On Fri, Mar 19, 2004 at 12:21:51PM -0500, Shan Jing wrote:
> I have 22  EMC symmetrix disks on two HBA links.  At this moment,
there is no working copy of EMC powerpath or HP Securepath available for
my kernel (2.4.21-112-itanium2-smp); therefore, I have to use the luns
directly with lvm without any DMP software in the middle.
>
> Without DMP software, there are two set of 22 luns, each representing
one HBA path.  See blow.

Can you set your HBA driver to do failover?   ie: something like this
in modules.conf:

options qla2300 ql2xfailover=1 ql2xmaxqdepth=64


This works well for my servers connected to a CX-series array. :)   The
driver handles failing between HBAs (active-passive), and only has the
LUN presented once to the system.

--
Randomly Generated Tagline:
"I didn't know Allman was a stand-up comedian ..."
         - Theo about laughter from sendmail tutorial

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

The following file(s) have been deleted by: Shan Jing on 3/19/2004
1:58:20 PM

C.DTF
attq5vht.dat





--

This e-mail may contain confidential and/or privileged information. If
you are not the intended recipient (or have received this e-mail in
error) please notify the sender immediately and destroy this e-mail. Any
unauthorized copying, disclosure or distribution of the material in this
e-mail is strictly forbidden.


_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/

_______________________________________________
linux-lvm mailing list
linux-lvm@redhat.com
https://www.redhat.com/mailman/listinfo/linux-lvm
read the LVM HOW-TO at http://tldp.org/HOWTO/LVM-HOWTO/





--

This e-mail may contain confidential and/or privileged information. If you are not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorized copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2004-03-23 15:10 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2004-03-19 17:21 [linux-lvm] How to associate two paths for each EMC device? Shan Jing
2004-03-19 17:44 ` Theo Van Dinter
  -- strict thread matches above, loose matches on Subject: below --
2004-03-19 18:50 Shan Jing
2004-03-19 19:01 Shan Jing
2004-03-23 15:09 ` Thomas Meller
2004-03-19 20:43 Dan_Caulfield
2004-03-19 20:54 Shan Jing

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.