* DM-MP, Xen 5.0.0 and NetApp
@ 2009-04-01 14:31 Rodrigo Nascimento
2009-04-01 14:36 ` Hannes Reinecke
0 siblings, 1 reply; 4+ messages in thread
From: Rodrigo Nascimento @ 2009-04-01 14:31 UTC (permalink / raw)
To: device-mapper development
[-- Attachment #1.1: Type: text/plain, Size: 5191 bytes --]
Hi All,
I'm having problems to put the Multipathing tools to work correctly with
NetApp Cluster Solution.
I have a xenpool with 3 machines the multipathing are enabled on them, the
HBA are Qlogic, two FAS3140 controllers in Cluster. I created a lun to be a
storage repository (SR), the lun is visible and was recognized on the 3
machines, but when we say to the XenServer to build a SR with this Lun, all
I/O operations arrive to the FAS3140 by the non-optimized path.
*XenServer multipath.conf*
defaults {
user_friendly_names no
}
devices {
device {
vendor "NETAPP"
product "LUN"
path_grouping_policy group_by_prio
getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
prio_callout "/sbin/mpath_prio_netapp /dev/%n"
features "1 queue_if_no_path"
path_checker directio
failback immediate
}
}
We made two tests, one with prio_callout as /sbin/mpath_prio_alua and other
with prio_callout as /sbin/mpath_prio_netapp, both returns the same results.
*Lun Stats from the Storage Controller*
Read Write Other QFull Read Write Average Queue Partner Lun
Ops Ops Ops kB kB Latency Length Ops kB
0 0 0 0 0 0 0.00 0.00 0 0
/vol/vol_cmprh01/cmprh01_disk01.lun
---
0 0 0 0 0 0 0.00 0.00 0 0
/vol/vol_cmprh01/cmprh01_disk01.lun
---
0 0 0 0 0 0 0.00 0.00 0 0
/vol/vol_cmprh01/cmprh01_disk01.lun
---
10 0 41 0 5 0 0.37 5.04 23 3
/vol/vol_cmprh01/cmprh01_disk01.lun
---
0 0 0 0 0 0 0.00 0.00 0 0
/vol/vol_cmprh01/cmprh01_disk01.lun
---
0 0 0 0 0 0 0.00 0.00 0 0
/vol/vol_cmprh01/cmprh01_disk01.lun
---
0 0 0 0 0 0 0.00 0.00 0 0
/vol/vol_cmprh01/cmprh01_disk01.lun
---
0 0 0 0 0 0 0.00 0.00 0 0
/vol/vol_cmprh01/cmprh01_disk01.lun
---
7 0 10 0 3 0 0.52 5.01 5 1
/vol/vol_cmprh01/cmprh01_disk01.lun
---
7 0 11 0 28 0 0.44 0.08 6 12
/vol/vol_cmprh01/cmprh01_disk01.lun
---
0 0 0 0 0 0 0.00 0.00 0 0
/vol/vol_cmprh01/cmprh01_disk01.lun
When we have an I/O operation, more than 50% arrives to the storage
controller by the Partner interface.
*igroup configuration*
NETAPP314001> igroup show -v
xenpool_ntap01 (FCP):
OS Type: linux
Member: 50:01:43:80:03:b9:36:f8 (logged in on: vtic, 0c, 0a)
Member: 50:01:43:80:03:b9:36:fa (logged in on: 0d, 0b, vtic)
Member: 50:01:43:80:03:b9:36:d8 (logged in on: 0c, 0a, vtic)
Member: 50:01:43:80:03:b9:36:da (logged in on: 0d, 0b, vtic)
Member: 50:01:43:80:03:b9:36:d4 (logged in on: 0c, 0a, vtic)
Member: 50:01:43:80:03:b9:36:d6 (logged in on: 0d, 0b, vtic)
Member: 50:01:43:80:03:b9:37:54 (logged in on: 0c, 0a, vtic)
Member: 50:01:43:80:03:b9:37:56 (logged in on: 0d, 0b, vtic)
Member: 50:01:43:80:03:b9:37:58 (logged in on: 0c, 0a, vtic)
Member: 50:01:43:80:03:b9:37:5a (logged in on: 0d, 0b, vtic)
Member: 50:01:43:80:03:b9:37:10 (logged in on: 0c, 0a, vtic)
Member: 50:01:43:80:03:b9:37:12 (logged in on: 0d, 0b, vtic)
Member: 50:01:43:80:03:b9:37:20 (logged in on: 0c, 0a, vtic)
Member: 50:01:43:80:03:b9:37:22 (logged in on: 0d, 0b, vtic)
Member: 50:01:43:80:03:b9:37:0c (logged in on: vtic, 0c, 0a)
Member: 50:01:43:80:03:b9:37:0e (logged in on: 0d, 0b, vtic)
Member: 50:01:43:80:03:b9:79:b8 (logged in on: 0c, 0a, vtic)
Member: 50:01:43:80:03:b9:79:ba (logged in on: 0d, 0b, vtic)
Member: 50:01:43:80:03:b9:35:54 (logged in on: vtic, 0c, 0a)
Member: 50:01:43:80:03:b9:35:56 (logged in on: 0d, 0b, vtic)
Member: 50:01:43:80:03:ba:0c:c8 (logged in on: vtic, 0c, 0a)
Member: 50:01:43:80:03:ba:0c:ca (logged in on: 0d, 0b, vtic)
Member: 50:01:43:80:03:b9:35:50 (logged in on: 0c, 0a, vtic)
Member: 50:01:43:80:03:b9:35:52 (logged in on: 0d, 0b, vtic)
Member: 50:01:43:80:03:ba:0c:d0 (logged in on: vtic, 0c, 0a)
Member: 50:01:43:80:03:ba:0c:d2 (logged in on: 0d, 0b, vtic)
Member: 50:01:43:80:03:b9:79:5c (logged in on: 0c, 0a, vtic)
Member: 50:01:43:80:03:b9:79:5e (logged in on: 0d, 0b, vtic)
Member: 50:01:43:80:03:ba:0c:dc (logged in on: 0c, 0a, vtic)
Member: 50:01:43:80:03:ba:0c:de (logged in on: 0d, 0b, vtic)
Member: 50:01:43:80:03:b9:35:28 (logged in on: vtic, 0c, 0a)
Member: 50:01:43:80:03:b9:35:2a (logged in on: 0d, 0b, vtic)
ALUA: Yes
The ALUA was configured at the igroup.
We made another test putting a Windows machine with NetApp DSM(Multipathing
software) and everything works fine.
Anyone could help me?
Rgds,
--
Rodrigo Nascimento
[-- Attachment #1.2: Type: text/html, Size: 7225 bytes --]
[-- Attachment #2: Type: text/plain, Size: 0 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: DM-MP, Xen 5.0.0 and NetApp
2009-04-01 14:31 DM-MP, Xen 5.0.0 and NetApp Rodrigo Nascimento
@ 2009-04-01 14:36 ` Hannes Reinecke
2009-04-01 14:45 ` Rodrigo Nascimento
0 siblings, 1 reply; 4+ messages in thread
From: Hannes Reinecke @ 2009-04-01 14:36 UTC (permalink / raw)
To: device-mapper development
Rodrigo Nascimento wrote:
> Hi All,
>
> I'm having problems to put the Multipathing tools to work correctly with
> NetApp Cluster Solution.
>
> I have a xenpool with 3 machines the multipathing are enabled on them,
> the HBA are Qlogic, two FAS3140 controllers in Cluster. I created a lun
> to be a storage repository (SR), the lun is visible and was recognized
> on the 3 machines, but when we say to the XenServer to build a SR with
> this Lun, all I/O operations arrive to the FAS3140 by the non-optimized
> path.
>
> *XenServer multipath.conf*
>
> defaults {
> user_friendly_names no
> }
> devices {
> device {
> vendor "NETAPP"
> product "LUN"
> path_grouping_policy group_by_prio
> getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
> prio_callout "/sbin/mpath_prio_netapp /dev/%n"
> features "1 queue_if_no_path"
> path_checker directio
> failback immediate
> }
> }
>
> We made two tests, one with prio_callout as /sbin/mpath_prio_alua and
> other with prio_callout as /sbin/mpath_prio_netapp, both returns the
> same results.
>
What's the output of 'multipath -ll'?
Looks like all paths are configured into one group here.
Cheers,
Hannes
--
Dr. Hannes Reinecke zSeries & Storage
hare@suse.de +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Markus Rex, HRB 16746 (AG Nürnberg)
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: DM-MP, Xen 5.0.0 and NetApp
2009-04-01 14:36 ` Hannes Reinecke
@ 2009-04-01 14:45 ` Rodrigo Nascimento
2009-04-02 18:19 ` James Tanner
0 siblings, 1 reply; 4+ messages in thread
From: Rodrigo Nascimento @ 2009-04-01 14:45 UTC (permalink / raw)
To: device-mapper development
[-- Attachment #1.1: Type: text/plain, Size: 2918 bytes --]
multipath -ll output:
[root@CMPXEN02 ~]# multipath -ll
360a98000572d42774e6f4f2f682f6330dm-0 NETAPP,LUN
[size=20G][features=1 queue_if_no_path][hwhandler=0]
\_ round-robin 0 [prio=32][active]
\_ 0:0:0:0 sda 8:0 [active][ready]
\_ 0:0:1:0 sdb 8:16 [active][ready]
\_ 1:0:0:0 sdd 8:48 [active][ready]
\_ 1:0:1:0 sde 8:64 [active][ready]
\_ 2:0:0:0 sdh 8:112 [active][ready]
\_ 3:0:0:0 sdi 8:128 [active][ready]
\_ 3:0:1:0 sdj 8:144 [active][ready]
\_ 2:0:1:0 sdk 8:160 [active][ready]
\_ round-robin 0 [prio=7][enabled]
\_ 0:0:2:0 sdc 8:32 [active][ready]
\_ 1:0:2:0 sdf 8:80 [active][ready]
\_ 1:0:3:0 sdg 8:96 [active][ready]
\_ 3:0:2:0 sdl 8:176 [active][ready]
\_ 3:0:3:0 sdn 8:208 [active][ready]
\_ 0:0:3:0 sdo 8:224 [active][ready]
\_ 2:0:3:0 sdp 8:240 [active][ready]
\_ round-robin 0 [prio=0][enabled]
\_ 2:0:2:0 sdm 8:192 [failed][faulty]
On Wed, Apr 1, 2009 at 11:36 AM, Hannes Reinecke <hare@suse.de> wrote:
> Rodrigo Nascimento wrote:
>
>> Hi All,
>> I'm having problems to put the Multipathing tools to work correctly with
>> NetApp Cluster Solution.
>> I have a xenpool with 3 machines the multipathing are enabled on them,
>> the HBA are Qlogic, two FAS3140 controllers in Cluster. I created a lun to
>> be a storage repository (SR), the lun is visible and was recognized on the 3
>> machines, but when we say to the XenServer to build a SR with this Lun, all
>> I/O operations arrive to the FAS3140 by the non-optimized path.
>> *XenServer multipath.conf*
>> defaults {
>> user_friendly_names no
>> }
>> devices {
>> device {
>> vendor "NETAPP"
>> product "LUN"
>> path_grouping_policy group_by_prio
>> getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
>> prio_callout "/sbin/mpath_prio_netapp /dev/%n"
>> features "1 queue_if_no_path"
>> path_checker directio
>> failback immediate
>> }
>> }
>> We made two tests, one with prio_callout as /sbin/mpath_prio_alua and
>> other with prio_callout as /sbin/mpath_prio_netapp, both returns the same
>> results.
>>
>>
>
> What's the output of 'multipath -ll'?
> Looks like all paths are configured into one group here.
>
> Cheers,
>
> Hannes
> --
> Dr. Hannes Reinecke zSeries & Storage
> hare@suse.de +49 911 74053 688
> SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
> GF: Markus Rex, HRB 16746 (AG Nürnberg)
>
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
>
--
Rodrigo Nascimento
[-- Attachment #1.2: Type: text/html, Size: 3785 bytes --]
[-- Attachment #2: Type: text/plain, Size: 0 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread* Re: DM-MP, Xen 5.0.0 and NetApp
2009-04-01 14:45 ` Rodrigo Nascimento
@ 2009-04-02 18:19 ` James Tanner
0 siblings, 0 replies; 4+ messages in thread
From: James Tanner @ 2009-04-02 18:19 UTC (permalink / raw)
To: device-mapper development
Netapp support has a host setup guide which covers the settings you
should be using in multipath.conf:
defaults
{
user_friendly_names yes
max_fds 4096
}
blacklist
{
wwid <DevId>
devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
devnode "^hd[a-z]"
devnode "^cciss!c[0-9]d[0-9]*[p[0-9]*]"
}
devices
{
device
{
vendor "NETAPP"
product "LUN"
getuid_callout "/sbin/scsi_id -g -u -s /block/%n"
prio_callout "/sbin/mpath_prio_ontap /dev/%n"
features "1 queue_if_no_path"
hardware_handler "0"
path_grouping_policy group_by_prio
failback immediate
rr_weight uniform
rr_min_io 128
path_checker directio
}
}
Try those out and see if it helps.
Rodrigo Nascimento wrote:
> multipath -ll output:
>
> [root@CMPXEN02 ~]# multipath -ll
> 360a98000572d42774e6f4f2f682f6330dm-0 NETAPP,LUN
> [size=20G][features=1 queue_if_no_path][hwhandler=0]
> \_ round-robin 0 [prio=32][active]
> \_ 0:0:0:0 sda 8:0 [active][ready]
> \_ 0:0:1:0 sdb 8:16 [active][ready]
> \_ 1:0:0:0 sdd 8:48 [active][ready]
> \_ 1:0:1:0 sde 8:64 [active][ready]
> \_ 2:0:0:0 sdh 8:112 [active][ready]
> \_ 3:0:0:0 sdi 8:128 [active][ready]
> \_ 3:0:1:0 sdj 8:144 [active][ready]
> \_ 2:0:1:0 sdk 8:160 [active][ready]
> \_ round-robin 0 [prio=7][enabled]
> \_ 0:0:2:0 sdc 8:32 [active][ready]
> \_ 1:0:2:0 sdf 8:80 [active][ready]
> \_ 1:0:3:0 sdg 8:96 [active][ready]
> \_ 3:0:2:0 sdl 8:176 [active][ready]
> \_ 3:0:3:0 sdn 8:208 [active][ready]
> \_ 0:0:3:0 sdo 8:224 [active][ready]
> \_ 2:0:3:0 sdp 8:240 [active][ready]
> \_ round-robin 0 [prio=0][enabled]
> \_ 2:0:2:0 sdm 8:192 [failed][faulty]
>
>
> On Wed, Apr 1, 2009 at 11:36 AM, Hannes Reinecke <hare@suse.de
> <mailto:hare@suse.de>> wrote:
>
> Rodrigo Nascimento wrote:
>
> Hi All,
> I'm having problems to put the Multipathing tools to work
> correctly with NetApp Cluster Solution.
> I have a xenpool with 3 machines the multipathing are enabled
> on them, the HBA are Qlogic, two FAS3140 controllers in
> Cluster. I created a lun to be a storage repository (SR), the
> lun is visible and was recognized on the 3 machines, but when
> we say to the XenServer to build a SR with this Lun, all I/O
> operations arrive to the FAS3140 by the non-optimized path.
> *XenServer multipath.conf*
> defaults {
> user_friendly_names no
> }
> devices {
> device {
> vendor "NETAPP"
> product "LUN"
> path_grouping_policy group_by_prio
> getuid_callout "/sbin/scsi_id -g -u -s
> /block/%n"
> prio_callout
> "/sbin/mpath_prio_netapp /dev/%n"
> features "1 queue_if_no_path"
> path_checker directio
> failback immediate
> }
> }
> We made two tests, one with prio_callout as
> /sbin/mpath_prio_alua and other with prio_callout as
> /sbin/mpath_prio_netapp, both returns the same results.
>
>
>
> What's the output of 'multipath -ll'?
> Looks like all paths are configured into one group here.
>
> Cheers,
>
> Hannes
> --
> Dr. Hannes Reinecke zSeries & Storage
> hare@suse.de <mailto:hare@suse.de> +49
> 911 74053 688
> SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
> GF: Markus Rex, HRB 16746 (AG Nürnberg)
>
> --
> dm-devel mailing list
> dm-devel@redhat.com <mailto:dm-devel@redhat.com>
> https://www.redhat.com/mailman/listinfo/dm-devel
>
>
>
>
> --
> Rodrigo Nascimento
> ------------------------------------------------------------------------
>
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2009-04-02 18:19 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2009-04-01 14:31 DM-MP, Xen 5.0.0 and NetApp Rodrigo Nascimento
2009-04-01 14:36 ` Hannes Reinecke
2009-04-01 14:45 ` Rodrigo Nascimento
2009-04-02 18:19 ` James Tanner
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.