* Configuring Path Groups
@ 2012-04-07 6:20 Ray Van Dolson
0 siblings, 0 replies; 5+ messages in thread
From: Ray Van Dolson @ 2012-04-07 6:20 UTC (permalink / raw)
To: dm-devel
I am setting up multipathing on RHEL6 via iSCSI to an IBM XIV. My
iSCSI target looks like this:
172.16.10.1:3260,1793 iqn.2005-10.com.xivstorage:003974
172.16.8.22:3260,2048 iqn.2005-10.com.xivstorage:003974
172.16.10.22:3260,2049 iqn.2005-10.com.xivstorage:003974
172.16.8.23:3260,2304 iqn.2005-10.com.xivstorage:003974
172.16.10.23:3260,2305 iqn.2005-10.com.xivstorage:003974
172.16.8.1:3260,1792 iqn.2005-10.com.xivstorage:003974
(This is one LUN). Networks are 172.16.10.0/23 and 172.16.8.0/23, so
as you can see above there are three target IP's on each subnet.
My RHEL6 host has two physical uplinks... one to each subnet. When I
configure multipath, by default it creates one big path group with all
of the devices above within it and round-robins.
mpathb (2001738000f860110) dm-2 IBM,2810XIV
size=48G features='1 queue_if_no_path' hwhandler='0' wp=rw
`-+- policy='queue-length 0' prio=1 status=active
|- 26:0:0:1 sdb 8:16 active ready running
|- 25:0:0:1 sdc 8:32 active ready running
|- 27:0:0:1 sdd 8:48 active ready running
|- 24:0:0:1 sde 8:64 active ready running
|- 23:0:0:1 sdf 8:80 active ready running
`- 22:0:0:1 sdg 8:96 active ready running
This results in less than ideal performance.
Ideally I want to only have one path per subnet active at any given
time, with the other paths only activating if there's a failure.
The only other option I seem to have is to set path_grouping_policy to
failover. This creates six separate path groups of which only one is
active at a time.
I thought perhaps I could do group_by_prio, but am not sure how I can
manually set priorities for my underlying physical iSCSI devices, and
the default "const" method just gives them all priority 1 so they all
end up in the same path group again.
It looks like perhaps I could configure path groups as I want manually
with dmsetup, but I'm sure there's a better way...
As it is I am currently just configuring my iSCSI initiator to log in
to only one target per subnet. This gets me the best performance but I
lose some (automated) redundancy.
Thoughts appreciated.
Ray
^ permalink raw reply [flat|nested] 5+ messages in thread
* Configuring Path Groups
@ 2012-04-07 6:29 Ray Van Dolson
2012-04-07 6:39 ` Ray Van Dolson
0 siblings, 1 reply; 5+ messages in thread
From: Ray Van Dolson @ 2012-04-07 6:29 UTC (permalink / raw)
To: dm-devel@redhat.com
I am setting up multipathing on RHEL6 via iSCSI to an IBM XIV. My
iSCSI target looks like this:
172.16.10.1:3260,1793 iqn.2005-10.com.xivstorage:003974
172.16.8.22:3260,2048 iqn.2005-10.com.xivstorage:003974
172.16.10.22:3260,2049 iqn.2005-10.com.xivstorage:003974
172.16.8.23:3260,2304 iqn.2005-10.com.xivstorage:003974
172.16.10.23:3260,2305 iqn.2005-10.com.xivstorage:003974
172.16.8.1:3260,1792 iqn.2005-10.com.xivstorage:003974
(This is one LUN). Networks are 172.16.10.0/23 and 172.16.8.0/23, so
as you can see above there are three target IP's on each subnet.
My RHEL6 host has two physical uplinks... one to each subnet. When I
configure multipath, by default it creates one big path group with all
of the devices above within it and round-robins.
mpathb (2001738000f860110) dm-2 IBM,2810XIV
size=48G features='1 queue_if_no_path' hwhandler='0' wp=rw
`-+- policy='queue-length 0' prio=1 status=active
|- 26:0:0:1 sdb 8:16 active ready running
|- 25:0:0:1 sdc 8:32 active ready running
|- 27:0:0:1 sdd 8:48 active ready running
|- 24:0:0:1 sde 8:64 active ready running
|- 23:0:0:1 sdf 8:80 active ready running
`- 22:0:0:1 sdg 8:96 active ready running
This results in less than ideal performance.
Ideally I want to only have one path per subnet active at any given
time, with the other paths only activating if there's a failure.
The only other option I seem to have is to set path_grouping_policy to
failover. This creates six separate path groups of which only one is
active at a time.
I thought perhaps I could do group_by_prio, but am not sure how I can
manually set priorities for my underlying physical iSCSI devices, and
the default "const" method just gives them all priority 1 so they all
end up in the same path group again.
It looks like perhaps I could configure path groups as I want manually
with dmsetup, but I'm sure there's a better way...
As it is I am currently just configuring my iSCSI initiator to log in
to only one target per subnet. This gets me the best performance but I
lose some (automated) redundancy.
Thoughts appreciated.
Ray
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Configuring Path Groups
2012-04-07 6:29 Configuring Path Groups Ray Van Dolson
@ 2012-04-07 6:39 ` Ray Van Dolson
2012-04-07 6:52 ` Ray Van Dolson
0 siblings, 1 reply; 5+ messages in thread
From: Ray Van Dolson @ 2012-04-07 6:39 UTC (permalink / raw)
To: dm-devel
On Fri, Apr 06, 2012 at 11:29:47PM -0700, Ray Van Dolson wrote:
> I am setting up multipathing on RHEL6 via iSCSI to an IBM XIV. My
> iSCSI target looks like this:
>
> 172.16.10.1:3260,1793 iqn.2005-10.com.xivstorage:003974
> 172.16.8.22:3260,2048 iqn.2005-10.com.xivstorage:003974
> 172.16.10.22:3260,2049 iqn.2005-10.com.xivstorage:003974
> 172.16.8.23:3260,2304 iqn.2005-10.com.xivstorage:003974
> 172.16.10.23:3260,2305 iqn.2005-10.com.xivstorage:003974
> 172.16.8.1:3260,1792 iqn.2005-10.com.xivstorage:003974
>
> (This is one LUN). Networks are 172.16.10.0/23 and 172.16.8.0/23, so
> as you can see above there are three target IP's on each subnet.
>
> My RHEL6 host has two physical uplinks... one to each subnet. When I
> configure multipath, by default it creates one big path group with all
> of the devices above within it and round-robins.
>
> mpathb (2001738000f860110) dm-2 IBM,2810XIV
> size=48G features='1 queue_if_no_path' hwhandler='0' wp=rw
> `-+- policy='queue-length 0' prio=1 status=active
> |- 26:0:0:1 sdb 8:16 active ready running
> |- 25:0:0:1 sdc 8:32 active ready running
> |- 27:0:0:1 sdd 8:48 active ready running
> |- 24:0:0:1 sde 8:64 active ready running
> |- 23:0:0:1 sdf 8:80 active ready running
> `- 22:0:0:1 sdg 8:96 active ready running
>
> This results in less than ideal performance.
>
> Ideally I want to only have one path per subnet active at any given
> time, with the other paths only activating if there's a failure.
>
> The only other option I seem to have is to set path_grouping_policy to
> failover. This creates six separate path groups of which only one is
> active at a time.
>
> I thought perhaps I could do group_by_prio, but am not sure how I can
> manually set priorities for my underlying physical iSCSI devices, and
> the default "const" method just gives them all priority 1 so they all
> end up in the same path group again.
>
> It looks like perhaps I could configure path groups as I want manually
> with dmsetup, but I'm sure there's a better way...
>
> As it is I am currently just configuring my iSCSI initiator to log in
> to only one target per subnet. This gets me the best performance but I
> lose some (automated) redundancy.
>
> Thoughts appreciated.
>
> Ray
It seems like what I'm after is "prio_callout". However, it doesn't
seem to be documented in the man pages (though it is in one of the
sample configurations).
I tried setting path_grouping_policy to group_by_prio and then defined
prio_callout to point to a bash script on my system that echo's "2".
This doesn't seem to actually get called though.
Actually, after doing a bit more reading it sounds like prio_callout
was replaced by "prio" in RHEL6's version of the tool. "prio" doesn't
appear to have an option to call a custom script. :(
Ray
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Configuring Path Groups
2012-04-07 6:39 ` Ray Van Dolson
@ 2012-04-07 6:52 ` Ray Van Dolson
2012-04-08 23:47 ` Alexander Murashkin
0 siblings, 1 reply; 5+ messages in thread
From: Ray Van Dolson @ 2012-04-07 6:52 UTC (permalink / raw)
To: dm-devel
On Fri, Apr 06, 2012 at 11:39:30PM -0700, Ray Van Dolson wrote:
> On Fri, Apr 06, 2012 at 11:29:47PM -0700, Ray Van Dolson wrote:
> > I am setting up multipathing on RHEL6 via iSCSI to an IBM XIV. My
> > iSCSI target looks like this:
> >
> > 172.16.10.1:3260,1793 iqn.2005-10.com.xivstorage:003974
> > 172.16.8.22:3260,2048 iqn.2005-10.com.xivstorage:003974
> > 172.16.10.22:3260,2049 iqn.2005-10.com.xivstorage:003974
> > 172.16.8.23:3260,2304 iqn.2005-10.com.xivstorage:003974
> > 172.16.10.23:3260,2305 iqn.2005-10.com.xivstorage:003974
> > 172.16.8.1:3260,1792 iqn.2005-10.com.xivstorage:003974
> >
> > (This is one LUN). Networks are 172.16.10.0/23 and 172.16.8.0/23, so
> > as you can see above there are three target IP's on each subnet.
> >
> > My RHEL6 host has two physical uplinks... one to each subnet. When I
> > configure multipath, by default it creates one big path group with all
> > of the devices above within it and round-robins.
> >
> > mpathb (2001738000f860110) dm-2 IBM,2810XIV
> > size=48G features='1 queue_if_no_path' hwhandler='0' wp=rw
> > `-+- policy='queue-length 0' prio=1 status=active
> > |- 26:0:0:1 sdb 8:16 active ready running
> > |- 25:0:0:1 sdc 8:32 active ready running
> > |- 27:0:0:1 sdd 8:48 active ready running
> > |- 24:0:0:1 sde 8:64 active ready running
> > |- 23:0:0:1 sdf 8:80 active ready running
> > `- 22:0:0:1 sdg 8:96 active ready running
> >
> > This results in less than ideal performance.
> >
> > Ideally I want to only have one path per subnet active at any given
> > time, with the other paths only activating if there's a failure.
> >
> > The only other option I seem to have is to set path_grouping_policy to
> > failover. This creates six separate path groups of which only one is
> > active at a time.
> >
> > I thought perhaps I could do group_by_prio, but am not sure how I can
> > manually set priorities for my underlying physical iSCSI devices, and
> > the default "const" method just gives them all priority 1 so they all
> > end up in the same path group again.
> >
> > It looks like perhaps I could configure path groups as I want manually
> > with dmsetup, but I'm sure there's a better way...
> >
> > As it is I am currently just configuring my iSCSI initiator to log in
> > to only one target per subnet. This gets me the best performance but I
> > lose some (automated) redundancy.
> >
> > Thoughts appreciated.
> >
> > Ray
>
> It seems like what I'm after is "prio_callout". However, it doesn't
> seem to be documented in the man pages (though it is in one of the
> sample configurations).
>
> I tried setting path_grouping_policy to group_by_prio and then defined
> prio_callout to point to a bash script on my system that echo's "2".
> This doesn't seem to actually get called though.
>
> Actually, after doing a bit more reading it sounds like prio_callout
> was replaced by "prio" in RHEL6's version of the tool. "prio" doesn't
> appear to have an option to call a custom script. :(
>
> Ray
Found this[1] post and sounds like perhaps prio_callout does still work
but doesn't play nice with bash scripts for lots of valid reasons.
I did try:
devices {
device {
vendor "dummy"
product "dummy"
prio_callout "/sbin/ray_prio"
}
device {
path_checker tur
product "2810XIV"
vendor "IBM"
rr_min_io 15
path_grouping_policy group_by_prio
prio_callout "/sbin/ray_prio /dev/%n"
path_selector "round-robin 0"
no_path_retry queue
failback 15
}
}
Per the suggestion in the article, but still no go. Perhaps if I
rewrote my prio callout in C? :)
[1] http://permalink.gmane.org/gmane.linux.kernel.device-mapper.devel/7904
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: Configuring Path Groups
2012-04-07 6:52 ` Ray Van Dolson
@ 2012-04-08 23:47 ` Alexander Murashkin
0 siblings, 0 replies; 5+ messages in thread
From: Alexander Murashkin @ 2012-04-08 23:47 UTC (permalink / raw)
To: 'device-mapper development'
> On Fri, Apr 06, 2012 at 11:39:30PM -0700, Ray Van Dolson wrote:
> > On Fri, Apr 06, 2012 at 11:29:47PM -0700, Ray Van Dolson wrote:
> > > I am setting up multipathing on RHEL6 via iSCSI to an IBM XIV. My
> > > iSCSI target looks like this:
> > >
> > > 172.16.10.1:3260,1793 iqn.2005-10.com.xivstorage:003974
> > > 172.16.8.22:3260,2048 iqn.2005-10.com.xivstorage:003974
> > > 172.16.10.22:3260,2049 iqn.2005-10.com.xivstorage:003974
> > > 172.16.8.23:3260,2304 iqn.2005-10.com.xivstorage:003974
> > > 172.16.10.23:3260,2305 iqn.2005-10.com.xivstorage:003974
> > > 172.16.8.1:3260,1792 iqn.2005-10.com.xivstorage:003974
> > >
> > > (This is one LUN). Networks are 172.16.10.0/23 and 172.16.8.0/23, so
> > > as you can see above there are three target IP's on each subnet.
> > >
> > > My RHEL6 host has two physical uplinks... one to each subnet. When I
> > > configure multipath, by default it creates one big path group with all
> > > of the devices above within it and round-robins.
> > >
> > > mpathb (2001738000f860110) dm-2 IBM,2810XIV
> > > size=48G features='1 queue_if_no_path' hwhandler='0' wp=rw
> > > `-+- policy='queue-length 0' prio=1 status=active
> > > |- 26:0:0:1 sdb 8:16 active ready running
> > > |- 25:0:0:1 sdc 8:32 active ready running
> > > |- 27:0:0:1 sdd 8:48 active ready running
> > > |- 24:0:0:1 sde 8:64 active ready running
> > > |- 23:0:0:1 sdf 8:80 active ready running
> > > `- 22:0:0:1 sdg 8:96 active ready running
> > >
> > > This results in less than ideal performance.
> > >
> > > Ideally I want to only have one path per subnet active at any given
> > > time, with the other paths only activating if there's a failure.
> > >
> > > The only other option I seem to have is to set path_grouping_policy to
> > > failover. This creates six separate path groups of which only one is
> > > active at a time.
> > >
> > > I thought perhaps I could do group_by_prio, but am not sure how I can
> > > manually set priorities for my underlying physical iSCSI devices, and
> > > the default "const" method just gives them all priority 1 so they all
> > > end up in the same path group again.
> > >
> > > It looks like perhaps I could configure path groups as I want manually
> > > with dmsetup, but I'm sure there's a better way...
> > >
> > > As it is I am currently just configuring my iSCSI initiator to log in
> > > to only one target per subnet. This gets me the best performance but
I
> > > lose some (automated) redundancy.
> > >
> > > Thoughts appreciated.
> > >
> > > Ray
> >
> > It seems like what I'm after is "prio_callout". However, it doesn't
> > seem to be documented in the man pages (though it is in one of the
> > sample configurations).
> >
> > I tried setting path_grouping_policy to group_by_prio and then defined
> > prio_callout to point to a bash script on my system that echo's "2".
> > This doesn't seem to actually get called though.
> >
> > Actually, after doing a bit more reading it sounds like prio_callout
> > was replaced by "prio" in RHEL6's version of the tool. "prio" doesn't
> > appear to have an option to call a custom script. :(
> >
> > Ray
>
> Found this[1] post and sounds like perhaps prio_callout does still work
> but doesn't play nice with bash scripts for lots of valid reasons.
>
> I did try:
>
> devices {
> device {
> vendor "dummy"
> product "dummy"
> prio_callout "/sbin/ray_prio"
> }
>
> device {
> path_checker tur
> product "2810XIV"
> vendor "IBM"
> rr_min_io 15
> path_grouping_policy group_by_prio
> prio_callout "/sbin/ray_prio /dev/%n"
> path_selector "round-robin 0"
> no_path_retry queue
> failback 15
> }
> }
>
> Per the suggestion in the article, but still no go. Perhaps if I
> rewrote my prio callout in C? :)
>
> [1] http://permalink.gmane.org/gmane.linux.kernel.device-
> mapper.devel/7904
See https://www.redhat.com/archives/dm-devel/2012-March/msg00186.html
It seems that prio_callout feature was removed in recent versions of
multipath. BTW Do you get any error message from multipath? In my case see
multipath.conf line NN, invalid keyword: prio_callout
IMHO prio_callout shall be added back even it does not work well in all
cases (just describe when it does not work). Also there are use cases when /
and /usr are on regular disks (for example, we keep OS on local SCSI disks
and use iSCSI for application data).
Best regards,
Alexander Murashkin
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2012-04-08 23:47 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2012-04-07 6:29 Configuring Path Groups Ray Van Dolson
2012-04-07 6:39 ` Ray Van Dolson
2012-04-07 6:52 ` Ray Van Dolson
2012-04-08 23:47 ` Alexander Murashkin
-- strict thread matches above, loose matches on Subject: below --
2012-04-07 6:20 Ray Van Dolson
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.