RE: dm-devel Digest, Vol 18, Issue 2

RE: dm-devel Digest, Vol 18, Issue 2

[goggin, edward]

On Tue, 02 Aug 2005 09:37:14 -0500
"Christopher C. Weis" <ccweis@gmail.com> wrote

> I have a multipath SAN environment with storage controllers that are
> active/active.  However, the controllers are not active/active at the
> LUN-level without a performance penalty, meaning if two 
> servers want to
> see the same LUN (as in a clustered filesystem environment), they both
> need to be using the same controller.  I'm trying to figure 
> out a way to
> statically "order" the paths so that I can copy a config to all of the
> nodes using the CFS.
> 
> >From what I've read, in a single-server environment with controllers
> such as the ones I'm dealing with, the path_grouping_policy should be
> set to "group_by_serial", which should work fine, but in a clustered
> environment, I need to be sure that the path ordering is the same.
> 
> Are there any path_selectors, other than round-robin, that might
> accomplish this?  Any other ideas?
> 

One was is to configure each multipath to have two groups with one group
having a higher priority than the other based on whether the path accesses
the fast path controller.  The assignment of the highest priority path group
is non deterministic when using the "group_by_serial" path grouping policy.

Seems like you want to use the "group_by_priority" path grouping policy and
create and get_priority executable which when invoked will return a 1 for
fast path and 0 for slow path.  See the code for mpath_prio_emc, the
get_priority executable for the EMC CLARiiON array in
multipath-tools/path_priority/pp_emc/pp_emc.c.

Re: RE: dm-devel Digest, Vol 18, Issue 2

[christophe varoqui]

On mar, 2005-08-02 at 15:07 -0400, goggin, edward wrote:
> On Tue, 02 Aug 2005 09:37:14 -0500
> "Christopher C. Weis" <ccweis@gmail.com> wrote
> 
> > I have a multipath SAN environment with storage controllers that are
> > active/active.  However, the controllers are not active/active at the
> > LUN-level without a performance penalty, meaning if two 
> > servers want to
> > see the same LUN (as in a clustered filesystem environment), they both
> > need to be using the same controller.  I'm trying to figure 
> > out a way to
> > statically "order" the paths so that I can copy a config to all of the
> > nodes using the CFS.
> > 
> > >From what I've read, in a single-server environment with controllers
> > such as the ones I'm dealing with, the path_grouping_policy should be
> > set to "group_by_serial", which should work fine, but in a clustered
> > environment, I need to be sure that the path ordering is the same.
> > 
> > Are there any path_selectors, other than round-robin, that might
> > accomplish this?  Any other ideas?
> > 
> 
> One was is to configure each multipath to have two groups with one group
> having a higher priority than the other based on whether the path accesses
> the fast path controller.  The assignment of the highest priority path group
> is non deterministic when using the "group_by_serial" path grouping policy.
> 
> Seems like you want to use the "group_by_priority" path grouping policy and
> create and get_priority executable which when invoked will return a 1 for
> fast path and 0 for slow path.  See the code for mpath_prio_emc, the
> get_priority executable for the EMC CLARiiON array in
> multipath-tools/path_priority/pp_emc/pp_emc.c.
> 
Yes, also note "group_by_priority" path grouping policy may be overkill
for the context. PG produced by "group_by_serial" can be sorted with an
adequate prioritizer too.

Regards,
-- 
christophe varoqui <christophe.varoqui@free.fr>

Re: RE: dm-devel Digest, Vol 18, Issue 2

[Christopher Weis]

[-- Attachment #1.1: Type: text/plain, Size: 2262 bytes --]

On 8/2/05, christophe varoqui <christophe.varoqui@free.fr> wrote:
> 
> On mar, 2005-08-02 at 15:07 -0400, goggin, edward wrote:
> > On Tue, 02 Aug 2005 09:37:14 -0500
> > "Christopher C. Weis" <ccweis@gmail.com> wrote
> >
> > > I have a multipath SAN environment with storage controllers that are
> > > active/active. However, the controllers are not active/active at the
> > > LUN-level without a performance penalty, meaning if two
> > > servers want to
> > > see the same LUN (as in a clustered filesystem environment), they both
> > > need to be using the same controller. I'm trying to figure
> > > out a way to
> > > statically "order" the paths so that I can copy a config to all of the
> > > nodes using the CFS.
> > >
> > > >From what I've read, in a single-server environment with controllers
> > > such as the ones I'm dealing with, the path_grouping_policy should be
> > > set to "group_by_serial", which should work fine, but in a clustered
> > > environment, I need to be sure that the path ordering is the same.
> > >
> > > Are there any path_selectors, other than round-robin, that might
> > > accomplish this? Any other ideas?
> > >
> >
> > One was is to configure each multipath to have two groups with one group
> > having a higher priority than the other based on whether the path 
> accesses
> > the fast path controller. The assignment of the highest priority path 
> group
> > is non deterministic when using the "group_by_serial" path grouping 
> policy.
> >
> > Seems like you want to use the "group_by_priority" path grouping policy 
> and
> > create and get_priority executable which when invoked will return a 1 
> for
> > fast path and 0 for slow path. See the code for mpath_prio_emc, the
> > get_priority executable for the EMC CLARiiON array in
> > multipath-tools/path_priority/pp_emc/pp_emc.c.
> >
> Yes, also note "group_by_priority" path grouping policy may be overkill
> for the context. PG produced by "group_by_serial" can be sorted with an
> adequate prioritizer too.
> 
> 
> 
Does this mean that "group_by_serial" utilizes a "default_prio_callout" 
program/script as well, or is there another callout (or something totally 
different that I'm missing)?

Thx.

~Chris

[-- Attachment #1.2: Type: text/html, Size: 2901 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: RE: dm-devel Digest, Vol 18, Issue 2

[Christopher Weis]

[-- Attachment #1.1: Type: text/plain, Size: 3090 bytes --]

On 8/2/05, Christopher Weis <ccweis@gmail.com> wrote:
> 
> On 8/2/05, christophe varoqui <christophe.varoqui@free.fr> wrote:
> > 
> > On mar, 2005-08-02 at 15:07 -0400, goggin, edward wrote:
> > > On Tue, 02 Aug 2005 09:37:14 -0500
> > > "Christopher C. Weis" <ccweis@gmail.com> wrote
> > >
> > > > I have a multipath SAN environment with storage controllers that are 
> > 
> > > > active/active. However, the controllers are not active/active at the
> > > > LUN-level without a performance penalty, meaning if two
> > > > servers want to
> > > > see the same LUN (as in a clustered filesystem environment), they 
> > both 
> > > > need to be using the same controller. I'm trying to figure
> > > > out a way to
> > > > statically "order" the paths so that I can copy a config to all of 
> > the
> > > > nodes using the CFS. 
> > > >
> > > > >From what I've read, in a single-server environment with 
> > controllers
> > > > such as the ones I'm dealing with, the path_grouping_policy should 
> > be
> > > > set to "group_by_serial", which should work fine, but in a clustered 
> > 
> > > > environment, I need to be sure that the path ordering is the same.
> > > >
> > > > Are there any path_selectors, other than round-robin, that might
> > > > accomplish this? Any other ideas?
> > > >
> > >
> > > One was is to configure each multipath to have two groups with one 
> > group
> > > having a higher priority than the other based on whether the path 
> > accesses
> > > the fast path controller. The assignment of the highest priority path 
> > group 
> > > is non deterministic when using the "group_by_serial" path grouping 
> > policy.
> > >
> > > Seems like you want to use the "group_by_priority" path grouping 
> > policy and
> > > create and get_priority executable which when invoked will return a 1 
> > for 
> > > fast path and 0 for slow path. See the code for mpath_prio_emc, the
> > > get_priority executable for the EMC CLARiiON array in
> > > multipath-tools/path_priority/pp_emc/pp_emc.c.
> > >
> > Yes, also note "group_by_priority" path grouping policy may be overkill 
> > for the context. PG produced by "group_by_serial" can be sorted with an
> > adequate prioritizer too.
> > 
> > 
> > 
> Does this mean that "group_by_serial" utilizes a "default_prio_callout" 
> program/script as well, or is there another callout (or something totally 
> different that I'm missing)?


I've verified that by writing my own "default_prio_callout" script I can 
achieve what I want with both the "group_by_priority" and "failover" 
policies. Once I get done testing/tweaking the script, I'll post it in case 
anyone else wants it.

I can't seem to make group_by_serial work, although I'm not sure I 
understand this policy very well. It sounds like it's supposed to make all 
paths that go to a particular controller (serial?) be in the same priority 
group. For me, it's no different than multibus. Probably my 
misunderstanding, so I'm open to any suggestions...

Thanks for all the help everyone.

~Chris

[-- Attachment #1.2: Type: text/html, Size: 4105 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: RE: dm-devel Digest, Vol 18, Issue 2

[christophe varoqui]

>                 
>         
>         
>         Does this mean that "group_by_serial" utilizes a
>         "default_prio_callout" program/script as well, or is there
>         another callout (or something totally different that I'm
>         missing)?
> 
> I've verified that by writing my own "default_prio_callout" script I
> can achieve what I want with both the "group_by_priority" and
> "failover" policies.  Once I get done testing/tweaking the script,
> I'll post it in case anyone else wants it.
> 
> I can't seem to make group_by_serial work, although I'm not sure I
> understand this policy very well.  It sounds like it's supposed to
> make all paths that go to a particular controller (serial?) be in the
> same priority group.  For me, it's no different than multibus.
> Probably my misunderstanding, so I'm open to any suggestions...
> 
"/sbin/multipath -v4" will verbosely show you the path discovery phase,
including the discovered serials.

May be you'll understand better what happens with your hardware.

Regards,
-- 
christophe varoqui <christophe.varoqui@free.fr>

Re: RE: dm-devel Digest, Vol 18, Issue 2

[Christopher C. Weis]

christophe varoqui wrote:

>>                
>>        
>>        
>>        Does this mean that "group_by_serial" utilizes a
>>        "default_prio_callout" program/script as well, or is there
>>        another callout (or something totally different that I'm
>>        missing)?
>>
>>I've verified that by writing my own "default_prio_callout" script I
>>can achieve what I want with both the "group_by_priority" and
>>"failover" policies.  Once I get done testing/tweaking the script,
>>I'll post it in case anyone else wants it.
>>
>>I can't seem to make group_by_serial work, although I'm not sure I
>>understand this policy very well.  It sounds like it's supposed to
>>make all paths that go to a particular controller (serial?) be in the
>>same priority group.  For me, it's no different than multibus.
>>Probably my misunderstanding, so I'm open to any suggestions...
>>
>>    
>>
>"/sbin/multipath -v4" will verbosely show you the path discovery phase,
>including the discovered serials.
>
>May be you'll understand better what happens with your hardware.
>
>Regards,	
>  
>
Ahhh, that makes things much clearer.  Thanks a bunch everyone.

~Chris

Re: RE: dm-devel Digest, Vol 18, Issue 2

[christophe varoqui]

>         >
>         > Seems like you want to use the "group_by_priority" path
>         grouping policy and
>         > create and get_priority executable which when invoked will
>         return a 1 for 
>         > fast path and 0 for slow path.  See the code for
>         mpath_prio_emc, the
>         > get_priority executable for the EMC CLARiiON array in
>         > multipath-tools/path_priority/pp_emc/pp_emc.c.
>         >
>         Yes, also note "group_by_priority" path grouping policy may be
>         overkill 
>         for the context. PG produced by "group_by_serial" can be
>         sorted with an
>         adequate prioritizer too.
> 
> Does this mean that "group_by_serial" utilizes a
> "default_prio_callout" program/script as well, or is there another
> callout (or something totally different that I'm missing)?
> 
Path prio are fetched in the path discovery phase (via callout, or
defaulting to 1 if no callout specified).

PG policies just decide the path grouping.

There is a implicit notion of PG prio, which is sum(path pio).

group_by_prio is special here : it's a mean for admins to implement a
new pgpolicy without having to hack the tools ... just with shell script
as prio callouts.

Regards,
-- 
christophe varoqui <christophe.varoqui@free.fr>