All of lore.kernel.org
 help / color / mirror / Atom feed
* RH 4 and HSG80
@ 2007-08-01 14:53 Eric Ritchie
  2007-08-01 15:39 ` Stefan Bader
  0 siblings, 1 reply; 10+ messages in thread
From: Eric Ritchie @ 2007-08-01 14:53 UTC (permalink / raw)
  To: dm-devel

I trying to setup multipathing with HSG80s on a Redhat 4 update 5 host. 
I have it mostly working, the problem I'm having is that the 2 disks 
I've created on the HSGs only go through one controller. I set the 
preferred path for each disk to use a different controller but dm always 
sets them to go through just one. I also can't find a way to move a disk 
to the other controller, I thought multipathd -k switch would work but 
it doesn't. Does anyone have RedHat4 and HSG80s working properly with 
multipathing? Here is my multipath.conf file and multipath -ll output:

defaults {
        user_friendly_names yes
}

defaults {
        udev_dir                /dev
        polling_interval        10
        selector                "round-robin 0"
        path_grouping_policy    multibus
        getuid_callout          "/sbin/scsi_id -g -u -s /block/%n"
        prio_callout            /bin/true
        path_checker            readsector0
        rr_min_io               100
        rr_weight               priorities
        failback                immediate
        no_path_retry           fail
        user_friendly_name      yes
        features                "1 queue_if_no_path"
}
devnode_blacklist {
        devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
        devnode "^hd[a-z]"
        devnode "^cciss!c[0-9]d[0-9]*"
}

multipaths {
        multipath {
                wwid                    360001fe1000d85f000039341397602d2
                alias                   disk1
                path_grouping_policy    multibus
                path_checker            readsector0
                path_selector           "round-robin 0"
                failback                manual
                rr_weight               priorities
                no_path_retry           5
        }
        multipath {
                wwid                    360001fe1000d85f0000393413976024b
                alias                   disk2
                path_grouping_policy    multibus
                path_checker            readsector0
                path_selector           "round-robin 0"
                failback                manual
                rr_weight               priorities
                no_path_retry           5
        }
}
devices {
        device {
                vendor                  DEC
                product                 HSG80
                path_grouping_policy    group_by_serial
                getuid_callout          "/sbin/scsi_id -g -u -s /block/%n"
                path_checker            hp_sw
                features                "1 queue_if_no_path"
        }
        device {
                vendor                  DEC
                product                 "HSG80    (C) DEC"
                path_grouping_policy    group_by_serial
                getuid_callout          "/sbin/scsi_id -g -u -s /block/%n"
                path_checker            hp_sw
                features                "1 queue_if_no_path"
        }
}

[root@lin380g5-ib etc]# multipath -ll
disk2 (360001fe1000d85f0000393413976024b)
[size=339 GB][features="1 queue_if_no_path"][hwhandler="0"]
\_ round-robin 0 [active]
 \_ 0:0:4:11 sdb 8:16  [active][ready]
 \_ 0:0:5:11 sdd 8:48  [active][ghost]
 \_ 1:0:4:11 sdf 8:80  [active][ready]
 \_ 1:0:5:11 sdh 8:112 [active][ghost]

disk1 (360001fe1000d85f000039341397602d2)
[size=67 GB][features="1 queue_if_no_path"][hwhandler="0"]
\_ round-robin 0 [active]
 \_ 0:0:4:20 sda 8:0   [active][ready]
 \_ 0:0:5:20 sdc 8:32  [active][ghost]
 \_ 1:0:4:20 sde 8:64  [active][ready]
 \_ 1:0:5:20 sdg 8:96  [active][ghost]


Thanks

-- 
Eric Ritchie
Interactive Brokers LLC

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: RH 4 and HSG80
  2007-08-01 14:53 RH 4 and HSG80 Eric Ritchie
@ 2007-08-01 15:39 ` Stefan Bader
  2007-08-01 17:11   ` Eric Ritchie
  2007-08-01 17:59   ` malahal
  0 siblings, 2 replies; 10+ messages in thread
From: Stefan Bader @ 2007-08-01 15:39 UTC (permalink / raw)
  To: device-mapper development

I have no access to HSGs but maybe this generic statements (plus some 
wisdom from hwtable.c) helps:

First, you are actually forcing multipath-tools to use all paths by 
stating "multibus" in you multipath section.
Everything you put there will replace the defaults and the device 
sections. Just be minimalistic there. In the
latest multipath-tools code there actually is a configuration for 
"DEC"/"HSG80". From this, does the following
multipath-conf help you?

defaults {
        rr_min_io                       64
        user_friendly_names     yes
}

multipaths {
        multipath {
                wwid    360001fe1000d85f000039341397602d2
                alias   disk1
        }
        multipath {
                wwid    360001fe1000d85f0000393413976024b
                alias   disk2
        }
}

devices {
        device {
                vendor                  "DEC"
                product                 "HSG80"
                path_grouping_policy    group_by_prio
                prio_callout            "/sbin/mpath_prio_hp_sw /dev/%n"
                hardware_handler                "1 hp_sw"
                path_checker            hp_sw
                no_path_retry           5
        }
        device {
                vendor                  "DEC"
                product                 "HSG80    (C) DEC"
                path_grouping_policy    group_by_prio
                prio_callout            "/sbin/mpath_prio_hp_sw /dev/%n"
                hardware_handler                "1 hp_sw"
                path_checker            hp_sw
                no_path_retry           5
        }
}


Mit freundlichem Gruß / Regards,
Stefan Bader

SW Linux on zSeries Development
Stefan.Bader@de.ibm.com

IBM Deutschland Entwicklung GmbH
Vorsitzender des Aufsichtsrats: Martin Jetter
Geschäftsführung: Herbert Kircher
Sitz der Gesellschaft: Böblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294
----------------------------------------------------------------------------------
  When all other means of communication fail, try words.

dm-devel-bounces@redhat.com wrote on 01.08.2007 16:53:55:

> I trying to setup multipathing with HSG80s on a Redhat 4 update 5 host. 
> I have it mostly working, the problem I'm having is that the 2 disks 
> I've created on the HSGs only go through one controller. I set the 
> preferred path for each disk to use a different controller but dm always 

> sets them to go through just one. I also can't find a way to move a disk 

> to the other controller, I thought multipathd -k switch would work but 
> it doesn't. Does anyone have RedHat4 and HSG80s working properly with 
> multipathing? Here is my multipath.conf file and multipath -ll output:
> 
> defaults {
>         user_friendly_names yes
> }
> 
> defaults {
>         udev_dir                /dev
>         polling_interval        10
>         selector                "round-robin 0"
>         path_grouping_policy    multibus
>         getuid_callout          "/sbin/scsi_id -g -u -s /block/%n"
>         prio_callout            /bin/true
>         path_checker            readsector0
>         rr_min_io               100
>         rr_weight               priorities
>         failback                immediate
>         no_path_retry           fail
>         user_friendly_name      yes
>         features                "1 queue_if_no_path"
> }
> devnode_blacklist {
>         devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
>         devnode "^hd[a-z]"
>         devnode "^cciss!c[0-9]d[0-9]*"
> }
> 
> multipaths {
>         multipath {
>                 wwid 360001fe1000d85f000039341397602d2
>                 alias                   disk1
>                 path_grouping_policy    multibus
>                 path_checker            readsector0
>                 path_selector           "round-robin 0"
>                 failback                manual
>                 rr_weight               priorities
>                 no_path_retry           5
>         }
>         multipath {
>                 wwid 360001fe1000d85f0000393413976024b
>                 alias                   disk2
>                 path_grouping_policy    multibus
>                 path_checker            readsector0
>                 path_selector           "round-robin 0"
>                 failback                manual
>                 rr_weight               priorities
>                 no_path_retry           5
>         }
> }
> devices {
>         device {
>                 vendor                  DEC
>                 product                 HSG80
>                 path_grouping_policy    group_by_serial
>                 getuid_callout          "/sbin/scsi_id -g -u -s 
/block/%n"
>                 path_checker            hp_sw
>                 features                "1 queue_if_no_path"
>         }
>         device {
>                 vendor                  DEC
>                 product                 "HSG80    (C) DEC"
>                 path_grouping_policy    group_by_serial
>                 getuid_callout          "/sbin/scsi_id -g -u -s 
/block/%n"
>                 path_checker            hp_sw
>                 features                "1 queue_if_no_path"
>         }
> }
> 
> [root@lin380g5-ib etc]# multipath -ll
> disk2 (360001fe1000d85f0000393413976024b)
> [size=339 GB][features="1 queue_if_no_path"][hwhandler="0"]
> \_ round-robin 0 [active]
>  \_ 0:0:4:11 sdb 8:16  [active][ready]
>  \_ 0:0:5:11 sdd 8:48  [active][ghost]
>  \_ 1:0:4:11 sdf 8:80  [active][ready]
>  \_ 1:0:5:11 sdh 8:112 [active][ghost]
> 
> disk1 (360001fe1000d85f000039341397602d2)
> [size=67 GB][features="1 queue_if_no_path"][hwhandler="0"]
> \_ round-robin 0 [active]
>  \_ 0:0:4:20 sda 8:0   [active][ready]
>  \_ 0:0:5:20 sdc 8:32  [active][ghost]
>  \_ 1:0:4:20 sde 8:64  [active][ready]
>  \_ 1:0:5:20 sdg 8:96  [active][ghost]
> 
> 
> Thanks
> 
> -- 
> Eric Ritchie
> Interactive Brokers LLC
> 
> 
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: RH 4 and HSG80
  2007-08-01 15:39 ` Stefan Bader
@ 2007-08-01 17:11   ` Eric Ritchie
  2007-08-01 18:52     ` Chandra Seetharaman
  2007-08-01 20:20     ` Bernd Zeimetz
  2007-08-01 17:59   ` malahal
  1 sibling, 2 replies; 10+ messages in thread
From: Eric Ritchie @ 2007-08-01 17:11 UTC (permalink / raw)
  To: device-mapper development

Thanks for the help. Unfortunately, I don't have /sbin/mpath_prio_hp_sw. 
I have other mpath_prio files but not the hp one. I guess that means I 
need a newer version of multipath-tools. How can I upgrade to the latest 
version?

Eric

Stefan Bader wrote:
> I have no access to HSGs but maybe this generic statements (plus some 
> wisdom from hwtable.c) helps:
>
> First, you are actually forcing multipath-tools to use all paths by 
> stating "multibus" in you multipath section.
> Everything you put there will replace the defaults and the device 
> sections. Just be minimalistic there. In the
> latest multipath-tools code there actually is a configuration for 
> "DEC"/"HSG80". From this, does the following
> multipath-conf help you?
>
> defaults {
>         rr_min_io                       64
>         user_friendly_names     yes
> }
>
> multipaths {
>         multipath {
>                 wwid    360001fe1000d85f000039341397602d2
>                 alias   disk1
>         }
>         multipath {
>                 wwid    360001fe1000d85f0000393413976024b
>                 alias   disk2
>         }
> }
>
> devices {
>         device {
>                 vendor                  "DEC"
>                 product                 "HSG80"
>                 path_grouping_policy    group_by_prio
>                 prio_callout            "/sbin/mpath_prio_hp_sw /dev/%n"
>                 hardware_handler                "1 hp_sw"
>                 path_checker            hp_sw
>                 no_path_retry           5
>         }
>         device {
>                 vendor                  "DEC"
>                 product                 "HSG80    (C) DEC"
>                 path_grouping_policy    group_by_prio
>                 prio_callout            "/sbin/mpath_prio_hp_sw /dev/%n"
>                 hardware_handler                "1 hp_sw"
>                 path_checker            hp_sw
>                 no_path_retry           5
>         }
> }
>
>
> Mit freundlichem Gruß / Regards,
> Stefan Bader
>
> SW Linux on zSeries Development
> Stefan.Bader@de.ibm.com
>
> IBM Deutschland Entwicklung GmbH
> Vorsitzender des Aufsichtsrats: Martin Jetter
> Geschäftsführung: Herbert Kircher
> Sitz der Gesellschaft: Böblingen
> Registergericht: Amtsgericht Stuttgart, HRB 243294
> ----------------------------------------------------------------------------------
>   When all other means of communication fail, try words.
>
> dm-devel-bounces@redhat.com wrote on 01.08.2007 16:53:55:
>
>   
>> I trying to setup multipathing with HSG80s on a Redhat 4 update 5 host. 
>> I have it mostly working, the problem I'm having is that the 2 disks 
>> I've created on the HSGs only go through one controller. I set the 
>> preferred path for each disk to use a different controller but dm always 
>>     
>
>   
>> sets them to go through just one. I also can't find a way to move a disk 
>>     
>
>   
>> to the other controller, I thought multipathd -k switch would work but 
>> it doesn't. Does anyone have RedHat4 and HSG80s working properly with 
>> multipathing? Here is my multipath.conf file and multipath -ll output:
>>
>> defaults {
>>         user_friendly_names yes
>> }
>>
>> defaults {
>>         udev_dir                /dev
>>         polling_interval        10
>>         selector                "round-robin 0"
>>         path_grouping_policy    multibus
>>         getuid_callout          "/sbin/scsi_id -g -u -s /block/%n"
>>         prio_callout            /bin/true
>>         path_checker            readsector0
>>         rr_min_io               100
>>         rr_weight               priorities
>>         failback                immediate
>>         no_path_retry           fail
>>         user_friendly_name      yes
>>         features                "1 queue_if_no_path"
>> }
>> devnode_blacklist {
>>         devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
>>         devnode "^hd[a-z]"
>>         devnode "^cciss!c[0-9]d[0-9]*"
>> }
>>
>> multipaths {
>>         multipath {
>>                 wwid 360001fe1000d85f000039341397602d2
>>                 alias                   disk1
>>                 path_grouping_policy    multibus
>>                 path_checker            readsector0
>>                 path_selector           "round-robin 0"
>>                 failback                manual
>>                 rr_weight               priorities
>>                 no_path_retry           5
>>         }
>>         multipath {
>>                 wwid 360001fe1000d85f0000393413976024b
>>                 alias                   disk2
>>                 path_grouping_policy    multibus
>>                 path_checker            readsector0
>>                 path_selector           "round-robin 0"
>>                 failback                manual
>>                 rr_weight               priorities
>>                 no_path_retry           5
>>         }
>> }
>> devices {
>>         device {
>>                 vendor                  DEC
>>                 product                 HSG80
>>                 path_grouping_policy    group_by_serial
>>                 getuid_callout          "/sbin/scsi_id -g -u -s 
>>     
> /block/%n"
>   
>>                 path_checker            hp_sw
>>                 features                "1 queue_if_no_path"
>>         }
>>         device {
>>                 vendor                  DEC
>>                 product                 "HSG80    (C) DEC"
>>                 path_grouping_policy    group_by_serial
>>                 getuid_callout          "/sbin/scsi_id -g -u -s 
>>     
> /block/%n"
>   
>>                 path_checker            hp_sw
>>                 features                "1 queue_if_no_path"
>>         }
>> }
>>
>> [root@lin380g5-ib etc]# multipath -ll
>> disk2 (360001fe1000d85f0000393413976024b)
>> [size=339 GB][features="1 queue_if_no_path"][hwhandler="0"]
>> \_ round-robin 0 [active]
>>  \_ 0:0:4:11 sdb 8:16  [active][ready]
>>  \_ 0:0:5:11 sdd 8:48  [active][ghost]
>>  \_ 1:0:4:11 sdf 8:80  [active][ready]
>>  \_ 1:0:5:11 sdh 8:112 [active][ghost]
>>
>> disk1 (360001fe1000d85f000039341397602d2)
>> [size=67 GB][features="1 queue_if_no_path"][hwhandler="0"]
>> \_ round-robin 0 [active]
>>  \_ 0:0:4:20 sda 8:0   [active][ready]
>>  \_ 0:0:5:20 sdc 8:32  [active][ghost]
>>  \_ 1:0:4:20 sde 8:64  [active][ready]
>>  \_ 1:0:5:20 sdg 8:96  [active][ghost]
>>
>>
>> Thanks
>>
>> -- 
>> Eric Ritchie
>> Interactive Brokers LLC
>>
>>
>> --
>> dm-devel mailing list
>> dm-devel@redhat.com
>> https://www.redhat.com/mailman/listinfo/dm-devel
>>     
>
>
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
>
>   


-- 
Eric Ritchie
Interactive Brokers LLC
203-618-5868

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: RH 4 and HSG80
  2007-08-01 15:39 ` Stefan Bader
  2007-08-01 17:11   ` Eric Ritchie
@ 2007-08-01 17:59   ` malahal
  2007-08-01 18:50     ` Chandra Seetharaman
  1 sibling, 1 reply; 10+ messages in thread
From: malahal @ 2007-08-01 17:59 UTC (permalink / raw)
  To: Stefan Bader; +Cc: device-mapper development

Stefan Bader [Stefan.Bader@de.ibm.com] wrote:
> I have no access to HSGs but maybe this generic statements (plus some 
> wisdom from hwtable.c) helps:
> 
> First, you are actually forcing multipath-tools to use all paths by 
> stating "multibus" in you multipath section.
> Everything you put there will replace the defaults and the device 
> sections. Just be minimalistic there. In the

One would expect that whatever is specified in the 'device' section would
be taken for that device no matter what is in the 'defaults' section. If
device section is missing, then it should use from the 'defaults'
section. Why is it implemented to have the 'defaults' section override the
'device' section? Looks odd to me.

Thanks, Malahal.

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: RH 4 and HSG80
  2007-08-01 17:59   ` malahal
@ 2007-08-01 18:50     ` Chandra Seetharaman
  0 siblings, 0 replies; 10+ messages in thread
From: Chandra Seetharaman @ 2007-08-01 18:50 UTC (permalink / raw)
  To: device-mapper development

On Wed, 2007-08-01 at 10:59 -0700, malahal@us.ibm.com wrote:
> Stefan Bader [Stefan.Bader@de.ibm.com] wrote:
> > I have no access to HSGs but maybe this generic statements (plus some 
> > wisdom from hwtable.c) helps:
> > 
> > First, you are actually forcing multipath-tools to use all paths by 
> > stating "multibus" in you multipath section.
> > Everything you put there will replace the defaults and the device 
> > sections. Just be minimalistic there. In the
> 
> One would expect that whatever is specified in the 'device' section would
> be taken for that device no matter what is in the 'defaults' section. If
> device section is missing, then it should use from the 'defaults'
> section. Why is it implemented to have the 'defaults' section override the
> 'device' section? Looks odd to me.

You are right, Malahal. Defaults section does not override the device
section. It just overrides internal default (hwtable.c).

To my understanding the order of attribute override is as under:

internal default (hwtable.c)
defaults section (in /etc/multipath.conf)
device section   (in /etc/multipath.conf)
multipaths section (in /etc/multipath.conf)

There were two things in Eric's multipath.conf which caused his problems
 - setting multibus as the path grouping policy (in multipaths section) 
   for a device that is active/passive (it should be ideally
    group_by_prio, but group_by_serial also works).
 - using /bin/true as prio_callout. As Stefan pointed prio_callout
   should be an appropriate one for the device.

> Thanks, Malahal.
> 
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
-- 

----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - sekharan@us.ibm.com   |      .......you may get it.
----------------------------------------------------------------------

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: RH 4 and HSG80
  2007-08-01 17:11   ` Eric Ritchie
@ 2007-08-01 18:52     ` Chandra Seetharaman
  2007-08-01 18:59       ` Eric Ritchie
  2007-08-02  6:27       ` Hannes Reinecke
  2007-08-01 20:20     ` Bernd Zeimetz
  1 sibling, 2 replies; 10+ messages in thread
From: Chandra Seetharaman @ 2007-08-01 18:52 UTC (permalink / raw)
  To: device-mapper development

On Wed, 2007-08-01 at 13:11 -0400, Eric Ritchie wrote:
> Thanks for the help. Unfortunately, I don't have /sbin/mpath_prio_hp_sw. 
> I have other mpath_prio files but not the hp one. I guess that means I 
> need a newer version of multipath-tools. How can I upgrade to the latest 
> version?

Try out different mpath_prio's (especially the _tpc one). One of them
_might_ work.

> 
> Eric
> 
> Stefan Bader wrote:
> > I have no access to HSGs but maybe this generic statements (plus some 
> > wisdom from hwtable.c) helps:
> >
> > First, you are actually forcing multipath-tools to use all paths by 
> > stating "multibus" in you multipath section.
> > Everything you put there will replace the defaults and the device 
> > sections. Just be minimalistic there. In the
> > latest multipath-tools code there actually is a configuration for 
> > "DEC"/"HSG80". From this, does the following
> > multipath-conf help you?
> >
> > defaults {
> >         rr_min_io                       64
> >         user_friendly_names     yes
> > }
> >
> > multipaths {
> >         multipath {
> >                 wwid    360001fe1000d85f000039341397602d2
> >                 alias   disk1
> >         }
> >         multipath {
> >                 wwid    360001fe1000d85f0000393413976024b
> >                 alias   disk2
> >         }
> > }
> >
> > devices {
> >         device {
> >                 vendor                  "DEC"
> >                 product                 "HSG80"
> >                 path_grouping_policy    group_by_prio
> >                 prio_callout            "/sbin/mpath_prio_hp_sw /dev/%n"
> >                 hardware_handler                "1 hp_sw"
> >                 path_checker            hp_sw
> >                 no_path_retry           5
> >         }
> >         device {
> >                 vendor                  "DEC"
> >                 product                 "HSG80    (C) DEC"
> >                 path_grouping_policy    group_by_prio
> >                 prio_callout            "/sbin/mpath_prio_hp_sw /dev/%n"
> >                 hardware_handler                "1 hp_sw"
> >                 path_checker            hp_sw
> >                 no_path_retry           5
> >         }
> > }
> >
> >
> > Mit freundlichem Gruß / Regards,
> > Stefan Bader
> >
> > SW Linux on zSeries Development
> > Stefan.Bader@de.ibm.com
> >
> > IBM Deutschland Entwicklung GmbH
> > Vorsitzender des Aufsichtsrats: Martin Jetter
> > Geschäftsführung: Herbert Kircher
> > Sitz der Gesellschaft: Böblingen
> > Registergericht: Amtsgericht Stuttgart, HRB 243294
> > ----------------------------------------------------------------------------------
> >   When all other means of communication fail, try words.
> >
> > dm-devel-bounces@redhat.com wrote on 01.08.2007 16:53:55:
> >
> >   
> >> I trying to setup multipathing with HSG80s on a Redhat 4 update 5 host. 
> >> I have it mostly working, the problem I'm having is that the 2 disks 
> >> I've created on the HSGs only go through one controller. I set the 
> >> preferred path for each disk to use a different controller but dm always 
> >>     
> >
> >   
> >> sets them to go through just one. I also can't find a way to move a disk 
> >>     
> >
> >   
> >> to the other controller, I thought multipathd -k switch would work but 
> >> it doesn't. Does anyone have RedHat4 and HSG80s working properly with 
> >> multipathing? Here is my multipath.conf file and multipath -ll output:
> >>
> >> defaults {
> >>         user_friendly_names yes
> >> }
> >>
> >> defaults {
> >>         udev_dir                /dev
> >>         polling_interval        10
> >>         selector                "round-robin 0"
> >>         path_grouping_policy    multibus
> >>         getuid_callout          "/sbin/scsi_id -g -u -s /block/%n"
> >>         prio_callout            /bin/true
> >>         path_checker            readsector0
> >>         rr_min_io               100
> >>         rr_weight               priorities
> >>         failback                immediate
> >>         no_path_retry           fail
> >>         user_friendly_name      yes
> >>         features                "1 queue_if_no_path"
> >> }
> >> devnode_blacklist {
> >>         devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
> >>         devnode "^hd[a-z]"
> >>         devnode "^cciss!c[0-9]d[0-9]*"
> >> }
> >>
> >> multipaths {
> >>         multipath {
> >>                 wwid 360001fe1000d85f000039341397602d2
> >>                 alias                   disk1
> >>                 path_grouping_policy    multibus
> >>                 path_checker            readsector0
> >>                 path_selector           "round-robin 0"
> >>                 failback                manual
> >>                 rr_weight               priorities
> >>                 no_path_retry           5
> >>         }
> >>         multipath {
> >>                 wwid 360001fe1000d85f0000393413976024b
> >>                 alias                   disk2
> >>                 path_grouping_policy    multibus
> >>                 path_checker            readsector0
> >>                 path_selector           "round-robin 0"
> >>                 failback                manual
> >>                 rr_weight               priorities
> >>                 no_path_retry           5
> >>         }
> >> }
> >> devices {
> >>         device {
> >>                 vendor                  DEC
> >>                 product                 HSG80
> >>                 path_grouping_policy    group_by_serial
> >>                 getuid_callout          "/sbin/scsi_id -g -u -s 
> >>     
> > /block/%n"
> >   
> >>                 path_checker            hp_sw
> >>                 features                "1 queue_if_no_path"
> >>         }
> >>         device {
> >>                 vendor                  DEC
> >>                 product                 "HSG80    (C) DEC"
> >>                 path_grouping_policy    group_by_serial
> >>                 getuid_callout          "/sbin/scsi_id -g -u -s 
> >>     
> > /block/%n"
> >   
> >>                 path_checker            hp_sw
> >>                 features                "1 queue_if_no_path"
> >>         }
> >> }
> >>
> >> [root@lin380g5-ib etc]# multipath -ll
> >> disk2 (360001fe1000d85f0000393413976024b)
> >> [size=339 GB][features="1 queue_if_no_path"][hwhandler="0"]
> >> \_ round-robin 0 [active]
> >>  \_ 0:0:4:11 sdb 8:16  [active][ready]
> >>  \_ 0:0:5:11 sdd 8:48  [active][ghost]
> >>  \_ 1:0:4:11 sdf 8:80  [active][ready]
> >>  \_ 1:0:5:11 sdh 8:112 [active][ghost]
> >>
> >> disk1 (360001fe1000d85f000039341397602d2)
> >> [size=67 GB][features="1 queue_if_no_path"][hwhandler="0"]
> >> \_ round-robin 0 [active]
> >>  \_ 0:0:4:20 sda 8:0   [active][ready]
> >>  \_ 0:0:5:20 sdc 8:32  [active][ghost]
> >>  \_ 1:0:4:20 sde 8:64  [active][ready]
> >>  \_ 1:0:5:20 sdg 8:96  [active][ghost]
> >>
> >>
> >> Thanks
> >>
> >> -- 
> >> Eric Ritchie
> >> Interactive Brokers LLC
> >>
> >>
> >> --
> >> dm-devel mailing list
> >> dm-devel@redhat.com
> >> https://www.redhat.com/mailman/listinfo/dm-devel
> >>     
> >
> >
> > --
> > dm-devel mailing list
> > dm-devel@redhat.com
> > https://www.redhat.com/mailman/listinfo/dm-devel
> >
> >   
> 
> 
-- 

----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - sekharan@us.ibm.com   |      .......you may get it.
----------------------------------------------------------------------

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: RH 4 and HSG80
  2007-08-01 18:52     ` Chandra Seetharaman
@ 2007-08-01 18:59       ` Eric Ritchie
  2007-08-01 20:58         ` Chandra Seetharaman
  2007-08-02  6:27       ` Hannes Reinecke
  1 sibling, 1 reply; 10+ messages in thread
From: Eric Ritchie @ 2007-08-01 18:59 UTC (permalink / raw)
  To: sekharan, device-mapper development

Chandra Seetharaman wrote:
> On Wed, 2007-08-01 at 13:11 -0400, Eric Ritchie wrote:
>   
>> Thanks for the help. Unfortunately, I don't have /sbin/mpath_prio_hp_sw. 
>> I have other mpath_prio files but not the hp one. I guess that means I 
>> need a newer version of multipath-tools. How can I upgrade to the latest 
>> version?
>>     
>
> Try out different mpath_prio's (especially the _tpc one). One of them
> _might_ work.
>   
I only have _alua, _emc, _hds_modular and _netapp. None of these work.

Thanks
Eric
>   
>> Eric
>>
>> Stefan Bader wrote:
>>     
>>> I have no access to HSGs but maybe this generic statements (plus some 
>>> wisdom from hwtable.c) helps:
>>>
>>> First, you are actually forcing multipath-tools to use all paths by 
>>> stating "multibus" in you multipath section.
>>> Everything you put there will replace the defaults and the device 
>>> sections. Just be minimalistic there. In the
>>> latest multipath-tools code there actually is a configuration for 
>>> "DEC"/"HSG80". From this, does the following
>>> multipath-conf help you?
>>>
>>> defaults {
>>>         rr_min_io                       64
>>>         user_friendly_names     yes
>>> }
>>>
>>> multipaths {
>>>         multipath {
>>>                 wwid    360001fe1000d85f000039341397602d2
>>>                 alias   disk1
>>>         }
>>>         multipath {
>>>                 wwid    360001fe1000d85f0000393413976024b
>>>                 alias   disk2
>>>         }
>>> }
>>>
>>> devices {
>>>         device {
>>>                 vendor                  "DEC"
>>>                 product                 "HSG80"
>>>                 path_grouping_policy    group_by_prio
>>>                 prio_callout            "/sbin/mpath_prio_hp_sw /dev/%n"
>>>                 hardware_handler                "1 hp_sw"
>>>                 path_checker            hp_sw
>>>                 no_path_retry           5
>>>         }
>>>         device {
>>>                 vendor                  "DEC"
>>>                 product                 "HSG80    (C) DEC"
>>>                 path_grouping_policy    group_by_prio
>>>                 prio_callout            "/sbin/mpath_prio_hp_sw /dev/%n"
>>>                 hardware_handler                "1 hp_sw"
>>>                 path_checker            hp_sw
>>>                 no_path_retry           5
>>>         }
>>> }
>>>
>>>
>>> Mit freundlichem Gruß / Regards,
>>> Stefan Bader
>>>
>>> SW Linux on zSeries Development
>>> Stefan.Bader@de.ibm.com
>>>
>>> IBM Deutschland Entwicklung GmbH
>>> Vorsitzender des Aufsichtsrats: Martin Jetter
>>> Geschäftsführung: Herbert Kircher
>>> Sitz der Gesellschaft: Böblingen
>>> Registergericht: Amtsgericht Stuttgart, HRB 243294
>>> ----------------------------------------------------------------------------------
>>>   When all other means of communication fail, try words.
>>>
>>> dm-devel-bounces@redhat.com wrote on 01.08.2007 16:53:55:
>>>
>>>   
>>>       
>>>> I trying to setup multipathing with HSG80s on a Redhat 4 update 5 host. 
>>>> I have it mostly working, the problem I'm having is that the 2 disks 
>>>> I've created on the HSGs only go through one controller. I set the 
>>>> preferred path for each disk to use a different controller but dm always 
>>>>     
>>>>         
>>>   
>>>       
>>>> sets them to go through just one. I also can't find a way to move a disk 
>>>>     
>>>>         
>>>   
>>>       
>>>> to the other controller, I thought multipathd -k switch would work but 
>>>> it doesn't. Does anyone have RedHat4 and HSG80s working properly with 
>>>> multipathing? Here is my multipath.conf file and multipath -ll output:
>>>>
>>>> defaults {
>>>>         user_friendly_names yes
>>>> }
>>>>
>>>> defaults {
>>>>         udev_dir                /dev
>>>>         polling_interval        10
>>>>         selector                "round-robin 0"
>>>>         path_grouping_policy    multibus
>>>>         getuid_callout          "/sbin/scsi_id -g -u -s /block/%n"
>>>>         prio_callout            /bin/true
>>>>         path_checker            readsector0
>>>>         rr_min_io               100
>>>>         rr_weight               priorities
>>>>         failback                immediate
>>>>         no_path_retry           fail
>>>>         user_friendly_name      yes
>>>>         features                "1 queue_if_no_path"
>>>> }
>>>> devnode_blacklist {
>>>>         devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
>>>>         devnode "^hd[a-z]"
>>>>         devnode "^cciss!c[0-9]d[0-9]*"
>>>> }
>>>>
>>>> multipaths {
>>>>         multipath {
>>>>                 wwid 360001fe1000d85f000039341397602d2
>>>>                 alias                   disk1
>>>>                 path_grouping_policy    multibus
>>>>                 path_checker            readsector0
>>>>                 path_selector           "round-robin 0"
>>>>                 failback                manual
>>>>                 rr_weight               priorities
>>>>                 no_path_retry           5
>>>>         }
>>>>         multipath {
>>>>                 wwid 360001fe1000d85f0000393413976024b
>>>>                 alias                   disk2
>>>>                 path_grouping_policy    multibus
>>>>                 path_checker            readsector0
>>>>                 path_selector           "round-robin 0"
>>>>                 failback                manual
>>>>                 rr_weight               priorities
>>>>                 no_path_retry           5
>>>>         }
>>>> }
>>>> devices {
>>>>         device {
>>>>                 vendor                  DEC
>>>>                 product                 HSG80
>>>>                 path_grouping_policy    group_by_serial
>>>>                 getuid_callout          "/sbin/scsi_id -g -u -s 
>>>>     
>>>>         
>>> /block/%n"
>>>   
>>>       
>>>>                 path_checker            hp_sw
>>>>                 features                "1 queue_if_no_path"
>>>>         }
>>>>         device {
>>>>                 vendor                  DEC
>>>>                 product                 "HSG80    (C) DEC"
>>>>                 path_grouping_policy    group_by_serial
>>>>                 getuid_callout          "/sbin/scsi_id -g -u -s 
>>>>     
>>>>         
>>> /block/%n"
>>>   
>>>       
>>>>                 path_checker            hp_sw
>>>>                 features                "1 queue_if_no_path"
>>>>         }
>>>> }
>>>>
>>>> [root@lin380g5-ib etc]# multipath -ll
>>>> disk2 (360001fe1000d85f0000393413976024b)
>>>> [size=339 GB][features="1 queue_if_no_path"][hwhandler="0"]
>>>> \_ round-robin 0 [active]
>>>>  \_ 0:0:4:11 sdb 8:16  [active][ready]
>>>>  \_ 0:0:5:11 sdd 8:48  [active][ghost]
>>>>  \_ 1:0:4:11 sdf 8:80  [active][ready]
>>>>  \_ 1:0:5:11 sdh 8:112 [active][ghost]
>>>>
>>>> disk1 (360001fe1000d85f000039341397602d2)
>>>> [size=67 GB][features="1 queue_if_no_path"][hwhandler="0"]
>>>> \_ round-robin 0 [active]
>>>>  \_ 0:0:4:20 sda 8:0   [active][ready]
>>>>  \_ 0:0:5:20 sdc 8:32  [active][ghost]
>>>>  \_ 1:0:4:20 sde 8:64  [active][ready]
>>>>  \_ 1:0:5:20 sdg 8:96  [active][ghost]
>>>>
>>>>
>>>> Thanks
>>>>
>>>> -- 
>>>> Eric Ritchie
>>>> Interactive Brokers LLC
>>>>
>>>>
>>>> --
>>>> dm-devel mailing list
>>>> dm-devel@redhat.com
>>>> https://www.redhat.com/mailman/listinfo/dm-devel
>>>>     
>>>>         
>>> --
>>> dm-devel mailing list
>>> dm-devel@redhat.com
>>> https://www.redhat.com/mailman/listinfo/dm-devel
>>>
>>>   
>>>       
>>     


-- 
Eric Ritchie
Interactive Brokers LLC
203-618-5868

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: RH 4 and HSG80
  2007-08-01 17:11   ` Eric Ritchie
  2007-08-01 18:52     ` Chandra Seetharaman
@ 2007-08-01 20:20     ` Bernd Zeimetz
  1 sibling, 0 replies; 10+ messages in thread
From: Bernd Zeimetz @ 2007-08-01 20:20 UTC (permalink / raw)
  To: device-mapper development

Eric Ritchie wrote:
> Thanks for the help. Unfortunately, I don't have /sbin/mpath_prio_hp_sw.
> I have other mpath_prio files but not the hp one. I guess that means I
> need a newer version of multipath-tools. How can I upgrade to the latest
> version?

you'll have to build it from git.
Also you need to build the hp_sw module for your kernel, patches which
at least build with 2.6.21 and .22 were posted on this list some days ago.


Cheers,

Bernd

-- 
Bernd Zeimetz
<bernd@bzed.de>                         <http://bzed.de/>

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: RH 4 and HSG80
  2007-08-01 18:59       ` Eric Ritchie
@ 2007-08-01 20:58         ` Chandra Seetharaman
  0 siblings, 0 replies; 10+ messages in thread
From: Chandra Seetharaman @ 2007-08-01 20:58 UTC (permalink / raw)
  To: Eric Ritchie; +Cc: device-mapper development

On Wed, 2007-08-01 at 14:59 -0400, Eric Ritchie wrote:
> Chandra Seetharaman wrote:
> > On Wed, 2007-08-01 at 13:11 -0400, Eric Ritchie wrote:
> >   
> >> Thanks for the help. Unfortunately, I don't have /sbin/mpath_prio_hp_sw. 
> >> I have other mpath_prio files but not the hp one. I guess that means I 
> >> need a newer version of multipath-tools. How can I upgrade to the latest 
> >> version?
> >>     
> >
> > Try out different mpath_prio's (especially the _tpc one). One of them
> > _might_ work.
> >   
> I only have _alua, _emc, _hds_modular and _netapp. None of these work.

sorry, I did not know what mpath_prio's were available in RHEL.

You can download the latest multipath tools tree from
http://git.kernel.org/?p=linux/storage/multipath-tools/.git;a=tree,
built the tools, copy only the _hp_sw or _tpc to /sbin and if it helps.

BTW, do note that doing this may invalidate your support contract with
your vendor (linux distributor). 
> 
> Thanks
> Eric
> >   
> >> Eric
> >>
> >> Stefan Bader wrote:
> >>     
> >>> I have no access to HSGs but maybe this generic statements (plus some 
> >>> wisdom from hwtable.c) helps:
> >>>
> >>> First, you are actually forcing multipath-tools to use all paths by 
> >>> stating "multibus" in you multipath section.
> >>> Everything you put there will replace the defaults and the device 
> >>> sections. Just be minimalistic there. In the
> >>> latest multipath-tools code there actually is a configuration for 
> >>> "DEC"/"HSG80". From this, does the following
> >>> multipath-conf help you?
> >>>
> >>> defaults {
> >>>         rr_min_io                       64
> >>>         user_friendly_names     yes
> >>> }
> >>>
> >>> multipaths {
> >>>         multipath {
> >>>                 wwid    360001fe1000d85f000039341397602d2
> >>>                 alias   disk1
> >>>         }
> >>>         multipath {
> >>>                 wwid    360001fe1000d85f0000393413976024b
> >>>                 alias   disk2
> >>>         }
> >>> }
> >>>
> >>> devices {
> >>>         device {
> >>>                 vendor                  "DEC"
> >>>                 product                 "HSG80"
> >>>                 path_grouping_policy    group_by_prio
> >>>                 prio_callout            "/sbin/mpath_prio_hp_sw /dev/%n"
> >>>                 hardware_handler                "1 hp_sw"
> >>>                 path_checker            hp_sw
> >>>                 no_path_retry           5
> >>>         }
> >>>         device {
> >>>                 vendor                  "DEC"
> >>>                 product                 "HSG80    (C) DEC"
> >>>                 path_grouping_policy    group_by_prio
> >>>                 prio_callout            "/sbin/mpath_prio_hp_sw /dev/%n"
> >>>                 hardware_handler                "1 hp_sw"
> >>>                 path_checker            hp_sw
> >>>                 no_path_retry           5
> >>>         }
> >>> }
> >>>
> >>>
> >>> Mit freundlichem Gruß / Regards,
> >>> Stefan Bader
> >>>
> >>> SW Linux on zSeries Development
> >>> Stefan.Bader@de.ibm.com
> >>>
> >>> IBM Deutschland Entwicklung GmbH
> >>> Vorsitzender des Aufsichtsrats: Martin Jetter
> >>> Geschäftsführung: Herbert Kircher
> >>> Sitz der Gesellschaft: Böblingen
> >>> Registergericht: Amtsgericht Stuttgart, HRB 243294
> >>> ----------------------------------------------------------------------------------
> >>>   When all other means of communication fail, try words.
> >>>
> >>> dm-devel-bounces@redhat.com wrote on 01.08.2007 16:53:55:
> >>>
> >>>   
> >>>       
> >>>> I trying to setup multipathing with HSG80s on a Redhat 4 update 5 host. 
> >>>> I have it mostly working, the problem I'm having is that the 2 disks 
> >>>> I've created on the HSGs only go through one controller. I set the 
> >>>> preferred path for each disk to use a different controller but dm always 
> >>>>     
> >>>>         
> >>>   
> >>>       
> >>>> sets them to go through just one. I also can't find a way to move a disk 
> >>>>     
> >>>>         
> >>>   
> >>>       
> >>>> to the other controller, I thought multipathd -k switch would work but 
> >>>> it doesn't. Does anyone have RedHat4 and HSG80s working properly with 
> >>>> multipathing? Here is my multipath.conf file and multipath -ll output:
> >>>>
> >>>> defaults {
> >>>>         user_friendly_names yes
> >>>> }
> >>>>
> >>>> defaults {
> >>>>         udev_dir                /dev
> >>>>         polling_interval        10
> >>>>         selector                "round-robin 0"
> >>>>         path_grouping_policy    multibus
> >>>>         getuid_callout          "/sbin/scsi_id -g -u -s /block/%n"
> >>>>         prio_callout            /bin/true
> >>>>         path_checker            readsector0
> >>>>         rr_min_io               100
> >>>>         rr_weight               priorities
> >>>>         failback                immediate
> >>>>         no_path_retry           fail
> >>>>         user_friendly_name      yes
> >>>>         features                "1 queue_if_no_path"
> >>>> }
> >>>> devnode_blacklist {
> >>>>         devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
> >>>>         devnode "^hd[a-z]"
> >>>>         devnode "^cciss!c[0-9]d[0-9]*"
> >>>> }
> >>>>
> >>>> multipaths {
> >>>>         multipath {
> >>>>                 wwid 360001fe1000d85f000039341397602d2
> >>>>                 alias                   disk1
> >>>>                 path_grouping_policy    multibus
> >>>>                 path_checker            readsector0
> >>>>                 path_selector           "round-robin 0"
> >>>>                 failback                manual
> >>>>                 rr_weight               priorities
> >>>>                 no_path_retry           5
> >>>>         }
> >>>>         multipath {
> >>>>                 wwid 360001fe1000d85f0000393413976024b
> >>>>                 alias                   disk2
> >>>>                 path_grouping_policy    multibus
> >>>>                 path_checker            readsector0
> >>>>                 path_selector           "round-robin 0"
> >>>>                 failback                manual
> >>>>                 rr_weight               priorities
> >>>>                 no_path_retry           5
> >>>>         }
> >>>> }
> >>>> devices {
> >>>>         device {
> >>>>                 vendor                  DEC
> >>>>                 product                 HSG80
> >>>>                 path_grouping_policy    group_by_serial
> >>>>                 getuid_callout          "/sbin/scsi_id -g -u -s 
> >>>>     
> >>>>         
> >>> /block/%n"
> >>>   
> >>>       
> >>>>                 path_checker            hp_sw
> >>>>                 features                "1 queue_if_no_path"
> >>>>         }
> >>>>         device {
> >>>>                 vendor                  DEC
> >>>>                 product                 "HSG80    (C) DEC"
> >>>>                 path_grouping_policy    group_by_serial
> >>>>                 getuid_callout          "/sbin/scsi_id -g -u -s 
> >>>>     
> >>>>         
> >>> /block/%n"
> >>>   
> >>>       
> >>>>                 path_checker            hp_sw
> >>>>                 features                "1 queue_if_no_path"
> >>>>         }
> >>>> }
> >>>>
> >>>> [root@lin380g5-ib etc]# multipath -ll
> >>>> disk2 (360001fe1000d85f0000393413976024b)
> >>>> [size=339 GB][features="1 queue_if_no_path"][hwhandler="0"]
> >>>> \_ round-robin 0 [active]
> >>>>  \_ 0:0:4:11 sdb 8:16  [active][ready]
> >>>>  \_ 0:0:5:11 sdd 8:48  [active][ghost]
> >>>>  \_ 1:0:4:11 sdf 8:80  [active][ready]
> >>>>  \_ 1:0:5:11 sdh 8:112 [active][ghost]
> >>>>
> >>>> disk1 (360001fe1000d85f000039341397602d2)
> >>>> [size=67 GB][features="1 queue_if_no_path"][hwhandler="0"]
> >>>> \_ round-robin 0 [active]
> >>>>  \_ 0:0:4:20 sda 8:0   [active][ready]
> >>>>  \_ 0:0:5:20 sdc 8:32  [active][ghost]
> >>>>  \_ 1:0:4:20 sde 8:64  [active][ready]
> >>>>  \_ 1:0:5:20 sdg 8:96  [active][ghost]
> >>>>
> >>>>
> >>>> Thanks
> >>>>
> >>>> -- 
> >>>> Eric Ritchie
> >>>> Interactive Brokers LLC
> >>>>
> >>>>
> >>>> --
> >>>> dm-devel mailing list
> >>>> dm-devel@redhat.com
> >>>> https://www.redhat.com/mailman/listinfo/dm-devel
> >>>>     
> >>>>         
> >>> --
> >>> dm-devel mailing list
> >>> dm-devel@redhat.com
> >>> https://www.redhat.com/mailman/listinfo/dm-devel
> >>>
> >>>   
> >>>       
> >>     
> 
> 
-- 

----------------------------------------------------------------------
    Chandra Seetharaman               | Be careful what you choose....
              - sekharan@us.ibm.com   |      .......you may get it.
----------------------------------------------------------------------

^ permalink raw reply	[flat|nested] 10+ messages in thread

* Re: RH 4 and HSG80
  2007-08-01 18:52     ` Chandra Seetharaman
  2007-08-01 18:59       ` Eric Ritchie
@ 2007-08-02  6:27       ` Hannes Reinecke
  1 sibling, 0 replies; 10+ messages in thread
From: Hannes Reinecke @ 2007-08-02  6:27 UTC (permalink / raw)
  To: sekharan, device-mapper development

Chandra Seetharaman wrote:
> On Wed, 2007-08-01 at 13:11 -0400, Eric Ritchie wrote:
>> Thanks for the help. Unfortunately, I don't have /sbin/mpath_prio_hp_sw. 
>> I have other mpath_prio files but not the hp one. I guess that means I 
>> need a newer version of multipath-tools. How can I upgrade to the latest 
>> version?
> 
> Try out different mpath_prio's (especially the _tpc one). One of them
> _might_ work.
> 

mpath_prio_tpc doesn't work. The HSG80 is basically the same firmware than
the older HP MSA boxes have (active/passive), so you'd need the hp_sw related
things.

Cheers,

Hannes
-- 
Dr. Hannes Reinecke		      zSeries & Storage
hare@suse.de			      +49 911 74053 688
SUSE LINUX Products GmbH, Maxfeldstr. 5, 90409 Nürnberg
GF: Markus Rex, HRB 16746 (AG Nürnberg)

^ permalink raw reply	[flat|nested] 10+ messages in thread

end of thread, other threads:[~2007-08-02  6:27 UTC | newest]

Thread overview: 10+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2007-08-01 14:53 RH 4 and HSG80 Eric Ritchie
2007-08-01 15:39 ` Stefan Bader
2007-08-01 17:11   ` Eric Ritchie
2007-08-01 18:52     ` Chandra Seetharaman
2007-08-01 18:59       ` Eric Ritchie
2007-08-01 20:58         ` Chandra Seetharaman
2007-08-02  6:27       ` Hannes Reinecke
2007-08-01 20:20     ` Bernd Zeimetz
2007-08-01 17:59   ` malahal
2007-08-01 18:50     ` Chandra Seetharaman

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.