Multipath setup and rdac questions

* Multipath setup and rdac questions
@ 2009-02-05 13:58 Christian Røsnes
  2009-02-05 15:07 ` Bryn M. Reeves
  0 siblings, 1 reply; 15+ messages in thread
From: Christian Røsnes @ 2009-02-05 13:58 UTC (permalink / raw)
  To: dm-devel

[-- Attachment #1.1: Type: text/plain, Size: 6333 bytes --]

Hi

I have got a couple of MD3000i with dual controllers,
and two M600e blades connected two both of them.

I've setup multipath, and would appreciate if
someone could tell me whether the setup described
below looks correct, or if you'd do it any
other way.
The reason I'm asking, is that I cannot see any
reference to 'rdac hardware handler' when
I do "multipath -ll".

However, when I change the module controller
ownership one of the LUNs (while copying files to that
particular LUN), the failover seems to work -
the gilr copying continues after the controller switch.

I also see these messages in /var/log/messages when
I change the module controller ownership:

 494 [RAIDarray.mpp]MD3000i_A:0:1:0 Cmnd-failed try alt ctrl 1. vcmnd SN
351494
pdev H2:C0:T0:L0 0x05/0x94/0x01 0x08000002 mpp_status:1

Q1: What could be the reason that I'm not using the rdac hwhandler ?
 (see output from "multipath -ll" below)

Q2: Is using rdac for this kind of setup preferred ?

Q3: I've created a GPFS filesystem using the /dev/dm-0, /dev/dm-1, etc
device. Is it OK to use the dm-x devices ? I see that there
are other alternatives such as /dev/mapper/mpath<x>.
However, /dev/mapper/mpath<x> is not visible in /proc/partitions
(see below), but the dm-<x> devices are - and the GPFS filesystem complains
when
it cannot see the devices in /proc/partitions.

Q4: Does my setup described here look OK ?

The blades are running:

  SLES 10 SP2 (x86-64)

From the MD3000i Dell Resource CD the following rpm was installed:

  linuxrdac-09.03.0C06.0030-1dkms

The following modules are running on my blades:

# lsmod | egrep 'rdac|mpp|multi'
dm_rdac                24832  0
dm_multipath           42640  3  dm_rdac,dm_round_robin
dm_mod                 80528  26 dm_multipath
mppVhba               145120  24
mppUpper              127436  1  mppVhba
scsi_mod              170936  12 dm_rdac,iscsi_tcp,
                                 libiscsi,
                                 scsi_transport_iscsi,
                                 mptctl,mppVhba,mptsas,
                                 mptscsih,
                                 scsi_transport_sas,
                                 mppUpper,sg,sd_mod

My multipath.conf file contains:

#cat /etc/multipath.conf

defaults {
        udev_dir                /dev
        user_friendly_names     yes
}
blacklist {
        devnode "^(ram|raw|loop|fd|md|dm-|sr|scd|st)[0-9]*"
        devnode "^hd[a-z]"
        devnode "^sda"
        devnode "^sda[0-9]"
        device {
                vendor DELL
                product "PERC|Universal"
        }
}
devices {
  device {
                vendor                  "DELL"
                product                 "MD3000i"
                product_blacklist       "Universal Xport"
                features                "1 queue_if_no_path"
                path_grouping_policy    group_by_prio
                hardware_handler        "1 rdac"
                path_checker            rdac
                prio                    "rdac"
                failback                immediate
        }
}
multipaths {
        mulitpath {
                device {
                        vendor DELL
                        product MD3000i
                }
        }
}

After changing the multipath.conf I did:

multipath -F
service multipathd restart

Now, if I've do:

# multipath -ll
mpathe (36002219000918b4c00005b6c497e6632) dm-4 DELL,MD Virtual Disk
[size=5.5T][features=0][hwhandler=0]
\_ round-robin 0 [prio=1][active]
 \_ 9:0:0:1  sdi 8:128 [active][ready]
mpathd (36002219000918b3300002e6e497e6315) dm-3 DELL,MD Virtual Disk
[size=5.5T][features=0][hwhandler=0]
\_ round-robin 0 [prio=1][active]
 \_ 9:0:0:2  sdj 8:144 [active][ready]
mpathc (36002219000918b4c00005b6f497e6695) dm-2 DELL,MD Virtual Disk
[size=5.5T][features=0][hwhandler=0]
\_ round-robin 0 [prio=1][active]
 \_ 9:0:0:3  sdk 8:160 [active][ready]
mpathb (36002219000918b3300002e71497e6378) dm-1 DELL,MD Virtual Disk
[size=5.5T][features=0][hwhandler=0]
\_ round-robin 0 [prio=1][active]
 \_ 9:0:0:4  sdl 8:176 [active][ready]
mpatha (36002219000918b4c00005b72497e66f3) dm-0 DELL,MD Virtual Disk
[size=5.5T][features=0][hwhandler=0]
\_ round-robin 0 [prio=1][active]
 \_ 9:0:0:5  sdm 8:192 [active][ready]
mpathl (36002219000918b3300002e6b497e62a6) dm-11 DELL,MD Virtual Disk
[size=5.5T][features=0][hwhandler=0]
\_ round-robin 0 [prio=1][active]
 \_ 9:0:0:0  sdb 8:16  [active][ready]
mpathk (36002219000922cdf000004a4497e53f5) dm-10 DELL,MD Virtual Disk
[size=5.5T][features=0][hwhandler=0]
\_ round-robin 0 [prio=1][active]
 \_ 9:0:1:10 sdc 8:32  [active][ready]
mpathj (36002219000918b2e000004b2497e581c) dm-9 DELL,MD Virtual Disk
[size=5.5T][features=0][hwhandler=0]
\_ round-robin 0 [prio=1][active]
 \_ 9:0:1:11 sdd 8:48  [active][ready]
mpathi (36002219000922cdf000004a7497e54dd) dm-8 DELL,MD Virtual Disk
[size=5.5T][features=0][hwhandler=0]
\_ round-robin 0 [prio=1][active]
 \_ 9:0:1:12 sde 8:64  [active][ready]
mpathh (36002219000918b2e000004b5497e587e) dm-7 DELL,MD Virtual Disk
[size=5.5T][features=0][hwhandler=0]
\_ round-robin 0 [prio=1][active]
 \_ 9:0:1:13 sdf 8:80  [active][ready]
mpathg (36002219000922cdf000004aa497e5532) dm-6 DELL,MD Virtual Disk
[size=5.5T][features=0][hwhandler=0]
\_ round-robin 0 [prio=1][active]
 \_ 9:0:1:14 sdg 8:96  [active][ready]
mpathf (36002219000918b2e000004b8497e58dc) dm-5 DELL,MD Virtual Disk
[size=5.5T][features=0][hwhandler=0]
\_ round-robin 0 [prio=1][active]
 \_ 9:0:1:15 sdh 8:112 [active][ready]

 cat /proc/partitions
major minor  #blocks  name

   8     0   71041024 sda
   8     1     530113 sda1
   8     2    8393962 sda2
   8     3   62115322 sda3
   8    16 5857419264 sdb
   8    32 5857419264 sdc
   8    48 5857419264 sdd
   8    64 5857419264 sde
   8    80 5857419264 sdf
   8    96 5857419264 sdg
   8   112 5857419264 sdh
   8   128 5857419264 sdi
   8   144 5857419264 sdj
   8   160 5857419264 sdk
   8   176 5857419264 sdl
   8   192 5857419264 sdm
 253     0 5857419264 dm-0
 253     1 5857419264 dm-1
 253     2 5857419264 dm-2
 253     3 5857419264 dm-3
 253     4 5857419264 dm-4
 253     5 5857419264 dm-5
 253     6 5857419264 dm-6
 253     7 5857419264 dm-7
 253     8 5857419264 dm-8
 253     9 5857419264 dm-9
 253    10 5857419264 dm-10
 253    11 5857419264 dm-11

Thank you

Christian

[-- Attachment #1.2: Type: text/html, Size: 11726 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

^ permalink raw reply	[flat|nested] 15+ messages in thread