Problem with Multipath (devices not ready)

Problem with Multipath (devices not ready)

[João Daniel]

[-- Attachment #1.1: Type: text/plain, Size: 4809 bytes --]

Hi,

 

 

 

I have upgraded from SLES9-SP2 to SLES9-SP3.

The purpose was to fix some problems related to OCFS2.

 

I have many warnings (as you can see below) in my /var/log/messages related
to multipath (i think), indicating that the devices aren't ready. 

Maybe the multipath.conf needs some extra configuration (please see my
multipath.conf).

The sdc and sdd devices are disks out on the EMC.

 

 

Can anyone help me with this problem ?

 

 

 

 

 

 

ERROR MESSAGES:

 

(...)

Jan 27 16:16:18 siipstestes kernel: Device sdc not ready.

Jan 27 16:16:18 siipstestes kernel: Device sdd not ready.

Jan 27 16:16:29 siipstestes kernel: Device sdc not ready.

Jan 27 16:16:29 siipstestes kernel: Device sdd not ready.

Jan 27 16:16:40 siipstestes kernel: Device sdc not ready.

Jan 27 16:16:40 siipstestes kernel: Device sdd not ready.

Jan 27 16:16:51 siipstestes kernel: Device sdc not ready.

Jan 27 16:16:51 siipstestes kernel: Device sdd not ready.

Jan 27 16:17:02 siipstestes kernel: Device sdc not ready.

Jan 27 16:17:02 siipstestes kernel: Device sdd not ready.

Jan 27 16:17:13 siipstestes kernel: Device sdc not ready.

Jan 27 16:17:13 siipstestes kernel: Device sdd not ready.

Jan 27 16:17:24 siipstestes kernel: Device sdc not ready.

Jan 27 16:17:24 siipstestes kernel: Device sdd not ready.

Jan 27 16:17:35 siipstestes kernel: Device sdc not ready.

Jan 27 16:17:35 siipstestes kernel: Device sdd not ready.

Jan 27 16:17:46 siipstestes kernel: Device sdc not ready.

Jan 27 16:17:46 siipstestes kernel: Device sdd not ready.

(...)

 

 

 

 

 

server:~ # multipath -v2 -l

dm names   N  

dm table LUN_TESTES  N  

dm table LUN_TESTES  N  

dm status LUN_TESTES  N  

dm info LUN_TESTES  O  

LUN_TESTES (360060160256014006adcbf64a644da11)

[size=458 GB][features="0"][hwhandler="0"]

\_ round-robin 0 [active]

 \_ 3:0:0:0 sdc 8:32 [failed][faulty]

 \_ 3:0:1:0 sdd 8:48 [failed][faulty]

 \_ 3:0:2:0 sde 8:64 [active][ready]

 \_ 3:0:3:0 sdf 8:80 [active][ready]

 

 

 

 

 

 

 

 

 

Here is my MULTIPATH.CONF file:

 

 

defaults {

            multipath_tool   "/sbin/multipath -v 0 -S"

            udev_dir            /dev

            polling_interval 10

            default_selector "round-robin 0"

    #default_path_grouping_policy           failover

            default_path_grouping_policy     multibus

            default_getuid_callout    "/sbin/scsi_id -g -u -s /block/%n"

            #default_prio_callout      "/bin/false"

    failback        immediate

}

 

#

# name    : devnode_blacklist

# scope   : multipath & multipathd

# desc    : list of device names to discard as not multipath candidates

# default : cciss, fd, hd, md, dm, sr, scd, st, ram, raw, loop

#

devnode_blacklist {

            devnode cciss

            devnode fd

            devnode hd

            devnode md

            devnode dm-

            devnode sr

            devnode scd

            devnode st

            devnode ram

            devnode raw

            devnode loop

            devnode sda

            devnode sdb

}

 

#

# name    : multipaths

# scope   : multipath & multipathd

# desc    : list of multipaths finest-grained settings

#

multipaths {

            #

            # name  : multipath

            # scope : multipath & multipathd

            # desc  : container for settings that apply to one specific
multipath

            #

            multipath {

                        wwid
360060160256014006adcbf64a644da11

                        alias                             LUN_TESTES

            }

}

 

 

 

 

 

 

 

 

 

 

 

 

 

 

HARDWARE:

 

- DELL 2850

- 1x QLogic 2340 (BIOS 1.47) to acess my Storage (EMC Clariion CX300)

 

 

 

 

SOFTWARE:

 

- SLES9 (32 bit) + SP3 (with Multipath and OCFS2 configured)

- Oracle Clusterware Release 2 (10.2.0.1.0)

- Oracle Database 10g Release 2 (10.2.0.1.0)

- Qlogic driver: qlafc-linux-8.01.00-4-install.tgz.gz

- Naviagent CLI: naviagentcli-6.16.0.4.63-1.i386.rpm

- SANsurfer: emc_sansurfer2.0.30b52_linux_install.bin

 

- Multipath 0.4.5-0.11

 

 

 

 

 

 

Best regards,

 

 

 

 





João Daniel




Apoio Informático 

Sistema de Informação do Instituto Politécnico de Setúbal 

 <mailto:jd-apinf@est.ips.pt> 

Instituto Politécnico de Setúbal 
Esc. Sup. de Tec. de Setúbal 

Campus do IPS, Estefanilha
2910-761 Setúbal 





tel:   


+351 265 790 133
+351 265 790 119

 

 


[-- Attachment #1.2: Type: text/html, Size: 34604 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problem with Multipath (devices not ready)

[Christophe Varoqui]

> 
>  
> 
> defaults {
> 
>             multipath_tool   "/sbin/multipath -v 0 -S"
> 
>             udev_dir            /dev
> 
>             polling_interval 10
> 
>             default_selector "round-robin 0"
> 
>     #default_path_grouping_policy           failover
> 
>             default_path_grouping_policy     multibus
> 

CX300 is not a multibus-capable hardware.
You should remove this directive and let the in-core defaults apply.

>             default_getuid_callout    "/sbin/scsi_id -g -u -s /block/%
> n"
> 
remove too : the CX need a special flag passed to scsi_id if I
remember ...


>             #default_prio_callout      "/bin/false"
> 
>     failback        immediate
> 
> }
> 
Regards,
cvaroqui
> 

Re: Problem with Multipath (devices not ready)

[Bernd Zeimetz]

Hi,

>>            default_getuid_callout    "/sbin/scsi_id -g -u -s /block/%
>>n"
>>
>>    
>>
>remove too : the CX need a special flag passed to scsi_id if I
>remember ...
>  
>
this was mentioned in a thread about the EMC CX a few weeks ago. But I
think there's also a working default setting for this in multipath (at
least it works very well with the default settings here). Also check
http://christophe.varoqui.free.fr/wiki/wakka.php?wiki=TestedEnvironments
- I've mentioned the neccessary settings on the CX there.


Bernd

Re: Problem with Multipath (devices not ready)

[João Daniel]

[-- Attachment #1: Type: text/html, Size: 41014 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]

Re: Problem with Multipath (devices not ready)

[Bernd Zeimetz]

Hi,
>     I've tried: update OS, update drivers, update multipath, re-configurate 
> multipath...
>   
did you update to multipath >= 0.4.6 ?

>     What do you think I should try now ?
>   
Probably I've missed it - but how do you access the CX? Via a switch?
direct? How many HBAs do you use? Are they all allowed to connect to the
machine?

Did you try it without any multipath.conf at all? The defaults in 0.4.6
work well.

> LUN_TESTES (360060160256014006adcbf64a644da11)
>
> [size=458 GB][features="0"][hwhandler="0"]
>
> \_ round-robin 0 [active]
>
>  \_ 3:0:0:0 sdc 8:32 [failed][faulty]
>
>  \_ 3:0:1:0 sdd 8:48 [failed][faulty]
>
>  \_ 3:0:2:0 sde 8:64 [active][ready]
>
>  \_ 3:0:3:0 sdf 8:80 [active][ready]
>   
although sdc and sdd are faulty you should be able to access the LUN -
you have 2 active and ready paths to it. Look for LUN_TESTES in your
/dev directory.

Do you have the right udev rules for multipath? As all distros do handle
this on a different way - which distro and udev version are you using?

Make sure dm events are not discarded.
On debian sarge for example you ahve to change this in udev.rules:

#KERNEL="dm-[0-9]*",    OPTIONS="ignore_device"
KERNEL="dm-[0-9]*", PROGRAM="/sbin/devmap_name %M %m", NAME="%k",
SYMLINK="%c"

Are the multipath rules for udev installed? Do you have any errors from
udev in your log?


I hope I could give you some more hints.


Best regards,

Bernd

Re: Problem with Multipath (devices not ready)

[João Daniel]

Hi,

   
    The problem is the QLogic driver.
    I use the driver below.

    EMC CLARiiON CXx00 and EMC Symmetrix approved common Linux installer 
for v2.6.x kernels -> 8.01.00-4
    [ 
http://support.qlogic.com/support/oem_product_detail.asp?p_id=253&oemid=65&oemname=QLA2340 
]
   
    This driver is not working properly under SLES9 SP3.
    I have connections to a LUN through different SPs, but only the 
paths to the SP that controls the LUN work.

    QLogic does not support its drivers for SLES9 SP3.
    The QLogic support forwarded this problem to EMC.
    I'm waiting for EMC...




Best Regards,

João Daniel






Bernd Zeimetz wrote:
> Hi,
>   
>>     I've tried: update OS, update drivers, update multipath, re-configurate 
>> multipath...
>>   
>>     
> did you update to multipath >= 0.4.6 ?
>
>   
>>     What do you think I should try now ?
>>   
>>     
> Probably I've missed it - but how do you access the CX? Via a switch?
> direct? How many HBAs do you use? Are they all allowed to connect to the
> machine?
>
> Did you try it without any multipath.conf at all? The defaults in 0.4.6
> work well.
>
>   
>> LUN_TESTES (360060160256014006adcbf64a644da11)
>>
>> [size=458 GB][features="0"][hwhandler="0"]
>>
>> \_ round-robin 0 [active]
>>
>>  \_ 3:0:0:0 sdc 8:32 [failed][faulty]
>>
>>  \_ 3:0:1:0 sdd 8:48 [failed][faulty]
>>
>>  \_ 3:0:2:0 sde 8:64 [active][ready]
>>
>>  \_ 3:0:3:0 sdf 8:80 [active][ready]
>>   
>>     
> although sdc and sdd are faulty you should be able to access the LUN -
> you have 2 active and ready paths to it. Look for LUN_TESTES in your
> /dev directory.
>
> Do you have the right udev rules for multipath? As all distros do handle
> this on a different way - which distro and udev version are you using?
>
> Make sure dm events are not discarded.
> On debian sarge for example you ahve to change this in udev.rules:
>
> #KERNEL="dm-[0-9]*",    OPTIONS="ignore_device"
> KERNEL="dm-[0-9]*", PROGRAM="/sbin/devmap_name %M %m", NAME="%k",
> SYMLINK="%c"
>
> Are the multipath rules for udev installed? Do you have any errors from
> udev in your log?
>
>
> I hope I could give you some more hints.
>
>
> Best regards,
>
> Bernd
>
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
>
>
>