Re: Multipath ID not equal to LUN scsi ID

["liuqing@huayun.com" <liuqing@huayun.com>]

[-- Attachment #1.1: Type: text/plain, Size: 11736 bytes --]

Hi Martin,
Thanks for the response.
I built multipath using the latest code to test, but I also see this issue in our product environment(multipath-tools v0.4.9 (05/33, 2016)).

I use getuid_callout is due to the id_serial is not correct as following:
[root@localhost ~]# multipath -ll
36005076300810eadf800000000000156 dm-3 IBM,2145
size=8.0G features='2 queue_if_no_path retain_attached_hw_handler' hwhandler='1 alua' wp=rw
|-+- policy='service-time 0' prio=50 status=active
| `- 2:0:0:0 sdp 8:240 active ready running
`-+- policy='service-time 0' prio=10 status=enabled
  `- 2:0:1:0 sdq 65:0  active ready running

[root@localhost ~]# udevadm info /dev/sdp
P: /devices/pci0000:40/0000:40:02.0/0000:42:00.1/host2/rport-2:0-0/target2:0:0/2:0:0:0/block/sdp
N: sdp
S: disk/by-id/scsi-36005076300810eadf800000000000156
S: disk/by-id/wwn-0x6005076300810eadf800000000000156
S: disk/by-path/pci-0000:42:00.1-fc-0x500507680307ab7c-lun-0
E: DEVLINKS=/dev/disk/by-id/scsi-36005076300810eadf800000000000156 /dev/disk/by-id/wwn-0x6005076300810eadf800000000000156 /dev/disk/by-path/pci-0000:42:00.1-fc-0x500507680307ab7c-lun-0
E: DEVNAME=/dev/sdp
E: DEVPATH=/devices/pci0000:40/0000:40:02.0/0000:42:00.1/host2/rport-2:0-0/target2:0:0/2:0:0:0/block/sdp
E: DEVTYPE=disk
E: ID_BUS=scsi
E: ID_MODEL=2145
E: ID_MODEL_ENC=2145\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20\x20
E: ID_PATH=pci-0000:42:00.1-fc-0x500507680307ab7c-lun-0
E: ID_PATH_TAG=pci-0000_42_00_1-fc-0x500507680307ab7c-lun-0
E: ID_REVISION=0000
E: ID_SCSI=1
E: ID_SCSI_SERIAL=00c02043ab7eXX00
E: ID_SERIAL=36005076300810eadf800000000000156
E: ID_SERIAL_SHORT=6005076300810eadf800000000000156
E: ID_TARGET_PORT=0
E: ID_TYPE=disk
E: ID_VENDOR=IBM
E: ID_VENDOR_ENC=IBM\x20\x20\x20\x20\x20
E: ID_WWN=0x6005076300810ead
E: ID_WWN_VENDOR_EXTENSION=0xf800000000000156
E: ID_WWN_WITH_EXTENSION=0x6005076300810eadf800000000000156
E: MAJOR=8
E: MINOR=240
E: MPATH_SBIN_PATH=/sbin
E: SUBSYSTEM=block
E: TAGS=:systemd:
E: USEC_INITIALIZED=262761901185

[root@localhost ~]# /lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/sdp
36005076300810eadf800000000000155

multipath -v3
===========
Jul 11 10:08:49 | libdevmapper version 1.02.135-RHEL7 (2016-11-16)
Jul 11 10:08:49 | DM multipath kernel driver v1.9.0
Jul 11 10:08:49 | loading /lib64/multipath/libchecktur.so checker
Jul 11 10:08:49 | loading /lib64/multipath/libprioconst.so prioritizer
Jul 11 10:08:49 | sda: udev property ID_WWN whitelisted
Jul 11 10:08:49 | sda: device node name blacklisted
Jul 11 10:08:49 | sdb: udev property ID_WWN whitelisted
Jul 11 10:08:49 | sdb: device node name blacklisted
Jul 11 10:08:49 | sdk: udev property ID_WWN whitelisted
Jul 11 10:08:49 | sdk: device node name blacklisted
Jul 11 10:08:49 | sdl: udev property ID_WWN whitelisted
Jul 11 10:08:49 | sdl: device node name blacklisted
Jul 11 10:08:49 | sdm: udev property ID_WWN whitelisted
Jul 11 10:08:49 | sdm: device node name blacklisted
Jul 11 10:08:49 | sdc: udev property ID_WWN whitelisted
Jul 11 10:08:49 | sdc: device node name blacklisted
Jul 11 10:08:49 | sdd: udev property ID_WWN whitelisted
Jul 11 10:08:49 | sdd: device node name blacklisted
Jul 11 10:08:49 | sde: udev property ID_WWN whitelisted
Jul 11 10:08:49 | sde: device node name blacklisted
Jul 11 10:08:49 | sdf: udev property ID_WWN whitelisted
Jul 11 10:08:49 | sdf: device node name blacklisted
Jul 11 10:08:49 | sdg: udev property ID_WWN whitelisted
Jul 11 10:08:49 | sdg: device node name blacklisted
Jul 11 10:08:49 | sdh: udev property ID_WWN whitelisted
Jul 11 10:08:49 | sdh: device node name blacklisted
Jul 11 10:08:49 | sdi: udev property ID_WWN whitelisted
Jul 11 10:08:49 | sdi: device node name blacklisted
Jul 11 10:08:49 | sdj: udev property ID_WWN whitelisted
Jul 11 10:08:49 | sdj: device node name blacklisted
Jul 11 10:08:49 | sdp: udev property ID_WWN whitelisted
Jul 11 10:08:49 | sdp: mask = 0x1f
Jul 11 10:08:49 | sdp: dev_t = 8:240
Jul 11 10:08:49 | sdp: size = 16777216
Jul 11 10:08:49 | sdp: vendor = IBM
Jul 11 10:08:49 | sdp: product = 2145
Jul 11 10:08:49 | sdp: rev = 0000
Jul 11 10:08:49 | sdp: h:b:t:l = 2:0:0:0
Jul 11 10:08:49 | SCSI target 2:0:0 -> FC rport 2:0-0
Jul 11 10:08:49 | sdp: tgt_node_name = 0x500507680303ab7c
Jul 11 10:08:49 | sdp: path state = running
Jul 11 10:08:49 | sdp: 8192 cyl, 64 heads, 32 sectors/track, start at 0
Jul 11 10:08:49 | sdp: serial = 00c02043ab7eXX00
Jul 11 10:08:49 | sdp: get_state
Jul 11 10:08:49 | sdp: detect_checker = yes (setting: multipath.conf defaults/devices section)
Jul 11 10:08:49 | sdp: path_checker = tur (setting: array autodetected)
Jul 11 10:08:49 | sdp: checker timeout = 30 ms (setting: multipath internal)
Jul 11 10:08:49 | sdp: state = up
Jul 11 10:08:49 | sdp: getuid = "/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/%n" (setting: multipath.conf defaults/devices section)
Jul 11 10:08:49 | sdp: using deprecated getuid callout
Jul 11 10:08:49 | formatted callout = /lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/sdp
Jul 11 10:08:49 | sdp: uid = 36005076300810eadf800000000000155 (callout)
Jul 11 10:08:49 | sdp: detect_prio = yes (setting: multipath.conf defaults/devices section)
Jul 11 10:08:49 | loading /lib64/multipath/libprioalua.so prioritizer
Jul 11 10:08:49 | sdp: prio = alua (setting: array autodetected)
Jul 11 10:08:49 | sdp: prio args = "" (setting: array autodetected)
Jul 11 10:08:49 | sdp: reported target port group is 0
Jul 11 10:08:49 | sdp: aas = 00 [active/optimized]
Jul 11 10:08:49 | sdp: alua prio = 50
Jul 11 10:08:49 | sdq: udev property ID_WWN whitelisted
Jul 11 10:08:49 | sdq: mask = 0x1f
Jul 11 10:08:49 | sdq: dev_t = 65:0
Jul 11 10:08:49 | sdq: size = 16777216
Jul 11 10:08:49 | sdq: vendor = IBM
Jul 11 10:08:49 | sdq: product = 2145
Jul 11 10:08:49 | sdq: rev = 0000
Jul 11 10:08:49 | sdq: h:b:t:l = 2:0:1:0
Jul 11 10:08:49 | SCSI target 2:0:1 -> FC rport 2:0-1
Jul 11 10:08:49 | sdq: tgt_node_name = 0x500507680303ab7d
Jul 11 10:08:49 | sdq: path state = running
Jul 11 10:08:49 | sdq: 8192 cyl, 64 heads, 32 sectors/track, start at 0
Jul 11 10:08:49 | sdq: serial = 00c02043ab7eXX00
Jul 11 10:08:49 | sdq: get_state
Jul 11 10:08:49 | sdq: detect_checker = yes (setting: multipath.conf defaults/devices section)
Jul 11 10:08:49 | sdq: path_checker = tur (setting: array autodetected)
Jul 11 10:08:49 | sdq: checker timeout = 30 ms (setting: multipath internal)
Jul 11 10:08:49 | sdq: state = up
Jul 11 10:08:49 | sdq: getuid = "/lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/%n" (setting: multipath.conf defaults/devices section)
Jul 11 10:08:49 | sdq: using deprecated getuid callout
Jul 11 10:08:49 | formatted callout = /lib/udev/scsi_id --whitelisted --replace-whitespace --device=/dev/sdq
Jul 11 10:08:49 | sdq: uid = 36005076300810eadf800000000000155 (callout)
Jul 11 10:08:49 | sdq: detect_prio = yes (setting: multipath.conf defaults/devices section)
Jul 11 10:08:49 | sdq: prio = alua (setting: array autodetected)
Jul 11 10:08:49 | sdq: prio args = "" (setting: array autodetected)
Jul 11 10:08:49 | sdq: reported target port group is 1
Jul 11 10:08:49 | sdq: aas = 01 [active/non-optimized]
Jul 11 10:08:49 | sdq: alua prio = 10
Jul 11 10:08:49 | dm-0: blacklisted, udev property missing
Jul 11 10:08:49 | dm-1: blacklisted, udev property missing
Jul 11 10:08:49 | dm-2: blacklisted, udev property missing
Jul 11 10:08:49 | dm-3: blacklisted, udev property missing
===== paths list =====
uuid                              hcil    dev dev_t pri dm_st chk_st vend/prod
36005076300810eadf800000000000155 2:0:0:0 sdp 8:240 50  undef undef  IBM,2145
36005076300810eadf800000000000155 2:0:1:0 sdq 65:0  10  undef undef  IBM,2145
Jul 11 10:08:49 | params = 2 queue_if_no_path retain_attached_hw_handler 1 alua 2 1 service-time 0 1 2 8:240 1 1 service-time 0 1 2 65:0 1 1
Jul 11 10:08:49 | status = 2 0 1 0 2 1 A 0 1 2 8:240 A 5 0 1 E 0 1 2 65:0 A 6 0 1
Jul 11 10:08:49 | 36005076300810eadf800000000000156: disassemble map [2 queue_if_no_path retain_attached_hw_handler 1 alua 2 1 service-time 0 1 2 8:240 1 1 service-time 0 1 2 65:0 1 1 ]
Jul 11 10:08:49 | 8:240: path wwid appears to have changed. Using map wwid.

Jul 11 10:08:49 | 65:0: path wwid appears to have changed. Using map wwid.

Jul 11 10:08:49 | 36005076300810eadf800000000000156: disassemble status [2 0 1 0 2 1 A 0 1 2 8:240 A 5 0 1 E 0 1 2 65:0 A 6 0 1 ]
Jul 11 10:08:49 | sdp: udev property ID_WWN whitelisted
Jul 11 10:08:49 | sdq: udev property ID_WWN whitelisted
Jul 11 10:08:49 | tur checker refcount 2
Jul 11 10:08:49 | alua prioritizer refcount 2
Jul 11 10:08:49 | tur checker refcount 1
Jul 11 10:08:49 | alua prioritizer refcount 1
Jul 11 10:08:49 | unloading alua prioritizer
Jul 11 10:08:49 | unloading const prioritizer
Jul 11 10:08:49 | unloading tur checker


liuqing@huayun.com
 
From: Martin Wilck
Date: 2017-07-10 21:55
To: dm-devel; liuqing
Subject: Re: [dm-devel] Multipath ID not equal to LUN scsi ID
On Mon, 2017-07-10 at 17:58 +0800, liuqing@huayun.com wrote:
> Dear list,
> We have a FC storage and using multipathd to manager the FC paths.
> I've met an issue in this environment. The following is how to
> recreate the issue.
 
Which version of multipath-tools have you been using? Could you please
run multipathd with "-v 3" and provide the logs of the procedure below?
 
Is there a specific reason why you are using the "getuid_callout"
option rather than "uid_attribute"?
 
Regards,
Martin
 
> 
> =======
> 1. Map a LUN to host with LUN ID 0,
> 2. rescan fc_host, a new path will be found by multipath.
> 3. Unmap LUN 0.  path will failed as following.
> [root@localhost sys]# multipath -ll
> Jul 10 18:41:50 | sdp: couldn't get asymmetric access state
> Jul 10 18:41:50 | sdq: couldn't get asymmetric access state
> 36005076300810eadf800000000000156 dm-3 IBM,2145
> size=8.0G features='2 queue_if_no_path retain_attached_hw_handler' hw
> handler='1 alua' wp=rw
> |-+- policy='service-time 0' prio=0 status=enabled
> | `- 2:0:0:0 sdp 8:240 failed faulty running
> `-+- policy='service-time 0' prio=0 status=enabled
>   `- 2:0:1:0 sdq 65:0  failed faulty running
> 4. Map another LUN which have different ID_SERIAL but with the same
> LUN ID(0). Multipath(check_path function) will set the paths up, tur
> checker will successfully test the LUN. 
> [root@localhost sys]# /lib/udev/scsi_id --whitelisted --replace-
> whitespace --device=/dev/mapper/36005076300810eadf800000000000156
> 36005076300810eadf800000000000155
>  I think this is not right, although I do not remove the device by
> 'echo 1> /sys/block/sdp/device/delete'. A multipath is identified by
> LUN scsi ID, in this situation the multipath
> ID is not equal to LUN scsi ID. And if I add the previous LUN back,
> the paths will also be added to that multipath
> id(36005076300810eadf800000000000156).
> 
> Any suggestion on how to make it create a new node in /dev with
> correct scsi ID? Thanks.
> 
> 
> ===========
> [root@localhost sys]# cat /etc/multipath.conf
> defaults {
>     features "0"
>     no_path_retry           queue
>     getuid_callout "/lib/udev/scsi_id --whitelisted --replace-
> whitespace --device=/dev/%n"
> }
> blacklist {
>     devnode "sd[a-m]"
> }
> 
> liuqing@huayun.com
> --
> dm-devel mailing list
> dm-devel@redhat.com
> https://www.redhat.com/mailman/listinfo/dm-devel
 
-- 
Dr. Martin Wilck <mwilck@suse.com>, Tel. +49 (0)911 74053 2107
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard, Graham Norton
HRB 21284 (AG Nürnberg)
 

[-- Attachment #1.2: Type: text/html, Size: 20738 bytes --]

[-- Attachment #2: Type: text/plain, Size: 0 bytes --]