From mboxrd@z Thu Jan 1 00:00:00 1970 From: "liuqing@huayun.com" Subject: Re: Multipath ID not equal to LUN scsi ID Date: Tue, 11 Jul 2017 11:49:57 +0800 Message-ID: <2017071111495659851849@huayun.com> Mime-Version: 1.0 Content-Type: multipart/mixed; boundary="===============8327867091457571325==" Return-path: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: dm-devel-bounces@redhat.com Errors-To: dm-devel-bounces@redhat.com To: maier , dm-devel Cc: liuqing List-Id: dm-devel.ids This is a multi-part message in MIME format. --===============8327867091457571325== Content-Type: multipart/alternative; boundary="----=_001_NextPart427568875283_=----" This is a multi-part message in MIME format. ------=_001_NextPart427568875283_=---- Content-Type: text/plain; charset="GB2312" Content-Transfer-Encoding: base64 SGkgU3RlZmZlbiwNCk9uY2UgYSBuZXcgYXJyaXZlZCBMVU4gbWFwcGVkIHdlIHdpbGwgZG8gcmVz Y2FuIGJ5ICJlY2hvICctIC0gLScgPiAvc3lzL2NsYXNzL3Njc2lfaG9zdC9ob3N0Mi9zY2FuIi4g QWZ0ZXIgcmVzY2FuIG9ubHkgdGhlIHNjc2lfaWQgdG9vbCBnaXZlIHRoZSByaWdodCBzZXJpYWwg aWQsIHVkZXZhZG0gc3RpbGwgZ290IHRoZSBwcnZpb3VzIG9uZS4NCkkgaGF2ZSBtb25pdG9yIHRo ZSB1ZGV2IGV2ZW50IGJ5IHVkZXZhZG0gbW9uaXRvciB3aGlsZSBtYXBwaW5nIGEgbmV3IExVTiB0 byB0aGUgaG9zdCwgd2hvIHdpbGwgcmV1c2UgdGhlIG9yaWdpbmFsIHBhdGguIE5vIGFkZCBldmVu dCBpcyB0cmlnZ2VycmVkLCBvbmx5IGRtLVggZW1pdHMgYSBjaGFuZ2UgZXZlbnQuIA0KQnV0IGlm IHRoZSBvcmlnaW5hbCBwYXRoIGlzIGRlbGV0ZWQocmVtb3ZlZCkgdGhlbiBhZGQgZXZlbnQgd2ls bCBiZSB0cmlnZ2VycmVkLg0KDQpGbHVzaCB0aGUgb2xkIFdXSUQgY291bGQgbWFrZSB0aGUgV1dJ RCBjb3JyZWN0IGJ1dCB0aGUgc2l6ZSBpcyBzdGlsbCBpbmNvcnJlY3QgYXMgZm9sbG93aW5nLiBB bmQgMzYwMDUwNzYzMDA4MTBlYWRmODAwMDAwMDAwMDAwMTU1IGlzIGFjdHVhbGx5IDVHQi4gDQpb cm9vdEBsb2NhbGhvc3Qgfl0jIG11bHRpcGF0aCAtdjINCkp1bCAxMSAxMjozNjowNiB8IHNkcDog dXNpbmcgZGVwcmVjYXRlZCBnZXR1aWQgY2FsbG91dA0KSnVsIDExIDEyOjM2OjA2IHwgc2RxOiB1 c2luZyBkZXByZWNhdGVkIGdldHVpZCBjYWxsb3V0DQpjcmVhdGU6IDM2MDA1MDc2MzAwODEwZWFk ZjgwMDAwMDAwMDAwMDE1NSB1bmRlZiBJQk0sMjE0NQ0Kc2l6ZT04LjBHIGZlYXR1cmVzPScxIHF1 ZXVlX2lmX25vX3BhdGgnIGh3aGFuZGxlcj0nMCcgd3A9dW5kZWYNCg0KDQpJIGJ1aWx0IHRoZSB0 b29sIHVzaW5nIGxhdGVzdCBjb2RlIGFuZCB0cmllZCBib3RoIGF0dHJpYnV0ZV9pZCBhbmQgZ2V0 dWlkX2NhbGxvdXQuIFRoZSBpc3N1ZSBleGlzdCBpbiBib3RoIGNvbmZpZ3VyYXRpb24uDQoNCg0K PiBEZWFyIGxpc3QsDQo+IFdlIGhhdmUgYSBGQyBzdG9yYWdlIGFuZCB1c2luZyBtdWx0aXBhdGhk IHRvIG1hbmFnZXIgdGhlIEZDIHBhdGhzLg0KPiBJJ3ZlIG1ldCBhbiBpc3N1ZSBpbiB0aGlzIGVu dmlyb25tZW50LiBUaGUgZm9sbG93aW5nIGlzIGhvdyB0byByZWNyZWF0ZSB0aGUgaXNzdWUuDQo+ DQo+ID09PT09PT0NCj4gMS4gTWFwIGEgTFVOIHRvIGhvc3Qgd2l0aCBMVU4gSUQgMCwNCj4gMi4g cmVzY2FuIGZjX2hvc3QsIGEgbmV3IHBhdGggd2lsbCBiZSBmb3VuZCBieSBtdWx0aXBhdGguDQo+ IDMuIFVubWFwIExVTiAwLiAgcGF0aCB3aWxsIGZhaWxlZCBhcyBmb2xsb3dpbmcuDQo+IFtyb290 QGxvY2FsaG9zdCBzeXNdIyBtdWx0aXBhdGggLWxsDQo+IEp1bCAxMCAxODo0MTo1MCB8IHNkcDog Y291bGRuJ3QgZ2V0IGFzeW1tZXRyaWMgYWNjZXNzIHN0YXRlDQo+IEp1bCAxMCAxODo0MTo1MCB8 IHNkcTogY291bGRuJ3QgZ2V0IGFzeW1tZXRyaWMgYWNjZXNzIHN0YXRlDQo+IDM2MDA1MDc2MzAw ODEwZWFkZjgwMDAwMDAwMDAwMDE1NiBkbS0zIElCTSwyMTQ1DQo+IHNpemU9OC4wRyBmZWF0dXJl cz0nMiBxdWV1ZV9pZl9ub19wYXRoIHJldGFpbl9hdHRhY2hlZF9od19oYW5kbGVyJyBod2hhbmRs ZXI9JzEgYWx1YScgd3A9cncNCj4gfC0rLSBwb2xpY3k9J3NlcnZpY2UtdGltZSAwJyBwcmlvPTAg c3RhdHVzPWVuYWJsZWQNCj4gfCBgLSAyOjA6MDowIHNkcCA4OjI0MCBmYWlsZWQgZmF1bHR5IHJ1 bm5pbmcNCj4gYC0rLSBwb2xpY3k9J3NlcnZpY2UtdGltZSAwJyBwcmlvPTAgc3RhdHVzPWVuYWJs ZWQNCj4gICAgYC0gMjowOjE6MCBzZHEgNjU6MCAgZmFpbGVkIGZhdWx0eSBydW5uaW5nDQo+IDQu IE1hcCBhbm90aGVyIExVTiB3aGljaCBoYXZlIGRpZmZlcmVudCBJRF9TRVJJQUwgYnV0IHdpdGgg dGhlIHNhbWUgTFVOIElEKDApLg0KIA0KRGlkIHlvdSAicmVzY2FuIiB0aGUgU0NTSSBkZXZpY2Ug dmlhIHN5c2ZzIHRvIGxldCBMaW51eCBrbm93IHRoYXQgdGhpcw0KaXMgbm93IGluIGZhY3QgYSBk aWZmZXJlbnQgZGV2aWNlPw0KIA0KQUZBSUssIExpbnV4IGRlY29kZXMgU0NTSSBzZW5zZSBkYXRh IGZvciBMVU5zIHJlbWFwcGVkIG9uIHRoZSBzdG9yYWdlDQp0YXJnZXQgYW5kIGVtaXRzIGEgdWRl diBldmVudCwgYnV0IEknbSBub3QgYXdhcmUgb2YgYW55IGRlZmF1bHQgdWRldg0KcnVsZSB0aGF0 IHdvdWxkIGFjdHVhbGx5IHJlYWN0LiBUaGUga2VybmVsIGl0c2VsZiBkb2VzIG5vdCByZWFjdCBv dGhlcg0KdGhhbiBjcmVhdGluZyB0aGUgdWV2ZW50Lg0KIA0KPiBNdWx0aXBhdGgoY2hlY2tfcGF0 aCBmdW5jdGlvbikgd2lsbCBzZXQgdGhlIHBhdGhzIHVwLCB0dXIgY2hlY2tlciB3aWxsIHN1Y2Nl c3NmdWxseSB0ZXN0IHRoZSBMVU4uDQo+IFtyb290QGxvY2FsaG9zdCBzeXNdIyAvbGliL3VkZXYv c2NzaV9pZCAtLXdoaXRlbGlzdGVkIC0tcmVwbGFjZS13aGl0ZXNwYWNlIC0tZGV2aWNlPS9kZXYv bWFwcGVyLzM2MDA1MDc2MzAwODEwZWFkZjgwMDAwMDAwMDAwMDE1Ng0KPiAzNjAwNTA3NjMwMDgx MGVhZGY4MDAwMDAwMDAwMDAxNTUNCj4gICBJIHRoaW5rIHRoaXMgaXMgbm90IHJpZ2h0LCBhbHRo b3VnaCBJIGRvIG5vdCByZW1vdmUgdGhlIGRldmljZSBieSAnZWNobyAxPiAvc3lzL2Jsb2NrL3Nk cC9kZXZpY2UvZGVsZXRlJy4gQSBtdWx0aXBhdGggaXMgaWRlbnRpZmllZCBieSBMVU4gc2NzaSBJ RCwgaW4gdGhpcyBzaXR1YXRpb24gdGhlIG11bHRpcGF0aA0KPiBJRCBpcyBub3QgZXF1YWwgdG8g TFVOIHNjc2kgSUQuIEFuZCBpZiBJIGFkZCB0aGUgcHJldmlvdXMgTFVOIGJhY2ssIHRoZSBwYXRo cyB3aWxsIGFsc28gYmUgYWRkZWQgdG8gdGhhdCBtdWx0aXBhdGggaWQoMzYwMDUwNzYzMDA4MTBl YWRmODAwMDAwMDAwMDAwMTU2KS4NCj4NCj4gQW55IHN1Z2dlc3Rpb24gb24gaG93IHRvIG1ha2Ug aXQgY3JlYXRlIGEgbmV3IG5vZGUgaW4gL2RldiB3aXRoIGNvcnJlY3Qgc2NzaSBJRD8gVGhhbmtz Lg0KIA0KSWYgeW91IGFzc3VtZSB0aGUgb2xkIG1hcHBlZCB2b2x1bWUgd291bGQgYmUgZ29uZSwg eW91IGNvdWxkIGFsc28gZmx1c2gNCnRoZSBtcGF0aCBtYXAgd2l0aCB0aGUgb2xkIFdXSUQuDQog DQpBZnRlciB0aGUgU0NTSSBkZXZpY2UgcmVzY2FuIGl0IHNob3VsZCBjcmVhdGUgYSBuZXcgbWFw IHdpdGggdGhlIG5ldyBXV0lELg0KIA0KPg0KPiA9PT09PT09PT09PQ0KPiBbcm9vdEBsb2NhbGhv c3Qgc3lzXSMgY2F0IC9ldGMvbXVsdGlwYXRoLmNvbmYNCj4gZGVmYXVsdHMgew0KPiAgICAgIGZl YXR1cmVzICIwIg0KPiAgICAgIG5vX3BhdGhfcmV0cnkgICAgICAgICAgIHF1ZXVlDQo+ICAgICAg Z2V0dWlkX2NhbGxvdXQgIi9saWIvdWRldi9zY3NpX2lkIC0td2hpdGVsaXN0ZWQgLS1yZXBsYWNl LXdoaXRlc3BhY2UgLS1kZXZpY2U9L2Rldi8lbiINCiANCklJUkMsIHVzaW5nIGFuIGV4cGxpY2l0 IGdldHVpZF9jYWxsb3V0IGhhcyBiZWVuIGRlcHJlY2F0ZWQgaW4gbGF0ZXINCnJlbGVhc2VzIG9m IG11bHRpcGF0aCB0b29scyBhbmQgaXQgd291bGQgdXNlIGJ1aWx0aW4gY29kZSB0byBkZXRlcm1p bmUNCnRoZSBuZWNlc3NhcnkgaW5mb3JtYXRpb24sIHBvdGVudGlhbGx5IGV2ZW4gdXNpbmcgaW5x dWlyeSBkYXRhIGNhY2hlZCBpbg0KdGhlIGtlcm5lbCAoZXhwb3J0ZWQgdmlhIHN5c2ZzKS4NCiAN CklmIHlvdXIgdG9vbGNoYWluIGlzIHJlY2VudCBlbm91Z2gsIHdvdWxkIGl0IHdvcmsgd2l0aG91 dCB0aGlzIGNvbmYgc2V0dGluZz8NCiANCj4gfQ0KPiBibGFja2xpc3Qgew0KPiAgICAgIGRldm5v ZGUgInNkW2EtbV0iDQo+IH0NCiANCi0tDQpNaXQgZnJldW5kbGljaGVuIEdyPz9lbiAvIEtpbmQg cmVnYXJkcw0KU3RlZmZlbiBNYWllcg0KIA0KTGludXggb24geiBTeXN0ZW1zIERldmVsb3BtZW50 DQogDQpJQk0gRGV1dHNjaGxhbmQgUmVzZWFyY2ggJiBEZXZlbG9wbWVudCBHbWJIDQpWb3JzaXR6 ZW5kZSBkZXMgQXVmc2ljaHRzcmF0czogTWFydGluYSBLb2VkZXJpdHoNCkdlc2NoYWVmdHNmdWVo cnVuZzogRGlyayBXaXR0a29wcA0KU2l0eiBkZXIgR2VzZWxsc2NoYWZ0OiBCb2VibGluZ2VuDQpS ZWdpc3RlcmdlcmljaHQ6IEFtdHNnZXJpY2h0IFN0dXR0Z2FydCwgSFJCIDI0MzI5NA0KDQoNCmxp dXFpbmdAaHVheXVuLmNvbQ0K ------=_001_NextPart427568875283_=---- Content-Type: text/html; charset="GB2312" Content-Transfer-Encoding: quoted-printable =0AHi Steffen,
Once a new arrived LUN mapped we will d= o rescan by "echo '- - -' > /sys/class/sc= si_host/host2/scan". After rescan only the scsi_id tool give the right ser= ial id, udevadm still got the prvious one.
I have = monitor the udev event by udevadm monitor while mapping a new LUN to the h= ost, who will reuse the original path. No add event is triggerred, only dm= -X emits a change event. 
But if the o= riginal path is deleted(removed) then add event will be triggerred.=

Flush the old= WWID could make the WWID correct but the size is still incorrect as follo= wing. And 36005076300810eadf800000000000155 is actually 5GB. 
= [root@localhost ~]# multipath -v2
Jul 11 12:36= :06 | sdp: using deprecated getuid calloutJul 11 12:36:06 | sdq: using deprecated&nb= sp;getuid callout
create: 36005076300810eadf800000000000155&n= bsp;undef IBM,2145
size=3D8.0G features=3D'1 queue_if_no= _path' hwhandler=3D'0' wp=3Dundef


I built the tool using latest code and tried both attribute_id and g= etuid_callout. The issue exist in both configuration.

<= /div>

=0A
> Dear list,
> We have a FC storage and using multipathd= to manager the FC paths.
> I've met an= issue in this environment. The following is how to recreate the issue.
>
> = =3D=3D=3D=3D=3D=3D=3D
> 1. Map a LUN t= o host with LUN ID 0,
> 2. rescan fc_h= ost, a new path will be found by multipath.
> 3. Unmap LUN 0.  path will failed as following.
> [root@localhost sys]# multipath -ll
> Jul 10 18:41:50 | sdp: couldn't get asymmetric access= state
> Jul 10 18:41:50 | sdq: couldn't= get asymmetric access state
> 36005076= 300810eadf800000000000156 dm-3 IBM,2145
&= gt; size=3D8.0G features=3D'2 queue_if_no_path retain_attached_hw_handler'= hwhandler=3D'1 alua' wp=3Drw
> |-+- pol= icy=3D'service-time 0' prio=3D0 status=3Denabled
> | `- 2:0:0:0 sdp 8:240 failed faulty running
> `-+- policy=3D'service-time 0' prio=3D0 status=3Denabled=
>    `- 2:0:1:0 sdq 65:0=   failed faulty running
> 4. Map a= nother LUN which have different ID_SERIAL but with the same LUN ID(0).
 
Did you= "rescan" the SCSI device via sysfs to let Linux know that this
is now in fact a different device?
 
AFAIK, Linux decodes S= CSI sense data for LUNs remapped on the storage
target and emits a udev event, but I'm not aware of any default udev<= /div>
rule that would actually react. The kernel = itself does not react other
than creating= the uevent.
 
> Multipath(check_path function) will set the paths up, tur che= cker will successfully test the LUN.
>= [root@localhost sys]# /lib/udev/scsi_id --whitelisted --replace-whitespac= e --device=3D/dev/mapper/36005076300810eadf800000000000156
> 36005076300810eadf800000000000155
>   I think this is not right, although I do not = remove the device by 'echo 1> /sys/block/sdp/device/delete'. A multipat= h is identified by LUN scsi ID, in this situation the multipath
> ID is not equal to LUN scsi ID. And if I add the= previous LUN back, the paths will also be added to that multipath id(3600= 5076300810eadf800000000000156).
>
=
> Any suggestion on how to make it create a n= ew node in /dev with correct scsi ID? Thanks.
 
If you assume the old mapped volu= me would be gone, you could also flush
the = mpath map with the old WWID.
 
<= div style=3D"font-family: =CE=A2=C8=ED=D1=C5=BA=DA, Tahoma; font-size: 14p= x; line-height: normal;">After the SCSI device rescan it should create a n= ew map with the new WWID.
 
>
> =3D=3D=3D= =3D=3D=3D=3D=3D=3D=3D=3D
> [root@local= host sys]# cat /etc/multipath.conf
> de= faults {
>     = features "0"
>    &= nbsp; no_path_retry         &= nbsp; queue
>    &nb= sp; getuid_callout "/lib/udev/scsi_id --whitelisted --replace-whitespace -= -device=3D/dev/%n"
 
IIRC, using an explicit getuid_callout has been deprecated= in later
releases of multipath tools and i= t would use builtin code to determine
the = necessary information, potentially even using inquiry data cached in
=
the kernel (exported via sysfs).
 
If your toolchain i= s recent enough, would it work without this conf setting?
 
> }
> blacklist {
>&nb= sp;     devnode "sd[a-m]"
> }
 
--
Mit freundlichen Gr??en / Kind rega= rds
Steffen Maier
 
Linux on z Systems Development=
 
IBM = Deutschland Research & Development GmbH
Vorsitzende des Aufsichtsrats: Martina Koederitz
Geschaeftsfuehrung: Dirk Wittkopp
S= itz der Gesellschaft: Boeblingen
Registerge= richt: Amtsgericht Stuttgart, HRB 243294
=0A
liuqing@huayun.com
=0A
------=_001_NextPart427568875283_=------ --===============8327867091457571325== Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Content-Disposition: inline --===============8327867091457571325==--