path checker open() flags, devmapper ioctl()

path checker open() flags, devmapper ioctl()

[Christophe Varoqui]

Notice to reader : real questions are hidden in this obfuscated
message ! I'll mark them with ** to augment the chances for them being
answered :)

Path checker maintainers,

** would you object to my changing the checkers open() flags from O_RDWR
to O_RDONLY ? ** Directio and readsector0 would not object, and all
other checkers use the file descriptor to submit SG_IO ioctls which do
not require Write capability.
Re-reading the SG documentation at http://sg.torque.net/sg/sg_io.html, I
understand it should be ok. In real life, scsi_id does exactly that for
example.

The particular issue I'm facing is that on a RHEL5 box presented with
EMC Symmetrix R2 Logical Units (write-protected LU receiving SRDF
updates from R1 LU), the checker bails out on open(/dev/sd?, O_RDWR|
O_NONBLOCK) returning EROFS. opening with O_RDONLY grants us a usable
fd, which give the checker a chance to report the actual path state.

I'd like to fix the checker inconsistency, but we'll hit another wall
short after that : because the device mapper refuses to honor the
R2-multipath-tables loading without the read-only devmap option.

This behavior is new to RHEL5. R2 LU could perfectly be multipathed in
RHEL4.
** Does someone have insight about this new kernel behaviour, and about
the multipath layer desired behaviour in this context ? ** I personnaly
would rather not care for promoting a devmap from RO to RW upon R1-R2
split, and the even more tricky demoting from RW to RO (what about io
flush ?) upon un-spliting.

Regards,
cvaroqui

Re: path checker open() flags, devmapper ioctl()

[Christophe Varoqui]

Le jeudi 17 avril 2008 à 22:55 +0200, Christophe Varoqui a écrit :
> Notice to reader : real questions are hidden in this obfuscated
> message ! I'll mark them with ** to augment the chances for them being
> answered :)
> 
> Path checker maintainers,
> 
> ** would you object to my changing the checkers open() flags from O_RDWR
> to O_RDONLY ? ** Directio and readsector0 would not object, and all
> other checkers use the file descriptor to submit SG_IO ioctls which do
> not require Write capability.
> Re-reading the SG documentation at http://sg.torque.net/sg/sg_io.html, I
> understand it should be ok. In real life, scsi_id does exactly that for
> example.
> 
Please disregard this first part : O_RDONLY is already the flag use in
libmultipath/discovey.c

> The particular issue I'm facing is that on a RHEL5 box presented with
> EMC Symmetrix R2 Logical Units (write-protected LU receiving SRDF
> updates from R1 LU), the checker bails out on open(/dev/sd?, O_RDWR|
> O_NONBLOCK) returning EROFS. opening with O_RDONLY grants us a usable
> fd, which give the checker a chance to report the actual path state.
> 
> I'd like to fix the checker inconsistency, but we'll hit another wall
> short after that : because the device mapper refuses to honor the
> R2-multipath-tables loading without the read-only devmap option.
> 
> This behavior is new to RHEL5. R2 LU could perfectly be multipathed in
> RHEL4.
> ** Does someone have insight about this new kernel behaviour, and about
> the multipath layer desired behaviour in this context ? ** I personnaly
> would rather not care for promoting a devmap from RO to RW upon R1-R2
> split, and the even more tricky demoting from RW to RO (what about io
> flush ?) upon un-spliting.
> 
> Regards,
> cvaroqui

Re: path checker open() flags, devmapper ioctl()

[Christophe Varoqui]

Here is an update with my findings of today.

Paths to R2 LU are flaged read-only upon detection, as can be seen with
"hdparm -r". hdparm is able to flag them rw (hdparm -r0). Once paths are
switched to rw, I still can't use multipath to load maps upon them.

But, issuing "blockdev --rereadpt" on the paths finally resolves this
issue.

I'm confused. What is expected from the multipath tools there ? I'd like
to have the multipath maps up and running for R2 LU, for them to be
ready the moment they get activated by the R1/R2 split. But I'm not sure
how to do that in the described context. Input welcome.

Regards,
cvaroqui