WWID / SerialNo in sysfs?

WWID / SerialNo in sysfs?

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 687 bytes --]

Hi,

I would like to add a few lines to the SCSI scanning code to retrieve
WWID and/or serial number from pages INQUIRY pages 0x83 and 0x80.

I'm aware that this can be done from userspace. I've done it before
(scsidev). However, you need some code to relate device nodes 
(sg, sd) to device IDs (C:B:T:U), thus it's non-trivial to do.

Thus, IMHO, it would be good to retrieve WWID and/or serial no
and export it via sysfs to make persistent device naming in userspace
easier. The USB subsystem already does it.

Regards,
-- 
Kurt Garloff  <garloff@suse.de>                            Cologne, DE 
SUSE LINUX AG, Nuernberg, DE                          SUSE Labs (Head)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: WWID / SerialNo in sysfs?

[James Bottomley]

On Mon, 2004-02-09 at 11:08, Kurt Garloff wrote:
> I would like to add a few lines to the SCSI scanning code to retrieve
> WWID and/or serial number from pages INQUIRY pages 0x83 and 0x80.
> 
> I'm aware that this can be done from userspace. I've done it before
> (scsidev). However, you need some code to relate device nodes 
> (sg, sd) to device IDs (C:B:T:U), thus it's non-trivial to do.
> 
> Thus, IMHO, it would be good to retrieve WWID and/or serial no
> and export it via sysfs to make persistent device naming in userspace
> easier. The USB subsystem already does it.

We tried that before.  The code is just too fragile and ridden with
policy to be workable (it was finally taken out because it annoyed too
many usb devices).

The SCSI part of udev already has most of this implemented at the user
level, so you should just be able to reuse that code.

James

Re: WWID / SerialNo in sysfs?

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 632 bytes --]

Hi,

On Mon, Feb 09, 2004 at 11:20:26AM -0500, James Bottomley wrote:
> The SCSI part of udev already has most of this implemented at the user
> level, so you should just be able to reuse that code.

As you know I have everything in scsidev as well already since long ;-)
Nice code duplication though ...

The nontrivial thing was always to make the relation sg/sd/... device
to C:B:T:U tuple. With sysfs it's at least possible to do that in
a reliable way.

Regards,
-- 
Kurt Garloff  <garloff@suse.de>                            Cologne, DE 
SUSE LINUX AG, Nuernberg, DE                          SUSE Labs (Head)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: WWID / SerialNo in sysfs?

[James Bottomley]

On Mon, 2004-02-09 at 11:29, Kurt Garloff wrote:
> The nontrivial thing was always to make the relation sg/sd/... device
> to C:B:T:U tuple. With sysfs it's at least possible to do that in
> a reliable way.

Actually, it should be unnecessary in 2.6; the sd devices accept SG_IO
directly.

James

Re: WWID / SerialNo in sysfs?

[Patrick Mansfield]

On Mon, Feb 09, 2004 at 05:29:06PM +0100, Kurt Garloff wrote:
> Hi,
> 
> On Mon, Feb 09, 2004 at 11:20:26AM -0500, James Bottomley wrote:
> > The SCSI part of udev already has most of this implemented at the user
> > level, so you should just be able to reuse that code.
> 
> As you know I have everything in scsidev as well already since long ;-)
> Nice code duplication though ...

Yes ;-) 

I was hoping you were supporting scsidev as backward compatible with 2.6,
and you could get users to migrate to udev + scsi_id.

I made scsi_id as an only functional with sysfs (only 2.6), and it must be
disconnected from the naming done by udev. Plus, disconnecting it from the
naming means it can be used by multipath configuration code.

Also it gets model + vendor via sysfs, mainly to avoid potential issues
avoided in scsi_scan.c when sending a standard INQUIRY (well, AFAICT no
one is using BLIST_INQUIRY_36 or BLIST_INQUIRY_58, but if they did, it
avoids those issue). This still saves an extra INQUIRY command from being
sent to the device.

Being able to store the serial number in the udev database would be nice,
since we potentially have multiple users: standard interface (upper level
driver sd/sr/st/osst), sg, and multipath. We could store it into a sysfs
scsi_device attribute, but I am not pushing for that, and for now would
rather have the same commands sent multiple times.

Also has these items that could be done in scsidev:

- a white/black list text file, so we don't need to rebuild a kernel or
  recompile the user binary to add or remove devices from the lists

- page 0x80 support

- supports page 0x83 types 0 (vendor specific) and 1 (T10 vendor
  identification).

> The nontrivial thing was always to make the relation sg/sd/... device
> to C:B:T:U tuple. With sysfs it's at least possible to do that in
> a reliable way.

As James said we can send SG_IO commands to sd, plus all of the upper
level drivers, the only one not yet completed is osst.

-- Patrick Mansfield

Re: WWID / SerialNo in sysfs?

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 3598 bytes --]

Hi Patrick,

On Mon, Feb 09, 2004 at 09:04:21AM -0800, Patrick Mansfield wrote:
> On Mon, Feb 09, 2004 at 05:29:06PM +0100, Kurt Garloff wrote:
> > As you know I have everything in scsidev as well already since long ;-)
> > Nice code duplication though ...
> 
> Yes ;-) 
> 
> I was hoping you were supporting scsidev as backward compatible with 2.6,
> and you could get users to migrate to udev + scsi_id.

Yes, that's indeed my plan.

Make sure previous scsidev users can live with it on 2.6 without much
change and make sure future users go to udev + scsi_id.

> I made scsi_id as an only functional with sysfs (only 2.6), and it must be
> disconnected from the naming done by udev. Plus, disconnecting it from the
> naming means it can be used by multipath configuration code.

Well, accessing devices by persistent names, you'll normally want 
to offer the three:
(a) naming by path
(b) naming by device identifier: Serial/WWID
(c) naming by contents (uuid)

All of them could be useful, think about backups, multipathing, ...

scsidev provided (a) and (b).

But for the name by path (a), scsidev used the driver name to have 
some protection against scsi host renumbering (or PCI bus changes ...)

Thus my patch to export the names.

> Also it gets model + vendor via sysfs, mainly to avoid potential issues
> avoided in scsi_scan.c when sending a standard INQUIRY (well, AFAICT no
> one is using BLIST_INQUIRY_36 or BLIST_INQUIRY_58, but if they did, it
> avoids those issue). This still saves an extra INQUIRY command from being
> sent to the device.

scsidev does parse sysfs as well nowadays. 
I'll release 2.32 in a few seconds, where it even works well with 2.6.x.
(2.31 was geared to work with 2.6.0-test1, but too much changed since
 then.)

In 2.4, it parsed /proc/scsi/scsi, but I had to add some extensions to 
relate the information there to highlevel drivers ;-)

> Being able to store the serial number in the udev database would be nice,
> since we potentially have multiple users: standard interface (upper level
> driver sd/sr/st/osst), sg, and multipath. We could store it into a sysfs
> scsi_device attribute, but I am not pushing for that, and for now would
> rather have the same commands sent multiple times.

Well, the relation sg device to highlevel driver is useful to know.

> Also has these items that could be done in scsidev:
> 
> - a white/black list text file, so we don't need to rebuild a kernel or
>   recompile the user binary to add or remove devices from the lists

I don't understand what you're telling me here.

> - page 0x80 support

It's there.

> - supports page 0x83 types 0 (vendor specific) and 1 (T10 vendor
>   identification).

scsidev currently supports EUI-64 (2) and the 8byte NAA formats (the 
latter by just treating them as opaque 64bit quantities). The limitation
is that it's internally represented as 64bit number. I would need to
convert this to a string ...

> > The nontrivial thing was always to make the relation sg/sd/... device
> > to C:B:T:U tuple. With sysfs it's at least possible to do that in
> > a reliable way.
> 
> As James said we can send SG_IO commands to sd, plus all of the upper
> level drivers, the only one not yet completed is osst.

As long as sg is there and has it's uses, you want to know the relations
between the sg devices and the high level scsi devices.

Regards,
-- 
Kurt Garloff  <garloff@suse.de>                            Cologne, DE 
SUSE LINUX AG, Nuernberg, DE                          SUSE Labs (Head)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: WWID / SerialNo in sysfs?

[Patrick Mansfield]

On Tue, Feb 10, 2004 at 01:19:28AM +0100, Kurt Garloff wrote:
> Hi Patrick,
> 
> On Mon, Feb 09, 2004 at 09:04:21AM -0800, Patrick Mansfield wrote:
> > On Mon, Feb 09, 2004 at 05:29:06PM +0100, Kurt Garloff wrote:

> Well, accessing devices by persistent names, you'll normally want 
> to offer the three:
> (a) naming by path
> (b) naming by device identifier: Serial/WWID
> (c) naming by contents (uuid)

Is (c) the mount -U uuid?

> All of them could be useful, think about backups, multipathing, ...
> 
> scsidev provided (a) and (b).
> 
> But for the name by path (a), scsidev used the driver name to have 
> some protection against scsi host renumbering (or PCI bus changes ...)

Yes, and (a) is harder with the host number being always incremented.

> Thus my patch to export the names.

Does that only narrow the scope of the problem? If you have two host
adapters of the same name, the name is not enough to identify a particular
adapter, and we have to fall back to something else. Using the full sysfs
path with a wildcard for the host number would cover some cases (for
example, if the PCI bus id were constant).

> > Also has these items that could be done in scsidev:
> > 
> > - a white/black list text file, so we don't need to rebuild a kernel or
> >   recompile the user binary to add or remove devices from the lists
> 
> I don't understand what you're telling me here.

That we should have a white and black list, since there are devices that
do not properly support page 0x80 or page 0x83, and that it should not be
compiled into the command.

If you're system has all known good devices (high end server, and no low
cost USB mass storage devices will be attached to it), you might want to
use a black list; if you're system is generic, you might want a white
list.

> > - page 0x80 support
> 
> It's there.

OK, sorry I missed it.

-- Patrick Mansfield

Re: WWID / SerialNo in sysfs?

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 2100 bytes --]

On Mon, Feb 09, 2004 at 04:48:02PM -0800, Patrick Mansfield wrote:
> On Tue, Feb 10, 2004 at 01:19:28AM +0100, Kurt Garloff wrote:
> Yes, and (a) is harder with the host number being always incremented.
> 
> > Thus my patch to export the names.
> 
> Does that only narrow the scope of the problem? If you have two host
> adapters of the same name, the name is not enough to identify a particular
> adapter, and we have to fall back to something else. Using the full sysfs
> path with a wildcard for the host number would cover some cases (for
> example, if the PCI bus id were constant).

scsidev the host number and the unique_id (ioport) by default for 
historical reasons. But you can construct path names using scsi.alias.
You can encode the hostname and the unique_id (hostid) in there
to construct a path.

> > > Also has these items that could be done in scsidev:
> > > 
> > > - a white/black list text file, so we don't need to rebuild a kernel or
> > >   recompile the user binary to add or remove devices from the lists
> > 
> > I don't understand what you're telling me here.
> 
> That we should have a white and black list, since there are devices that
> do not properly support page 0x80 or page 0x83, and that it should not be
> compiled into the command.

Ah, I see.

> If you're system has all known good devices (high end server, and no low
> cost USB mass storage devices will be attached to it), you might want to
> use a black list; if you're system is generic, you might want a white
> list.

Lucky enough, all devices I tested so far survived scsidev ...
And I did not get bug reports. 

Maybe the patch to do it in the kernel should have just avoided it
for usb-storage and everything would have been fine ;-)

More seriously: I wonder whether a devices_are_likely_broken flag in
the host template that is set by usb-storage wouldn't save us from
a lot of hacks in SCSI.

Regards,
-- 
Kurt Garloff  <garloff@suse.de>                            Cologne, DE 
SUSE LINUX AG, Nuernberg, DE                          SUSE Labs (Head)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]