Re: /proc/scsi/map

Re: /proc/scsi/map

[Douglas Gilbert]

Andries.Brouwer@cwi.nl wrote:

> > How can one assign stable device names to SCSI devices in
> > case there are devices that may or may not be switched on or connected.
> 
> An interesting unsolved problem.
> [Your discussion confuses a few things, especially in the context
> of removable devices: a uuid lives on the disk, the C,B,T,U tends
> to identify the drive rather than the disk.]

In the lsml "[RFC] Persistent naming of scsi devices" thread
Martin K. Petersen <mkp@mkp.net> summarized scsi addressing issues
thus:
#
# What we want is (at least) three ways of addressing a device:
#
# 1. By content.  This is the persistent naming.  Think
#    filesystem/MD/LVM UUID.  This is what you put in /etc/fstab and
#    what metadisk systems use to assemble logical volumes.
#
#    Content referencing is used for accessing data.
#
# 2. By physical path.  This naming is not persistent.  Not even runtime
#    because hotplug, iSCSI and whatnot may mess things up.
#
#    Path naming is for discovery and recovery.  When you add an
#    unlabeled disk you want to reference it by path to give it a name.
#    When you have a failed disk on your system you want to know which
#    physical device to pull from the array.
#
# 3. By enumeration.  This is what the kernel happens to be using to
#    reference the device.  diskN.  Certainly not persistent.
#
#    Enumeration is for the kernel.

The /proc/scsi/map facility proposed by Kurt does a very
good job at tying together points 2) and 3) . As a bonus
it also shows the sg device to primary device mapping
(which is a major headache for me as the sg maintainer).

As for the physical path getting fooled by a re-plug, I would 
like to present the idea of a "device attach event number" 
which would be uniquely issued (ascending order) to each newly 
attached scsi device [an attach timestamp could be useful as well].
This is easy to implement, has a many-to-one
relationship with the physical path (i.e. C,B,T,U) but, at
any instant, has a one-to-one relationship. So the attach
event number can be used as an alias for C,B,T,U** and makes
no pretensions to be persistent from one kernel lifetime
to the next (on the same machine). The scsimon driver supports
a device "attach event number". If the idea is workable the
number should probably be put in the scsi mid-level (i.e.
Scsi_Device structure).

** The C,B,T,U tuple (or nexus) is already a handful and
will probably get uglier with 8-byte luns and iSCSI extensions.
The SCSI_IOCTL_GET_IDLUN tries to pack C,B,T,U into one integer
and obviously won't scale to an 8 byte lun. A new ioctl such as
SCSI_IOCTL_GET_PHYSICAL_PATH (for example) could fix "GET_IDLUN"'s
shortcomings and yield the device attach event number. 


Mike Sullivan (who started the above-mentioned "[RFC] Persistent 
naming of scsi devices" thread) recently presented patches on
lsml for a devnaming (and scsiname) utility. These patches  
address Martin's point 1) [content addressing].


There remains the issue of removable media (i.e. device stays
put but the media is changed). In the absence of asynchronous
event notification, the next (normal) scsi command receives a
"unit attention" to alert the kernel. I'm not sure if Mike's
patch addresses this issue.

> > Life would be easier if the scsi subsystem would just report which
> > SCSI device (uniquely identified by the controller,bus,target,unit tuple)
> > belongs to which high-level device.
> 
> Yes. I took your patch, ported it to 2.5, and tried it out.
> 
> # cat /proc/scsi/map
> # C,B,T,U       Type    onl     sg_nm   sg_dev  nm      dev(hex)
> 1,0,06,00       0x00    1       sg0     c:15:00 sda     b:08:00
> 2,0,00,00       0x00    1       sg1     c:15:01 sdb     b:08:10
> 2,0,00,01       0x00    1       sg2     c:15:02 sdc     b:08:20
> 3,0,00,00       0x00    1       sg3     c:15:03 sdd     b:08:30
> 3,0,00,01       0x00    1       sg4     c:15:04 sde     b:08:40
> 
> Very good - in combination with /proc/scsi/scsi this gives
> good information. I like it.

Adding INQUIRY strings would make it harder to parse and more
cluttered for humans to read.

> But just "cat /proc/scsi/map" is not good enough.
> From the above output alone one cannot easily guess which is which.
> One would need a small utility that reads /proc/scsi/map and
> /proc/scsi/scsi and produces something readable.

Note the possible race condition: reading /proc/scsi/scsi
then /proc/scsi/map (or vice versa) when a hotplug is
occurring...

BTW In lk 2.5 we store the whole INQUIRY response in the
Scsi_Device structure. Currently we have no ioctl to yield
this information to the user space :-(

> Will add sth to util-linux in case this gets accepted.

IMO as soon as lk 2.4.19 comes out, this patch should
be presented to Marcelo (for inclusion in lk 2.4.20). The 
sooner we get it into the lk 2.5 series the better. Could you 
forward your lk 2.5 version of Kurt's patch to Linus (and the
linux-scsi list)?

Doug Gilbert

Re: /proc/scsi/map

[Chris Adams]

Once upon a time, Doug Ledford  <dledford@redhat.com> said:
>Umm, this patently fails to meet the criteria you posted of "stable device 
>name".  Adding a controller to a system is just as likely to blow this 
>naming scheme to hell as it is to blow the traditional linux /dev/sd? 
>scheme.  IOW, even though the /proc/scsi/map file looks nice and usefull, 
>it fails to solve the very problem you are trying to solve.

I use OSF/1^WDigital Unix^WCompaq^WHP Tru64 Unix here at work, and with
version 5, they've got a nice persistent device naming system (I don't
claim to know how it works however).  The first hard drive found is
/dev/disk/dsk0, the second is /dev/disk/dsk1, etc.  It doesn't matter
how you get to the drive (which is important if you want multipathing -
the controller and bus should NOT figure into the device name or you are
going to have problems).  If you remove dsk0, there won't be another
dsk0 (unless you clean it out of the device database).

-- 
Chris Adams <cmadams@hiwaay.net>
Systems and Network Administrator - HiWAAY Internet Services
I don't speak for anybody but myself - that's enough trouble.

Re: /proc/scsi/map

[Andreas Dilger]

On Jun 18, 2002  15:06 -0500, Chris Adams wrote:
> I use OSF/1^WDigital Unix^WCompaq^WHP Tru64 Unix here at work, and with
> version 5, they've got a nice persistent device naming system (I don't
> claim to know how it works however).  The first hard drive found is
> /dev/disk/dsk0, the second is /dev/disk/dsk1, etc.  It doesn't matter
> how you get to the drive (which is important if you want multipathing -
> the controller and bus should NOT figure into the device name or you are
> going to have problems).  If you remove dsk0, there won't be another
> dsk0 (unless you clean it out of the device database).

Yes, this is the same as AIX - the physical connection is irrelevant.
Any attempts to identify disk devices by their physical connection
are only marginally better than the current "probe order" naming.

I have written a library which allows identifying devices by content.
It uses the UUIDs found in ext2/ext3/XFS/JFS/MD. I added support for
UUIDs in reiserfs and swap also - the reiserfs UUID support is in
recent kernels/tools, but I never got around to submitting a patch
for swap (just to reserve the fields in the swap header definition,
the kernel doesn't really care about the contents).

The code was built as part of e2fsprogs (so fsck could use it), but
again I didn't push my changes back to the stock e2fsprogs yet.  What
I'm really waiting for is to ensure the current library works well
with mount (so you can use "mount LABEL=blah" for all filesystem types).

The library is at the following URL if anyone is interested in hacking
mount to use it.  I have an agreement-in-principle from Andries Brouwer
to accept changes to the mount/swapon code to use libblkid.

http://www-mddsp.enel.ucalgary.ca/People/adilger/blkid/

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/

RE: /proc/scsi/map

[Heinz, Michael]

>From an infiniband point of view, this sounds great!

--
Michael Heinz
Infinicon Systems, Inc.
http://www.infiniconsys.com 

> -----Original Message-----
> From: Sancho Dauskardt [mailto:sancho@dauskardt.de]
> Sent: Saturday, June 15, 2002 3:50 PM
> To: Kurt Garloff; Linux kernel list; Linux SCSI list
> Cc: linux-usb-devel@lists.sourceforge.net;
> linux1394-devel@lists.sourceforge.net
> Subject: Re: /proc/scsi/map
> 
> 
> 
> >Life would be easier if the scsi subsystem would just report 
> which SCSI
> >device (uniquely identified by the 
> controller,bus,target,unit tuple) belongs
> >to which high-level device. The information is available in 
> the kernel.
> >
> >Attached patch does this:
> >garloff@pckurt:/raid5/Kernel/src $ cat /proc/scsi/map
> ># C,B,T,U       Type    onl     sg_nm   sg_dev  nm      dev(hex)
> >0,0,00,00       0x05    1       sg0     c:15:00 sr0     b:0b:00
> [...]
> 
> Great, this was really missing badly.
> 
> But how about adding another column: GUID.
> Most usb-storage and (all?) FireWire devices have such a 
> unique identitiy.
> In contrast to native SCSI devices, these emulated SCSI devices on 
> hot-plugging busses will change their LUNs/IDs. Therefor the 
> GUID is really 
> a must to be able to create stable names (laptop suspend, etc.).
> 
> Both usb-storage and iee1394-sbp2 know the GUID. It only needs to be 
> communicated..
> 
> I'd guess that FibreChannel has similar problems ?
> 
> - sda
> 
> -
> To unsubscribe from this list: send the line "unsubscribe 
> linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

Re: /proc/scsi/map

[Borsenkow Andrej]

> # cat /proc/scsi/map
> # C,B,T,U Type onl sg_nm sg_dev nm dev(hex)
> 1,0,06,00 0x00 1 sg0 c:15:00 sda b:08:00
> 2,0,00,00 0x00 1 sg1 c:15:01 sdb b:08:10
> 2,0,00,01 0x00 1 sg2 c:15:02 sdc b:08:20
> 3,0,00,00 0x00 1 sg3 c:15:03 sdd b:08:30
> 3,0,00,01 0x00 1 sg4 c:15:04 sde b:08:40

The device names <-> SCSI addresses is just a part of problem. You still
are not able to assign permanent controller numbers. If you have two
SCSI controllers there is no (general) way to assure that one of them
gets 1 and another 2. (It is possible to some extent with scsihosts
parameter but only if two controllers use different drivers). Which
means that if for some reason one of them is not present (failed) you
suddenly get wrong addresses - _totally_ wrong addresses.

In some cases (LAN interfaces) it may even lead to interesting security
problem.

What is needed is a _generic_ way to assign logical controller numbers
for physical devices. Legacy devices may be ignored in this respect, but
when you have unambiguous device address (like PCI) it is possible. 

-andrej

Re: /proc/scsi/map

[Andries.Brouwer]

    From dougg@torque.net  Sat Jun 15 23:10:06 2002

    In the lsml "[RFC] Persistent naming of scsi devices" thread
    Martin K. Petersen <mkp@mkp.net> summarized scsi addressing issues
    thus:
    #
    # What we want is (at least) three ways of addressing a device:
    #
    # 1. By content.  This is the persistent naming.  Think
    #    filesystem/MD/LVM UUID.  This is what you put in /etc/fstab and
    #    what metadisk systems use to assemble logical volumes.
    #
    #    Content referencing is used for accessing data.
    #
    # 2. By physical path.  This naming is not persistent.  Not even runtime
    #    because hotplug, iSCSI and whatnot may mess things up.
    #
    #    Path naming is for discovery and recovery.  When you add an
    #    unlabeled disk you want to reference it by path to give it a name.
    #    When you have a failed disk on your system you want to know which
    #    physical device to pull from the array.
    #
    # 3. By enumeration.  This is what the kernel happens to be using to
    #    reference the device.  diskN.  Certainly not persistent.
    #
    #    Enumeration is for the kernel.

Yes, a very nice summary. Maybe also

# 4. By fingerprint.  This naming is persistent, but need not
#    identify the device uniquely.  Think device type, vendor,
#    serial number, capacity.
#
#    Fingerprints are convenient for the user. "My ZIP drive".


    The /proc/scsi/map facility proposed by Kurt does a very
    good job at tying together points 2) and 3) .

Yes. Or, more precisely, it ties together the name sde and the
path (C,B,T,U) = (3,0,00,01). However, this path is not very
"physical". Indeed, these four numbers all arose by enumeration -
they are kernel numbers for some usb-storage device.

I can find the physical path, but that requires quite some digging in
/proc/scsi/scsi and /proc/scsi/sg/host_strs and /proc/scsi/usb-storage*/*
and /proc/bus/usb/devices.


    > Very good - in combination with /proc/scsi/scsi this gives
    > good information. I like it.

    Adding INQUIRY strings would make it harder to parse and more
    cluttered for humans to read.

Yes, I would like to leave that to a user-space utility.
Writing that will also show what kernel facilities are missing.

    > But just "cat /proc/scsi/map" is not good enough.
    > From the above output alone one cannot easily guess which is which.
    > One would need a small utility that reads /proc/scsi/map and
    > /proc/scsi/scsi and produces something readable.

    Note the possible race condition: reading /proc/scsi/scsi
    then /proc/scsi/map (or vice versa) when a hotplug is
    occurring...

Already reading /proc/scsi/map has a race: no locking is done.

    > Will add sth to util-linux in case this gets accepted.

    IMO as soon as lk 2.4.19 comes out, this patch should
    be presented to Marcelo (for inclusion in lk 2.4.20).

Yes, perhaps. I would like to get some experience first,
play with it, see what information is missing.

I have to admit that 2.5 is not very suitable for development
these days, it is not stable enough, but the appropriate path
is to try things out in 2.5 first, and when something crystallizes
out, backport.

    The sooner we get it into the lk 2.5 series the better. Could you 
    forward your lk 2.5 version of Kurt's patch to Linus (and the
    linux-scsi list)?

Yes, but I'll wait for 2.5.22 so that it is clear against
what tree to diff.

Andries

Re: /proc/scsi/map

[Andries.Brouwer]

    >Life would be easier if the scsi subsystem would just report which SCSI
    >device (uniquely identified by the controller,bus,target,unit tuple) belongs
    >to which high-level device. The information is available in the kernel.
    >
    >Attached patch does this:
    >garloff@pckurt:/raid5/Kernel/src $ cat /proc/scsi/map
    ># C,B,T,U       Type    onl     sg_nm   sg_dev  nm      dev(hex)
    >0,0,00,00       0x05    1       sg0     c:15:00 sr0     b:0b:00
    [...]

    Great, this was really missing badly.

    But how about adding another column: GUID.
    Most usb-storage and (all?) FireWire devices have such a unique identitiy.
    In contrast to native SCSI devices, these emulated SCSI devices on 
    hot-plugging busses will change their LUNs/IDs. Therefor the GUID is
    really a must to be able to create stable names (laptop suspend, etc.).

    Both usb-storage and iee1394-sbp2 know the GUID. It only needs to be 
    communicated..

The usb-storage GUID is just one random item of information.
One might wish for much more.

And: this information is already somewhere:

% cat /proc/scsi/sg/host_strs
SCSI host adapter emulation for IDE ATAPI devices
Iomega VPI2 (imm) interface
SCSI emulation for USB Mass Storage devices
SCSI emulation for USB Mass Storage devices
%

This tells me that host 0 will be in ide-scsi, host 1 in imm,
host 2 in usb-storage-0, host 3 in usb-storage-1.
And

% cat /proc/scsi/ide-scsi/0
SCSI host adapter emulation for IDE ATAPI devices
% cat /proc/scsi/imm/1
Version : 2.05 (for Linux 2.4.0)
Parport : parport0
Mode    : SPP
% cat /proc/scsi/usb-storage-0/2
   Host scsi2: usb-storage
       Vendor: DataFab Systems Inc.
      Product: USB CF+SM
Serial Number: 5DC69477C6
     Protocol: Transparent SCSI
    Transport: Datafab Bulk-Only
         GUID: 07c4a1090000005dc69477c6
     Attached: Yes
% cat /proc/scsi/usb-storage-1/3
   Host scsi3: usb-storage
       Vendor: SCM Microsystems Inc.
      Product: eUSB SmartMedia / CompactFlash 
Serial Number: None
     Protocol: Transparent SCSI
    Transport: Control/Bulk-EUSB/SDDR09
         GUID: 04e600050000000000000000
     Attached: Yes
%

A small utility that looks around in /proc is able to
find the GUID. Of course it would be better when fewer
heuristics were required.

Finally, the GUIDs you see here do not determine the LUN.
So, there is no well-defined line in /proc/scsi/map
where they would belong.

Andries

Re: /proc/scsi/map

[Douglas Gilbert]

Andries.Brouwer@cwi.nl wrote:
> 
>     >Life would be easier if the scsi subsystem would just report which SCSI
>     >device (uniquely identified by the controller,bus,target,unit tuple) belongs
>     >to which high-level device. The information is available in the kernel.
>     >
>     >Attached patch does this:
>     >garloff@pckurt:/raid5/Kernel/src $ cat /proc/scsi/map
>     ># C,B,T,U       Type    onl     sg_nm   sg_dev  nm      dev(hex)
>     >0,0,00,00       0x05    1       sg0     c:15:00 sr0     b:0b:00
>     [...]
> 
>     Great, this was really missing badly.
> 
>     But how about adding another column: GUID.
>     Most usb-storage and (all?) FireWire devices have such a unique identitiy.
>     In contrast to native SCSI devices, these emulated SCSI devices on
>     hot-plugging busses will change their LUNs/IDs. Therefor the GUID is
>     really a must to be able to create stable names (laptop suspend, etc.).
> 
>     Both usb-storage and iee1394-sbp2 know the GUID. It only needs to be
>     communicated..
> 
> The usb-storage GUID is just one random item of information.
> One might wish for much more.
> 
> And: this information is already somewhere:
> 
> % cat /proc/scsi/sg/host_strs
> SCSI host adapter emulation for IDE ATAPI devices
> Iomega VPI2 (imm) interface
> SCSI emulation for USB Mass Storage devices
> SCSI emulation for USB Mass Storage devices
> %
> 
> This tells me that host 0 will be in ide-scsi, host 1 in imm,
> host 2 in usb-storage-0, host 3 in usb-storage-1.
> And
> 
> % cat /proc/scsi/ide-scsi/0
> SCSI host adapter emulation for IDE ATAPI devices
> % cat /proc/scsi/imm/1
> Version : 2.05 (for Linux 2.4.0)
> Parport : parport0
> Mode    : SPP
> % cat /proc/scsi/usb-storage-0/2
>    Host scsi2: usb-storage
>        Vendor: DataFab Systems Inc.
>       Product: USB CF+SM
> Serial Number: 5DC69477C6
>      Protocol: Transparent SCSI
>     Transport: Datafab Bulk-Only
>          GUID: 07c4a1090000005dc69477c6
>      Attached: Yes
> % cat /proc/scsi/usb-storage-1/3
>    Host scsi3: usb-storage
>        Vendor: SCM Microsystems Inc.
>       Product: eUSB SmartMedia / CompactFlash
> Serial Number: None
>      Protocol: Transparent SCSI
>     Transport: Control/Bulk-EUSB/SDDR09
>          GUID: 04e600050000000000000000
>      Attached: Yes
> %
> 
> A small utility that looks around in /proc is able to
> find the GUID. Of course it would be better when fewer
> heuristics were required.
> 
> Finally, the GUIDs you see here do not determine the LUN.
> So, there is no well-defined line in /proc/scsi/map
> where they would belong.

In lk 2.5 we are hoping that driverfs will give us an
"information bridge" between scsi pseudo devices
and other driver subsystems such as ide, usb and iee1394.
Mike Sullivan's persistent naming patch (that I mentioned
in my previous post on this thread) adds driverfs capability
into the scsi subsystem. Driverfs capability is already
in the ide and usb subsystems.

procfs, driverfs and devfs ...

Doug Gilbert

Re: /proc/scsi/map

[Sancho Dauskardt]

>
>In lk 2.5 we are hoping that driverfs will give us an
>"information bridge" between scsi pseudo devices
>and other driver subsystems such as ide, usb and iee1394.
>Mike Sullivan's persistent naming patch (that I mentioned
>in my previous post on this thread) adds driverfs capability
>into the scsi subsystem. Driverfs capability is already
>in the ide and usb subsystems.
Driverfs will hopefully solve the problem, of "oh there's a SCSI device. 
how is it connected ?".

But to date, SCSI doesn't know about the GUID's, right ?
And without this, we won't get a uniform way of creating stable names for 
hot-plugable devices...

- sda

Re: /proc/scsi/map

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 2370 bytes --]

Hi Sancho,

On Sun, Jun 16, 2002 at 12:40:50AM +0200, Sancho Dauskardt wrote:
> >In lk 2.5 we are hoping that driverfs will give us an
> >"information bridge" between scsi pseudo devices
> >and other driver subsystems such as ide, usb and iee1394.
> >Mike Sullivan's persistent naming patch (that I mentioned
> >in my previous post on this thread) adds driverfs capability
> >into the scsi subsystem. Driverfs capability is already
> >in the ide and usb subsystems.
> Driverfs will hopefully solve the problem, of "oh there's a SCSI device. 
> how is it connected ?".
> 
> But to date, SCSI doesn't know about the GUID's, right ?
> And without this, we won't get a uniform way of creating stable names for 
> hot-plugable devices...

For the SCSI code, a device is uniquely identified by the CBTU tuple.
Every device has one and it's known to the SCSI code.
If you don't change your SCSI IDs (which rarely happens in practice) and
make sure the SCSI host adpters are loaded in the same order always (by
using scsihosts= or by just doing the modprobe in a certain order) you 
even a stable way to address devices.

The SCSI code does not know about GUIDs or such things. It's up to the
lowlevel code to report such things. Or change the SCSI code to collect
such information.

Low-level drivers that are supposed to have a lot of SCSI devices attached,
such as FC drivers, should provide a way of making a stable mapping from the
identifiers in FC Land (Unique IDs) to the identifiers in SCSI land (CBTU).
If not, they should at least report the mapping in some proc file.

A more general approach could be to build a unique identifier from a host
adapter identifier and the BTU and use this unique identifier (which would
be the UID of FC devices for FC devices) inside the SCSI code instead of
CBTU. Then the first column of /proc/scsi/map would report this identifier
instead ...

But that would be a more invasive change than the simple code that I added
to report the SCSI CBTU to SCSI high-level devices mapping.

Regards,
-- 
Kurt Garloff                   <kurt@garloff.de>         [Eindhoven, NL]
Physics: Plasma simulations    <K.Garloff@TUE.NL>     [TU Eindhoven, NL]
Linux: SCSI, Security          <garloff@suse.de>    [SuSE Nuernberg, DE]
 (See mail header or public key servers for PGP2 and GPG public keys.)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: /proc/scsi/map

[Sancho Dauskardt]

>
>     Both usb-storage and iee1394-sbp2 know the GUID. It only needs to be
>     communicated..
>
>The usb-storage GUID is just one random item of information.

Why is this 'one random item' ??
For usb-storage devices the GUID is built using Vendor ID & Device ID and 
the device's Serial Nr.

For identification purposes, the serial number is useless without Vendor ID 
& Device ID.
Ofcourse we'll never have a change of creating stable name for devices that 
don't have a serialnr.


>One might wish for much more.
>
>And: this information is already somewhere:
Sure. But
a) not easily readable
b) totall different for FireWire devices
c) race-conditions (reading multiple files).



>% cat /proc/scsi/usb-storage-0/2
>    Host scsi2: usb-storage
>        Vendor: DataFab Systems Inc.
>       Product: USB CF+SM
>Serial Number: 5DC69477C6
>      Protocol: Transparent SCSI
>     Transport: Datafab Bulk-Only
>          GUID: 07c4a1090000005dc69477c6
>      Attached: Yes
Exactly, but finding this out at the moment involves reading: 
/proc/scsi/scsi, scanning /proc/scsi/usb-storage-*, scanning 
/proc/scsi/usb-storage-X/*, reading /proc/scsi/usb-storage-X/Y.


>% cat /proc/scsi/usb-storage-1/3
>    Host scsi3: usb-storage
>        Vendor: SCM Microsystems Inc.
>       Product: eUSB SmartMedia / CompactFlash
>Serial Number: None
>      Protocol: Transparent SCSI
>     Transport: Control/Bulk-EUSB/SDDR09
>          GUID: 04e600050000000000000000
>      Attached: Yes

Well that's just SCM, huh ? The newer SCM-Orca chipset has a S/N.



>Finally, the GUIDs you see here do not determine the LUN.
>So, there is no well-defined line in /proc/scsi/map
>where they would belong.

But the LUN-map for such a device will never change ?
Incase a device has 4 LUN's, well have 4 /proc/scsi/map entries with the 
same GUID.

The hotplug agent will indeed be watching GUID+Lun.

- sda

Re: /proc/scsi/map

[Andries.Brouwer]

> How can one assign stable device names to SCSI devices in
> case there are devices that may or may not be switched on or connected.

An interesting unsolved problem.
[Your discussion confuses a few things, especially in the context
of removable devices: a uuid lives on the disk, the C,B,T,U tends
to identify the drive rather than the disk.]

> Life would be easier if the scsi subsystem would just report which
> SCSI device (uniquely identified by the controller,bus,target,unit tuple)
> belongs to which high-level device.

Yes. I took your patch, ported it to 2.5, and tried it out.

# cat /proc/scsi/map
# C,B,T,U       Type    onl     sg_nm   sg_dev  nm      dev(hex)
1,0,06,00       0x00    1       sg0     c:15:00 sda     b:08:00
2,0,00,00       0x00    1       sg1     c:15:01 sdb     b:08:10
2,0,00,01       0x00    1       sg2     c:15:02 sdc     b:08:20
3,0,00,00       0x00    1       sg3     c:15:03 sdd     b:08:30
3,0,00,01       0x00    1       sg4     c:15:04 sde     b:08:40

Very good - in combination with /proc/scsi/scsi this gives
good information. I like it.

But just "cat /proc/scsi/map" is not good enough.
>From the above output alone one cannot easily guess which is which.
One would need a small utility that reads /proc/scsi/map and
/proc/scsi/scsi and produces something readable.
Will add sth to util-linux in case this gets accepted.

Andries

Re: /proc/scsi/map

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 2253 bytes --]

Hi Andries,

On Sat, Jun 15, 2002 at 06:04:19PM +0200, Andries Brouwer wrote:
> > How can one assign stable device names to SCSI devices in
> > case there are devices that may or may not be switched on or connected.
> 
> An interesting unsolved problem.
> [Your discussion confuses a few things, especially in the context
> of removable devices: a uuid lives on the disk, the C,B,T,U tends
> to identify the drive rather than the disk.]

Sure. But those are the things that are normally proposed to relieve the
situation. (And yes, I got the uuid thing confused.)
I actually forgot one. LVM. There, signatures are stored on the disks.

> > Life would be easier if the scsi subsystem would just report which
> > SCSI device (uniquely identified by the controller,bus,target,unit tuple)
> > belongs to which high-level device.
> 
> Yes. I took your patch, ported it to 2.5, and tried it out.

Oh, I expected to do it myself if it does not get bashed too much ...
THANKS!
And, yes, I agree, I would prefer to know it's accepted in 2.5 before it's
applied to 2.4. It's just that I develop on 2.4 currently :-(

> Very good - in combination with /proc/scsi/scsi this gives
> good information. I like it.
> 
> But just "cat /proc/scsi/map" is not good enough.

I did not want to duplicate the information from /proc/scsi/scsi. The 
idea was that it should be straightforward to make device nodes from 
the information provided in map and to allow more elaborated code in
userspace to know where to collect information from.

> >From the above output alone one cannot easily guess which is which.
> One would need a small utility that reads /proc/scsi/map and
> /proc/scsi/scsi and produces something readable.
> Will add sth to util-linux in case this gets accepted.

That would be great! 
Because the mapping between sg and the other device is known, you can use
the sg device to do ioctls on it (or even send SCSI commands) or to use
the information from /proc/scsi/sg/ as well.

Regards,
-- 
Kurt Garloff  <garloff@suse.de>                          Eindhoven, NL
GPG key: See mail header, key servers         Linux kernel development
SuSE Linux AG, Nuernberg, DE                            SCSI, Security

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

/proc/scsi/map

[Kurt Garloff]

[-- Attachment #1.1: Type: text/plain, Size: 4733 bytes --]

Hi SCSI users,

from people using SCSI devices, there's one question that turns up again 
and again: How can one assign stable device names to SCSI devices in
case there are devices that may or may not be switched on or connected.

There are a couple of ways to address this problem:
(a) mount by uuid
(b) userspace programs that collect information to create
    alternative and persistent names / device nodes, such
    as Eric Youngdale's scsidev[1], Doug Gilbert's sg_map[2], scsimon[3],
    or Mike Sullivan's scsiname/devnaming[4]
(c) devfs

[1] http://www.garloff.de/kurt/linux/scsidev/
[2] http://www.torque.net/sg/
[3] http://www.torque.net/scsi/scsimon.html
[4] http://oss.software.ibm.com/devreg/

Unfortunately, those approaches all have some deficiencies.
Ad (a): Does only work for ext2 filesystems. For locating
        / one needs additional initrd work.
Ad (b): A considerable amount of work needs to be done in userspace:
        - For all devices types you need to probe possible devices
	- You need to do SCSI_IOCTL_GET_IDLUN to get controller,bus,
	  target and unit number
	The problem is that the collection of this information is 
	not always successful. If a medium is not inserted, the
	open() fails for some device types, despite O_NONBLOCK.
	Assumptions about the order of devices OTOH are not safe,
	as remove-single-device/add-single-device may result in a
	non-straightforward ordering.
Ad (c): devfs is currently not (yet?) an option for distributions
        due to security and stability considerations.

Life would be easier if the scsi subsystem would just report which SCSI
device (uniquely identified by the controller,bus,target,unit tuple) belongs
to which high-level device. The information is available in the kernel.

Attached patch does this:
garloff@pckurt:/raid5/Kernel/src $ cat /proc/scsi/map 
# C,B,T,U       Type    onl     sg_nm   sg_dev  nm      dev(hex)
0,0,00,00       0x05    1       sg0     c:15:00 sr0     b:0b:00
1,0,01,00       0x05    1       sg1     c:15:01 sr1     b:0b:01
1,0,02,00       0x01    1       sg2     c:15:02 osst0   c:ce:00
1,0,03,00       0x05    1       sg3     c:15:03 sr2     b:0b:02
1,0,05,00       0x00    1       sg4     c:15:04 sda     b:08:00
1,0,09,00       0x00    1       sg5     c:15:05 sdb     b:08:10
2,0,01,00       0x05    1       sg6     c:15:06 sr3     b:0b:03
2,0,02,00       0x01    1       sg7     c:15:07 osst1   c:ce:01
2,0,03,00       0x05    1       sg8     c:15:08 sr4     b:0b:04
2,0,05,00       0x00    1       sg9     c:15:09 sdc     b:08:20
2,0,09,00       0x00    1       sg10    c:15:0a sdd     b:08:30
3,0,10,00       0x00    1       sg11    c:15:0b sde     b:08:40
3,0,12,00       0x00    1       sg12    c:15:0c sdf     b:08:50

This allows a simple script to parse the map and create device nodes as
needed.

The patch does work the following way.
- Add a find_kdev() function pointer to the high-level driver template
  structure. The function takes a Scsi_Device pointer (points to a 
  low-level device) and returns a name and a kdev_t if the device
  is attached to this high-level driver.
- Implement the function for sg, sd, sr, st, osst
- Make scsi/scsi_proc iterate over all devices and calls the high-level
  drivers find_kdev() to find out about it.

Obviously, it can only report the assignment of high-level drivers,
if they are loaded, otherwise the last two columns will stay empty.
(sg is handled especially, as we know it supports all devices.)
If we attach a third high-level device driver, two more columns would show 
up. 
(Is this variable column number format a problem?)

The patch also includes a simple shell script that does assign
/dev/scsi/sdc2b0t9u0 type names to those devices and making a device nodes
(or optionally symlinks to the old name devices) with this name.

The design allows for two more things:
* using root=/dev/scsi/sdc1b0t9u0p5 without much additional code
  (patch follows in another mail)
* in case we want to support more than 128 scsi disks, the information
  about additional majors can be reported by /proc/scsi/map without further
  change

Patch is against 2.4.19pre10.

I'd like to get it accepted into the kernel. 
So please give your criticism ... 
I already got some by Doug Gilbert :-)

A patch for 2.5 should be done as well, if the design is OK, of course.

Regards,
-- 
Kurt Garloff                   <kurt@garloff.de>         [Eindhoven, NL]
Physics: Plasma simulations    <K.Garloff@TUE.NL>     [TU Eindhoven, NL]
Linux: SCSI, Security          <garloff@suse.de>    [SuSE Nuernberg, DE]
 (See mail header or public key servers for PGP2 and GPG public keys.)

[-- Attachment #1.2: scsi-map6.diff --]
[-- Type: text/plain, Size: 17108 bytes --]

diff -uNr linux-2.4.18.S18.fixed/Documentation/scsi-devs.sh linux-2.4.18.S18.sdmany/Documentation/scsi-devs.sh
--- linux-2.4.18.S18.fixed/Documentation/scsi-devs.sh	Thu Jan  1 01:00:00 1970
+++ linux-2.4.18.S18.sdmany/Documentation/scsi-devs.sh	Fri Jun 14 22:37:11 2002
@@ -0,0 +1,154 @@
+#!/bin/bash
+# Script to create SCSI device nodes according to their physical
+# address (host controller no,bus/channel,target SCSI ID,unit SCSI LUN)
+# using the /proc/scsi/map introduced in Linux 2.4.20pre/2.5.2x
+# (c) Kurt Garloff <garloff@suse.de>, 2002-06-13
+# License: GNU GPL v2
+
+# Settings
+
+# Target directory for device nodes/links
+TDIR=/dev/scsi
+
+# Make symbolic links or create fresh nodes in $TDIR
+# For link mode, the dev nodes will still be made if needed
+MODE=
+
+# In link mode, where is the real /dev dir ?
+DEVDIR=..
+
+# Ownership and permissions of newly created device nodes
+OWNER="root.disk"
+PERM="0660"
+SDIRPERM="0755"
+
+# Make scsi disk partitions (0 = off)
+SDPART=15
+
+# End of settings
+
+# Options
+# -f: Remove all old devs
+if test "$1" = "-f"; then
+  rm -f $TDIR/*
+  shift
+fi
+# -l: Make links 
+if test "$1" = "-l"; then
+  MODE=LINK
+  shift
+fi
+
+# Sanitize environment
+export LANG=posix
+unset IFS
+#umask `printf %o $[0777-$PERM]`
+PATH=/bin:/usr/bin:/sbin:/usr/sbin
+
+# Functions
+function exit_fail 
+{
+  echo $1
+  exit $2
+}
+
+# Build new scsi name based on oldname (prefix) and 
+# append host,channel,id,lun numbers that are unique
+# and persistent identifiers for the device.
+function builddevnm
+{ 
+  if test "${5:0:2}" = "sd"; then
+    NM="sdc$1b$2t${3#0}u${4#0}"
+  else
+    NM="${5%%[0-9]*}c$1b$2t${3#0}u${4#0}"
+  fi
+}
+
+# Make device node
+# Parameters filename type maj(hex) min(hex) minadd(dec)
+function mk_nod
+{
+  rm -f $1
+  mknod -m $PERM $1 $2 $[0x$3] $[0x$4+$5]
+  chown $OWNER $1
+}
+
+# Make device by creating node or symlinking
+# Parameters oldname device(tp:maj:min) newname minadd
+function mk_dev 
+{
+  IFS=":"
+  if test "$MODE" = "LINK"; then
+    if test ! -e $TDIR/$DEVDIR/$1; then
+      mk_nod $TDIR/$DEVDIR/$1 $2 $4
+    fi
+    ln -sf $DEVDIR/$1 $TDIR/$3
+  else
+    mk_nod $TDIR/$3 $2 $4
+  fi
+  unset IFS
+}
+
+# Make multiple devices in case we do have sd partitions
+# Parameters oldname device(tp:maj:min) newname
+function mk_devs
+{
+  mk_dev $1 $2 $3 0
+  # Handle partitions
+  if test "${1:0:2}" = "sd" -a "$SDPART" != "0"; then
+    #unset IFS
+    for no in `seq 1 $SDPART`; do
+      mk_dev $1$no $2 $3p$no $no
+    done
+  fi
+  # Handle nst/nosst
+  if test "${1:0:2}" = "st" -o "${1:0:4}" = "osst"; then
+    mk_dev n$1 $2 n$3 128
+  fi
+}
+
+# Main() Script
+if ! test -e /proc/scsi/map; then
+  exit_fail "/proc/scsi/map does not exist" 1
+fi
+if ! test -d $TDIR; then
+  mkdir -m $SDIRMODE $TDIR || exit_fail "Failed to create $TDIR" 2
+  chown $OWNER $TDIR
+fi
+
+# We might have been called by some sort of hotplug event
+# and are only support to add a single device
+# Syntax for this is [-l] add host channel id lun
+if test "$1" = "add"; then
+  CMPAGAINST="$2,$3,`printf %02i $4`,`printf %02i $5`"
+else
+  unset CMPAGAINST
+fi
+# The main processing loop
+while read cbtu tp onl sgnm sgdev othnm othdev oothnm oothdev rest; do
+  # Skip comment line(s)
+  if test "${cbtu:0:1}" = "#"; then continue; fi
+  # If we're just dealing with one device, do skip the others
+  if test ! -z "$CMPAGAINST" -a "$CMPAGAINST" != "$cbtu"; then continue; fi
+  # now parse line
+  IFS=","
+  # Test for validity of sg device
+  if test "$sgnm" != "sg?"; then
+    builddevnm $cbtu $sgnm
+    mk_dev $sgnm $sgdev $NM 0
+  fi
+  # There is possibly a second device
+  if test ! -z "$othnm" -a ! "$othnm" = "none"; then
+    IFS=","
+    builddevnm $cbtu $othnm
+    mk_devs $othnm $othdev $NM
+    # Maybe even a third one
+    if test ! -z "$oothnm" -a ! "$oothnm" = "none"; then
+      IFS=","
+      builddevnm $cbtu $oothnm
+      mk_devs $oothnm $oothdev $NM
+    fi
+  fi
+  unset IFS
+done < /proc/scsi/map
+
diff -uNr linux-2.4.18.S18.fixed/drivers/scsi/hosts.h linux-2.4.18.S18.sdmany/drivers/scsi/hosts.h
--- linux-2.4.18.S18.fixed/drivers/scsi/hosts.h	Wed Jun 12 11:37:09 2002
+++ linux-2.4.18.S18.sdmany/drivers/scsi/hosts.h	Wed Jun 12 11:53:54 2002
@@ -530,6 +530,7 @@
     void (*detach)(Scsi_Device *);
     int (*init_command)(Scsi_Cmnd *);     /* Used by new queueing code. 
                                            Selects command for blkdevs */
+    int (*find_kdev)(Scsi_Device *, char*, kdev_t*);  /* find back dev. */
 };
 
 void  scsi_initialize_queue(Scsi_Device * SDpnt, struct Scsi_Host * SHpnt);
diff -uNr linux-2.4.18.S18.fixed/drivers/scsi/osst.c linux-2.4.18.S18.sdmany/drivers/scsi/osst.c
--- linux-2.4.18.S18.fixed/drivers/scsi/osst.c	Fri Dec 21 18:41:55 2001
+++ linux-2.4.18.S18.sdmany/drivers/scsi/osst.c	Wed Jun 12 11:39:47 2002
@@ -156,6 +156,7 @@
 static int osst_attach(Scsi_Device *);
 static int osst_detect(Scsi_Device *);
 static void osst_detach(Scsi_Device *);
+static int osst_find_kdev(Scsi_Device *, char*, kdev_t*);
 
 struct Scsi_Device_Template osst_template =
 {
@@ -166,7 +167,8 @@
        detect:		osst_detect,
        init:		osst_init,
        attach:		osst_attach,
-       detach:		osst_detach
+       detach:		osst_detach,
+       find_kdev:	osst_find_kdev,
 };
 
 static int osst_int_ioctl(OS_Scsi_Tape *STp, Scsi_Request ** aSRpnt, unsigned int cmd_in,unsigned long arg);
@@ -5417,6 +5419,24 @@
 	return 0;
 }
 
+static int osst_find_kdev(Scsi_Device *sdp, char* nm, kdev_t *dev)
+{
+	int i;
+	OS_Scsi_Tape *ostp;
+	
+	if (sdp && sdp->type == TYPE_TAPE && osst_supports(sdp)) {
+		for (ostp = os_scsi_tapes[i = 0]; i < osst_template.dev_max;
+		     ostp = os_scsi_tapes[++i]) {
+			if (ostp && ostp->device == sdp) {
+				sprintf (nm, "osst%i", i);
+				*dev = MKDEV(OSST_MAJOR, i);
+				return 0;
+			}
+		}
+	}
+	return 1;
+}
+
 static int osst_attach(Scsi_Device * SDp)
 {
 	OS_Scsi_Tape * tpnt;
diff -uNr linux-2.4.18.S18.fixed/drivers/scsi/scsi.c linux-2.4.18.S18.sdmany/drivers/scsi/scsi.c
--- linux-2.4.18.S18.fixed/drivers/scsi/scsi.c	Wed Jun 12 11:37:07 2002
+++ linux-2.4.18.S18.sdmany/drivers/scsi/scsi.c	Fri Jun 14 12:49:15 2002
@@ -80,6 +80,7 @@
 
 #ifdef CONFIG_PROC_FS
 static int scsi_proc_info(char *buffer, char **start, off_t offset, int length);
+static int scsi_proc_map (char *buffer, char **start, off_t offset, int length);
 static void scsi_dump_status(int level);
 #endif
 
@@ -1554,6 +1555,70 @@
 }
 
 #ifdef CONFIG_PROC_FS
+/* 
+ * Output the mapping of physical devices (contorller,channel.id,lun)
+ * to devices (sg and other highlevel drivers) to /proc/scsi/map.
+ * Caveat: No locking is done, so if your scsi config changes during
+ * reading this file, you may read garbled data.	KG, 2002-06-14
+ */
+static int scsi_proc_map(char *buffer, char **start, off_t offset, int length)
+{
+	Scsi_Device *scd;
+	struct Scsi_Host *HBA_ptr;
+	int size = 0, len = 0, repeat = 0;
+	off_t begin = 0;
+	off_t pos = 0;
+
+	struct Scsi_Device_Template *sg_t;
+	do {
+		sg_t = scsi_devicelist;
+		while (sg_t && !(sg_t->tag && !strcmp(sg_t->tag, "sg")))
+			sg_t = sg_t->next;
+#ifdef CONFIG_KMOD
+		if (!repeat && !sg_t)
+			request_module("sg");
+	} while (!repeat++);
+#else
+	} while (0);
+#endif	
+	/*
+	 * First, see if there are any attached devices or not.
+	 */
+	for (HBA_ptr = scsi_hostlist; HBA_ptr; HBA_ptr = HBA_ptr->next) {
+		if (HBA_ptr->host_queue != NULL) {
+			break;
+		}
+	}
+	if (!HBA_ptr)
+		goto stop_map_output;
+	
+	size = sprintf(buffer + len, "# C,B,T,U\tType\tonl\tsg_nm\tsg_dev\tnm\tdev(hex)\n");
+	len += size;
+	pos = begin + len;
+
+	for (HBA_ptr = scsi_hostlist; HBA_ptr; HBA_ptr = HBA_ptr->next) {
+		for (scd = HBA_ptr->host_queue; scd; scd = scd->next) {
+			proc_print_scsimap(scd, buffer, &size, len, sg_t);
+			len += size;
+			pos = begin + len;
+
+			if (pos < offset) {
+				len = 0;
+				begin = pos;
+			}
+			if (pos > offset + length)
+				goto stop_map_output;
+		}
+	}
+
+ stop_map_output:
+	*start = buffer + (offset - begin);	/* Start of wanted data */
+	len -= (offset - begin);	/* Start slop */
+	if (len > length)
+		len = length;	/* Ending slop */
+	return (len);
+}
+
 static int scsi_proc_info(char *buffer, char **start, off_t offset, int length)
 {
 	Scsi_Device *scd;
@@ -2578,6 +2643,7 @@
 static int __init init_scsi(void)
 {
 	struct proc_dir_entry *generic;
+	struct proc_dir_entry *map;
 
 	printk(KERN_INFO "SCSI subsystem driver " REVISION "\n");
 
@@ -2602,6 +2668,13 @@
 		return -ENOMEM;
 	}
 	generic->write_proc = proc_scsi_gen_write;
+	
+	map = create_proc_info_entry ("scsi/map", 0, 0, scsi_proc_map);
+	if (!map) {
+		printk (KERN_ERR "cannot init /proc/scsi/map\n");
+		remove_proc_entry("scsi", 0);
+		return -ENOMEM;
+	}
 #endif
 
         scsi_devfs_handle = devfs_mk_dir (NULL, "scsi", NULL);
@@ -2636,6 +2709,7 @@
 
 #ifdef CONFIG_PROC_FS
 	/* No, we're not here anymore. Don't show the /proc/scsi files. */
+	remove_proc_entry ("scsi/map", 0);
 	remove_proc_entry ("scsi/scsi", 0);
 	remove_proc_entry ("scsi", 0);
 #endif
diff -uNr linux-2.4.18.S18.fixed/drivers/scsi/scsi.h linux-2.4.18.S18.sdmany/drivers/scsi/scsi.h
--- linux-2.4.18.S18.fixed/drivers/scsi/scsi.h	Wed Jun 12 11:37:07 2002
+++ linux-2.4.18.S18.sdmany/drivers/scsi/scsi.h	Wed Jun 12 11:53:54 2002
@@ -517,6 +517,8 @@
  * Prototypes for functions in scsi_proc.c
  */
 extern void proc_print_scsidevice(Scsi_Device *, char *, int *, int);
+extern void proc_print_scsimap(Scsi_Device *, char *, int *, int, 
+			       struct Scsi_Device_Template *);
 extern struct proc_dir_entry *proc_scsi;
 
 /*
diff -uNr linux-2.4.18.S18.fixed/drivers/scsi/scsi_proc.c linux-2.4.18.S18.sdmany/drivers/scsi/scsi_proc.c
--- linux-2.4.18.S18.fixed/drivers/scsi/scsi_proc.c	Thu Jun 28 02:10:55 2001
+++ linux-2.4.18.S18.sdmany/drivers/scsi/scsi_proc.c	Fri Jun 14 00:30:42 2002
@@ -301,11 +301,66 @@
 	return;
 }
 
+
+
+
+void proc_print_scsimap(Scsi_Device *scd, char *buffer, int *size, int len,
+			struct Scsi_Device_Template *sg_t)
+{
+	int y, err = 0;
+	char nm[16];
+	kdev_t kdev; 
+	int att = scd->attached;
+	struct Scsi_Device_Template *sd_t = scsi_devicelist;
+
+	y = sprintf(buffer + len,
+	     "%i,%i,%02i,%02i\t0x%02x\t%i",
+		    scd->host->host_no, scd->channel, scd->id, scd->lun,
+		    scd->type, scd->online);
+	if (sg_t && sg_t->find_kdev) {
+		err = sg_t->find_kdev(scd, nm, &kdev);
+		if (!err) {
+			y += sprintf(buffer + len + y,
+				     "\t%s\tc:%02x:%02x", nm, MAJOR(kdev), MINOR(kdev));
+			--att;
+		} else {
+			printk (KERN_ERR "scsimap: sg device not found?!\n");
+			y += sprintf(buffer + len + y,
+				     "\tsg?\tc:NN:NN");
+		}
+	} else {
+		printk (KERN_INFO "scsimap: need sg support to report sg devices\n");
+		y += sprintf(buffer + len + y,
+			     "\tsg?\tc:NN:NN");
+	}
+	while (att && sd_t) {
+		if (sd_t->scsi_type != 0xff && sd_t->find_kdev) {
+			err = sd_t->find_kdev(scd, nm, &kdev);
+			if (!err) {
+				y += sprintf(buffer + len + y,
+					     "\t%s\t%c:%02x:%02x",
+					     nm, (sd_t->blk? 'b': 'c'),
+					     MAJOR(kdev), MINOR(kdev));
+				--att;
+			}
+		}
+		sd_t = sd_t->next;
+	}
+
+ map_out:
+	y += sprintf(buffer + len + y, "\n");
+	*size = y;
+	return;
+}
+
 #else				/* if !CONFIG_PROC_FS */
 
 void proc_print_scsidevice(Scsi_Device * scd, char *buffer, int *size, int len)
 {
 }
+void proc_print_scsimap(Scsi_Device *scd, char *buffer, int *size, int len)
+{
+}
 
 #endif				/* CONFIG_PROC_FS */
 
diff -uNr linux-2.4.18.S18.fixed/drivers/scsi/sd.c linux-2.4.18.S18.sdmany/drivers/scsi/sd.c
--- linux-2.4.18.S18.fixed/drivers/scsi/sd.c	Wed Jun 12 11:37:13 2002
+++ linux-2.4.18.S18.sdmany/drivers/scsi/sd.c	Wed Jun 12 11:39:47 2002
@@ -109,6 +109,7 @@
 static int sd_detect(Scsi_Device *);
 static void sd_detach(Scsi_Device *);
 static int sd_init_command(Scsi_Cmnd *);
+static int sd_find_kdev(Scsi_Device*, char*, kdev_t*);
 
 static struct Scsi_Device_Template sd_template = {
 	name:"disk",
@@ -127,6 +128,7 @@
 	attach:sd_attach,
 	detach:sd_detach,
 	init_command:sd_init_command,
+	find_kdev:sd_find_kdev,
 };
 
 
@@ -281,6 +283,23 @@
 	}
 }
 
+static int sd_find_kdev(Scsi_Device *sdp, char* nm, kdev_t *dev)
+{
+	Scsi_Disk *dp; 
+	int i;
+	
+	if (sdp && (sdp->type == TYPE_DISK || sdp->type == TYPE_MOD)) {
+		for (dp = rscsi_disks, i = 0; i < sd_template.dev_max; ++i, ++dp) {
+			if (dp->device == sdp) {
+				sd_devname(i, nm);
+				*dev = MKDEV_SD(i);
+				return 0;
+			}
+		}
+	}
+	return 1;
+}
+
 static request_queue_t *sd_find_queue(kdev_t dev)
 {
 	Scsi_Disk *dpnt;
diff -uNr linux-2.4.18.S18.fixed/drivers/scsi/sg.c linux-2.4.18.S18.sdmany/drivers/scsi/sg.c
--- linux-2.4.18.S18.fixed/drivers/scsi/sg.c	Wed Jun 12 11:37:04 2002
+++ linux-2.4.18.S18.sdmany/drivers/scsi/sg.c	Wed Jun 12 15:27:55 2002
@@ -115,6 +115,7 @@
 static void sg_finish(void);
 static int sg_detect(Scsi_Device *);
 static void sg_detach(Scsi_Device *);
+static int sg_find_kdev(Scsi_Device *, char*, kdev_t*);
 
 static Scsi_Request * dummy_cmdp;	/* only used for sizeof */
 
@@ -123,6 +124,7 @@
 
 static struct Scsi_Device_Template sg_template =
 {
+      name:"generic",
       tag:"sg",
       scsi_type:0xff,
       major:SCSI_GENERIC_MAJOR,
@@ -130,7 +132,8 @@
       init:sg_init,
       finish:sg_finish,
       attach:sg_attach,
-      detach:sg_detach
+      detach:sg_detach,
+      find_kdev:sg_find_kdev
 };
 
 
@@ -2696,6 +2699,36 @@
 }
 
 #ifdef CONFIG_PROC_FS
+static int sg_find_kdev(Scsi_Device* sdp, char *nm, kdev_t *dev)
+{
+    unsigned long iflags;
+    int err = 1; 
+
+    if (sdp && sg_dev_arr) {
+	int k;
+	read_lock_irqsave(&sg_dev_arr_lock, iflags);
+	for (k = 0; k < sg_template.dev_max; ++k) {
+	    if (sg_dev_arr[k] && sg_dev_arr[k]->device == sdp) {
+		sprintf (nm, "sg%i", k);
+	        *dev = sg_dev_arr[k]->i_rdev;
+		err = 0;
+		break;
+	    }
+	}
+	read_unlock_irqrestore(&sg_dev_arr_lock, iflags);
+    }
+    return err;
+}
+#else
+/* Not needed without procfs support */
+static int sg_find_kdev(Scsi_Device* sdp, char *nm, kdev_t *dev)
+{
+    *nm = 0; *kdev = MKDEV(255,255);
+    return 1;
+}
+#endif
+
+#ifdef CONFIG_PROC_FS
 
 static struct proc_dir_entry * sg_proc_sgp = NULL;
 
diff -uNr linux-2.4.18.S18.fixed/drivers/scsi/sr.c linux-2.4.18.S18.sdmany/drivers/scsi/sr.c
--- linux-2.4.18.S18.fixed/drivers/scsi/sr.c	Wed Jun 12 11:37:15 2002
+++ linux-2.4.18.S18.sdmany/drivers/scsi/sr.c	Wed Jun 12 11:39:47 2002
@@ -69,6 +69,8 @@
 
 static int sr_init_command(Scsi_Cmnd *);
 
+static int sr_find_kdev(Scsi_Device*, char*, kdev_t*);
+
 static struct Scsi_Device_Template sr_template =
 {
 	name:"cdrom",
@@ -81,7 +83,8 @@
 	finish:sr_finish,
 	attach:sr_attach,
 	detach:sr_detach,
-	init_command:sr_init_command
+	init_command:sr_init_command,
+	find_kdev:sr_find_kdev,
 };
 
 Scsi_CD *scsi_CDs;
@@ -586,6 +589,22 @@
 	return 0;
 }
 
+static int sr_find_kdev(Scsi_Device *sdp, char* nm, kdev_t *dev)
+{
+	Scsi_CD *srp;
+	int i;
+	
+	if (sdp && (sdp->type == TYPE_ROM || sdp->type == TYPE_WORM)) {
+		for (srp = scsi_CDs, i = 0; i < sr_template.dev_max; ++i, ++srp) {
+			if (srp->device == sdp) {
+				sprintf(nm, "sr%i", i);
+				*dev = MKDEV(SCSI_CDROM_MAJOR,i);
+				return 0;
+			}
+		}
+	}
+	return 1;
+}
 
 void get_sectorsize(int i)
 {
diff -uNr linux-2.4.18.S18.fixed/drivers/scsi/st.c linux-2.4.18.S18.sdmany/drivers/scsi/st.c
--- linux-2.4.18.S18.fixed/drivers/scsi/st.c	Mon Feb 25 20:38:04 2002
+++ linux-2.4.18.S18.sdmany/drivers/scsi/st.c	Wed Jun 12 11:39:47 2002
@@ -164,6 +164,7 @@
 static int st_attach(Scsi_Device *);
 static int st_detect(Scsi_Device *);
 static void st_detach(Scsi_Device *);
+static int st_find_kdev(Scsi_Device*, char*, kdev_t*);
 
 static struct Scsi_Device_Template st_template =
 {
@@ -174,7 +175,8 @@
 	detect:st_detect, 
 	init:st_init,
 	attach:st_attach, 
-	detach:st_detach
+	detach:st_detach,
+	find_kdev:st_find_kdev,
 };
 
 static int st_compression(Scsi_Tape *, int);
@@ -3827,6 +3829,23 @@
 	return 1;
 }
 
+static int st_find_kdev(Scsi_Device * sdp, char* nm, kdev_t *dev)
+{
+	int i;
+	Scsi_Tape *stp;
+	
+	if (sdp && sdp->type == TYPE_TAPE && !st_incompatible(sdp)) {
+		for (stp = scsi_tapes[0], i = 0; i < st_template.dev_max; stp=scsi_tapes[++i]) {
+			if (stp && stp->device == sdp) {
+				sprintf(nm, "st%i", i);
+				*dev = MKDEV (SCSI_TAPE_MAJOR, i);
+				return 0;
+			}
+		}
+	}
+	return 1;
+}
+
 static int st_registered = 0;
 
 /* Driver initialization (not __init because may be called later) */

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: /proc/scsi/map

[John Summerfield]

> 
> Life would be easier if the scsi subsystem would just report which SCSI
> device (uniquely identified by the controller,bus,target,unit tuple) belongs
> to which high-level device. The information is available in the kernel.


Does this not fail if I pull a device off, change its ID (perhaps to fit into another system), then plug it in again? Or if I move it from one adaptor to another?





-- 
Cheers
John Summerfield

Microsoft's most solid OS: http://www.geocities.com/rcwoolley/

Note: mail delivered to me is deemed to be intended for me, for my disposition.

==============================
If you don't like being told you're wrong,
	be right!

Re: /proc/scsi/map

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 1496 bytes --]

Hi John,

On Sat, Jun 15, 2002 at 10:08:54PM +0800, John Summerfield wrote:
> > Life would be easier if the scsi subsystem would just report which SCSI
> > device (uniquely identified by the controller,bus,target,unit tuple) belongs
> > to which high-level device. The information is available in the kernel.
> 
> Does this not fail if I pull a device off, change its ID (perhaps to fit
> into another system), then plug it in again? Or if I move it from one
> adaptor to another?  

Sure it does.
The kernel can offer you the knowledge of a hardware path to your device,
which is given by controller,bust,SCSI target and unit numbers. This is
pretty stable in most configurations.

If you want to have more, you will probably use some sort of signatures.

But that's nothing which happens at SCSI layer.
For plain old SCSI devices, you may e.g. inquire the serial number (INQUIRY,
page code 0x80) which gives you a unique identifier if combined with
vendor and model strings. scsidev does this for you. But it occasinally
fails, as the open on scsi device may fail and we don't know the relation
between the sg devices (that can be used reliably to collect such
information) and the other high level devices. /proc/scsi/map solves this.

Regards,
-- 
Kurt Garloff  <garloff@suse.de>                          Eindhoven, NL
GPG key: See mail header, key servers         Linux kernel development
SuSE Linux AG, Nuernberg, DE                            SCSI, Security

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: /proc/scsi/map

[Richard Gooch]

Kurt Garloff writes:
> Hi SCSI users,
> 
> from people using SCSI devices, there's one question that turns up again=20
> and again: How can one assign stable device names to SCSI devices in
> case there are devices that may or may not be switched on or connected.
> 
> There are a couple of ways to address this problem:
[...]
> (c) devfs
[...]
> Unfortunately, those approaches all have some deficiencies.
[...]
> Ad (c): devfs is currently not (yet?) an option for distributions
>         due to security and stability considerations.

Mandrake is using devfs. And the security and stability issues have
been fixed many months ago. The "devfs races" that Al used to complain
about regularly have been fixed. I haven't heard from Al for many
months (I see that as a positive sign:-). The current devfs code is in
maintenance mode. The next release of code will be a new devfs core
which uses the VFS for tree maintenance, making the code much smaller.
I.e. not a bugfixing release.

If there *are* remaining bugs with the devfs core (or devfsd for that
matter), I've not been made aware of them. If you know something I
don't, please let me know. AFAICT, all the bugs are long since solved.

				Regards,

					Richard....
Permanent: rgooch@atnf.csiro.au
Current:   rgooch@ras.ucalgary.ca

Re: /proc/scsi/map

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 947 bytes --]

Hi Richard,

I was in no way intending to trigger a discussion about devfs.
Some of the things addressed by the scsi/map patch indeed are no issue if
you use devfs; that's why I mentioned devfs at all.
I don't want to bash devfs and I think it's nice that it's in the kernel, 
so users have the choice to use it and the motivation to improve it.

But the problem that I wanted to address IMHO should also be solved
for those people that for one or another reason decided not to use
devfs. 

And face it: I do not think that all major Linux distributions will
start to use devfs within short future. The example you mentioned
(Mandrake) is certainly not a good one: Look at their update kernel.

Best regards,
-- 
Kurt Garloff  <garloff@suse.de>                          Eindhoven, NL
GPG key: See mail header, key servers         Linux kernel development
SuSE Linux AG, Nuernberg, DE                            SCSI, Security

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: /proc/scsi/map

[Ingo Oeser]

On Sun, Jun 16, 2002 at 09:41:12PM +0200, Kurt Garloff wrote:
> I don't want to bash devfs and I think it's nice that it's in the kernel, 
> so users have the choice to use it and the motivation to improve it.
> 
> But the problem that I wanted to address IMHO should also be solved
> for those people that for one or another reason decided not to use
> devfs. 

So make it optional too. I don't need this kind of code
duplication, because I use devfs and I consider it bloat.

And if you do this kind of change, why not implementing
persistent naming for ALL devices?

Other devices have the very same problem.

Regards

Ingo Oeser
-- 
Science is what we can tell a computer. Art is everything else. --- D.E.Knuth

Re: /proc/scsi/map

[Sancho Dauskardt]

>Life would be easier if the scsi subsystem would just report which SCSI
>device (uniquely identified by the controller,bus,target,unit tuple) belongs
>to which high-level device. The information is available in the kernel.
>
>Attached patch does this:
>garloff@pckurt:/raid5/Kernel/src $ cat /proc/scsi/map
># C,B,T,U       Type    onl     sg_nm   sg_dev  nm      dev(hex)
>0,0,00,00       0x05    1       sg0     c:15:00 sr0     b:0b:00
[...]

Great, this was really missing badly.

But how about adding another column: GUID.
Most usb-storage and (all?) FireWire devices have such a unique identitiy.
In contrast to native SCSI devices, these emulated SCSI devices on 
hot-plugging busses will change their LUNs/IDs. Therefor the GUID is really 
a must to be able to create stable names (laptop suspend, etc.).

Both usb-storage and iee1394-sbp2 know the GUID. It only needs to be 
communicated..

I'd guess that FibreChannel has similar problems ?

- sda

Re: /proc/scsi/map

[Albert D. Cahalan]

Kurt Garloff writes:
> garloff@pckurt:/raid5/Kernel/src $ cat /proc/scsi/map
> # C,B,T,U       Type    onl     sg_nm   sg_dev  nm      dev(hex)
> 0,0,00,00       0x05    1       sg0     c:15:00 sr0     b:0b:00
> 1,0,01,00       0x05    1       sg1     c:15:01 sr1     b:0b:01
> 1,0,02,00       0x01    1       sg2     c:15:02 osst0   c:ce:00
> 1,0,03,00       0x05    1       sg3     c:15:03 sr2     b:0b:02
> 1,0,05,00       0x00    1       sg4     c:15:04 sda     b:08:00
> 1,0,09,00       0x00    1       sg5     c:15:05 sdb     b:08:10
> 2,0,01,00       0x05    1       sg6     c:15:06 sr3     b:0b:03
> 2,0,02,00       0x01    1       sg7     c:15:07 osst1   c:ce:01
> 2,0,03,00       0x05    1       sg8     c:15:08 sr4     b:0b:04
> 2,0,05,00       0x00    1       sg9     c:15:09 sdc     b:08:20
> 2,0,09,00       0x00    1       sg10    c:15:0a sdd     b:08:30
> 3,0,10,00       0x00    1       sg11    c:15:0b sde     b:08:40
> 3,0,12,00       0x00    1       sg12    c:15:0c sdf     b:08:50
>
> This allows a simple script to parse the map and create device
> nodes as needed.
...
> Obviously, it can only report the assignment of high-level drivers,
> if they are loaded, otherwise the last two columns will stay empty.
> (sg is handled especially, as we know it supports all devices.)
> If we attach a third high-level device driver, two more columns
> would show up. (Is this variable column number format a problem?)

The variable column format is of course annoying, but use
it if you must. The also-annoying alternative is to pick
a fill character that would be easy for a beginner to
handle in a script. Maybe one of:  @ - . / ?

The header line is far worse. It's too terse to be very helpful.
It gets in the way of every person writing a parser. Even in
your example script, you had to hack your way around it:

> +while read cbtu tp onl sgnm sgdev othnm othdev oothnm oothdev rest; do
> +  # Skip comment line(s)
> +  if test "${cbtu:0:1}" = "#"; then continue; fi
> +  # If we're just dealing with one device, do skip the others
> +  if test ! -z "$CMPAGAINST" -a "$CMPAGAINST" != "$cbtu"; then continue;
>  fi

Re: /proc/scsi/map

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 1861 bytes --]

Hi Albert,

On Sun, Jun 16, 2002 at 03:24:33PM -0400, Albert D. Cahalan wrote:
> Kurt Garloff writes:
> > If we attach a third high-level device driver, two more columns
> > would show up. (Is this variable column number format a problem?)
> 
> The variable column format is of course annoying, but use
> it if you must. The also-annoying alternative is to pick
> a fill character that would be easy for a beginner to
> handle in a script. Maybe one of:  @ - . / ?

Yes, as you correctly mention in your other mail, this would make it easier
to add more columns later.
But the problem then would be that we would need to fix (and limit) the
number of high-level devices that may be reported this way, which is not so
nice either. At this moment it's not a problem, of course, AFAIK.

> The header line is far worse. It's too terse to be very helpful.
> It gets in the way of every person writing a parser. Even in
> your example script, you had to hack your way around it:

I would not call it a hack. Ignoring comment lines is one of the basic
things each parser needs to do. Defining a format that does not allow
for comments actually would not be a very clever move.

But for a file exported from kernel, you may have a valid point.

Actually, the exact format of /proc/scsi/map is certainly something 
that can be discussed separately from the basic idea of adding a file
that does expose the mapping of a SCSI address (CBTU) and the attached
high level drivers. 
The way I designed it just should make it easy for a shell script to use
it. And keeping it simple certainly is a good thing.

Regards,
-- 
Kurt Garloff  <garloff@suse.de>                          Eindhoven, NL
GPG key: See mail header, key servers         Linux kernel development
SuSE Linux AG, Nuernberg, DE                            SCSI, Security

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: /proc/scsi/map

[Patrick Mansfield]

On Sat, Jun 15, 2002 at 03:36:06PM +0200, Kurt Garloff wrote:
> Life would be easier if the scsi subsystem would just report which SCSI
> device (uniquely identified by the controller,bus,target,unit tuple) belongs
> to which high-level device. The information is available in the kernel.

I prefer we refer to the tuple as host, channel, id, lun (H, C, I, L), so
as to more closely match /proc/scsi/scsi, /proc/scsi/sg, and attached
messages:

[root@elm3a50 root]# cat /proc/scsi/scsi | head -4
Attached devices: 
Host: scsi0 Channel: 00 Id: 00 Lun: 00
  Vendor: IBM-PSG  Model: ST318203LC    !# Rev: B222
  Type:   Direct-Access                    ANSI SCSI revision: 02

root # cat /proc/scsi/sg/device_hdr  /proc/scsi/sg/devices | head -2
host    chan    id      lun     type    opens   qdepth  busy    online
0       0       0       0       0       4       253     0       1

Attached messages are of the form:

Attached scsi disk sdn at scsi2, channel 0, id 2, lun 0

> Attached patch does this:
> garloff@pckurt:/raid5/Kernel/src $ cat /proc/scsi/map 
> # C,B,T,U       Type    onl     sg_nm   sg_dev  nm      dev(hex)
> 0,0,00,00       0x05    1       sg0     c:15:00 sr0     b:0b:00
> 1,0,01,00       0x05    1       sg1     c:15:01 sr1     b:0b:01
> 1,0,02,00       0x01    1       sg2     c:15:02 osst0   c:ce:00
> 1,0,03,00       0x05    1       sg3     c:15:03 sr2     b:0b:02
> 1,0,05,00       0x00    1       sg4     c:15:04 sda     b:08:00
> 1,0,09,00       0x00    1       sg5     c:15:05 sdb     b:08:10
> 2,0,01,00       0x05    1       sg6     c:15:06 sr3     b:0b:03
> 2,0,02,00       0x01    1       sg7     c:15:07 osst1   c:ce:01
> 2,0,03,00       0x05    1       sg8     c:15:08 sr4     b:0b:04
> 2,0,05,00       0x00    1       sg9     c:15:09 sdc     b:08:20
> 2,0,09,00       0x00    1       sg10    c:15:0a sdd     b:08:30
> 3,0,10,00       0x00    1       sg11    c:15:0b sde     b:08:40
> 3,0,12,00       0x00    1       sg12    c:15:0c sdf     b:08:50

Why not treat each upper layer driver the same? Type is already
in /proc/scsi/scsi, or implied by the upper level drivers attached.
Online should really be part of /proc/scsi/scsi.

Then, each line is a path followed by a list of upper level devices.
This would also simplify the code, although the ordering of the upper
level devices becomes link or module load order dependent.

And similiar to sg (someone commented on parsing '^#'), have a _hdr
entry; something like:

$ cat /proc/scsi/map_hdr /proc/scsi/map

H:C:I:L      online  type:name:block/char:maj:min
00:00:00:00     1       sg:sg0:c:15:00      sr:sr0:b:0b:00
01:00:01:00     1       sg:sg1:c:15:01      sr:sr1:b:0b:01
01:00:02:00     1       sg:sg2:c:15:02      osst:osst0:c:ce:00
02:00:09:00     1       sg:sg3:c:15:03      sd:sdd:b:08:30

Or:

H:C:I:L      online  type:enumeration:block/char:maj:min
00:00:00:00     1      sg:0:c:15:00      sr:0:b:0b:00
01:00:01:00     1      sg:1:c:15:01      sr:1:b:0b:01
01:00:02:00     1      sg:2:c:15:02      osst:0:c:ce:00
02:00:09:00     1      sg:3:c:15:03      sd:d:b:08:30


> A patch for 2.5 should be done as well, if the design is OK, of course.
> 

IMO, we should use driverfs for this in 2.5. Mike Sullivan's scsi driverfs
patch currently ends up with a driverfs layout (showing one Scsi_Device
with two partitions, sg and sd attached) like this:

[root@elm3a50 devices]# tree ./root/pci1/01\:06.0/scsi2/2\:0\:2\:0/
./root/pci1/01:06.0/scsi2/2:0:2:0/
|-- 2:0:2:0:disc
|   |-- kdev
|   |-- name
|   |-- power
|   `-- type
|-- 2:0:2:0:gen
|   |-- kdev
|   |-- name
|   |-- power
|   `-- type
|-- 2:0:2:0:p1
|   |-- kdev
|   |-- name
|   |-- power
|   `-- type
|-- 2:0:2:0:p2
|   |-- kdev
|   |-- name
|   |-- power
|   `-- type
|-- name
|-- power
`-- type

Right now, the name is storing an ID; the ID is retrieved in the kernel (using
page 0x80 or page 0x83 or the path). For example disc has:

[root@elm3a50 2:0:2:0:disc]# pwd
/devices/root/pci1/01:06.0/scsi2/2:0:2:0/2:0:2:0:disc
[root@elm3a50 2:0:2:0:disc]# ls  
kdev  name  power  type
[root@elm3a50 2:0:2:0:disc]# cat *
8d0
U20000020371719e8disc
0
BLK

-- Patrick Mansfield

Re: /proc/scsi/map

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 3862 bytes --]

Hi Patrick,

On Mon, Jun 17, 2002 at 01:35:34PM -0700, Patrick Mansfield wrote:
> On Sat, Jun 15, 2002 at 03:36:06PM +0200, Kurt Garloff wrote:
> > Life would be easier if the scsi subsystem would just report which SCSI
> > device (uniquely identified by the controller,bus,target,unit tuple) belongs
> > to which high-level device. The information is available in the kernel.
> 
> I prefer we refer to the tuple as host, channel, id, lun (H, C, I, L), so
> as to more closely match /proc/scsi/scsi, /proc/scsi/sg, and attached
> messages:

You are refering to the naming of this 4-tuple, right: HCIL vs. CBTU?
I chose for CBTU, because that on's used in devfs. Actually, as you can see
from scsidev, I like HCIL more. But that's a detail the kernel should not
care about. The header line should be removed anyway as Albert remarked.
And helping those people who think that 200 bytes is unacceptable bloat.

[...]
> > 3,0,12,00       0x00    1       sg12    c:15:0c sdf     b:08:50
> 
> Why not treat each upper layer driver the same? Type is already
> in /proc/scsi/scsi, or implied by the upper level drivers attached.
> Online should really be part of /proc/scsi/scsi.

I'm not sure I know what you mean. The fact that I decided to put
the sg device name first independently of the (potentially) random
order in which high-level drivers are assigned?

> Then, each line is a path followed by a list of upper level devices.

It is. 

> This would also simplify the code, although the ordering of the upper
> level devices becomes link or module load order dependent.

Just I decided to report shg first. This has a very pratical reason:
I you want to use userspace tools to collect more advanced (and maybe type
dependant information), you will always want to use the sg device, which 
you can use to send SCSI commands and which you can open, even if there is
no medium or if the device is in use.

> And similiar to sg (someone commented on parsing '^#'), have a _hdr
> entry; something like:
> 
> $ cat /proc/scsi/map_hdr /proc/scsi/map
> 
> H:C:I:L      online  type:name:block/char:maj:min
> 00:00:00:00     1       sg:sg0:c:15:00      sr:sr0:b:0b:00
> 01:00:01:00     1       sg:sg1:c:15:01      sr:sr1:b:0b:01
> 01:00:02:00     1       sg:sg2:c:15:02      osst:osst0:c:ce:00
> 02:00:09:00     1       sg:sg3:c:15:03      sd:sdd:b:08:30

This looks find to me as well, by the way.
The reason why I chose to additionally report the device type reported by
inquiry is that you will only see the attached (and thus only the loaded)
high-level drivers of a device. With the device type, a userspace tool could
easily decide whether to trigger a modprobe and start again ...

> Or:
> 
> H:C:I:L      online  type:enumeration:block/char:maj:min
> 00:00:00:00     1      sg:0:c:15:00      sr:0:b:0b:00
> 01:00:01:00     1      sg:1:c:15:01      sr:1:b:0b:01
> 01:00:02:00     1      sg:2:c:15:02      osst:0:c:ce:00
> 02:00:09:00     1      sg:3:c:15:03      sd:d:b:08:30
> 
> 
> > A patch for 2.5 should be done as well, if the design is OK, of course.
> 
> IMO, we should use driverfs for this in 2.5. Mike Sullivan's scsi driverfs
> patch currently ends up with a driverfs layout (showing one Scsi_Device
> with two partitions, sg and sd attached) like this:

I still think the easy /proc/scsi/map format would be a nice basis to
inquire more information on the SCSI devices from userspace, even if you add
hierarchical attachment information via driverfs. And I think a solution
that works with both 2.4 and 2.5 would help most users, of course.

Regards,
-- 
Kurt Garloff  <garloff@suse.de>                          Eindhoven, NL
GPG key: See mail header, key servers         Linux kernel development
SuSE Linux AG, Nuernberg, DE                            SCSI, Security

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: /proc/scsi/map

[Patrick Mansfield]

Kurt -

On Mon, Jun 17, 2002 at 10:57:50PM +0200, Kurt Garloff wrote:
> Hi Patrick,
> 
> > I prefer we refer to the tuple as host, channel, id, lun (H, C, I, L), so
> > as to more closely match /proc/scsi/scsi, /proc/scsi/sg, and attached
> > messages:
> 
> You are refering to the naming of this 4-tuple, right: HCIL vs. CBTU?

Yes.

> I chose for CBTU, because that on's used in devfs. Actually, as you can see
> from scsidev, I like HCIL more. But that's a detail the kernel should not
> care about. The header line should be removed anyway as Albert remarked.
> And helping those people who think that 200 bytes is unacceptable bloat.
> 
> [...]
> > > 3,0,12,00       0x00    1       sg12    c:15:0c sdf     b:08:50
> > 
> > Why not treat each upper layer driver the same? Type is already
> > in /proc/scsi/scsi, or implied by the upper level drivers attached.
> > Online should really be part of /proc/scsi/scsi.
> 
> I'm not sure I know what you mean. The fact that I decided to put
> the sg device name first independently of the (potentially) random
> order in which high-level drivers are assigned?

Yes, I don't know why I took that to mean that sg was displayed differently.

> Just I decided to report shg first. This has a very pratical reason:
> I you want to use userspace tools to collect more advanced (and maybe type
> dependant information), you will always want to use the sg device, which 
> you can use to send SCSI commands and which you can open, even if there is
> no medium or if the device is in use.

No matter the column position sg can be found if each column includes
the upper level name (sg, sd etc.). Then you do not need to know or
check the scsi_type of the template, or explicitly locate the sg
template in scsi_proc_map().

And then without the header scsi_proc_map() gets really simple.

> Regards,
> -- 
> Kurt Garloff  <garloff@suse.de>                          Eindhoven, NL
> GPG key: See mail header, key servers         Linux kernel development
> SuSE Linux AG, Nuernberg, DE                            SCSI, Security

-- Patrick Mansfield

Re: /proc/scsi/map

[Doug Ledford]

On Sat, Jun 15, 2002 at 03:36:06PM +0200, Kurt Garloff wrote:
> Hi SCSI users,
> 
> from people using SCSI devices, there's one question that turns up again 
> and again: How can one assign stable device names to SCSI devices in
> case there are devices that may or may not be switched on or connected.


> Life would be easier if the scsi subsystem would just report which SCSI
> device (uniquely identified by the controller,bus,target,unit tuple) belongs
> to which high-level device. The information is available in the kernel.

Umm, this patently fails to meet the criteria you posted of "stable device 
name".  Adding a controller to a system is just as likely to blow this 
naming scheme to hell as it is to blow the traditional linux /dev/sd? 
scheme.  IOW, even though the /proc/scsi/map file looks nice and usefull, 
it fails to solve the very problem you are trying to solve.


-- 
  Doug Ledford <dledford@redhat.com>     919-754-3700 x44233
         Red Hat, Inc. 
         1801 Varsity Dr.
         Raleigh, NC 27606
  

Re: /proc/scsi/map

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 2452 bytes --]

Hi Doug,

On Mon, Jun 17, 2002 at 06:08:18PM -0400, Doug Ledford wrote:
> On Sat, Jun 15, 2002 at 03:36:06PM +0200, Kurt Garloff wrote:
> > Life would be easier if the scsi subsystem would just report which SCSI
> > device (uniquely identified by the controller,bus,target,unit tuple) belongs
> > to which high-level device. The information is available in the kernel.
> 
> Umm, this patently fails to meet the criteria you posted of "stable device 
> name".  Adding a controller to a system is just as likely to blow this 
> naming scheme to hell as it is to blow the traditional linux /dev/sd? 
> scheme.  IOW, even though the /proc/scsi/map file looks nice and usefull, 
> it fails to solve the very problem you are trying to solve.

In case you just add controllers, you just need to make sure you get them the
same numbers again. A solution for this exists already:
* For a kernel where SCSI low-level drivers are loaded as modules,
  you just need to keep the order constant
* For compiled in SCSI drivers, use scsihosts=


But actually, the patch is not meant to be the holy grail of persistent
device naming. But it enables userspace tools to collect information 
* reliably 
  (fails so far due to possible open() failures with unknown
   relation to the corresponding sg device (which could be opened))
* without too much trouble

Both things I consider important and useful.

The patch basically does provide two pieces of information:
* mapping between sg vs. other high level devices
* mapping CBTU to high-level devices
The latter one is enough for many setups, and the former can be used for
more elaborate solutions involving userspace tools more advanced than the
simple script I included in the patch.

If you want to go for the holy grail, you may either come up with a 
unique address at hardware level (which does currently not exist for all
types dealt with by the SCSI subsystem) and make it available to SCSI mid
level or use signatures that allows you to find devices back. LVM, e.g.
does the latter. 
But at this moment, I fear, neither of them are possible in all cases.

Regards,
-- 
Kurt Garloff                   <kurt@garloff.de>         [Eindhoven, NL]
Physics: Plasma simulations    <K.Garloff@TUE.NL>     [TU Eindhoven, NL]
Linux: SCSI, Security          <garloff@suse.de>    [SuSE Nuernberg, DE]
 (See mail header or public key servers for PGP2 and GPG public keys.)

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: /proc/scsi/map

[Doug Ledford]

On Tue, Jun 18, 2002 at 01:06:48AM +0200, Kurt Garloff wrote:
> Hi Doug,
> 
> On Mon, Jun 17, 2002 at 06:08:18PM -0400, Doug Ledford wrote:
> > On Sat, Jun 15, 2002 at 03:36:06PM +0200, Kurt Garloff wrote:
> > > Life would be easier if the scsi subsystem would just report which SCSI
> > > device (uniquely identified by the controller,bus,target,unit tuple) belongs
> > > to which high-level device. The information is available in the kernel.
> > 
> > Umm, this patently fails to meet the criteria you posted of "stable device 
> > name".  Adding a controller to a system is just as likely to blow this 
> > naming scheme to hell as it is to blow the traditional linux /dev/sd? 
> > scheme.  IOW, even though the /proc/scsi/map file looks nice and usefull, 
> > it fails to solve the very problem you are trying to solve.
> 
> In case you just add controllers, you just need to make sure you get them the
> same numbers again. A solution for this exists already:
> * For a kernel where SCSI low-level drivers are loaded as modules,
>   you just need to keep the order constant
> * For compiled in SCSI drivers, use scsihosts=

No, this is not true.  If you add a new controller (for some new disks in 
a new external enclosure or whatever), and that controller uses the same 
driver as other controller(s) in your system, then you have no guarantee 
of order.  For example, adding a 4th aic7xxx controller to your system 
might or might not place the new controller at the end of the list 
depending on PCI scan order, etc.  There simply is *no* guarantee here of 
any consistent naming, so don't bother trying to claim there is.

Now don't get me wrong, I'm not saying the patch isn't usefull, but the 
patch doesn't provide *any* guarantee of consistent device naming and so 
using that as a reason to put the patch into the mainstream kernel is 
utter crap.  Go ahead and make your case for why it should be in the 
kernel, but don't use reasons that aren't correct.
 
> But actually, the patch is not meant to be the holy grail of persistent
> device naming. But it enables userspace tools to collect information 
> * reliably 
>   (fails so far due to possible open() failures with unknown
>    relation to the corresponding sg device (which could be opened))

This can be done without your patch (the mapping from /dev/sg to /dev/sd? 
or /dev/st? or /dev/scd? or whatever is not impossible from user space 
without your patch, it just requires a user space tool to open the files 
and start comparing host/bus/id/lun combinations from dev file to dev 
file).

> * without too much trouble

This part is true enough, it is easier to read the map file than to 
program the information retrieval I mentioned above.

> Both things I consider important and useful.
> 
> The patch basically does provide two pieces of information:
> * mapping between sg vs. other high level devices

This I think is usefull.

> * mapping CBTU to high-level devices

This I don't think is usefull at all.  It's no more reliable than our 
current system and people that are depending on this to solve their "I 
can't tell what device is what" delima are going to be sorely upset when 
they realize that hardware changes can change this stuff around just as 
fast as it changes around the /dev/sd? mappings.

> The latter one is enough for many setups,

The latter one is just as broken in design as the original /dev/sd? 
enumeration problem (which stands to reason since this method also is an 
enumeration method, it's just that instead of enumerating the disks 
starting at 0, we are enumerating the SCSI controllers starting at the 
first one we find and going from there).

> and the former can be used for
> more elaborate solutions involving userspace tools more advanced than the
> simple script I included in the patch.

Which is the much better way to go.

> If you want to go for the holy grail, you may either come up with a 
> unique address at hardware level (which does currently not exist for all
> types dealt with by the SCSI subsystem) and make it available to SCSI mid
> level or use signatures that allows you to find devices back. LVM, e.g.
> does the latter. 
> But at this moment, I fear, neither of them are possible in all cases.
> 
> Regards,
> -- 
> Kurt Garloff                   <kurt@garloff.de>         [Eindhoven, NL]
> Physics: Plasma simulations    <K.Garloff@TUE.NL>     [TU Eindhoven, NL]
> Linux: SCSI, Security          <garloff@suse.de>    [SuSE Nuernberg, DE]
>  (See mail header or public key servers for PGP2 and GPG public keys.)



-- 
  Doug Ledford <dledford@redhat.com>     919-754-3700 x44233
         Red Hat, Inc. 
         1801 Varsity Dr.
         Raleigh, NC 27606
  

Re: /proc/scsi/map

[Douglas Gilbert]

Doug Ledford wrote:
> 
> On Tue, Jun 18, 2002 at 01:06:48AM +0200, Kurt Garloff wrote:
> > Hi Doug,
> >
> > On Mon, Jun 17, 2002 at 06:08:18PM -0400, Doug Ledford wrote:
> > > On Sat, Jun 15, 2002 at 03:36:06PM +0200, Kurt Garloff wrote:
> > > > Life would be easier if the scsi subsystem would just report which SCSI
> > > > device (uniquely identified by the controller,bus,target,unit tuple) belongs
> > > > to which high-level device. The information is available in the kernel.
> > >
> > > Umm, this patently fails to meet the criteria you posted of "stable device
> > > name".  Adding a controller to a system is just as likely to blow this
> > > naming scheme to hell as it is to blow the traditional linux /dev/sd?
> > > scheme.  IOW, even though the /proc/scsi/map file looks nice and usefull,
> > > it fails to solve the very problem you are trying to solve.
> >
> > In case you just add controllers, you just need to make sure you get them the
> > same numbers again. A solution for this exists already:
> > * For a kernel where SCSI low-level drivers are loaded as modules,
> >   you just need to keep the order constant
> > * For compiled in SCSI drivers, use scsihosts=
> 
> No, this is not true.  If you add a new controller (for some new disks in
> a new external enclosure or whatever), and that controller uses the same
> driver as other controller(s) in your system, then you have no guarantee
> of order.  For example, adding a 4th aic7xxx controller to your system
> might or might not place the new controller at the end of the list
> depending on PCI scan order, etc.  There simply is *no* guarantee here of
> any consistent naming, so don't bother trying to claim there is.

It only been two days and it seems appropriate to refer
to Martin Petersen's summation of persistent naming issues again:
http://marc.theaimsgroup.com/?l=linux-scsi&m=101840990116069&w=2


As for "scsihosts" its current syntax is:
 scsihosts=aic7xxx:sym53c8xx::aic7xxx

This could be extended to
  scsihosts=aic7xxx[pci=00:0c.0]:sym53c8xx::aic7xxx
and if "pci=" was assumed to be the default then this
would have the same effect as:
  scsihosts=[00:0c.0]:sym53c8xx::aic7xxx
assuming there was a aic7xxx controlled HBA at that PCI slot.

No consistent persistence here, just a little more precision.

> Now don't get me wrong, I'm not saying the patch isn't usefull, but the
> patch doesn't provide *any* guarantee of consistent device naming and so
> using that as a reason to put the patch into the mainstream kernel is
> utter crap.  Go ahead and make your case for why it should be in the
> kernel, but don't use reasons that aren't correct.
> 
> > But actually, the patch is not meant to be the holy grail of persistent
> > device naming. But it enables userspace tools to collect information
> > * reliably
> >   (fails so far due to possible open() failures with unknown
> >    relation to the corresponding sg device (which could be opened))
> 
> This can be done without your patch (the mapping from /dev/sg to /dev/sd?
> or /dev/st? or /dev/scd? or whatever is not impossible from user space
> without your patch, it just requires a user space tool to open the files
> and start comparing host/bus/id/lun combinations from dev file to dev
> file).
> 
> > * without too much trouble
> 
> This part is true enough, it is easier to read the map file than to
> program the information retrieval I mentioned above.

You can say that again. Joerg Schilling is still some way
from getting it right in cdrecord, SANE still has problems
in this area (and I rewrote the scsi scanning code). Kurt
and I have a fair amount of experience in this particular 
area. There are lots of gotchas ...


Kurt's implementation is worthy of note as well. When I have
pondered this problem I just wanted to poke each sg kdev_t into
the corresponding Scsi_Device. People objected that this was
a layering violation. Kurt has added a new upper level callback
find_kdev() which is cleaner. Based on this new callback we
could add a new mid level ioctl that yielded the principal
(k)dev_t (i.e. sd, st or sr) and the generic one. That would
make life easier for cdrecord and cdparanoia (and sg_utils). 
As for SANE, by looking at /proc/scsi/map it could only analyse 
sg devices that where "printer" or "processor" device types.

Doug Gilbert

Re: /proc/scsi/map

[Doug Ledford]

On Tue, Jun 18, 2002 at 12:32:16AM -0400, Douglas Gilbert wrote:
> As for "scsihosts" its current syntax is:
>  scsihosts=aic7xxx:sym53c8xx::aic7xxx
> 
> This could be extended to
>   scsihosts=aic7xxx[pci=00:0c.0]:sym53c8xx::aic7xxx
> and if "pci=" was assumed to be the default then this
> would have the same effect as:
>   scsihosts=[00:0c.0]:sym53c8xx::aic7xxx
> assuming there was a aic7xxx controlled HBA at that PCI slot.
> 
> No consistent persistence here, just a little more precision.

I think that relies upon the SCSI controller using the newer PCI scanning 
methods (which, for example, my aic7xxx driver does not use).

-- 
  Doug Ledford <dledford@redhat.com>     919-754-3700 x44233
         Red Hat, Inc. 
         1801 Varsity Dr.
         Raleigh, NC 27606
  

Re: /proc/scsi/map

[Kurt Garloff]

[-- Attachment #1: Type: text/plain, Size: 6339 bytes --]

Hi Doug,

On Mon, Jun 17, 2002 at 10:40:47PM -0400, Doug Ledford wrote:
> On Tue, Jun 18, 2002 at 01:06:48AM +0200, Kurt Garloff wrote:
> > * For compiled in SCSI drivers, use scsihosts=
> 
> No, this is not true.  If you add a new controller (for some new disks in 
> a new external enclosure or whatever), and that controller uses the same 
> driver as other controller(s) in your system, then you have no guarantee 
> of order.  For example, adding a 4th aic7xxx controller to your system 
> might or might not place the new controller at the end of the list 
> depending on PCI scan order, etc.  There simply is *no* guarantee here of 
> any consistent naming, so don't bother trying to claim there is.

You're right; there is no guarantee.

In scsidev, the IO port is added as identifier to the host number, but even
that may change in case your Plug'n'Pray BIOS assigns a different IO port
upon PCI configuration ...

> > But actually, the patch is not meant to be the holy grail of persistent
> > device naming. But it enables userspace tools to collect information 
> > * reliably 
> >   (fails so far due to possible open() failures with unknown
> >    relation to the corresponding sg device (which could be opened))
> 
> This can be done without your patch (the mapping from /dev/sg to /dev/sd? 
> or /dev/st? or /dev/scd? or whatever is not impossible from user space 
> without your patch, it just requires a user space tool to open the files 
> and start comparing host/bus/id/lun combinations from dev file to dev 
> file).

No, it unfortunately can't. The reason is you just can't reliably open a
device to do the SCSI_IOCTL_GET_IDLUN to find out about CBTU. 
Devices with removable media are a good example. The open on a tape
that is not inserted just fails. Or it may be in use

Or cause very nasty side effects, like stalling for a minute because the
tape driver rewinds the drive and looks for meta-information (osst).

This problem is really the reason why scsidev is not foolproof.
sg_map does not do magic either.

root@pckurt:~ $ sg_map 
sg_map: close error: Input/output error
/dev/sg0  /dev/scd0
/dev/sg1  /dev/sda
/dev/sg2  /dev/sdb
/dev/sg3  /dev/sdc
/dev/sg4  /dev/sdd
/dev/sg5  /dev/sde
/dev/sg6  /dev/sdf
/dev/sg7  /dev/scd1
/dev/sg8  /dev/osst0
/dev/sg9  /dev/scd2
/dev/sg10  /dev/scd3
/dev/sg11
/dev/sg12  /dev/scd4

The /proc/scsi/map does provide the missing piece of information.

root@pckurt:~ $ cat /proc/scsi/map
# C,B,T,U       Type    onl     sg_nm   sg_dev  nm      dev(hex)
0,0,00,00       0x05    1       sg0     c:15:00 sr0     b:0b:00
1,0,09,00       0x00    1       sg2     c:15:02 sdb     b:08:10
1,0,05,00       0x00    1       sg1     c:15:01 sda     b:08:00
1,0,01,00       0x05    1       sg7     c:15:07 sr1     b:0b:01
1,0,02,00       0x01    1       sg8     c:15:08 osst0   c:ce:00
1,0,03,00       0x05    1       sg9     c:15:09 sr2     b:0b:02
2,0,05,00       0x00    1       sg3     c:15:03 sdc     b:08:20
2,0,09,00       0x00    1       sg4     c:15:04 sdd     b:08:30
2,0,01,00       0x05    1       sg10    c:15:0a sr3     b:0b:03
2,0,02,00       0x01    1       sg11    c:15:0b osst1   c:ce:01
2,0,03,00       0x05    1       sg12    c:15:0c sr4     b:0b:04
3,0,10,00       0x00    1       sg5     c:15:05 sde     b:08:40
3,0,12,00       0x00    1       sg6     c:15:06 sdf     b:08:50

> > * without too much trouble
> 
> This part is true enough, it is easier to read the map file than to 
> program the information retrieval I mentioned above.
> 
> > Both things I consider important and useful.
> > 
> > The patch basically does provide two pieces of information:
> > * mapping between sg vs. other high level devices
> 
> This I think is usefull.
> 
> > * mapping CBTU to high-level devices
> 
> This I don't think is usefull at all.  It's no more reliable than our 
> current system and people that are depending on this to solve their "I 
> can't tell what device is what" delima are going to be sorely upset when 
> they realize that hardware changes can change this stuff around just as 
> fast as it changes around the /dev/sd? mappings.

You less often build in new host adpaters or jumper your drives' IDs
differently than you switch on/off an external ZIP drive/scanner/ e.g.

And if you take a disk from one controller to another one, you arguably even
want the address to change as your path to the device changed. At least you
can expect it.

CBTU still gives you reasonable information about the path to a device.

The "C" identification may need improvement ... and I'm certainly open
for suggestions.

And I personally do think that at SCSI midlayer in kernel, you want to
report about this path to a device.

> > The latter one is enough for many setups,
> 
> The latter one is just as broken in design as the original /dev/sd? 
> enumeration problem (which stands to reason since this method also is an 
> enumeration method, it's just that instead of enumerating the disks 
> starting at 0, we are enumerating the SCSI controllers starting at the 
> first one we find and going from there).

I don't think the original /dev/sd? design is broken. It's a choice that
makes some sense if you think about the fact that you only have a limited
amount of majors reserved for SCSI disks.
But it has some disadvantages ...

> > and the former can be used for
> > more elaborate solutions involving userspace tools more advanced than the
> > simple script I included in the patch.
> 
> Which is the much better way to go.

Expect a scsidev release (2.25) that does use /proc/scsi/map (if present) to
present a somewhat better mapping than the little script included.
It optionally does assign aliases based on the serial number
(INQUIRY,evpd=1,pg.cd. 0x80), which is rather stable in case your do provide
this piece of information. But not unique in case you provide multiple
paths to your devices (like having more than one SCSI host adapter on a bus,
which I BTW do for testing purposes).

Regards,
-- 
Kurt Garloff  <garloff@suse.de>                          Eindhoven, NL
GPG key: See mail header, key servers         Linux kernel development
SuSE Linux AG, Nuernberg, DE                            SCSI, Security

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]