Re: [linux-usb-devel] Re: /proc/scsi/map

Re: [linux-usb-devel] Re: /proc/scsi/map

[David Brownell]

Andries.Brouwer@cwi.nl wrote:
>     Both usb-storage and iee1394-sbp2 know the GUID. It only needs to be 
>     communicated..
> 
> The usb-storage GUID is just one random item of information.
> One might wish for much more.
> 
> And: this information is already somewhere:

For example, on the 2.5 kernels one might also wish for the physical
path to the USB device in question ... what usb_make_path() returns.
(Simple enough to backport this to 2.4, which some folk have wanted.)

It's a string like "usb-06:0f.1-3.2" indicating first that it's
USB, then that it's the bus with name/serial "06:0f.1" (which in
this case is a PCI function), then that it's connected to port 3 on
the root hub, and port 2 on the hub connected there.  That's a
stable name:  it won't change unless/until you re-cable things.

Right now all of the 2.5 USB network adapters with ethool support
return those strings in the bus_info field, FWIW -- someone else
on this thread mentioned the need to asssign network interface names
based on physical location, and for USB (and likely PCI) that can
be done already.

- Dave

Re: [linux-usb-devel] Re: /proc/scsi/map

[James Bottomley]

Since we already have a huge long list of different ways to identify different 
devices, I don't think coding any one or even a set of such methods into the 
kernel would satisfy everyone.

What about a different approach:

We already (nearly) have the scsimon patches to do hot plug events on SCSI 
devices incorporated.  Any identification could be done from the scsi device 
hotplug script (i.e. if you see it's USB, get the GID, if it's enterprise 
storage get the WWN, try the filesystem UUID etc).  Then all the hotplug 
script does is plug this device into some type of volume idenfication scheme 
like /dev/volume/<name>.

Any application needing to always know where the device is would refer to it 
by name, and since there's no prescription at all about what the <name> is, 
you could even alias horribly unfriendly things like the WWN to more palatable 
names using a user specified translation table.

This implementation doesn't even depend on SCSI, so it could potentially be 
used by any subsystem (IDE, block devices).

The only component that doesn't exist is the configurable /dev/volume piece.  
Since the name would be a property of the device node, it probably makes sense 
to place this into the new driverfs somehow.

James

Re: [linux-usb-devel] Re: /proc/scsi/map

[Oliver Neukum]

Am Sonntag, 16. Juni 2002 19:25 schrieb James Bottomley:
> Since we already have a huge long list of different ways to identify
> different devices, I don't think coding any one or even a set of such
> methods into the kernel would satisfy everyone.
>
> What about a different approach:
>
> We already (nearly) have the scsimon patches to do hot plug events on
> SCSI devices incorporated.  Any identification could be done from the
> scsi device hotplug script (i.e. if you see it's USB, get the GID, if
> it's enterprise storage get the WWN, try the filesystem UUID etc).  Then
> all the hotplug script does is plug this device into some type of volume
> idenfication scheme like /dev/volume/<name>.

How would you find out what a device is ?
If the kernel has to supply the information anyway, you could
just as well pass all information to the script or devfs.

	Regards
		Oliver

Re: [linux-usb-devel] Re: /proc/scsi/map

[James Bottomley]

oliver@neukum.name said:
> How would you find out what a device is ? If the kernel has to supply
> the information anyway, you could just as well pass all information to
> the script or devfs. 

But the kernel doesn't know this globally unique identifier information today, 
that's what the discussion is about.

The essence of the problem is that there's no one method that can get a unique 
identifier for every SCSI device (even though almost every device possesses 
one in one form or another).  So you have to implement a collection of ad hoc 
methods depending on what the device actually is.

The idea behind using hotplug to solve the problem is that with scsimon, a 
hotplug insertion event is generated for every SCSI device as it is added.  
The script is provided with the information the kernel knows (host, channel, 
pun lun, model and vendor inquiry strings---see www.torque.net/scsimon.html 
for details).  The hotplug script then does the remaining processing to 
extract the ID from the device (by ioctls, sending down SCSI commands etc.) 
and then binds it into the /dev/volume nodes using the identifier it 
determines.

The result is that however you move the device around (between controllers or 
even change its id), it will always show up as its unique /dev/volume name.

The key philosophy is that the code to make the policy decision for assigning 
a unique name isn't cluttering up the kernel, it's in user land where it can 
be easily customised.  Once scsimon is part of the kernel, we need no other 
kernel changes at all to implement this, since /dev/volume could just be done 
with symbolic links (although having the kernel know the name is useful).

James

Re: [linux-usb-devel] Re: /proc/scsi/map

[Oliver Neukum]

Am Montag, 17. Juni 2002 00:02 schrieb James Bottomley:
> oliver@neukum.name said:
> > How would you find out what a device is ? If the kernel has to supply
> > the information anyway, you could just as well pass all information to
> > the script or devfs.
>
> But the kernel doesn't know this globally unique identifier information
> today, that's what the discussion is about.

But the drivers already know, or would have to be taught to know
about it. Somewhence that information has to come. You cannot
avoid that effort.

> The essence of the problem is that there's no one method that can get a
> unique identifier for every SCSI device (even though almost every device
> possesses one in one form or another).  So you have to implement a
> collection of ad hoc methods depending on what the device actually is.

That is wrong. You'd need a common method to determine device type
and several methods of passing guid. You are better off in implementing
one common way of getting at that information, which shouldn't be
too hard, as all the generic layer would have to do is pass up that information.

> The idea behind using hotplug to solve the problem is that with scsimon,
> a hotplug insertion event is generated for every SCSI device as it is
> added. The script is provided with the information the kernel knows
> (host, channel, pun lun, model and vendor inquiry strings---see
> www.torque.net/scsimon.html for details).  The hotplug script then does
> the remaining processing to extract the ID from the device (by ioctls,
> sending down SCSI commands etc.) and then binds it into the /dev/volume
> nodes using the identifier it determines.

So implement a scsi low level method called 'get_guid' and pass its
result with the other hotplugging information. Simple, no callbacks needed
and no problems with simultaneous plugging events.

The only problem you now face is stale symlinks. Which is not trivial.
If you have a link /dev/volumes/scratchvol -> /dev/sdc and you issue
mkfs -t ext2 /dev/volumes/scratchvolume you want to be very sure
that the symlink is current.
IMHO you need a kernel part that kills the symlink upon device removal.

	Regards
		Oliver

Re: [linux-usb-devel] Re: /proc/scsi/map

[James Bottomley]

oliver@neukum.name said:
> But the drivers already know, or would have to be taught to know about
> it. Somewhence that information has to come. You cannot avoid that
> effort. 

Not necessarily: consider the SCSI WWN, which is supported by most modern SCSI 
devices.  The driver never probes for or asks for this.  Nowhere in the 
current SCSI code do we ask for this.  However user level commands (like 
sg_inq) can formulate the 0x83 page inquiry to get this and return the output. 
 This works today with the current driver.

> That is wrong. You'd need a common method to determine device type and
> several methods of passing guid. You are better off in implementing
> one common way of getting at that information, which shouldn't be too
> hard, as all the generic layer would have to do is pass up that
> information. 

but the complexity is in the "common method to determine device type and 
several methods of passing guid".  Even for a simple SCSI disk, there's no one 
universal way of getting a unique id (let alone when we add the usb devices 
masquerading as scsi disks into the mix).  That will lead us to scanning a 
list of inquiry strings to see what the disk is and determine what globally 
unique ID it supports.  Since we have to implement a lookup table just to 
determine how to get the unqiue id, it's far better off being done outside the 
kernel.

James

Re: [linux-usb-devel] Re: /proc/scsi/map

[Oliver Neukum]

Am Montag, 17. Juni 2002 01:14 schrieb James Bottomley:
> oliver@neukum.name said:
> > But the drivers already know, or would have to be taught to know about
> > it. Somewhence that information has to come. You cannot avoid that
> > effort.
>
> Not necessarily: consider the SCSI WWN, which is supported by most
> modern SCSI devices.  The driver never probes for or asks for this. 
> Nowhere in the current SCSI code do we ask for this.  However user level
> commands (like sg_inq) can formulate the 0x83 page inquiry to get this
> and return the output. This works today with the current driver.

These may be an exception. You usually want to get drivers involved
even if only for synchronisation. Failing to do so has already let to
problems with usb storage.

> > That is wrong. You'd need a common method to determine device type and
> > several methods of passing guid. You are better off in implementing
> > one common way of getting at that information, which shouldn't be too
> > hard, as all the generic layer would have to do is pass up that
> > information.
>
> but the complexity is in the "common method to determine device type and
> several methods of passing guid".  Even for a simple SCSI disk, there's
> no one universal way of getting a unique id (let alone when we add the
> usb devices masquerading as scsi disks into the mix).  That will lead us

That is the point. The driver knows best what kind of devices it works on.
You can forget about the whole identification method business, if you
go for the driver. In case of usb storage and firewire that data is already
sitting there ready for taking. I suspect the same for fibrechannel.

	Regards
		Oliver

Re: [linux-usb-devel] Re: /proc/scsi/map

[James Bottomley]

oliver@neukum.name said:
>  These may be an exception. You usually want to get drivers involved
> even if only for synchronisation. Failing to do so has already let to
> problems with usb storage. 

I don't deny that there may be devices that need some type of vendor specific 
probe to get the information.

> That is the point. The driver knows best what kind of devices it works
> on. You can forget about the whole identification method business, if
> you go for the driver. In case of usb storage and firewire that data
> is already sitting there ready for taking. I suspect the same for
> fibrechannel. 

But, in SCSI, you can't.  Just for a simple device showing up as a SCSI disc 
(that's a real SCSI disc, attached say by parallel connectors), there are 
several potential ways to get a unique ID.  No one way works for all such 
disks, that is the nub of the problem.

I think, by driver, we may mean different things.  A scsi disc attaches like 
this:

   +----+
   | sd |
   +----+
      |
 +-------+
 | mid   |
 | layer |
 +-------+
     |
  +-----+
  | lld |
  +-----+
     |
 +--------+
 | real   |
 | device |
 +--------+

but the only piece that has disc specific logic is sd.  The lld is specific to 
the host adapter card (not shown), not the real device, so it is not the right 
element to probe for an ID.

In USB, things look slightly different:

   +----+
   | sd |
   +----+
      |
 +-------+
 | mid   |
 | layer |
 +-------+
     |
+---------+
| usb     |
| storage |
+---------+
     |
 +--------+
 | real   |
 | device |
 +--------+

Here, the usb-storage driver does know about the real device (and already has 
a huge exception table), so it has enough knowledge to probe for an identifier.

The thing I think is a bad idea is having to code the logic to look up a 
unique identifier (plus all the exceptions) in sd.  But for the pure SCSI 
stack, there's nowhere else to place it.  Even if you get usb-storage to 
supply an API for providing the id, it will be one of the few llds that can 
retrieve this, so it will become just another exception sd has to cope with.

James

Re: [linux-usb-devel] Re: /proc/scsi/map

[Patrick Mansfield]

On Mon, Jun 17, 2002 at 09:54:56AM -0500, James Bottomley wrote:
> 
> But, in SCSI, you can't.  Just for a simple device showing up as a SCSI disc 
> (that's a real SCSI disc, attached say by parallel connectors), there are 
> several potential ways to get a unique ID.  No one way works for all such 
> disks, that is the nub of the problem.

I agree it would be nice to get the ID via user space, but it is not that
hard to get the ID, and trying the various SCSI INQUIRY pages that supply 
the ID is not complicated. Putting ID code (including hotplug hooks) into
user space is probably more complex. I would also like the ID at scan time
for multi-path use so we don't have to allocate extraneous Scsi_Devices,
and later coalesce them.

Mike Sullivan's driverfs patch for SCSI (without any device naming) includes
code to look up an ID, and store the ID in driverfs; he was working on
a patch for 2.5.21.

FYI the various SCSI ID pages and such are described in the SCSI primaray
command for example the following:

ftp://ftp.t10.org/t10/drafts/spc3/spc3r07.pdf

Near page 316 (section 8.6) are descriptions for INQUIRY VPD page 0x80
and 0x83.

> but the only piece that has disc specific logic is sd.  The lld is specific to 
> the host adapter card (not shown), not the real device, so it is not the right 
> element to probe for an ID.

Any SCSI device can return an ID (i.e. INQUIRY VPD page 0x80 or 0x83),
so the logic need not be in sd. I don't know how removable media should
be handled (not a SCSI device being added/removed from the system), for
tape this is probably not an issue.

> 
> Here, the usb-storage driver does know about the real device (and already has 
> a huge exception table), so it has enough knowledge to probe for an identifier.

> 
> The thing I think is a bad idea is having to code the logic to look up a 
> unique identifier (plus all the exceptions) in sd.  But for the pure SCSI 
> stack, there's nowhere else to place it.  Even if you get usb-storage to 
> supply an API for providing the id, it will be one of the few llds that can 
> retrieve this, so it will become just another exception sd has to cope with.
> 
> James

usb-storage could emulate VPD page 0x83 to return the GID, and that could
then be used by the mid-layer or a user level program to extract an ID.

-- Patrick Mansfield

Re: [linux-usb-devel] Re: /proc/scsi/map

[James Bottomley]

patmans@us.ibm.com said:
> I agree it would be nice to get the ID via user space, but it is not
> that hard to get the ID, and trying the various SCSI INQUIRY pages
> that supply  the ID is not complicated.

OK, how about some hardware scenarios:

I've a scsi array (an LSI 6298) which has no WWN page (0x83) but does have a 
serial number in the VPD page.  However, the VPD serial number is an 
identifier for the array *controller* and thus is the *same* for all the LUNs. 
 For this beast I have to add in the LUN number to the VPD serial number to 
get a unique identifier per device.  Worse, the LSI 6298 is multi path.  Now I 
get a different "unique" identifier depending on path (because different 
controllers give different VPD serial numbers), so if I want to identify it 
uniquely, I have to use some property of the LUN, like partion UUID.

Or, what about an old 8 bit EMC symmetrix.  They have no WWN page and they 
embed both a LUN and a *path* identifier in the VPD serial number.  To get the 
globally unique ID for this one, I have to strip off the path part.

> FYI the various SCSI ID pages and such are described in the SCSI
> primaray command for example the following:

> ftp://ftp.t10.org/t10/drafts/spc3/spc3r07.pdf 

Ah, but that's the scsi-3 spec which (finally) cleaned up this unique name 
business.  However, for SCSI-2 and before, it was an unholy mess, as the two 
examples above illustrate.  I agree that for all modern devices which are 
SCSI-3 SPC compliant, then just asking for the WWN page probably works.  The 
question is what to do about all the legacy hardware out there?

James

Re: [linux-usb-devel] Re: /proc/scsi/map

[David Brownell]

> Any SCSI device can return an ID (i.e. INQUIRY VPD page 0x80 or 0x83),
> so the logic need not be in sd. I don't know how removable media should
> be handled (not a SCSI device being added/removed from the system), for
> tape this is probably not an issue.
> ...
> 
>>Here, the usb-storage driver does know about the real device (and already has 
>>a huge exception table), so it has enough knowledge to probe for an identifier.
>  
> usb-storage could emulate VPD page 0x83 to return the GID, and that could
> then be used by the mid-layer or a user level program to extract an ID.

Except that as I recall, that's not practical for all devices; they may not
have built in IDs.  Hence there need to be topology-based IDs available.

Which is why I had pointed out usb_make_path() ... which returns a stable
controller ID for the "CBTU tuple".  It isn't going to get re-assigned by
changing the bus probe sequence or driver load order.  Even PCI has better
IDs available than those simple numbers (pci_device.slot_name).

If the driverfs names for all devices are supposed to be using stable IDs
(ones that don't change unless at least the hardware gets re-configured)
where possible, then those driverfs names will be better answers for the
"what is this device's topological ID" than those purely numeric CBTU tuples.

(And they won't address the volume/media/... ID level problems, which are
of course a separate issue.)

- Dave

Re: [linux-usb-devel] Re: /proc/scsi/map

[David Brownell]

> The idea behind using hotplug to solve the problem is that with scsimon, a 
> hotplug insertion event is generated for every SCSI device as it is added.  
> The script is provided with the information the kernel knows (host, channel, 
> pun lun, model and vendor inquiry strings---see www.torque.net/scsimon.html 

Make that http://www.torque.net/scsi/scsimon.html ... that page is linked
from the http://linux-hotplug.sourceforge.net links page, and I'm glad to
see it's been updated recently.


> for details).  The hotplug script then does the remaining processing to 
> extract the ID from the device (by ioctls, sending down SCSI commands etc.) 
> and then binds it into the /dev/volume nodes using the identifier it 
> determines.
> 
> The result is that however you move the device around (between controllers or 
> even change its id), it will always show up as its unique /dev/volume name.
> 
> The key philosophy is that the code to make the policy decision for assigning 
> a unique name isn't cluttering up the kernel, it's in user land where it can 
> be easily customised.  

I think that's a good approach for packaging that naming policy, though I'm
not quite sure where it stands in relation to "driverfs".  As with "usbfs"
it seems to me that there need to be both low-level "topological" and higher
level "logical/policy-driven" names.  And I'd prefer not to see a multiplicity
of approaches for anything except the bus-specific parts (which in this case
is "SCSI" even if the transport may be USB); I think I'm not alone in that.

Also, given what I noted earlier about stable IDs, I think SCSI_HOST should
be a stable string ID, not an integer that can easily get switched around
during system boot.

- Dave

Re: [linux-usb-devel] Re: /proc/scsi/map

[Patrick Mansfield]

On Mon, Jun 17, 2002 at 11:42:05AM -0500, James Bottomley wrote:

> OK, how about some hardware scenarios:
> 

> Ah, but that's the scsi-3 spec which (finally) cleaned up this unique name 
> business.  However, for SCSI-2 and before, it was an unholy mess, as the two 
> examples above illustrate.  I agree that for all modern devices which are 
> SCSI-3 SPC compliant, then just asking for the WWN page probably works.  The 
> question is what to do about all the legacy hardware out there?
> 
> James

Yes, legacy or broken devices need vendor specific code to get a unique
serial number or uid, but it is not clear to me that putting the code
in user space is better or worse than kernel space. Maybe we need to
see some implementations.

For user space get-id code:

What happens at boot time? Do we now need special stripped copies of user
space get-id commands to boot from scsi (for use with initrd)? Do we just
wait until the system is booted before getting/setting the ID's?

If I build without sg or /proc, should SCSI ID's still be available?

How can I efficiently handle multi-path code without overallocating
N Scsi_Devices (where N is the number of paths to each Scsi_Device)?

For kernel implementations, we could add a special entry in the device_list
or have a new list mapping vendor+product to a get-the-id function.

Module code could be created that contains the functional code to get an
ID, and adds a pointer to the function in device_list or some other list.

User or kernel code could supply a table with a VPD page to use, and an
offset or such within the page to a serial number, and flags for other
special usage such as appending the LUN value to the serial number.

-- Patrick Mansfield

Re: [linux-usb-devel] Re: /proc/scsi/map

[James Bottomley]

patmans@us.ibm.com said:
> Yes, legacy or broken devices need vendor specific code to get a
> unique serial number or uid, but it is not clear to me that putting
> the code in user space is better or worse than kernel space. Maybe we
> need to see some implementations. 

For me the advantage of doing this in user space is that we can use simple 
scripting to handle all the exceptions (and actually, allowing easy 
reconfiguration of the exceptions), but I agree, a reference implementation 
would be a useful demonstration.

> What happens at boot time? Do we now need special stripped copies of
> user space get-id commands to boot from scsi (for use with initrd)? Do
> we just wait until the system is booted before getting/setting the
> ID's? 

That depends on whether you really need the unique id at boot or not.  The 
boot code is really only looking for the root filesystem, which can be found 
by fs label, so there may not be a need to put all this in initrd.  However, 
if we have to it would go in as part of the diet{libc, hotplug} stuff.

If you don't put it in initrd, then you have an init script that runs to scan 
all the existing devices and configure them accordingly.

> If I build without sg or /proc, should SCSI ID's still be available? 

Without /proc, depending on how you do the sg<->sd mapping, possibly.  Without 
sg, no since I'd use that to formulate the SCSI cdb to get the ID.

> How can I efficiently handle multi-path code without overallocating N
> Scsi_Devices (where N is the number of paths to each Scsi_Device)? 

Since this is a one time configuration, is overallocation such a bad thing 
(particularly as the allocator is moving towards a much more dynamic model in 
2.5)?  You can allow the normal stuff to run and then trigger multi-path 
configuration adds as you see the same unique ID come in for different devices.

> For kernel implementations, we could add a special entry in the
> device_list or have a new list mapping vendor+product to a get-the-id
> function. 

device_list is what first got me thinking down this path:  I keep running into 
customer problems that boil down to XXX linux distribution doesn't find all 
the LUNS on my array.  The obvious answer is "that's because it isn't in 
device_list".  Now, I could solve the problem by giving the customer a kernel 
patch adding the device, but this runs into problems with inexpert users, or 
enterprise customers who won't deviate from the "standard" kernel.  Being 
lazy, the solution I ultimately adopted was to send them an init script that 
recognised the array from it's inquiry strings and used scsi-add-single-device 
to probe its LUNs.

James