[RFC]: 64 bit LUN/Tags, dummy device in host_queue, host_lock <-> LLDD reentrancy

[RFC]: 64 bit LUN/Tags, dummy device in host_queue, host_lock <-> LLDD reentrancy

[Luben Tuikov]

Here's some of the things which have been looming in my mind for
some time now. Nothing important really, but I felt at least to
share it with you guys, just to keep in mind, that's all.
Comment at will.


* SAM-3 specifies that a LUN is 64 bit. While there may not
be any devices which actually even come close to using this
huge space, it provides different _addressing__modes_.

This very nicely fits LUN virtualization (iSCSI, LUN masking, etc),
and would allow one to do magic for storage devices (IP SANs, etc).

A 64 bit LUN could be presented to userspace which may/can then
present a nice ``picture'' (GUI or what not) of the underlying
physical/virtual (depending on the SCSI port name/id) storage
network.

(E.g. iSCSI communicates 64 bit LUNs only...)


* SAM-3 specifies that tags are 64 bit. It is not inconcievable that
a Task Manager (LUN structure, SAM-3, 4.8, 4.12) would use
those tags to order/id/whatnot the tasks it receives for
execution.

A really, leally long shot: Now that they are being moved up to
the block layer, it may be convenient to define them as such.


* In scsi_scan.c :: void scan_scsis(), select_queue_depths() is called
when there is a _dummy_ device in host_queue. This comes from
doing that juggle with SDpnt and oldSDpnt, allocating it in one
place and freeing it in another...

So when select_queue_depths() is called, in the host_queue, there's
_one__more_ device than there really should be, because of that SDpnt/oldSDpnt.

I filter the dummy SDpnt in select_queue_depths() by checking
vendor, model, rev and attached. But I'd rather
just get only the _actual_ devices in the host_queue, by
probably getting rid of that SDpnt/oldSDpnt juggle logic.


* It seems to me that the host_lock exists just to indirectly
impose/hint/etc. reentrancy of the LLDD methods exposed to the kernel.

Isn't the next step to stipulate that all methods of LLDDs should
be renentrant in and of themselves (which are exposed to the kernel
of course). That is, they'll have to achieve their own reentrancy
via their own locks and queues. Wouldn't this be a cleaner approach?

The kernel will have one less thing to worry about.

I realize that some LLDD will be really easy to convert and
other will have to undergo deep surgery, but this kind
of reentrancy seems to be implied by the host lock.

-- 
Luben

Re: [RFC]: 64 bit LUN/Tags, dummy device in host_queue, host_lock <-> LLDD reentrancy

[Patrick Mansfield]

On Fri, Aug 23, 2002 at 04:27:46PM -0400, Luben Tuikov wrote:

> * In scsi_scan.c :: void scan_scsis(), select_queue_depths() is called
> when there is a _dummy_ device in host_queue. This comes from
> doing that juggle with SDpnt and oldSDpnt, allocating it in one
> place and freeing it in another...
> 
> So when select_queue_depths() is called, in the host_queue, there's
> _one__more_ device than there really should be, because of that SDpnt/oldSDpnt.
> 
> I filter the dummy SDpnt in select_queue_depths() by checking
> vendor, model, rev and attached. But I'd rather
> just get only the _actual_ devices in the host_queue, by
> probably getting rid of that SDpnt/oldSDpnt juggle logic.

The above is for the "hardcoded" scans; the cleaned-up scsi_scan.c has
this change and is in the latest linus bk tree, so it should show up
in 2.5.31.

-- Patrick Mansfield

Re: [RFC]: 64 bit LUN/Tags, dummy device in host_queue, host_lock <-> LLDD reentrancy

[Aron Zeh]

> * SAM-3 specifies that a LUN is 64 bit. While there may not
> be any devices which actually even come close to using this
> huge space, it provides different _addressing__modes_.
>
> This very nicely fits LUN virtualization (iSCSI, LUN masking, etc),
> and would allow one to do magic for storage devices (IP SANs, etc).
>
> A 64 bit LUN could be presented to userspace which may/can then
> present a nice ``picture'' (GUI or what not) of the underlying
> physical/virtual (depending on the SCSI port name/id) storage
> network.
>
> (E.g. iSCSI communicates 64 bit LUNs only...)

To me all of the proposed changes look good. The one above I'd be
interested in particularly. It would help sorting out a lot of woes we had
with writing a fibre-channel HBA driver. For fibre-channel LUNs and port
IDs (WWPN) are 64-bit and depending on configuration, high values can be
(and in our case are) common.

Aron

Re: [RFC]: 64 bit LUN/Tags, dummy device in host_queue, host_lock <-> LLDD reentrancy

[James Bottomley]

ARZEH@de.ibm.com said:
> To me all of the proposed changes look good. The one above I'd be
> interested in particularly. It would help sorting out a lot of woes we
> had with writing a fibre-channel HBA driver. For fibre-channel LUNs
> and port IDs (WWPN) are 64-bit and depending on configuration, high
> values can be (and in our case are) common. 

Actually, I'd like to see us moving towards adopting the capabilities of 
driverfs for this.

PUN is to all intents and purposes now an abstraction in the FC realm.  LUNs 
too, as long as they retain the grouping abstraction with PUNs.  The mid layer 
really doesn't need to know what these are (there are a few pieces of code 
that populate the LUN field for SCSI-1 devices that would need rework).  All 
the mid layer really needs is the Scsi_Device structure that describes an 
individual LUN (and some knowledge of LUN and PUN grouping for reset action 
prediction).  It doesn't need to know or care about the current PUN and LUN 
numbers.

If FC drivers move straight to using driverfs, you can populate the driverfs 
names with whatever is meaningful to you for PUN (WWN, portid etch) and LUN 
and thus avoid this problem (and also the mapping code most have to take these 
to and from the PUN/LUN numbers).  Since driverfs is a tree, the PUN/LUN 
division is done in a directory (and could, theoretically be done on more than 
just a two level split).

James

Re: [RFC]: 64 bit LUN/Tags, dummy device in host_queue, host_lock <-> LLDD reentrancy

[Patrick Mansfield]

On Mon, Aug 26, 2002 at 10:02:47AM +0200, Aron Zeh wrote:
> 
> 
> 
> > * SAM-3 specifies that a LUN is 64 bit. While there may not
> > be any devices which actually even come close to using this
> > huge space, it provides different _addressing__modes_.
> >
> > This very nicely fits LUN virtualization (iSCSI, LUN masking, etc),
> > and would allow one to do magic for storage devices (IP SANs, etc).
> >
> > A 64 bit LUN could be presented to userspace which may/can then
> > present a nice ``picture'' (GUI or what not) of the underlying
> > physical/virtual (depending on the SCSI port name/id) storage
> > network.
> >
> > (E.g. iSCSI communicates 64 bit LUNs only...)
> 
> To me all of the proposed changes look good. The one above I'd be
> interested in particularly. It would help sorting out a lot of woes we had
> with writing a fibre-channel HBA driver. For fibre-channel LUNs and port
> IDs (WWPN) are 64-bit and depending on configuration, high values can be
> (and in our case are) common.
> 
> Aron

Exactly what high values do have you in the LUN? What kind of storage is it? 

I thought the FC ID was 24 bits (or is it 32), why do you need the WWPN?
Or, is this internal to the adapter? If the code was setup in scsi_scan.c
and an adpater (some functional callout to an iterator), the FC ID could
be stored in the Scsi_Device::id.

If we are to add an 8 byte LUN:

We still need to allow a smaller lun for all of the current open source
adapter drivers (AFAIK all open source drivers except the qlogic drivers
use a one byte lun, and the qlogic uses a two byte lun) - either a field
in Scsi_Device and/or Scsi_Cmnd, or a functional interface to get the
equivalent four byte value (and then it can be assigned to a one or two
byte value as is done today) in addition to any new 8 byte lun field.

The Scsi_Host::max_lun usage would have to remain for the old four byte
Scsi_Device::lun, and perhaps for the 8 byte lun - it would be nice if the
adapter could just return "LUN out of bounds" or such, rather than having
to set (via adapter driver code) and check (in the mid-layer) the max_lun
value. Users of the 8 byte lun might be able to set a flag (or overload
max_lun with 0) such that max_lun is not checked. 

There might need to be some sort of result from the adapter that says
"I can't handle any more LUNs", otherwise any scan/probe (even via
REPORT LUNS) might end up with lots of errors (but that might not be
really bad since it would be a first or one-time occurence when hooking
up a many LUN target). This probably is not hit (ever?) today since the
max_lun values are always set so low - even for the qlogic, 255 (I was
not able to up their max without major problems in the driver some time
ago, I haven't tried this with any of the recent qla beta drivers).

Both an 8 byte LUN capable adapter and storage are needed to implement and
test such changes.

-- Patrick Mansfield

Re: [RFC]: 64 bit LUN/Tags, dummy device in host_queue, host_lock <-> LLDD reentrancy

[Luben Tuikov]

Patrick Mansfield wrote:
>
> If we are to add an 8 byte LUN:
> 
> We still need to allow a smaller lun for all of the current open source
> adapter drivers (AFAIK all open source drivers except the qlogic drivers
> use a one byte lun, and the qlogic uses a two byte lun) - either a field
> in Scsi_Device and/or Scsi_Cmnd, or a functional interface to get the
> equivalent four byte value (and then it can be assigned to a one or two
> byte value as is done today) in addition to any new 8 byte lun field.

For transition purposes, yes.

But, ideally, the LUN should be a function of the LLDD, since
it is the _LLDD_ which knows (then maybe not) about the transport
used (FCP, SPI, etc).

The kernel shouldn't care if the transport uses 5 bits (SPI) or
64 bits (iSCSI) for LUN identification, it should use the SAM-3
definition of 64 bits.

> The Scsi_Host::max_lun usage would have to remain for the old four byte

Certainly one can limit the scan, but we no more can enumerate it so
simply as it was done before.

There may be only 6 LU, with addresses scattered in the 64 bit space,
only to tell us that the transport is iSCSI and the target is
actually 4 daisy-chained SANs. (or something like that, you guys get the idea)

LUNs should be picked up from the REPORT LUNS report data, in order to
discover all the LUNs currently available, just because of the large
but sparse LUN space.

-- 
Luben

Re: [RFC]: 64 bit LUN/Tags, dummy device in host_queue, host_lock <-> LLDD reentrancy

[Aron Zeh]

> ARZEH@de.ibm.com said:
> > To me all of the proposed changes look good. The one above I'd be
> > interested in particularly. It would help sorting out a lot of woes we
> > had with writing a fibre-channel HBA driver. For fibre-channel LUNs
> > and port IDs (WWPN) are 64-bit and depending on configuration, high
> > values can be (and in our case are) common.
>
> Actually, I'd like to see us moving towards adopting the capabilities of
> driverfs for this.
>
> PUN is to all intents and purposes now an abstraction in the FC realm.
LUNs
> too, as long as they retain the grouping abstraction with PUNs.  The mid
layer
> really doesn't need to know what these are (there are a few pieces of
code
> that populate the LUN field for SCSI-1 devices that would need rework).
All
> the mid layer really needs is the Scsi_Device structure that describes an

> individual LUN (and some knowledge of LUN and PUN grouping for reset
action
> prediction).  It doesn't need to know or care about the current PUN and
LUN
> numbers.
>
> If FC drivers move straight to using driverfs, you can populate the
driverfs
> names with whatever is meaningful to you for PUN (WWN, portid etch) and
LUN
> and thus avoid this problem (and also the mapping code most have to take
these
> to and from the PUN/LUN numbers).  Since driverfs is a tree, the PUN/LUN
> division is done in a directory (and could, theoretically be done on more
than
> just a two level split).
>
> James

James, there are some things that I don't fully understand, yet.

Does the use of driverfs-names mean that the HBA driver somehow comes into
the discovery-loop for devices (e.g. scan_scsis somehow interacts with the
HBA driver to determine which driverfs entry to create.)?
Shouldn't there be some sort of rule for LUN (or unit) naming within at
least the same hardware class? That is for: parallel, iSCSI, FC, etc.
Will the SCSI stack automatically try to discover all devices behind the
HBA (e.g, on the FC SAN)? Or will there be an interface to restrict the
range, etc?
Lastly, how will you keep track of LUNs in the SCSI stack? Will the current
LUN field disappear in favour of a generic character string?

Cheers,
Aron

Re: [RFC]: 64 bit LUN/Tags, dummy device in host_queue, host_lock <-> LLDD reentrancy

[James Bottomley]

ARZEH@de.ibm.com said:
> Does the use of driverfs-names mean that the HBA driver somehow comes
> into the discovery-loop for devices (e.g. scan_scsis somehow interacts
> with the HBA driver to determine which driverfs entry to create.)?

Well, scanning is really only for legacy.  Way into the future, I'd like to 
see hotplug notification for device attach events, so no fibre controller ever 
need be troubled by the scan.  All they'd do is send a device attach 
notification into SCSI when they detect one.  (SCSI would probably then turn 
around and ask for a REPORT_LUNS).

> Shouldn't there be some sort of rule for LUN (or unit) naming within
> at least the same hardware class? That is for: parallel, iSCSI, FC,
> etc. Will the SCSI stack automatically try to discover all devices
> behind the HBA (e.g, on the FC SAN)? Or will there be an interface to
> restrict the range, etc?

Yes, there should.  But such a rule would only be for maintaining consistency 
within driverfs, and therefore could happily become Patric Mochel's problem.  
If we go the whole hotplug route, discovery would be up to the scsi hotplug 
scripts.

> Lastly, how will you keep track of LUNs in the SCSI stack? Will the
> current LUN field disappear in favour of a generic character string? 

Internally to Scsi_Device, LUNs will be linked by pointers.  How we know 
whether such a linkage needs to occur, I'm not yet sure about.  In the worst 
case we could use driverfs matching to see if a newly discovered device's PUN 
matches an existing device, or there might be a way for the HBA driver to 
supply the information.

James

Re: [RFC]: 64 bit LUN/Tags, dummy device in host_queue, host_lock <-> LLDD reentrancy

[Mike Anderson]

James Bottomley [James.Bottomley@SteelEye.com] wrote:
> ARZEH@de.ibm.com said:
> > Does the use of driverfs-names mean that the HBA driver somehow comes
> > into the discovery-loop for devices (e.g. scan_scsis somehow interacts
> > with the HBA driver to determine which driverfs entry to create.)?
> 
> Well, scanning is really only for legacy.  Way into the future, I'd like to 
> see hotplug notification for device attach events, so no fibre controller ever 
> need be troubled by the scan.  All they'd do is send a device attach 
> notification into SCSI when they detect one.  (SCSI would probably then turn 
> around and ask for a REPORT_LUNS).
> 

There will probably still need to be some form of scan through the
hotplug scripts based on a HAs capability (parallel SCSI, FC loop). A
CAM-3 like indication that the HA is capable of auto discovery or needs
a probe from min-max.

> > Shouldn't there be some sort of rule for LUN (or unit) naming within
> > at least the same hardware class? That is for: parallel, iSCSI, FC,
> > etc. Will the SCSI stack automatically try to discover all devices
> > behind the HBA (e.g, on the FC SAN)? Or will there be an interface to
> > restrict the range, etc?
> 
> Yes, there should.  But such a rule would only be for maintaining consistency 
> within driverfs, and therefore could happily become Patric Mochel's problem.  
> If we go the whole hotplug route, discovery would be up to the scsi hotplug 
> scripts.
> 
> > Lastly, how will you keep track of LUNs in the SCSI stack? Will the
> > current LUN field disappear in favour of a generic character string? 
> 
> Internally to Scsi_Device, LUNs will be linked by pointers.  How we know 
> whether such a linkage needs to occur, I'm not yet sure about.  In the worst 
> case we could use driverfs matching to see if a newly discovered device's PUN 
> matches an existing device, or there might be a way for the HBA driver to 
> supply the information.
> 

James I do not follow the usage of LUNs linked by pointers, can you
explain further?

-Mike
-- 
Michael Anderson
andmike@us.ibm.com

Re: [RFC]: 64 bit LUN/Tags, dummy device in host_queue, host_lock <-> LLDD reentrancy

[James Bottomley]

andmike@us.ibm.com said:
> James I do not follow the usage of LUNs linked by pointers, can you
> explain further? 

Yes:  the mid-layer could function only with Scsi_Devices and Scsi_Hosts.  
LUNs are currently only used explicitly for scanning.  However, I'd like to 
make the error handler handle the consequences of its action, thus it will 
need to know LUN relationships to handle a BDR correctly.  However, the mid 
layer doesn't need an arbitrary number associated with LUN or PUN.  Since the 
number, string or whatever is only really useful to the lld.  So, simply put, 
we'd probably have an entry in the Scsi_Device for lun_list.  This would be 
the usual doubly linked list so starting with any Scsi_Device we could always 
find the associated LUNs.

James

Re: [RFC]: 64 bit LUN/Tags, dummy device in host_queue, host_lock <-> LLDD reentrancy

[Mike Anderson]

James Bottomley [James.Bottomley@SteelEye.com] wrote:
> andmike@us.ibm.com said:
> > James I do not follow the usage of LUNs linked by pointers, can you
> > explain further? 
> 
> Yes:  the mid-layer could function only with Scsi_Devices and Scsi_Hosts.  
> LUNs are currently only used explicitly for scanning.  However, I'd like to 
> make the error handler handle the consequences of its action, thus it will 
> need to know LUN relationships to handle a BDR correctly.  However, the mid 
> layer doesn't need an arbitrary number associated with LUN or PUN.  Since the 
> number, string or whatever is only really useful to the lld.  So, simply put, 
> we'd probably have an entry in the Scsi_Device for lun_list.  This would be 
> the usual doubly linked list so starting with any Scsi_Device we could always 
> find the associated LUNs.
> 

Ok, so we are looking for a sibling check. We should alter our driverfs
registration so that we pick this up for free by comparing the
de->parent of our scsi_device. Example tree shown below. Our scsi_device
devfs_entry would be pointing the lun leaf node. TBD on where all the
devfs_entrys get stored an who creates / destroys them.

Example tree
devices/root/pci0/00:09.0/name "PCI device 1014:002e"
devices/root/pci0/00:09.0/0/name "scsi0"
devices/root/pci0/00:09.0/0/0/name "chan0"
devices/root/pci0/00:09.0/0/0/1/name "tgt1"
devices/root/pci0/00:09.0/0/0/1/1/name "lun1"
devices/root/pci0/00:09.0/0/0/1/0/name "lun0"
devices/root/pci0/00:09.0/0/0/0/name "tgt0"
devices/root/pci0/00:09.0/0/0/0/1/name "lun1"
devices/root/pci0/00:09.0/0/0/0/0/name "lun0"


-Mike
-- 
Michael Anderson
andmike@us.ibm.com

Re: [RFC]: 64 bit LUN/Tags, dummy device in host_queue, host_lock <-> LLDD reentrancy

[James Bottomley]

andmike@us.ibm.com said:
> Ok, so we are looking for a sibling check

Exactly.

> Example tree
> devices/root/pci0/00:09.0/name "PCI device 1014:002e"
> devices/root/pci0/00:09.0/0/name "scsi0"
> devices/root/pci0/00:09.0/0/0/name "chan0"
> devices/root/pci0/00:09.0/0/0/1/name "tgt1"
> devices/root/pci0/00:09.0/0/0/1/1/name "lun1"
> devices/root/pci0/00:09.0/0/0/1/0/name "lun0"
> devices/root/pci0/00:09.0/0/0/0/name "tgt0"
> devices/root/pci0/00:09.0/0/0/0/1/name "lun1"
> devices/root/pci0/00:09.0/0/0/0/0/name "lun0"

I'm not too happy with this tree.  The unique string 
devices/root/pci0/00:09.0/name "PCI device 1014:002e" is enough to enumerate 
the scsi card (as long as it's not multi-function).  If I follow my PUN/LUN 
position to its logical conclusion, we no longer need scsi host numbers, 
either.  tgt1 is really a fiction: For a device that has luns there's usually 
no such thing as the target address (traditionally, it's LUN 0).  However, 
fundamentally, the point is that moving all this entirely into driverfs means 
that scsi doesn't need to care what the tree looks like, it just has mappings 
into parts of it.

James

Re: [RFC]: 64 bit LUN/Tags, dummy device in host_queue, host_lock <-> LLDD reentrancy

[Mike Anderson]

James Bottomley [James.Bottomley@SteelEye.com] wrote:
> > Example tree
> > devices/root/pci0/00:09.0/name "PCI device 1014:002e"
> > devices/root/pci0/00:09.0/0/name "scsi0"
> > devices/root/pci0/00:09.0/0/0/name "chan0"
> > devices/root/pci0/00:09.0/0/0/1/name "tgt1"
> > devices/root/pci0/00:09.0/0/0/1/1/name "lun1"
> > devices/root/pci0/00:09.0/0/0/1/0/name "lun0"
> > devices/root/pci0/00:09.0/0/0/0/name "tgt0"
> > devices/root/pci0/00:09.0/0/0/0/1/name "lun1"
> > devices/root/pci0/00:09.0/0/0/0/0/name "lun0"
> 
> I'm not too happy with this tree.  The unique string 
> devices/root/pci0/00:09.0/name "PCI device 1014:002e" is enough to enumerate 
> the scsi card (as long as it's not multi-function).  If I follow my PUN/LUN 
> position to its logical conclusion, we no longer need scsi host numbers, 
> either.  
Agreed, this most likely can be collapsed. In looking at hosts.[ch] list
addition/cleanup and talking with Greg k-h in the future (once driverfs
ref count is in) we should not being doing our own host lists and ref
counting. 

> tgt1 is really a fiction: For a device that has luns there's usually 
> no such thing as the target address (traditionally, it's LUN 0).

Is there no need to represent a I_T nexus and a I_T_L nexus? It would
seem to be a lost of information if we dump everything into to large of
a bucket and then need to make extra links to regain this information.

The tgts where just examples, these could be port names, etc. My point
was to say if luns (which can have any name you want) where contained
under a port node than it would be easy to determine siblings. The only
drawback would be an extra lun0 node.

> However, fundamentally, the point is that moving all this entirely
> into driverfs means that scsi doesn't need to care what the tree looks
> like, it just has mappings into parts of it.

I am confused as the scsi subsystem is creating parts of this tree by
deciding when and what to register with driverfs. Driverfs is only
providing the interface and some default nodes. While driverfs is doing
a lot, how we register can make future code easier or hard.

-Mike

-- 
Michael Anderson
andmike@us.ibm.com

Re: [RFC]: 64 bit LUN/Tags, dummy device in host_queue, host_lock <-> LLDD reentrancy

[Patrick Mansfield]

On Mon, Aug 26, 2002 at 03:38:09PM -0700, Mike Anderson wrote:
> James Bottomley [James.Bottomley@SteelEye.com] wrote:
> > > Example tree
> > > devices/root/pci0/00:09.0/name "PCI device 1014:002e"
> > > devices/root/pci0/00:09.0/0/name "scsi0"
> > > devices/root/pci0/00:09.0/0/0/name "chan0"
> > > devices/root/pci0/00:09.0/0/0/1/name "tgt1"
> > > devices/root/pci0/00:09.0/0/0/1/1/name "lun1"
> > > devices/root/pci0/00:09.0/0/0/1/0/name "lun0"
> > > devices/root/pci0/00:09.0/0/0/0/name "tgt0"
> > > devices/root/pci0/00:09.0/0/0/0/1/name "lun1"
> > > devices/root/pci0/00:09.0/0/0/0/0/name "lun0"
> > 
> > I'm not too happy with this tree.  The unique string 
> > devices/root/pci0/00:09.0/name "PCI device 1014:002e" is enough to enumerate 
> > the scsi card (as long as it's not multi-function).  If I follow my PUN/LUN 
> > position to its logical conclusion, we no longer need scsi host numbers, 
> > either.  
> Agreed, this most likely can be collapsed. In looking at hosts.[ch] list
> addition/cleanup and talking with Greg k-h in the future (once driverfs
> ref count is in) we should not being doing our own host lists and ref
> counting. 

> > tgt1 is really a fiction: For a device that has luns there's usually 
> > no such thing as the target address (traditionally, it's LUN 0).
> 
> Is there no need to represent a I_T nexus and a I_T_L nexus? It would
> seem to be a lost of information if we dump everything into to large of
> a bucket and then need to make extra links to regain this information.
 
> The tgts where just examples, these could be port names, etc. My point
> was to say if luns (which can have any name you want) where contained
> under a port node than it would be easy to determine siblings. The only
> drawback would be an extra lun0 node.

James/Mike - 

For hotplug user level probe/scanning, you want a target identified: having
the device model (aka driverfs) expose a target would (should) eventually
lead to a hotplug callout that can then be used to trigger a probe (sequential
lun or report lun scans) of the target from user land.

A target representation is also nice for device (i.e. target) 
removal/replacment, where we want to shutdown and perhaps remove
everything on one target with multiple LUNs. Even if we have to manually
remove each LUN, having a target in the tree lets a user figure out
which LUNs have to be shutdown/removed.

The notion of a target could be useful for multi-path path selection, when
we want to round-robin across the target ports, rather than a LUN port, or
pick the least-busy target port. The notion of a bus (like a SCSI bus, FCP
fabric, or an FCP loop) could be usefull for similiar reasons.

Also - according to Pat Mochel the struct device "name" field is supposed
to be a description of the device, not an id or unique identifier. I don't
know if he has plans to add a id/uuid field.

-- Patrick Mansfield

Re: [RFC]: 64 bit LUN/Tags, dummy device in host_queue, host_lock <-> LLDD reentrancy

[Doug Ledford]

On Mon, Aug 26, 2002 at 03:38:09PM -0700, Mike Anderson wrote:
> James Bottomley [James.Bottomley@SteelEye.com] wrote:
> > > Example tree
> > > devices/root/pci0/00:09.0/name "PCI device 1014:002e"
> > > devices/root/pci0/00:09.0/0/name "scsi0"
> > > devices/root/pci0/00:09.0/0/0/name "chan0"
> > > devices/root/pci0/00:09.0/0/0/1/name "tgt1"
> > > devices/root/pci0/00:09.0/0/0/1/1/name "lun1"
> > > devices/root/pci0/00:09.0/0/0/1/0/name "lun0"
> > > devices/root/pci0/00:09.0/0/0/0/name "tgt0"
> > > devices/root/pci0/00:09.0/0/0/0/1/name "lun1"
> > > devices/root/pci0/00:09.0/0/0/0/0/name "lun0"
> > 
> > I'm not too happy with this tree.  The unique string 
> > devices/root/pci0/00:09.0/name "PCI device 1014:002e" is enough to enumerate 
> > the scsi card (as long as it's not multi-function).  If I follow my PUN/LUN 
> > position to its logical conclusion, we no longer need scsi host numbers, 
> > either.  
> Agreed, this most likely can be collapsed. In looking at hosts.[ch] list
> addition/cleanup and talking with Greg k-h in the future (once driverfs
> ref count is in) we should not being doing our own host lists and ref
> counting. 

The scsi host can be collapsed, the other stuff should really remain.  
After all, the scsi host number was nothing more than the previous way of 
expressing "PCI device 1014:002e", they both map to a controller 
somewhere.

> > tgt1 is really a fiction: For a device that has luns there's usually 
> > no such thing as the target address (traditionally, it's LUN 0).
> 
> Is there no need to represent a I_T nexus and a I_T_L nexus?

There is.  Not really for device needs, but for management needs.  It's 
connection specific and likely is just an implementation detail, but it 
would make like simpler for LLDDs if this was expressed.  What I'm 
thinking is that both the target level struct and the lun level struct 
need to have driver_private pointers for driver defined memory structs as 
well as a back pointer from the lun struct to the parent target struct.  
This is so drivers can easily do things like implement tagged queues 
properly, which are per lun, and also implement speed negotiations, which 
are per target.

> It would
> seem to be a lost of information if we dump everything into to large of
> a bucket and then need to make extra links to regain this information.

Having links to get the information isn't bad (it's currently better than 
the crap we do now which is search the linear list until we've found all 
the needed devices, very ugly).  Having the right links is actually quite 
nice ;-)

> The tgts where just examples, these could be port names, etc. My point
> was to say if luns (which can have any name you want) where contained
> under a port node than it would be easy to determine siblings. The only
> drawback would be an extra lun0 node.

It doesn't have to be that way.  It's entirely possible to make the target 
struct not require this extra lun0 node you refer to.  However, the target 
struct should actually be a fairly thin struct, only containing enough 
info to represent those items that are in fact target wide vs. lun 
specific.  Essentially, anything that the SAM spec calls out as using an 
I_T nexus should be here, and anything that goes on an I_T_L nexus should 
be in the lun struct.  It seems to me that the specs are likely to keep 
this level of heirarchy going forward and so it should be safe to model 
after it, but I could be wrong (and I have read the latest draft specs so 
I could already be wrong).  Feel free to correct me here if I am wrong.

> I am confused as the scsi subsystem is creating parts of this tree by
> deciding when and what to register with driverfs. Driverfs is only
> providing the interface and some default nodes. While driverfs is doing
> a lot, how we register can make future code easier or hard.

Hmmm...I'm not going to comment on this yet because I want to go look a 
few things up first....

-- 
  Doug Ledford <dledford@redhat.com>     919-754-3700 x44233
         Red Hat, Inc. 
         1801 Varsity Dr.
         Raleigh, NC 27606
  

Re: [RFC]: 64 bit LUN/Tags, dummy device in host_queue, host_lock <-> LLDD reentrancy

[James Bottomley]

andmike@us.ibm.com said:
> Is there no need to represent a I_T nexus and a I_T_L nexus? It would
> seem to be a lost of information if we dump everything into to large
> of a bucket and then need to make extra links to regain this
> information. 

As far as the mid-layer is concerned, no.  I_T vs I_T_L is really somewhat of 
a fiction even in SAM-2 since the M_IDENTIFY for I_T just looks like a 
M_IDENTIFY for I_T_L with LUN set to 0.

> The tgts where just examples, these could be port names, etc. My point
> was to say if luns (which can have any name you want) where contained
> under a port node than it would be easy to determine siblings. The
> only drawback would be an extra lun0 node. 

Oh, yes, I agree with that.   The mid-layer just uses a Scsi_Device with a 
driverfs pointer attached.  If the driverfs representation happens to be a 
simple tgt1 or tgt1/lun0, the mid-layer won't care.

> I am confused as the scsi subsystem is creating parts of this tree by
> deciding when and what to register with driverfs. Driverfs is only
> providing the interface and some default nodes. While driverfs is
> doing a lot, how we register can make future code easier or hard. 

OK, I see, conceptually what I'm aiming at is that the bulk of the mid-layer 
doesn't know or care about the name:  it thinks in terms of Scsi_Host or 
Scsi_Device.  When the mid-layer needs to print a name, it fishes it out of 
driverfs.

Currently, the code that generates these names lives in the mid-layer, but it 
is possible that the day may come when we need standardisation amongst all 
disc like devices (or tape like devices, etc.), and it may not live there any 
longer (or at least, parts of it may not).

The ideal now is that we separate these as cleanly as possible inside the 
mid-layer.  I'd like us to be to the point where if we don't like the current  
host/chan/pun/lun/... naming scheme, a few formatting characters get altered 
in one routine and the new naming scheme is implemented. In the long run, I 
think we probably turn over one piece (the host/chan/pun) to the Host driver 
so that it can be used for naming an mapping (with library routines in the mid 
layer for the usual pun numbering) and the LUN part probably always belongs in 
the mid layer since that's fairly universal.

James

Re: [RFC]: 64 bit LUN/Tags, dummy device in host_queue, host_lock <-> LLDD reentrancy

[Mike Anderson]

Sorry for replying to myself, but a mistake on the previous mail in that
I meant to reference the scsi_device sdev_driverfs_dev member and
associated sdev_driverfs_dev.parent pointer.

-Mike
Mike Anderson [andmike@us.ibm.com] wrote:
> James Bottomley [James.Bottomley@SteelEye.com] wrote:
> > andmike@us.ibm.com said:
> > > James I do not follow the usage of LUNs linked by pointers, can you
> > > explain further? 
> > 
> > Yes:  the mid-layer could function only with Scsi_Devices and Scsi_Hosts.  
> > LUNs are currently only used explicitly for scanning.  However, I'd like to 
> > make the error handler handle the consequences of its action, thus it will 
> > need to know LUN relationships to handle a BDR correctly.  However, the mid 
> > layer doesn't need an arbitrary number associated with LUN or PUN.  Since the 
> > number, string or whatever is only really useful to the lld.  So, simply put, 
> > we'd probably have an entry in the Scsi_Device for lun_list.  This would be 
> > the usual doubly linked list so starting with any Scsi_Device we could always 
> > find the associated LUNs.
> > 
> 
> Ok, so we are looking for a sibling check. We should alter our driverfs
> registration so that we pick this up for free by comparing the
> de->parent of our scsi_device. Example tree shown below. Our scsi_device
> devfs_entry would be pointing the lun leaf node. TBD on where all the
> devfs_entrys get stored an who creates / destroys them.
> 
> Example tree
> devices/root/pci0/00:09.0/name "PCI device 1014:002e"
> devices/root/pci0/00:09.0/0/name "scsi0"
> devices/root/pci0/00:09.0/0/0/name "chan0"
> devices/root/pci0/00:09.0/0/0/1/name "tgt1"
> devices/root/pci0/00:09.0/0/0/1/1/name "lun1"
> devices/root/pci0/00:09.0/0/0/1/0/name "lun0"
> devices/root/pci0/00:09.0/0/0/0/name "tgt0"
> devices/root/pci0/00:09.0/0/0/0/1/name "lun1"
> devices/root/pci0/00:09.0/0/0/0/0/name "lun0"
> 
> 
> -Mike
> -- 
> Michael Anderson
> andmike@us.ibm.com
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Michael Anderson
andmike@us.ibm.com

Re: [RFC]: 64 bit LUN/Tags, dummy device in host_queue, host_lock <-> LLDD reentrancy

[Aron Zeh]

> ARZEH@de.ibm.com said:
> > Does the use of driverfs-names mean that the HBA driver somehow comes
> > into the discovery-loop for devices (e.g. scan_scsis somehow interacts
> > with the HBA driver to determine which driverfs entry to create.)?
>
> Well, scanning is really only for legacy.  Way into the future, I'd like
to
> see hotplug notification for device attach events, so no fibre controller
ever
> need be troubled by the scan.  All they'd do is send a device attach
> notification into SCSI when they detect one.  (SCSI would probably then
turn
> around and ask for a REPORT_LUNS).

Blimey, that was a quick response. Only to spurn some more questions from
me though. Sorry.

If you say scanning is legacy only does that mean you expect it to be gone
by kernel 2.6?
I currently don't know of a way that SCSI devices (LUNs) generate hotplug
notifications by. (FC sends RSCNs and other nasty stuff for ports, which
are one level higher up. I don't think reconfiguring the LUNs behind a port
would generate any message, would it?)
Even of messages are generated, they'd probably differ according to the
underlying hardware. Would HBA drivers have to catch them and use more
generic callbacks to tell the SCSI stack?

Cheers,
Aron

Re: [RFC]: 64 bit LUN/Tags, dummy device in host_queue, host_lock <-> LLDD reentrancy

[James Bottomley]

ARZEH@de.ibm.com said:
> If you say scanning is legacy only does that mean you expect it to be
> gone by kernel 2.6? 

Not unless someone else is working on it...We currently don't have any of the 
replacement infrastructure in place.  It also depends on projects outside SCSI 
(hotplug, driverfs, initramfs etc.).

> I currently don't know of a way that SCSI devices (LUNs) generate
> hotplug notifications by. (FC sends RSCNs and other nasty stuff for
> ports, which are one level higher up. I don't think reconfiguring the
> LUNs behind a port would generate any message, would it?) Even of
> messages are generated, they'd probably differ according to the
> underlying hardware. Would HBA drivers have to catch them and use more
> generic callbacks to tell the SCSI stack? 

There would have to be a generic notify mechanism implemented within SCSI.  
Probably via a skeleton Scsi_Device with attached driverfs entry passed up.  
These would then trigger the hotplug events so we get hotplug event 
consistency.

The idea is that this would occur at the PUN (device) level, not the LUN (you 
don't want 65,535 notifications for a large array...).  Once the device has 
been noticed, it's up to the hotplug system to do something (or even if it 
chooses, nothing) about the device (like probe its LUNs, etc.).

James