IO transfer limits

IO transfer limits

[john clyne]

Can anyone give me some guidance on where in the IO stack I might be running
into a 512KB limit on IO transfer sizes to an external FC device? I've
checked IO scheduler parameter
(/sys/block/<dev>/queue/{max_sectors_kb,max_hw_sectors_kb}. Both are set to
32767. I'm using Qlogic HBAs (qla2312), but I don't see any relevent
parameters. I'm running RHEL 4.0 with a 2.6.9-34 kernel. Any pointers would
be greatly appreciated.

cheers - jc

-- 
View this message in context: http://www.nabble.com/IO-transfer-limits-tf2953389.html#a8260533
Sent from the linux-scsi mailing list archive at Nabble.com.

Re: IO transfer limits

[Mike Christie]

john clyne wrote:
> Can anyone give me some guidance on where in the IO stack I might be running
> into a 512KB limit on IO transfer sizes to an external FC device? I've
> checked IO scheduler parameter
> (/sys/block/<dev>/queue/{max_sectors_kb,max_hw_sectors_kb}. Both are set to
> 32767. I'm using Qlogic HBAs (qla2312), but I don't see any relevent
> parameters. I'm running RHEL 4.0 with a 2.6.9-34 kernel. Any pointers would
> be greatly appreciated.
> 

There are also scatterlist limits.

/sys/class/scsi_host/hostX/sg_tablesize is a limit for the number of
scatter list entries. For qla2xxx it is 255.

The scsi layer sets the queue's max_phys_segments to 128 by default. I
thought there was ia scsi compile time option to increase this, but
maybe you have to just modify the SCSI_MAX_PHYS_SEGMENTS define by hand.

So with the default value and with 4 K pages if you end up getting pages
that cannot be clustered you will end up with 4K * 128.

Re: IO transfer limits

[Douglas Gilbert]

john clyne wrote:
> Can anyone give me some guidance on where in the IO stack I might be running
> into a 512KB limit on IO transfer sizes to an external FC device? I've
> checked IO scheduler parameter
> (/sys/block/<dev>/queue/{max_sectors_kb,max_hw_sectors_kb}. Both are set to
> 32767. I'm using Qlogic HBAs (qla2312), but I don't see any relevent
> parameters. I'm running RHEL 4.0 with a 2.6.9-34 kernel. Any pointers would
> be greatly appreciated.

John,
I discuss the subject in this page:
   http://www.torque.net/sg/sg_io.html
in the section titled:
Maximum transfer size per command

Mike C. has given you the answer for the block device
interface (e.g. via /dev/sda); you should be able
to do about 8 times better via the scsi generic
interface (e.g. /dev/sg0).

Doug Gilbert

Re: IO transfer limits

[john clyne]

So given that the HBA max_sectors_kb looks sufficiently large, my
interpretation of your article is the next place I need to look is at
ajusting SCSI_MAX_PHYS_SEGMENTS. Sounds like i can only increase this to
256. Does that than give me a max transfer size of 256 * page_size (4k) =
1MB? Is 1MB the limit or am I missing something?

thanks - jc


Douglas Gilbert wrote:
> 
> john clyne wrote:
>> Can anyone give me some guidance on where in the IO stack I might be
>> running
>> into a 512KB limit on IO transfer sizes to an external FC device? I've
>> checked IO scheduler parameter
>> (/sys/block/<dev>/queue/{max_sectors_kb,max_hw_sectors_kb}. Both are set
>> to
>> 32767. I'm using Qlogic HBAs (qla2312), but I don't see any relevent
>> parameters. I'm running RHEL 4.0 with a 2.6.9-34 kernel. Any pointers
>> would
>> be greatly appreciated.
> 
> John,
> I discuss the subject in this page:
>    http://www.torque.net/sg/sg_io.html
> in the section titled:
> Maximum transfer size per command
> 
> Mike C. has given you the answer for the block device
> interface (e.g. via /dev/sda); you should be able
> to do about 8 times better via the scsi generic
> interface (e.g. /dev/sg0).
> 
> Doug Gilbert
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

-- 
View this message in context: http://www.nabble.com/IO-transfer-limits-tf2953389.html#a8268682
Sent from the linux-scsi mailing list archive at Nabble.com.

Re: IO transfer limits

[john clyne]

What do the different hostX in /sys/class/scsi_host corespond to? There are
seven hostX directories, 5 with sg_tablesize set to 128 and two set to 255.

Is the implication that the hard limit is 255 * page_size, or is page_size
simply the default?



Mike Christie wrote:
> 
> john clyne wrote:
>> Can anyone give me some guidance on where in the IO stack I might be
>> running
>> into a 512KB limit on IO transfer sizes to an external FC device? I've
>> checked IO scheduler parameter
>> (/sys/block/<dev>/queue/{max_sectors_kb,max_hw_sectors_kb}. Both are set
>> to
>> 32767. I'm using Qlogic HBAs (qla2312), but I don't see any relevent
>> parameters. I'm running RHEL 4.0 with a 2.6.9-34 kernel. Any pointers
>> would
>> be greatly appreciated.
>> 
> 
> There are also scatterlist limits.
> 
> /sys/class/scsi_host/hostX/sg_tablesize is a limit for the number of
> scatter list entries. For qla2xxx it is 255.
> 
> The scsi layer sets the queue's max_phys_segments to 128 by default. I
> thought there was ia scsi compile time option to increase this, but
> maybe you have to just modify the SCSI_MAX_PHYS_SEGMENTS define by hand.
> 
> So with the default value and with 4 K pages if you end up getting pages
> that cannot be clustered you will end up with 4K * 128.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

-- 
View this message in context: http://www.nabble.com/IO-transfer-limits-tf2953389.html#a8269227
Sent from the linux-scsi mailing list archive at Nabble.com.

Re: IO transfer limits

[Douglas Gilbert]

john clyne wrote:
> What do the different hostX in /sys/class/scsi_host corespond to? There are
> seven hostX directories, 5 with sg_tablesize set to 128 and two set to 255.

A Linux "host" is a SCSI initiator port (e.g. FC) or a
SCSI initiator device (e.g. SAS). Another way of looking
at a "host" is as a bridge between a computer bus (e.g. PCI)
and a storage transport. There is usually one (low level)
driver (LLD) controlling all hosts associated with a
specific class of hardware.

If you fetch the lsscsi utility and load it then you can
try 'lsscsi --hosts' to list the active hosts on a
system (numbered on the left) and see the names of the
various LLDs associated with them. Here is an example:

# lsscsi --hosts
[0]    sata_nv
[1]    sata_nv
[2]    sata_nv
[3]    sata_nv
[4]    mptsas
[5]    aic94xx
[6]    sbp2

The first four are SATA ports (connectors) on the motherboard,
all controlled by the same driver. Then there is a LSI SAS
HBA (whose driver is mptsas), an Adaptec SAS HBA (48300) and
finally an Adaptec IEEE 1394 controller.

> Is the implication that the hard limit is 255 * page_size, or is page_size
> simply the default?

There are "big" pages (around 1 MB in size) but I'm
unaware that anything in the SCSI subsystem uses them.
Otherwise the kernel page size is typically 4 KB.
When the scsi generic driver builds its scatter gather
lists, then it attempts to place 8 contiguous pages in
each scatter gather element.

"Arm waving" was a term used when I tried to explain
to several kernel people that there were users out there
that needed larger IO transfer limits. So I suggest that
you talk the the management and tell them why you
need higher limits. Linux is retarded in this area.

Doug Gilbert

> Mike Christie wrote:
>> john clyne wrote:
>>> Can anyone give me some guidance on where in the IO stack I might be
>>> running
>>> into a 512KB limit on IO transfer sizes to an external FC device? I've
>>> checked IO scheduler parameter
>>> (/sys/block/<dev>/queue/{max_sectors_kb,max_hw_sectors_kb}. Both are set
>>> to
>>> 32767. I'm using Qlogic HBAs (qla2312), but I don't see any relevent
>>> parameters. I'm running RHEL 4.0 with a 2.6.9-34 kernel. Any pointers
>>> would
>>> be greatly appreciated.
>>>
>> There are also scatterlist limits.
>>
>> /sys/class/scsi_host/hostX/sg_tablesize is a limit for the number of
>> scatter list entries. For qla2xxx it is 255.
>>
>> The scsi layer sets the queue's max_phys_segments to 128 by default. I
>> thought there was ia scsi compile time option to increase this, but
>> maybe you have to just modify the SCSI_MAX_PHYS_SEGMENTS define by hand.
>>
>> So with the default value and with 4 K pages if you end up getting pages
>> that cannot be clustered you will end up with 4K * 128.
>> -
>> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>>
> 

Re: IO transfer limits

[Stefan Richter]

Douglas Gilbert wrote:
> john clyne wrote:
>> What do the different hostX in /sys/class/scsi_host corespond to? There are
>> seven hostX directories, 5 with sg_tablesize set to 128 and two set to 255.
> 
> A Linux "host" is a SCSI initiator port (e.g. FC) or a
> SCSI initiator device (e.g. SAS). Another way of looking
> at a "host" is as a bridge between a computer bus (e.g. PCI)
> and a storage transport. There is usually one (low level)
> driver (LLD) controlling all hosts associated with a
> specific class of hardware.
> 
> If you fetch the lsscsi utility and load it then you can
> try 'lsscsi --hosts' to list the active hosts on a
> system (numbered on the left) and see the names of the
> various LLDs associated with them. Here is an example:
> 
> # lsscsi --hosts
> [0]    sata_nv
> [1]    sata_nv
> [2]    sata_nv
> [3]    sata_nv
> [4]    mptsas
> [5]    aic94xx
> [6]    sbp2
> 
> The first four are SATA ports (connectors) on the motherboard,
> all controlled by the same driver. Then there is a LSI SAS
> HBA (whose driver is mptsas), an Adaptec SAS HBA (48300) and
> finally an Adaptec IEEE 1394 controller.

A side note:
I don't think a Scsi_Host has a well-defined meaning beyond "the
kernel-internal resource which LLDs use to connect to the Linux SCSI mid
layer". It may have further meaning for many LLDs, but not for all.
Specifically, the host6 in your example is in the current implementation
 indeed nothing more than an internal resource. lsscsi is nevertheless
able to determine the actual initiator port by means of knowledge of the
implementation.
-- 
Stefan Richter
-=====-=-=== ---= -=-==
http://arcgraph.de/sr/

Re: IO transfer limits

[James Bottomley]

On Wed, 2007-01-10 at 15:07 -0800, john clyne wrote:
> So given that the HBA max_sectors_kb looks sufficiently large, my
> interpretation of your article is the next place I need to look is at
> ajusting SCSI_MAX_PHYS_SEGMENTS. Sounds like i can only increase this to
> 256. Does that than give me a max transfer size of 256 * page_size (4k) =
> 1MB? Is 1MB the limit or am I missing something?

Well ... that's not the maximum possible ... just the maximum number of
SG elements.  If the buffer is completely fragmented in physical memory,
then it will be 1MB ... however, linux does a reasonable job of getting
physically contiguous pages into a large memory allocation, so you could
get transfers that are larger than the 1MB limit by quite a way.  And,
of course, any system that has a larger page size or uses an iommu will
go over this 1MB as well.

Jmaes

no utility / method to show association between host bus adapter and non-sg BLOCK devices

[Thayne Harmon]

Gentlemen,

hwinfo, lshal, sysfs do not show the relationship for non-sg BLOCK devices with there 
associated Host Bus Adapter.

Do you know of a utility or method that can show this?

Example is the HP/Compaq CCISS block driver.

The HBA and devices are listed, but no association is given or can be determine, 
only by the user knowing which is which.

The kernel certainly knows, surely the above apps could be made to 
determine this or some utility exits that will show this?

Thanks for any help,

Thayne Harmon

Re: IO transfer limits

[john clyne]

So what's the best course of action for increasing the 512kb IO transfer size
limit, changing SCSI_MAX_PHYS_SEGMENTS (currently 128 and apparently can be
set no greater than 256), or increasing the page size? Or is there another
option.

thanks - jc


James Bottomley wrote:
> 
> On Wed, 2007-01-10 at 15:07 -0800, john clyne wrote:
>> So given that the HBA max_sectors_kb looks sufficiently large, my
>> interpretation of your article is the next place I need to look is at
>> ajusting SCSI_MAX_PHYS_SEGMENTS. Sounds like i can only increase this to
>> 256. Does that than give me a max transfer size of 256 * page_size (4k) =
>> 1MB? Is 1MB the limit or am I missing something?
> 
> Well ... that's not the maximum possible ... just the maximum number of
> SG elements.  If the buffer is completely fragmented in physical memory,
> then it will be 1MB ... however, linux does a reasonable job of getting
> physically contiguous pages into a large memory allocation, so you could
> get transfers that are larger than the 1MB limit by quite a way.  And,
> of course, any system that has a larger page size or uses an iommu will
> go over this 1MB as well.
> 
> Jmaes
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

-- 
View this message in context: http://www.nabble.com/IO-transfer-limits-tf2953389.html#a8284792
Sent from the linux-scsi mailing list archive at Nabble.com.

Re: no utility / method to show association between host bus adapter and non-sg BLOCK devices

[Douglas Gilbert]

Thayne Harmon wrote:
> Gentlemen,
> 
> hwinfo, lshal, sysfs do not show the relationship for non-sg BLOCK devices with there 
> associated Host Bus Adapter.

All devices (i.e. logical units) have a 4 element tuple
associated with them and the first element is the host
number. A HBA contains one or more hosts. Then you can
datamine in /sys/class/scsi_host/host<n> for whatever
information you want.

> Do you know of a utility or method that can show this?

May I suggest lsscsi. That won't help you in the lk 2.4
series and earlier though There are other methods by
which the sg device corresponding to a "non-sg" block
device (e.g. /dev/sdc) can be found.

> Example is the HP/Compaq CCISS block driver.
> 
> The HBA and devices are listed, but no association is given or can be determine, 
> only by the user knowing which is which.
> 
> The kernel certainly knows, surely the above apps could be made to 
> determine this or some utility exits that will show this?

See http://www.torque.net/scsi/lsscsi.html

Doug Gilbert

Re: IO transfer limits

[Douglas Gilbert]

Stefan Richter wrote:
> Douglas Gilbert wrote:
>> john clyne wrote:
>>> What do the different hostX in /sys/class/scsi_host corespond to? There are
>>> seven hostX directories, 5 with sg_tablesize set to 128 and two set to 255.
>> A Linux "host" is a SCSI initiator port (e.g. FC) or a
>> SCSI initiator device (e.g. SAS). Another way of looking
>> at a "host" is as a bridge between a computer bus (e.g. PCI)
>> and a storage transport. There is usually one (low level)
>> driver (LLD) controlling all hosts associated with a
>> specific class of hardware.
>>
>> If you fetch the lsscsi utility and load it then you can
>> try 'lsscsi --hosts' to list the active hosts on a
>> system (numbered on the left) and see the names of the
>> various LLDs associated with them. Here is an example:
>>
>> # lsscsi --hosts
>> [0]    sata_nv
>> [1]    sata_nv
>> [2]    sata_nv
>> [3]    sata_nv
>> [4]    mptsas
>> [5]    aic94xx
>> [6]    sbp2
>>
>> The first four are SATA ports (connectors) on the motherboard,
>> all controlled by the same driver. Then there is a LSI SAS
>> HBA (whose driver is mptsas), an Adaptec SAS HBA (48300) and
>> finally an Adaptec IEEE 1394 controller.
> 
> A side note:
> I don't think a Scsi_Host has a well-defined meaning beyond "the
> kernel-internal resource which LLDs use to connect to the Linux SCSI mid
> layer". It may have further meaning for many LLDs, but not for all.
> Specifically, the host6 in your example is in the current implementation
>  indeed nothing more than an internal resource. lsscsi is nevertheless
> able to determine the actual initiator port by means of knowledge of the
> implementation.

There are two sides to a "host": a kernel side (e.g. a
PCI device or virtual) and a storage transport side.
A host can be seen as a bridge between the two sides.

SCSI command sets need the concept of an initiator.
For queuing, mode page policy and reservations (i.e.
in multi-initiator environments) those initiators
(actually initiator ports) need domain unique identifiers,
preferably world wide unique. The identifier is an
attribute of the external side (i.e. the storage
transport side) of a linux "host".

So even if you consider the kernel side of a host
is a kludge, there is still the storage transport
side to consider.

In the case of sbp, the initiator (device and port) has
a EUI-64 wwn. SBP, USB mass storage, and iSCSI all set up
SCSI hosts in Linux on a session basis (just-in-time if
you like). As long as the initiator port identifiers are
stable (predictable from one session to the next) it seems
to me little different to SAS, SPI and FC which maintain
their hosts for as long as their HBAs are present.


There are cheap external boxes out there that have 1394,
USB and (e)SATA interfaces. I wonder what would happen
if one tried to use two interfaces connected to different
machines at the same time :-)

Doug Gilbert

Re: IO transfer limits

[Stefan Richter]

Douglas Gilbert wrote:
> There are cheap external boxes out there that have 1394,
> USB and (e)SATA interfaces. I wonder what would happen
> if one tried to use two interfaces connected to different
> machines at the same time :-)

AFAIK these are all based on the OXUF924 series of bridge chips. Their
FireWire side supports multiple initiators, at least if OxSemi's
firmware is used, like all but the very earliest OxSemi SBP-2 target
devices. However my _guess_ is the different interconnects---FireWire,
USB, eSATA---cannot be used together at once. (John, sorry for straying
off the subject.)
-- 
Stefan Richter
-=====-=-=== ---= -==--
http://arcgraph.de/sr/

Re: IO transfer limits

[James Bottomley]

On Thu, 2007-01-11 at 11:48 -0800, john clyne wrote:
> So what's the best course of action for increasing the 512kb IO transfer size
> limit, changing SCSI_MAX_PHYS_SEGMENTS (currently 128 and apparently can be
> set no greater than 256), or increasing the page size? Or is there another
> option.

Well, without knowing the details of what you want to do it's hard to
say.  Doug offered you a method via SG_IO (albeit involving buffer
copying); you can just increase the SCSI_MAX_PHYS_SEGMENTS, or you could
increase the PAGE_SIZE or use an IOMMU.  For a given workload, you'll
just have to see which works out best in practice.

James

Re: IO transfer limits

[john clyne]

Increasing SCSI_MAX_PHYS_SEGMENTS to 256 has doubled the IO size to 1MB.
Thanks to all!

cheers - jc


Mike Christie wrote:
> 
> john clyne wrote:
>> Can anyone give me some guidance on where in the IO stack I might be
>> running
>> into a 512KB limit on IO transfer sizes to an external FC device? I've
>> checked IO scheduler parameter
>> (/sys/block/<dev>/queue/{max_sectors_kb,max_hw_sectors_kb}. Both are set
>> to
>> 32767. I'm using Qlogic HBAs (qla2312), but I don't see any relevent
>> parameters. I'm running RHEL 4.0 with a 2.6.9-34 kernel. Any pointers
>> would
>> be greatly appreciated.
>> 
> 
> There are also scatterlist limits.
> 
> /sys/class/scsi_host/hostX/sg_tablesize is a limit for the number of
> scatter list entries. For qla2xxx it is 255.
> 
> The scsi layer sets the queue's max_phys_segments to 128 by default. I
> thought there was ia scsi compile time option to increase this, but
> maybe you have to just modify the SCSI_MAX_PHYS_SEGMENTS define by hand.
> 
> So with the default value and with 4 K pages if you end up getting pages
> that cannot be clustered you will end up with 4K * 128.
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 
> 

-- 
View this message in context: http://www.nabble.com/IO-transfer-limits-tf2953389.html#a8307532
Sent from the linux-scsi mailing list archive at Nabble.com.

Re: no utility / method to show association between HBA & non-sg BLOCK (scsi) devices - register_blkdev()

[Douglas Gilbert]

Thayne Harmon wrote:
>>>> On Thu, Jan 11, 2007 at  1:15 PM, in message <45A69AD6.60207@torque.net>,
> Douglas Gilbert <dougg@torque.net> wrote: 
>> Thayne Harmon wrote:
>>> Gentlemen,
>>>
>>> hwinfo, lshal, sysfs do not show the relationship for non- sg BLOCK devices 
>> with there 
>>> associated Host Bus Adapter.
>> All devices (i.e. logical units) have a 4 element tuple
>> associated with them and the first element is the host
>> number. A HBA contains one or more hosts. Then you can
>> datamine in /sys/class/scsi_host/host<n> for whatever
>> information you want.
>>
>>> Do you know of a utility or method that can show this?
>> May I suggest lsscsi. That won't help you in the lk 2.4
>> series and earlier though There are other methods by
>> which the sg device corresponding to a "non- sg" block
>> device (e.g. /dev/sdc) can be found.
> 
> [context - Linux testserver 2.6.16.21-0.8-smp i586]
> 
> There is no corresponding sg device. The device file is
> /dev/cciss/c0d1.

Ok, I'm not familiar with the cciss driver. It looks like
it lives outside the linux scsi subsystem but according
to Documentation/cciss.txt it can subsequently "engage"
the scsi subsystem??

If it is outside the scsi subsystem then it doesn't
get corresponding sg devices. However as part of the
block subsystem it might accept the SG_IO ioctl (if
it accepts SCSI commands and it is implemented).

> I tried lsscsi, however it would not print out the non-sg block devices.
> 
> I have attached the output of tree /sys and the output of lsscsi and uname.
> One can search for cciss to find the devices and the driver.
> I still cannot see a relationship.

<snip sysfs dump>

> [0:0:0:0]    storage COMPAQ   MSA1000          4.32  -       
> [0:0:0:3]    disk    COMPAQ   MSA1000 VOLUME   4.32  /dev/sda
> [0:0:0:4]    disk    COMPAQ   MSA1000 VOLUME   4.32  /dev/sdb
> [0:0:0:5]    disk    COMPAQ   MSA1000 VOLUME   4.32  /dev/sdc
> [0:0:0:6]    disk    COMPAQ   MSA1000 VOLUME   4.32  /dev/sdd
> [0:0:0:7]    disk    COMPAQ   MSA1000 VOLUME   4.32  /dev/sde
> [1:0:0:0]    storage COMPAQ   MSA1000          4.32  -       
> [1:0:0:3]    disk    COMPAQ   MSA1000 VOLUME   4.32  /dev/sdf
> [1:0:0:4]    disk    COMPAQ   MSA1000 VOLUME   4.32  /dev/sdg
> [1:0:0:5]    disk    COMPAQ   MSA1000 VOLUME   4.32  /dev/sdh
> [1:0:0:6]    disk    COMPAQ   MSA1000 VOLUME   4.32  /dev/sdi
> [1:0:0:7]    disk    COMPAQ   MSA1000 VOLUME   4.32  /dev/sdj

Well this looks like output from lsscsi. And those devices look
like they could be associated with cciss, especially the
compaq "storage" devices. These devices should have corresponding
sg device nodes. Try "lsscsi -g".

Still a bit unclear as hosts 0 and 1 are Fibre Channel
judging from the sysfs output for them.

Doug Gilbert

Re: no utility / method to show association between HBA & non-sg BLOCK (scsi) devices - register_blkdev()

[Andrew Patterson]

On Wed, 2007-01-17 at 16:04 -0500, Douglas Gilbert wrote:
> Thayne Harmon wrote:
> >>>> On Thu, Jan 11, 2007 at  1:15 PM, in message <45A69AD6.60207@torque.net>,
> > Douglas Gilbert <dougg@torque.net> wrote: 
> >> Thayne Harmon wrote:
> >>> Gentlemen,
> >>>
> >>> hwinfo, lshal, sysfs do not show the relationship for non- sg BLOCK devices 
> >> with there 
> >>> associated Host Bus Adapter.
> >> All devices (i.e. logical units) have a 4 element tuple
> >> associated with them and the first element is the host
> >> number. A HBA contains one or more hosts. Then you can
> >> datamine in /sys/class/scsi_host/host<n> for whatever
> >> information you want.
> >>
> >>> Do you know of a utility or method that can show this?
> >> May I suggest lsscsi. That won't help you in the lk 2.4
> >> series and earlier though There are other methods by
> >> which the sg device corresponding to a "non- sg" block
> >> device (e.g. /dev/sdc) can be found.
> > 
> > [context - Linux testserver 2.6.16.21-0.8-smp i586]
> > 
> > There is no corresponding sg device. The device file is
> > /dev/cciss/c0d1.
> 
> Ok, I'm not familiar with the cciss driver. It looks like
> it lives outside the linux scsi subsystem but according
> to Documentation/cciss.txt it can subsequently "engage"
> the scsi subsystem??
> 
> If it is outside the scsi subsystem then it doesn't
> get corresponding sg devices. However as part of the
> block subsystem it might accept the SG_IO ioctl (if
> it accepts SCSI commands and it is implemented).
> 

Cciss is a block driver mostly outside of SCSI. It does interface into
the SCSI system for tape drives.

> > I tried lsscsi, however it would not print out the non-sg block devices.
> > 
> > I have attached the output of tree /sys and the output of lsscsi and uname.
> > One can search for cciss to find the devices and the driver.
> > I still cannot see a relationship.
> 
> <snip sysfs dump>
> 
> > [0:0:0:0]    storage COMPAQ   MSA1000          4.32  -       
> > [0:0:0:3]    disk    COMPAQ   MSA1000 VOLUME   4.32  /dev/sda
> > [0:0:0:4]    disk    COMPAQ   MSA1000 VOLUME   4.32  /dev/sdb
> > [0:0:0:5]    disk    COMPAQ   MSA1000 VOLUME   4.32  /dev/sdc
> > [0:0:0:6]    disk    COMPAQ   MSA1000 VOLUME   4.32  /dev/sdd
> > [0:0:0:7]    disk    COMPAQ   MSA1000 VOLUME   4.32  /dev/sde
> > [1:0:0:0]    storage COMPAQ   MSA1000          4.32  -       
> > [1:0:0:3]    disk    COMPAQ   MSA1000 VOLUME   4.32  /dev/sdf
> > [1:0:0:4]    disk    COMPAQ   MSA1000 VOLUME   4.32  /dev/sdg
> > [1:0:0:5]    disk    COMPAQ   MSA1000 VOLUME   4.32  /dev/sdh
> > [1:0:0:6]    disk    COMPAQ   MSA1000 VOLUME   4.32  /dev/sdi
> > [1:0:0:7]    disk    COMPAQ   MSA1000 VOLUME   4.32  /dev/sdj
> 
> Well this looks like output from lsscsi. And those devices look
> like they could be associated with cciss, especially the
> compaq "storage" devices. These devices should have corresponding
> sg device nodes. Try "lsscsi -g".

These are Fibre-Channel Storage arrays, probably going through a QLogic
HBA.

> 
> Still a bit unclear as hosts 0 and 1 are Fibre Channel
> judging from the sysfs output for them.
> 
> Doug Gilbert
> 
> 
> -
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

-- 
Andrew Patterson
Hewlett-Packard Company