Re: sg driver and Fedora Core 2

[Douglas Gilbert <dougg@torque.net>]

Jens Axboe wrote:
> On Sat, May 29 2004, Jeff Garzik wrote:
> 
>>Really what needs to happen is a generic userland tap attaches to a 
>>chrdev, issues an ioctl(2) to attach to a request_queue, and then does 
>>read(2)/write(2)/mmap(2) to communicate with the request_queue.
>>
>>:)
>>
>>Call it /dev/bsg, or somesuch.
>>
>>The SG_IO ioctl won't work for all cases, since some block devices may 
>>rightly wish to limit (or block) multiple openers on the blkdev itself. 
>>  OTOH, the SG_IO ioctl does eliminate the userland app needing to 
>>worry about _any_ target/addressing information, since that is implicit 
>>in the dentry pointing to the blkdev's inode.
> 
> 
> Hear, hear!

Jeff + Jens,
Yes, that is what I was getting at with my "bind"
idea. Could a "misc" char device minor number for
/dev/bsg be made available? It isn't clear to me how sysfs
can be used to do what Jeff proposes.


I have no expertise in Fibre Channel but Serial Attached SCSI
(SAS) throws up more addressing problems. There can be up to
16,000 devices in a single SAS domain the bulk of which would
be targets **, expanders which are the interesting ones, and
other initiators (potentially on other machines) will
also be visible. Those expanders are critical to
configuring the SAS domain. Expanders are SAS devices but
not SCSI devices (i.e. they are neither initiators nor
targets ***). It seems obvious that the management of a SAS
domain should be done from the user space. Only those
expanders and devices directly attached to a SAS Host Bus
Adapter should automatically be visible.

Enumerating all the disks, and/or all the scsi devices
(as either devices or sysfs entries) makes less and
less sense. In the case of SAS, devices can be discovered
by recursive descent of the expanders (via a bind through
/dev/bsg for example). In a large system one possible policy
could be: only if the user is interested in a
particular device should it be presented as a device
node (in /dev, /udev or /sys).


My recent patch to scsi_debug (adding up to 4 partitions per
pseudo device) "allows" (6 * 32768) device nodes to be
created! To play this game, turn off udev, hotplug and set
delay=0 in scsi_debug:-) I didn't get far with 512 MB of ram,
Patrick Mansfield got further with a multi gigabyte machine.
Seems like sysfs eats up most of the ram. To isolate out the
impact of the block subsystem the pseudo devices can be set to
be enclosure peripheral type (for example) via a scsi_debug
driver argument.


** targets include both SATA and SAS disks

*** show should SAS expanders go in /sys/class/scsi_device ?

Doug Gilbert