allow_dio/SG_FLAG_DIRECT_IO for sr devices

allow_dio/SG_FLAG_DIRECT_IO for sr devices

[Adar Dembo]

When sending SG_IO ioctls to a device:
- /dev/sgX devices (using sg.c) follow the behavior described at http://tldp.org/HOWTO/SCSI-Generic-HOWTO/dio.html in terms of defaulting to indirect I/O, and using direct I/O only if both /proc/scsi/sg/allow_dio and SG_FLAG_DIRECT_IO in the request are set.
- /dev/srX devices (using sr.c) default to direct I/O. At least, that appears to be the case from inspecting the code, as sr_block_ioctl->cdrom_ioctl->scsi_cmd_ioctl->sg_io->blk_rq_map_user->__blk_rq_map_user will attempt to map the pages, regardless of what the user specified in the request or in the proc node.

This difference caught me by surprise; my expectation would be that an sg_io_hdr_t passed via SG_IO would be serviced in the same way regardless of the underlying driver. Is this a bug, or a feature? If the latter, is it documented somewhere?

Re: allow_dio/SG_FLAG_DIRECT_IO for sr devices

[Douglas Gilbert]

Adar Dembo wrote:
> When sending SG_IO ioctls to a device:
> - /dev/sgX devices (using sg.c) follow the behavior described at http://tldp.org/HOWTO/SCSI-Generic-HOWTO/dio.html in terms of defaulting to indirect I/O, and using direct I/O only if both /proc/scsi/sg/allow_dio and SG_FLAG_DIRECT_IO in the request are set.
> - /dev/srX devices (using sr.c) default to direct I/O. At least, that appears to be the case from inspecting the code, as sr_block_ioctl->cdrom_ioctl->scsi_cmd_ioctl->sg_io->blk_rq_map_user->__blk_rq_map_user will attempt to map the pages, regardless of what the user specified in the request or in the proc node.
> 
> This difference caught me by surprise; my expectation would be that an sg_io_hdr_t passed via SG_IO would be serviced in the same way regardless of the underlying driver. Is this a bug, or a feature? If the latter, is it documented somewhere?

As the name suggests the SG_IO ioctl was originally
implemented in the sg driver. Due to difficulties in
discovering which sg device corresponded to a particular
SCSI device node belonging to another driver (e.g. /dev/sr0
belonging to the sr driver), the SG_IO ioctl was
grafted on to various other drivers (in some cases via
the block layer).

The result is not always pretty and there are many
subtle differences. There is no 100% clean SCSI
pass-through in Linux (and if there was it could not
easily share a device with another high level SCSI driver
such as sr). The sg and the bsg drivers give the cleanest
pass-throughs. The rest tend to make assumptions appropriate
to their class of devices. And it is not just the SG_IO
ioctl that you need to take care of: some other drivers
will hang on open() if the device is not ready and the
O_NONBLOCK flag is not given.

Summary: the driver "owning" the file descriptor that
the SG_IO ioctl uses, is in control.

Doug Gilbert

RE: allow_dio/SG_FLAG_DIRECT_IO for sr devices

[Adar Dembo]

Douglas Gilbert wrote:
> Adar Dembo wrote:
> > When sending SG_IO ioctls to a device:
> > - /dev/sgX devices (using sg.c) follow the behavior described at
> > http://tldp.org/HOWTO/SCSI-Generic-HOWTO/dio.html in terms of
> > defaulting to indirect I/O, and using direct I/O only if both
> > /proc/scsi/sg/allow_dio and SG_FLAG_DIRECT_IO in the request are set.
> > - /dev/srX devices (using sr.c) default to direct I/O. At least, that
> > appears to be the case from inspecting the code, as
> > sr_block_ioctl->cdrom_ioctl->scsi_cmd_ioctl->sg_io->blk_rq_map_user-
> > >__blk_rq_map_user will attempt to map the pages, regardless of what
> > the user specified in the request or in the proc node.
> > 
> > This difference caught me by surprise; my expectation would be that an
> > sg_io_hdr_t passed via SG_IO would be serviced in the same way
> > regardless of the underlying driver. Is this a bug, or a feature? If
> > the latter, is it documented somewhere?
> 
> As the name suggests the SG_IO ioctl was originally
> implemented in the sg driver. Due to difficulties in
> discovering which sg device corresponded to a particular
> SCSI device node belonging to another driver (e.g. /dev/sr0
> belonging to the sr driver), the SG_IO ioctl was
> grafted on to various other drivers (in some cases via
> the block layer).
> 
> The result is not always pretty and there are many
> subtle differences. There is no 100% clean SCSI
> pass-through in Linux (and if there was it could not
> easily share a device with another high level SCSI driver
> such as sr). The sg and the bsg drivers give the cleanest
> pass-throughs. The rest tend to make assumptions appropriate
> to their class of devices. And it is not just the SG_IO
> ioctl that you need to take care of: some other drivers
> will hang on open() if the device is not ready and the
> O_NONBLOCK flag is not given.
> 
> Summary: the driver "owning" the file descriptor that
> the SG_IO ioctl uses, is in control.

Thanks for the response.

What you've described seems like a reasonable justification for why the different drivers should be allowed to handle SG_IO differently. That said, do you think it makes sense for the block layer's implementation to be extended to check allow_dio and/or SG_FLAG_DIRECT_IO, as the sg driver does?