SCSI over USB showstopper bug?

SCSI over USB showstopper bug?

[Joerg Schilling]

Hi,

it looks as if SG_GET_RESERVED_SIZE & SG_SET_RESERVED_SIZE
are not in interaction with the underlying SCSI transport.

Programs like readcd and cdda2wav that try to get very large SCSI
transfer buffers get a confirmation for nearly any SCSI transfer size 
but later when readcd/cdda2wav try to transfer data with an
actual SCSI command, they fail with ENOMEM.

Correct fix: let sg.c make a callback to the underlying SCSI transport
		and let it get a confirmation tfor the buffer size.

Quick and dirty fix: reduce the maximum allowed DMA size to the smallest
		max DMA size of all SCSI transports.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       schilling@fokus.fraunhofer.de     (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily

Re: SCSI over USB showstopper bug?

[Arjan van de Ven]

On Tue, 2006-10-31 at 23:08 +0100, Joerg Schilling wrote:
> Hi,
> 
> it looks as if SG_GET_RESERVED_SIZE & SG_SET_RESERVED_SIZE
> are not in interaction with the underlying SCSI transport.
> 
> Programs like readcd and cdda2wav that try to get very large SCSI
> transfer buffers get a confirmation for nearly any SCSI transfer size 
> but later when readcd/cdda2wav try to transfer data with an
> actual SCSI command, they fail with ENOMEM.
> 
> Correct fix: let sg.c make a callback to the underlying SCSI transport
> 		and let it get a confirmation tfor the buffer size.
> 
> Quick and dirty fix: reduce the maximum allowed DMA size to the smallest
> 		max DMA size of all SCSI transports.

real good fix:

use SG_IO on the device directly that checks this already

Re: SCSI over USB showstopper bug?

[Joerg Schilling]

Arjan van de Ven <arjan@infradead.org> wrote:

> On Tue, 2006-10-31 at 23:08 +0100, Joerg Schilling wrote:
> > Hi,
> > 
> > it looks as if SG_GET_RESERVED_SIZE & SG_SET_RESERVED_SIZE
> > are not in interaction with the underlying SCSI transport.
> > 
> > Programs like readcd and cdda2wav that try to get very large SCSI
> > transfer buffers get a confirmation for nearly any SCSI transfer size 
> > but later when readcd/cdda2wav try to transfer data with an
> > actual SCSI command, they fail with ENOMEM.
> > 
> > Correct fix: let sg.c make a callback to the underlying SCSI transport
> > 		and let it get a confirmation tfor the buffer size.
> > 
> > Quick and dirty fix: reduce the maximum allowed DMA size to the smallest
> > 		max DMA size of all SCSI transports.
>
> real good fix:
>
> use SG_IO on the device directly that checks this already

>From looking into the source, this claim seems to be wrong.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       schilling@fokus.fraunhofer.de     (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily

Re: SCSI over USB showstopper bug?

[Jens Axboe]

On Wed, Nov 01 2006, Joerg Schilling wrote:
> Arjan van de Ven <arjan@infradead.org> wrote:
> 
> > On Tue, 2006-10-31 at 23:08 +0100, Joerg Schilling wrote:
> > > Hi,
> > > 
> > > it looks as if SG_GET_RESERVED_SIZE & SG_SET_RESERVED_SIZE
> > > are not in interaction with the underlying SCSI transport.
> > > 
> > > Programs like readcd and cdda2wav that try to get very large SCSI
> > > transfer buffers get a confirmation for nearly any SCSI transfer size 
> > > but later when readcd/cdda2wav try to transfer data with an
> > > actual SCSI command, they fail with ENOMEM.
> > > 
> > > Correct fix: let sg.c make a callback to the underlying SCSI transport
> > > 		and let it get a confirmation tfor the buffer size.
> > > 
> > > Quick and dirty fix: reduce the maximum allowed DMA size to the smallest
> > > 		max DMA size of all SCSI transports.
> >
> > real good fix:
> >
> > use SG_IO on the device directly that checks this already
> 
> From looking into the source, this claim seems to be wrong.

The block layer SG_IO entry point does what Arjan describes - it checks
the queue settings, which must match the hardware limits. It needs to,
since it won't accept a command larger than what the path to that device
will allow in one go. The SCSI sg variant may be more restricted, since
it should handle partial completions of such commands.

-- 
Jens Axboe

Re: SCSI over USB showstopper bug?

[Joerg Schilling]

Jens Axboe <jens.axboe@oracle.com> wrote:

> > > > it looks as if SG_GET_RESERVED_SIZE & SG_SET_RESERVED_SIZE
> > > > are not in interaction with the underlying SCSI transport.
> > > > 
> > > > Programs like readcd and cdda2wav that try to get very large SCSI
> > > > transfer buffers get a confirmation for nearly any SCSI transfer size 
> > > > but later when readcd/cdda2wav try to transfer data with an
> > > > actual SCSI command, they fail with ENOMEM.
> > > > 
> > > > Correct fix: let sg.c make a callback to the underlying SCSI transport
> > > > 		and let it get a confirmation tfor the buffer size.
> > > > 
> > > > Quick and dirty fix: reduce the maximum allowed DMA size to the smallest
> > > > 		max DMA size of all SCSI transports.
> > >
> > > real good fix:
> > >
> > > use SG_IO on the device directly that checks this already
> > 
> > From looking into the source, this claim seems to be wrong.
>
> The block layer SG_IO entry point does what Arjan describes - it checks
> the queue settings, which must match the hardware limits. It needs to,
> since it won't accept a command larger than what the path to that device
> will allow in one go. The SCSI sg variant may be more restricted, since
> it should handle partial completions of such commands.

Then someone should change the source to match this statements.

>From a report I have from the k3b Author, readcd and cdda2wav only work
if you add a "ts=128k" option. 

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       schilling@fokus.fraunhofer.de     (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily

Re: SCSI over USB showstopper bug?

[Jens Axboe]

On Wed, Nov 01 2006, Joerg Schilling wrote:
> Jens Axboe <jens.axboe@oracle.com> wrote:
> 
> > > > > it looks as if SG_GET_RESERVED_SIZE & SG_SET_RESERVED_SIZE
> > > > > are not in interaction with the underlying SCSI transport.
> > > > > 
> > > > > Programs like readcd and cdda2wav that try to get very large SCSI
> > > > > transfer buffers get a confirmation for nearly any SCSI transfer size 
> > > > > but later when readcd/cdda2wav try to transfer data with an
> > > > > actual SCSI command, they fail with ENOMEM.
> > > > > 
> > > > > Correct fix: let sg.c make a callback to the underlying SCSI transport
> > > > > 		and let it get a confirmation tfor the buffer size.
> > > > > 
> > > > > Quick and dirty fix: reduce the maximum allowed DMA size to the smallest
> > > > > 		max DMA size of all SCSI transports.
> > > >
> > > > real good fix:
> > > >
> > > > use SG_IO on the device directly that checks this already
> > > 
> > > From looking into the source, this claim seems to be wrong.
> >
> > The block layer SG_IO entry point does what Arjan describes - it checks
> > the queue settings, which must match the hardware limits. It needs to,
> > since it won't accept a command larger than what the path to that device
> > will allow in one go. The SCSI sg variant may be more restricted, since
> > it should handle partial completions of such commands.
> 
> Then someone should change the source to match this statements.
> 
> From a report I have from the k3b Author, readcd and cdda2wav only work
> if you add a "ts=128k" option. 

Then please file (or have him/her file) a proper bug report. It may be a
usb specific bug, or it may just be something else.

-- 
Jens Axboe

Re: SCSI over USB showstopper bug?

[Joerg Schilling]

Jens Axboe <jens.axboe@oracle.com> wrote:

> > Then someone should change the source to match this statements.
> > 
> > From a report I have from the k3b Author, readcd and cdda2wav only work
> > if you add a "ts=128k" option. 
>
> Then please file (or have him/her file) a proper bug report. It may be a
> usb specific bug, or it may just be something else.

To me, it looks like a problem that happens with usb because there is 
no proper interaction with SG_?ET_RESETVED_SIZE and the usb layer.

I am still in hope that someone will fix this soon.

Jörg

-- 
 EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
       js@cs.tu-berlin.de                (uni)  
       schilling@fokus.fraunhofer.de     (work) Blog: http://schily.blogspot.com/
 URL:  http://cdrecord.berlios.de/old/private/ ftp://ftp.berlios.de/pub/schily

Re: SCSI over USB showstopper bug?

[Jens Axboe]

On Mon, Nov 06 2006, Joerg Schilling wrote:
> Jens Axboe <jens.axboe@oracle.com> wrote:
> 
> > > Then someone should change the source to match this statements.
> > > 
> > > From a report I have from the k3b Author, readcd and cdda2wav only work
> > > if you add a "ts=128k" option. 
> >
> > Then please file (or have him/her file) a proper bug report. It may be a
> > usb specific bug, or it may just be something else.
> 
> To me, it looks like a problem that happens with usb because there is 
> no proper interaction with SG_?ET_RESETVED_SIZE and the usb layer.

The limits are communicated from the usb layer to the block layer via
the SCSI layer, by setting proper limits in the scsi host adapter
template. SCSI then informs the block layer, by setting the appropriate
limits on the queue. Perhaps there's a usb-storage bug there, who knows,
so far there's been no real info posted.

> I am still in hope that someone will fix this soon.

Someone may very well fix it, but the odds of that happening when a real
bug report exists is a lot bigger. Who reported this issue to you? Get
him/her to file a proper bug report, as I wrote in my last mail as well.

-- 
Jens Axboe