SCSI commands order when using scsi_do_req

SCSI commands order when using scsi_do_req

[Liran Schour]

I am developing a new SCSI upper layer driver. (Object based SCSI device)
I noticed that scsi_do_req execute commands in LIFO order (add new commands
to the head of the Q).
This behavior can cause a starvation of commands (or at least long delays).
What is the proper solution to this?

Thanks,
- Liran

Re: SCSI commands order when using scsi_do_req

[James Bottomley]

On Thu, 2005-06-02 at 20:31 +0300, Liran Schour wrote:
> I am developing a new SCSI upper layer driver. (Object based SCSI device)
> I noticed that scsi_do_req execute commands in LIFO order (add new commands
> to the head of the Q).
> This behavior can cause a starvation of commands (or at least long delays).
> What is the proper solution to this?

Well, to use the proper interfaces.

scsi_do_req is deprecated, so don't use it.  scsi_wait_req is its
replacement.  Note: they're only really intended to allow ULDs to
execute commands that are necessary for controlling the device (that's
why they go at the head of the queue).  Block commands are executed in
order using ULD init_command() (or REQ_BLOCK_PC).

James

Re: SCSI commands order when using scsi_do_req

[Douglas Gilbert]

James Bottomley wrote:
> On Thu, 2005-06-02 at 20:31 +0300, Liran Schour wrote:
> 
>>I am developing a new SCSI upper layer driver. (Object based SCSI device)
>>I noticed that scsi_do_req execute commands in LIFO order (add new commands
>>to the head of the Q).
>>This behavior can cause a starvation of commands (or at least long delays).
>>What is the proper solution to this?
> 
> 
> Well, to use the proper interfaces.
> 
> scsi_do_req is deprecated, so don't use it.  scsi_wait_req is its
> replacement.

James,
scsi_wait_req() doesn't look like a replacement of
scsi_do_req() to me. The "wait" call is synchronous
while scsi_do_req() is asynchronous with a done()
callback.

  Note: they're only really intended to allow ULDs to
> execute commands that are necessary for controlling the device (that's
> why they go at the head of the queue).

The "char" ULDs use scsi_do_req() exclusively for
SCSI command injection. The sg driver could be sharing
the device with sd driver, but then again it could
be talking to a SES target. The block layer SG_IO ioctl
is the only use in "mixed" mode (e.g. smartmontools
usage). However the block layer SG_IO is not being
used in "mixed" mode when a CD/DVD is being burnt.
IMO the mid level should not arbitrarily adopt LIFO
order.

   Block commands are executed in
> order using ULD init_command() (or REQ_BLOCK_PC).

OSDs break the UNIX block paradigm. Perhaps
it's time to start thinking about how to properly
support them.

Doug Gilbert

Re: SCSI commands order when using scsi_do_req

[Liran Schour]

linux-scsi-owner@vger.kernel.org wrote on 03/06/2005 03:03:04:

> James Bottomley wrote:
> > On Thu, 2005-06-02 at 20:31 +0300, Liran Schour wrote:
> >
> >>I am developing a new SCSI upper layer driver. (Object based SCSI
device)
> >>I noticed that scsi_do_req execute commands in LIFO order (add new
commands
> >>to the head of the Q).
> >>This behavior can cause a starvation of commands (or at least long
delays).
> >>What is the proper solution to this?
> >
> >
> > Well, to use the proper interfaces.
> >
> > scsi_do_req is deprecated, so don't use it.  scsi_wait_req is its
> > replacement.
>
> James,
> scsi_wait_req() doesn't look like a replacement of
> scsi_do_req() to me. The "wait" call is synchronous
> while scsi_do_req() is asynchronous with a done()
> callback.
>
>   Note: they're only really intended to allow ULDs to
> > execute commands that are necessary for controlling the device (that's
> > why they go at the head of the queue).
>
> The "char" ULDs use scsi_do_req() exclusively for
> SCSI command injection. The sg driver could be sharing
> the device with sd driver, but then again it could
> be talking to a SES target. The block layer SG_IO ioctl
> is the only use in "mixed" mode (e.g. smartmontools
> usage). However the block layer SG_IO is not being
> used in "mixed" mode when a CD/DVD is being burnt.
> IMO the mid level should not arbitrarily adopt LIFO
> order.
>
>    Block commands are executed in
> > order using ULD init_command() (or REQ_BLOCK_PC).
>
> OSDs break the UNIX block paradigm. Perhaps
> it's time to start thinking about how to properly
> support them.
>
> Doug Gilbert

In order to run our OSD SCSI initiator on top of the Linux SCSI layer
we have needed to do the following changes to the SCSI code:
1. Redefine MAX_COMMAND_SIZE to 256 byte.
2. Define a new scatter gather structures that includes two virtual buffers
   on for data out and one for data in. (implementing bidirectional
commands)
   The SCSI middle layer doesn't now this new structure (only ULD and LLD).
3. The problem above - define a new scsi_do_req that inserts our commands
to
   the tail of the Q.

We intend to release our code as an open source project soon.

- Liran

Re: SCSI commands order when using scsi_do_req

[James Bottomley]

On Mon, 2005-06-06 at 11:32 +0300, Liran Schour wrote:
> In order to run our OSD SCSI initiator on top of the Linux SCSI layer
> we have needed to do the following changes to the SCSI code:
> 1. Redefine MAX_COMMAND_SIZE to 256 byte.

There's more than just you interested in this one, so separating this
feature out from all the rest would be helpful.

> 2. Define a new scatter gather structures that includes two virtual buffers
>    on for data out and one for data in. (implementing bidirectional
> commands)
>    The SCSI middle layer doesn't now this new structure (only ULD and LLD).

Yes, this is the nasty one ... and why OSD hasn't been incredibly
popular; bidirectional commands necessitate fairly sweeping changes to
all LLDs to support.  However, I don't see how we can get away without
the mid-layer knowing, since it's the entity that constructs the sglists
and the sglists have to be separated between the in and out pieces.

> 3. The problem above - define a new scsi_do_req that inserts our commands
> to
>    the tail of the Q.

Well, the solution, I think, is going to be the block execute one, which
will allow head or tail insertion.

James

Re: SCSI commands order when using scsi_do_req

[Liran Schour]

James Bottomley <James.Bottomley@SteelEye.com> wrote on 06/06/2005
17:32:53:

> On Mon, 2005-06-06 at 11:32 +0300, Liran Schour wrote:
> > In order to run our OSD SCSI initiator on top of the Linux SCSI layer
> > we have needed to do the following changes to the SCSI code:
> > 1. Redefine MAX_COMMAND_SIZE to 256 byte.
>
> There's more than just you interested in this one, so separating this
> feature out from all the rest would be helpful.
>
> > 2. Define a new scatter gather structures that includes two virtual
buffers
> >    on for data out and one for data in. (implementing bidirectional
> > commands)
> >    The SCSI middle layer doesn't now this new structure (only ULD and
LLD).
>
> Yes, this is the nasty one ... and why OSD hasn't been incredibly
> popular; bidirectional commands necessitate fairly sweeping changes to
> all LLDs to support.  However, I don't see how we can get away without
> the mid-layer knowing, since it's the entity that constructs the sglists
> and the sglists have to be separated between the in and out pieces.
>
In our driver we build both sg lists ( IN and OUT) in the OSD driver (ULD)
and send it
to the middlayer via scsi_do_req.

- Liran

Re: SCSI commands order when using scsi_do_req

[Vladislav Bolkhovitin]

James Bottomley wrote:
> On Thu, 2005-06-02 at 20:31 +0300, Liran Schour wrote:
> 
>>I am developing a new SCSI upper layer driver. (Object based SCSI device)
>>I noticed that scsi_do_req execute commands in LIFO order (add new commands
>>to the head of the Q).
>>This behavior can cause a starvation of commands (or at least long delays).
>>What is the proper solution to this?
> 
> 
> Well, to use the proper interfaces.
> 
> scsi_do_req is deprecated, so don't use it.  scsi_wait_req is its
> replacement.  Note: they're only really intended to allow ULDs to
> execute commands that are necessary for controlling the device (that's
> why they go at the head of the queue).  Block commands are executed in
> order using ULD init_command() (or REQ_BLOCK_PC).

Hm, st/osst/sg use scsi_do_req(), so should we consider those drivers 
broken in async mode? Apparently, a user in this mode could write not 
what he/she intented, the order could be wrong. Or do I miss something?

Vlad