Bidirectional data transfers from SCSI layer

Bidirectional data transfers from SCSI layer

[Madhavi Manchala]

Dear All,

I have seen that SCSI supports bi-directional commands
DMA_BIDIRECTIONAL. However, I did not get such type of commands in my
mass storage driver. My device supports bi-directional data transfers.
My question is, how can I get such type of requests to my mass storage
driver? If so, how can I handle the IN and OUT buffers? Are there
separate buffers for both? Is there any support for the bi-directional
in the current SCSI implementation? I am currently using the 2.6.28
kernel code.

We have only one pointer for data buffer instead of two, one for data
OUT and one for data IN in command block. There is only one length
variable. How can we use the OUT and IN buffers? How can I pass
different IN length and OUT length to lower stack from SCSI layer?

Is there any thing need to be changed in the SCSI layer code to handle
the bi-directional data transfers?

Please point to me some links which shows the bi-directional data transfers.

Thanks in advance.

With Regards,
Madhavi M.

Re: Bidirectional data transfers from SCSI layer

[Boaz Harrosh]

On 08/18/2009 09:22 AM, Madhavi Manchala wrote:
> Dear All,
> 
> I have seen that SCSI supports bi-directional commands
> DMA_BIDIRECTIONAL. However, I did not get such type of commands in my
> mass storage driver. My device supports bi-directional data transfers.
> My question is, how can I get such type of requests to my mass storage
> driver? If so, how can I handle the IN and OUT buffers? Are there
> separate buffers for both? Is there any support for the bi-directional
> in the current SCSI implementation? I am currently using the 2.6.28
> kernel code.
> 
> We have only one pointer for data buffer instead of two, one for data
> OUT and one for data IN in command block. There is only one length
> variable. How can we use the OUT and IN buffers? How can I pass
> different IN length and OUT length to lower stack from SCSI layer?
> 
> Is there any thing need to be changed in the SCSI layer code to handle
> the bi-directional data transfers?
> 
> Please point to me some links which shows the bi-directional data transfers.
> 
> Thanks in advance.
> 
> With Regards,
> Madhavi M.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Bidi is fully supported since 2.6.25

If you want example of how to issue bidi commands you can see these two files:
block/bsg.c
drivers/scsi/osd/osd_initiator.c

Basically you allocate two struct request one for OUT one for IN. You map
some memory on them and do "out_req->next_rq = in_req;" And you execute
the out_req.
These two request will go together as one bidi request.

On the scsi-driver side you do:

	if (scsi_bidi_cmnd(cmnd)) {
		in_sgl = scsi_in(cmnd)->table.sgl;
		in_nents = scsi_in(cmnd)->table.nents;
		in_bytes = scsi_in(cmnd)->length;

		/* Same thing for out */
		out_sgl = scsi_out(cmnd)->table.sgl;
		...
	}

scsi_in() and scsi_out() will return a pointer or NULL if the
cmnd does not have a buffer in this direction. So for example
scsi_in() is always used on receive regardless if it is a bidi
or a uni_read.

NOTE: in bidi_commands
	scsi_bidi_cmnd(cmnd) == true;
	but
	cmnd->sc_data_direction == DMA_TO_DEVICE;
	DMA_BIDIRECTIONAL is *never* returned from scsi
	and it means something else inside the DMA engines.

For example driver that supports bidi see:
drivers/scsi/iscsi_tcp.c
drivers/scsi/scsi_debug.c
(Search for "scsi_in" and "scsi_out")

What type of scsi-device are you using? what bidi commands?

Have a good day
Boaz

Re: Bidirectional data transfers from SCSI layer

[Madhavi Manchala]

On Tue, Aug 18, 2009 at 1:34 PM, Boaz Harrosh<bharrosh@panasas.com> wrote:

Dear Boaz,

Thanks for the information.

> drivers/scsi/osd/osd_initiator.c

I did not find this file in my 2.6.28 sources.

> On the scsi-driver side you do:
>
>        if (scsi_bidi_cmnd(cmnd)) {
>                in_sgl = scsi_in(cmnd)->table.sgl;
>                in_nents = scsi_in(cmnd)->table.nents;
>                in_bytes = scsi_in(cmnd)->length;
>
>                /* Same thing for out */
>                out_sgl = scsi_out(cmnd)->table.sgl;
>                ...
>        }

Do I need to modify the SCSI driver files to handle  the
bi-directional comamnds or Is this already supported?

> What type of scsi-device are you using?

I am working for USB Attached SCSI (UAS) device which supports
bi-directional commands. I'm modifying USB Mass storage driver
according to UAS requirements. Whenever there is command to process,
SCSI layer sends scsi_cmnd structure pointer to UAS class driver (my
driver - modified mass storage driver). But I did not find two data
pointers and two data length variables (one IN and one OUT) in this
structure to handle bi-directional data.

I found only one struct scsi_data_buffer pointer in scsi_cmnd
structure. So, how do SCSI layer sends IN and OUT buffer pointers and
their lengths to my class driver?

> what bidi commands?

I did not understand this. Could you please elaborate it? Are there
any bidi commands like READ_10 or WRITE_10 which are specific to any
device?

Thanks and Regards,
Madhavi M.
--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Bidirectional data transfers from SCSI layer

[Boaz Harrosh]

On 08/18/2009 12:03 PM, Madhavi Manchala wrote:
> On Tue, Aug 18, 2009 at 1:34 PM, Boaz Harrosh<bharrosh@panasas.com> wrote:
> 
> Dear Boaz,
> 
> Thanks for the information.
> 
>> drivers/scsi/osd/osd_initiator.c
> 
> I did not find this file in my 2.6.28 sources.
> 

If you are developing new code for Linux it is best
to base the work on latest git tree. Either Linus's or the
maintainer tree that will receive your work. See here:
http://git.kernel.org/

The file above you can see here:
http://git.open-osd.org/gitweb.cgi?p=open-osd.git;a=tree;f=drivers/scsi/osd;h=803721b1c52bcd52822976b515de817a8b765969;hb=ec34ca8bdef55a47c3b1f82b1021609570ccb283
Only 2.6.30 and up have osd support

>> On the scsi-driver side you do:
>>
>>        if (scsi_bidi_cmnd(cmnd)) {
>>                in_sgl = scsi_in(cmnd)->table.sgl;
>>                in_nents = scsi_in(cmnd)->table.nents;
>>                in_bytes = scsi_in(cmnd)->length;
>>
>>                /* Same thing for out */
>>                out_sgl = scsi_out(cmnd)->table.sgl;
>>                ...
>>        }
> 
> Do I need to modify the SCSI driver files to handle  the
> bi-directional comamnds or Is this already supported?
> 

Yes only very few drivers support bidi commands.
i.e. iscsi is the only one that supports them in general.

>> What type of scsi-device are you using?
> 
> I am working for USB Attached SCSI (UAS) device which supports
> bi-directional commands. I'm modifying USB Mass storage driver
> according to UAS requirements. 

Yes, Current "USB Mass storage driver" does not support bidi as well
as many other standard scsi things like NCQ.

> Whenever there is command to process,
> SCSI layer sends scsi_cmnd structure pointer to UAS class driver (my
> driver - modified mass storage driver). But I did not find two data
> pointers and two data length variables (one IN and one OUT) in this
> structure to handle bi-directional data.
> 

As I said use scsi_in() and scsi_out() at driver. But you need an initiator
to issue these type of commands. The Kernel does not use these type of commands
for normal I/O. Only osd initiator with osd-scsi targets does.

You can also use bsg driver to issue bidi commands from user mode. You will
need that your driver set the bidi queue-bit on initialization. (See iscsi_tcp)

(Tell me if you need example code for bsg use in user-mode).

> I found only one struct scsi_data_buffer pointer in scsi_cmnd
> structure. So, how do SCSI layer sends IN and OUT buffer pointers and
> their lengths to my class driver?
> 

Again: 
	if (scsi_bidi_cmnd(cmnd)) {
		in_sdb = scsi_in(cmnd);
		out_sdb = scsi_out(cmnd);
	}
It works, look at the code.

>> what bidi commands?
> 
> I did not understand this. Could you please elaborate it? Are there
> any bidi commands like READ_10 or WRITE_10 which are specific to any
> device?
> 

There is the XOR_10 I think and others. But I'm not sure. You'll have to look
it up yourself at the different T10 scsi standards. Also scsi_debug.c as some emulation
for a command or two. I only use scsi-osd type commands which are all potentially bidi
capable.

> Thanks and Regards,
> Madhavi M.

Boaz

Re: Bidirectional data transfers from SCSI layer

[Madhavi Manchala]

On Tue, Aug 18, 2009 at 3:30 PM, Boaz Harrosh<bharrosh@panasas.com> wrote:

Dear Boaz,

Thanks a lot.

>> Do I need to modify the SCSI driver files to handle  the
>> bi-directional comamnds or Is this already supported?
>>
>
> Yes only very few drivers support bidi commands.
> i.e. iscsi is the only one that supports them in general.

Currently, I am registering my driver to the SCSI layer. Now I
understand, I need to register my driver to iscsi to get
bi-directional commands. If this is the case, how can I register my
driver to iscsi driver to get bi-directional commands, and will I get
the same functionality from iscsi as I am currently getting from scsi.

> (Tell me if you need example code for bsg use in user-mode).

Could you please send me the link to the user mode bsg initiator code
which issues the bi-directional commands?

>> I found only one struct scsi_data_buffer pointer in scsi_cmnd
>> structure. So, how do SCSI layer sends IN and OUT buffer pointers and
>> their lengths to my class driver?
>>
>
> Again:
>        if (scsi_bidi_cmnd(cmnd)) {
>                in_sdb = scsi_in(cmnd);
>                out_sdb = scsi_out(cmnd);
>        }
> It works, look at the code.

Before using the above snippet in my class driver, do I need to fill
in_req and out_req in SCSI layer or Is it already filled? If I use the
iscsi driver, then there is no need to do any modifications to the
in_req and out_req.

Thanks and Regards,
Madhavi M.

Re: Bidirectional data transfers from SCSI layer

[Boaz Harrosh]

On 08/18/2009 01:35 PM, Madhavi Manchala wrote:
> On Tue, Aug 18, 2009 at 3:30 PM, Boaz Harrosh<bharrosh@panasas.com> wrote:
> 
> Dear Boaz,
> 
> Thanks a lot.
> 
>>> Do I need to modify the SCSI driver files to handle  the
>>> bi-directional comamnds or Is this already supported?
>>>
>>
>> Yes only very few drivers support bidi commands.
>> i.e. iscsi is the only one that supports them in general.
> 
> Currently, I am registering my driver to the SCSI layer. Now I
> understand, I need to register my driver to iscsi to get
> bi-directional commands. If this is the case, how can I register my
> driver to iscsi driver to get bi-directional commands, and will I get
> the same functionality from iscsi as I am currently getting from scsi.
> 

No!. You miss understood. iscsi is just a scsi LLD (Low Level Driver)
just as usb/storage/scsiglue.c is an LLD. You only register with scsi.

scsi-ml has full support for bidi. Only the LLDs do not. except two
LLDs: iscsi and scsi_debug.

>> (Tell me if you need example code for bsg use in user-mode).
> 
> Could you please send me the link to the user mode bsg initiator code
> which issues the bi-directional commands?
> 

Start here:
http://git.open-osd.org/gitweb.cgi?p=open-osd.git;a=blob;f=lib/bsgdev.c;h=f55a87879784f05950e7a2d88798605183cc660a;hb=ec34ca8bdef55a47c3b1f82b1021609570ccb283
and generally inspect this folder:
http://git.open-osd.org/gitweb.cgi?p=open-osd.git;a=tree;f=lib;h=5739f66f529c010755ab4bb05db4496d0ea4b229;hb=ec34ca8bdef55a47c3b1f82b1021609570ccb283

Above files are used to issue scsi-osd commands from user-mode, which can all be bidi.

>>> I found only one struct scsi_data_buffer pointer in scsi_cmnd
>>> structure. So, how do SCSI layer sends IN and OUT buffer pointers and
>>> their lengths to my class driver?
>>>
>>
>> Again:
>>        if (scsi_bidi_cmnd(cmnd)) {
>>                in_sdb = scsi_in(cmnd);
>>                out_sdb = scsi_out(cmnd);
>>        }
>> It works, look at the code.
> 
> Before using the above snippet in my class driver, do I need to fill
> in_req and out_req in SCSI layer or Is it already filled? If I use the
> iscsi driver, then there is no need to do any modifications to the
> in_req and out_req.
> 

Again: upper layers have full support. You only need the top most initiator
and a supporting LLD.

[ Initiator => block-layer => scsi-ml => LLD
  - initiators: bsg and osd support bidi. All other do not.
  - block-layer && scsi-ml fully support bidi.
  - LLDs: only iscsi currently supports bidi.
]

> Thanks and Regards,
> Madhavi M.

Boaz

Re: Bidirectional data transfers from SCSI layer

[Madhavi Manchala]

On Tue, Aug 18, 2009 at 5:16 PM, Boaz Harrosh<bharrosh@panasas.com> wrote:

Dear Boaz,

Thanks again for the information.

> No!. You miss understood. iscsi is just a scsi LLD (Low Level Driver)
> just as usb/storage/scsiglue.c is an LLD. You only register with scsi.
>
> scsi-ml has full support for bidi. Only the LLDs do not. except two
> LLDs: iscsi and scsi_debug.

I guess, iscsi you are referring to is nothing but
drivers/scsi/iscsi_tcp.c. If so, do I need to modify
drivers/usb/storage/scsiglue.c file in a way to handle bidirectional
commands. The reference code for the modification is iscsi_tcp.c file.

Please let me know whether my assumption is correct or not.

Thanks and Regards,
Madhavi M.

Re: Bidirectional data transfers from SCSI layer

[Boaz Harrosh]

On 08/18/2009 04:42 PM, Madhavi Manchala wrote:
> On Tue, Aug 18, 2009 at 5:16 PM, Boaz Harrosh<bharrosh@panasas.com> wrote:
> 
> Dear Boaz,
> 
> Thanks again for the information.
> 
>> No!. You miss understood. iscsi is just a scsi LLD (Low Level Driver)
>> just as usb/storage/scsiglue.c is an LLD. You only register with scsi.
>>
>> scsi-ml has full support for bidi. Only the LLDs do not. except two
>> LLDs: iscsi and scsi_debug.
> 
> I guess, iscsi you are referring to is nothing but
> drivers/scsi/iscsi_tcp.c. If so, do I need to modify
> drivers/usb/storage/scsiglue.c file in a way to handle bidirectional
> commands. The reference code for the modification is iscsi_tcp.c file.
> 

Yes
and also drivers/scsi/scsi_debug.c

> Please let me know whether my assumption is correct or not.
> 
> Thanks and Regards,
> Madhavi M.

Good luck
Boaz