iSCSI Expected Data Transfer Length for T10-PI

iSCSI Expected Data Transfer Length for T10-PI

[Sagi Grimberg]

Hey All,

Recently, iSER end-to-end T10-PI support maid it mainline.
I am wandering about the impact T10-PI should or shouldn't have on iSCSI 
header
field "Expected Data Transfer Length".

RFC-7143 states:
"the Expected Data Transfer Length field contains the number of bytes of 
data involved in this SCSI operation."
Since this field relates to *data bytes* I kept T10-PI implicit wrt this 
field. The iSCSI target calculates the
total transfer length (data + protection) from the cdb transfer length 
field and protect bits.

In FC, the fc_dl field was updated to relate to the total number of 
transfer bytes and includes
data and protection bytes. virtio_scsi was added with a header PI 
section (virtio_scsi_cmd_req_pi).

So my question is, should this field be updated to explicitly include 
T10-PI bytes like the FC equivalent fc_dl?
Or should T10-PI bytes be implicit?

I want to pin down this one to avoid a situation where the standard is 
open for interpretations.

Thanks,
Sagi.

Re: iSCSI Expected Data Transfer Length for T10-PI

[Julian Satran]

I have some trouble parsing you English/Question. I think the intent of SCSI PI was that wherever the PI exist it should be checked end-to-end and it may be checked in between.
A storage client (server) will have the PI appended in the memory when writing and reading and checking it. As the values are standardized storage may also check it both when reading and when writing but it should not change it. If by implicit you mean inclusive I assume iSCSI Expected data transfer length will take whatever is in the cdb.
Block transfer devices will likely add the PI length to the pure data length - i.e. inclusive. iSER may allow placing PI in diferent memory areas than the pure data but I think it has already provisions for that.

Regards,
Julo


> On May 25, 2014, at 18:30, Sagi Grimberg <sagig@dev.mellanox.co.il> wrote:
> 
> Hey All,
> 
> Recently, iSER end-to-end T10-PI support maid it mainline.
> I am wandering about the impact T10-PI should or shouldn't have on iSCSI header
> field "Expected Data Transfer Length".
> 
> RFC-7143 states:
> "the Expected Data Transfer Length field contains the number of bytes of data involved in this SCSI operation."
> Since this field relates to *data bytes* I kept T10-PI implicit wrt this field. The iSCSI target calculates the
> total transfer length (data + protection) from the cdb transfer length field and protect bits.
> 
> In FC, the fc_dl field was updated to relate to the total number of transfer bytes and includes
> data and protection bytes. virtio_scsi was added with a header PI section (virtio_scsi_cmd_req_pi).
> 
> So my question is, should this field be updated to explicitly include T10-PI bytes like the FC equivalent fc_dl?
> Or should T10-PI bytes be implicit?
> 
> I want to pin down this one to avoid a situation where the standard is open for interpretations.
> 
> Thanks,
> Sagi.

Re: iSCSI Expected Data Transfer Length for T10-PI

[Sagi Grimberg]

On 5/25/2014 10:39 PM, Julian Satran wrote:
Hi Julian,

> I have some trouble parsing you English/Question.

I'll try to clarify.

>   I think the intent of SCSI PI was that wherever the PI exist it should be checked end-to-end and it may be checked in between.
> A storage client (server) will have the PI appended in the memory when writing and reading and checking it. As the values are standardized storage may also check it both when reading and when writing but it should not change it.

All true.

>   If by implicit you mean inclusive I assume iSCSI Expected data transfer length will take whatever is in the cdb.
> Block transfer devices will likely add the PI length to the pure data length - i.e. inclusive.

My intention was that there is no explicit indication in the CDB that X 
additional PI bytes should be transferred,
it is implied that each block is appended with 8 bytes of PI.

For example:
Say we have an iSCSI target exposing a LUN which is formatted with PI. 
The iSCSI initiator supports PI transfer as well.
What do you expect iSCSI.ExpectedDataTransferLength field to be in the 
case of a single 512B block SCSI READ?  512?  520?
My understanding is that you think it should be the number of pure data 
bytes (i.e. 512).

As I mentioned, FC's fc_dl for example, was updated to be the *total* 
number of bytes (data + PI) in the SCSI operation (i.e. will be 520 in 
the above example).
virtio_scsi header was updated to additionally specify the number of IN 
PI bytes and the number of OUT PI bytes involved.

What I understood from the spec, is that 
iSCSI.ExpectedDataTransferLength corresponds to the number of
*data* bytes involved in this SCSI operation. That's why I kept 
iSCSI.ExpectedDataTransferLength to be
the number of pure data bytes.

So I guess I'm basically asking is, should iSCSI header include the 
number of PI bytes that should be transferred over the
wire as well?

Hope things are clearer,
Sagi.

RE: iSCSI Expected Data Transfer Length for T10-PI

[Black, David]

Hi Sagi,

> RFC-7143 states:
> "the Expected Data Transfer Length field contains the number of bytes of
> data involved in this SCSI operation."
> Since this field relates to *data bytes* I kept T10-PI implicit wrt this
> field. The iSCSI target calculates the
> total transfer length (data + protection) from the cdb transfer length
> field and protect bits.

That is wrong.  At the SCSI transport interface (both iSCSI and FCP are
SCSI transports), the concept of "data bytes" includes SCSI protection
information.

> In FC, the fc_dl field was updated to relate to the total number of
> transfer bytes and includes
> data and protection bytes. virtio_scsi was added with a header PI
> section (virtio_scsi_cmd_req_pi).

That is the correct approach.

At the SCSI transport interface (both FCP and iSCSI are SCSI transports),
no distinction is made between user data and protection information.
therefore, a transfer of 512 bytes of user data + 8 bytes of protection
information is a 520 byte transfer for both protocols.

The authority for this is SAM-5, or SAM-4 if one wants to refer to an
approved standard.  Neither are open to interpretation on this point.

All SCSI transports are supposed to behave the same way.  I hope this
can be corrected quickly.

Thanks,
--David


> -----Original Message-----
> From: Sagi Grimberg [mailto:sagig@dev.mellanox.co.il]
> Sent: Sunday, May 25, 2014 11:31 AM
> To: martin.petersen@oracle.com; Mike Christie; Nicholas A. Bellinger
> Cc: linux-scsi; target-devel; Oren Duer; james.smart@emulex.com; Or Gerlitz;
> cbm@chadalapaka.com; julians@infinidat.com; meth@il.ibm.com; Black, David
> Subject: iSCSI Expected Data Transfer Length for T10-PI
> 
> Hey All,
> 
> Recently, iSER end-to-end T10-PI support maid it mainline.
> I am wandering about the impact T10-PI should or shouldn't have on iSCSI
> header
> field "Expected Data Transfer Length".
> 
> RFC-7143 states:
> "the Expected Data Transfer Length field contains the number of bytes of
> data involved in this SCSI operation."
> Since this field relates to *data bytes* I kept T10-PI implicit wrt this
> field. The iSCSI target calculates the
> total transfer length (data + protection) from the cdb transfer length
> field and protect bits.
> 
> In FC, the fc_dl field was updated to relate to the total number of
> transfer bytes and includes
> data and protection bytes. virtio_scsi was added with a header PI
> section (virtio_scsi_cmd_req_pi).
> 
> So my question is, should this field be updated to explicitly include
> T10-PI bytes like the FC equivalent fc_dl?
> Or should T10-PI bytes be implicit?
> 
> I want to pin down this one to avoid a situation where the standard is
> open for interpretations.
> 
> Thanks,
> Sagi.

Re: iSCSI Expected Data Transfer Length for T10-PI

[Sagi Grimberg]

On 5/27/2014 2:58 PM, Black, David wrote:
> Hi Sagi,

Hey David,

>
>> RFC-7143 states:
>> "the Expected Data Transfer Length field contains the number of bytes of
>> data involved in this SCSI operation."
>> Since this field relates to *data bytes* I kept T10-PI implicit wrt this
>> field. The iSCSI target calculates the
>> total transfer length (data + protection) from the cdb transfer length
>> field and protect bits.
> That is wrong.  At the SCSI transport interface (both iSCSI and FCP are
> SCSI transports), the concept of "data bytes" includes SCSI protection
> information.
>
>> In FC, the fc_dl field was updated to relate to the total number of
>> transfer bytes and includes
>> data and protection bytes. virtio_scsi was added with a header PI
>> section (virtio_scsi_cmd_req_pi).
> That is the correct approach.
>
> At the SCSI transport interface (both FCP and iSCSI are SCSI transports),
> no distinction is made between user data and protection information.
> therefore, a transfer of 512 bytes of user data + 8 bytes of protection
> information is a 520 byte transfer for both protocols.
>
> The authority for this is SAM-5, or SAM-4 if one wants to refer to an
> approved standard.  Neither are open to interpretation on this point.

I see, thanks for clarifying this for me.

> All SCSI transports are supposed to behave the same way.  I hope this
> can be corrected quickly.

No problem, I'll fix it.

Sagi.