[LSF/MM ATTEND] iSCSI/iSER MQ + EXTENDED_COPY host support

[LSF/MM ATTEND] iSCSI/iSER MQ + EXTENDED_COPY host support

[Nicholas A. Bellinger]

Hi LSF-PC folks,

I'd like to attend LSF/MM 2015 this year in Boston.

The topics I'm interested in are iSCSI/iSER multi-queue (MC/S) support
within the open-iscsi initiator, and pushing forward the pieces
necessary for EXTENDED_COPY host side support so we can finally begin to
utilize copy-offload in KVM and Xen environments.

As maintainer of the SCSI target subsystem, I've contributed MC/S
support in iscsi-target, as well as EXTENDED_COPY(LID1) support within
the upstream LIO target.  Now that we've got upstream target support for
these features, I'd like to help facilitate support for these into the
initiator / host side stack.

Thanks!

--nab

Re: [LSF/MM ATTEND] iSCSI/iSER MQ + EXTENDED_COPY host support

[Douglas Gilbert]

On 15-01-07 04:21 PM, Nicholas A. Bellinger wrote:
> Hi LSF-PC folks,
>
> I'd like to attend LSF/MM 2015 this year in Boston.
>
> The topics I'm interested in are iSCSI/iSER multi-queue (MC/S) support
> within the open-iscsi initiator, and pushing forward the pieces
> necessary for EXTENDED_COPY host side support so we can finally begin to
> utilize copy-offload in KVM and Xen environments.
>
> As maintainer of the SCSI target subsystem, I've contributed MC/S
> support in iscsi-target, as well as EXTENDED_COPY(LID1) support within
> the upstream LIO target.  Now that we've got upstream target support for
> these features, I'd like to help facilitate support for these into the
> initiator / host side stack.

T10 have now dropped the LID1 and LID4 stuff (its the length
of the LIST IDENTIFIER field: 1 byte or 4 bytes) by obsoleting
all LID1 variants in SPC-5 revision 01. So the LID1 variants
are now gone and the LID4 appendage is dropped from the remaining
commands.

Obviously LID1 based XCOPY support is important for backward
compatibility (back to SPC-2) but for the future maybe you might
consider adding the equivalent LID4 support for a basic XCOPY.
For example it means using the 3PC VPD page instead of the
RECEIVE COPY RESULTS(operating parameters) command. I believe
FreeNAS now supports both LID1 and LID4 variants of a basic
XCOPY (plus a restricted version of ODX without snapshots).


Doug Gilbert

Re: [LSF/MM ATTEND] iSCSI/iSER MQ + EXTENDED_COPY host support

[Andy Grover]

On 01/07/2015 02:11 PM, Douglas Gilbert wrote:
> T10 have now dropped the LID1 and LID4 stuff (its the length
> of the LIST IDENTIFIER field: 1 byte or 4 bytes) by obsoleting
> all LID1 variants in SPC-5 revision 01. So the LID1 variants
> are now gone and the LID4 appendage is dropped from the remaining
> commands.
>
> Obviously LID1 based XCOPY support is important for backward
> compatibility (back to SPC-2) but for the future maybe you might
> consider adding the equivalent LID4 support for a basic XCOPY.
> For example it means using the 3PC VPD page instead of the
> RECEIVE COPY RESULTS(operating parameters) command. I believe
> FreeNAS now supports both LID1 and LID4 variants of a basic
> XCOPY (plus a restricted version of ODX without snapshots).

Hi Doug,

I was trying to learn more about ODX and saw this page:

http://msdn.microsoft.com/en-us/library/windows/desktop/hh848056%28v=vs.85%29.aspx

but it just muddied the waters for me. Is ODX proprietary to MS, or is 
it XCopy lite, or what? And what's the difference between xcopy lite and 
LID4 XCOPY?

Thanks -- Andy

Re: [LSF/MM ATTEND] iSCSI/iSER MQ + EXTENDED_COPY host support

[Nicholas A. Bellinger]

On Thu, 2015-01-08 at 10:58 -0800, Andy Grover wrote:
> On 01/07/2015 02:11 PM, Douglas Gilbert wrote:
> > T10 have now dropped the LID1 and LID4 stuff (its the length
> > of the LIST IDENTIFIER field: 1 byte or 4 bytes) by obsoleting
> > all LID1 variants in SPC-5 revision 01. So the LID1 variants
> > are now gone and the LID4 appendage is dropped from the remaining
> > commands.
> >
> > Obviously LID1 based XCOPY support is important for backward
> > compatibility (back to SPC-2) but for the future maybe you might
> > consider adding the equivalent LID4 support for a basic XCOPY.
> > For example it means using the 3PC VPD page instead of the
> > RECEIVE COPY RESULTS(operating parameters) command. I believe
> > FreeNAS now supports both LID1 and LID4 variants of a basic
> > XCOPY (plus a restricted version of ODX without snapshots).
> 
> Hi Doug,
> 
> I was trying to learn more about ODX and saw this page:
> 
> http://msdn.microsoft.com/en-us/library/windows/desktop/hh848056%28v=vs.85%29.aspx
> 
> but it just muddied the waters for me. Is ODX proprietary to MS, or is 
> it XCopy lite, or what? And what's the difference between xcopy lite and 
> LID4 XCOPY?
> 

xcopy-lite in MSFT speak and LID4 XCOPY in T10 speak are the same thing.
EG: Tokenized XCOPY where all reads from the copy source are done from a
point-in-time snapshot.

--nab

Re: [LSF/MM ATTEND] iSCSI/iSER MQ + EXTENDED_COPY host support

[Douglas Gilbert]

On 15-01-08 01:58 PM, Andy Grover wrote:
> On 01/07/2015 02:11 PM, Douglas Gilbert wrote:
>> T10 have now dropped the LID1 and LID4 stuff (its the length
>> of the LIST IDENTIFIER field: 1 byte or 4 bytes) by obsoleting
>> all LID1 variants in SPC-5 revision 01. So the LID1 variants
>> are now gone and the LID4 appendage is dropped from the remaining
>> commands.
>>
>> Obviously LID1 based XCOPY support is important for backward
>> compatibility (back to SPC-2) but for the future maybe you might
>> consider adding the equivalent LID4 support for a basic XCOPY.
>> For example it means using the 3PC VPD page instead of the
>> RECEIVE COPY RESULTS(operating parameters) command. I believe
>> FreeNAS now supports both LID1 and LID4 variants of a basic
>> XCOPY (plus a restricted version of ODX without snapshots).
>
> Hi Doug,
>
> I was trying to learn more about ODX and saw this page:
>
> http://msdn.microsoft.com/en-us/library/windows/desktop/hh848056%28v=vs.85%29.aspx
>
> but it just muddied the waters for me. Is ODX proprietary to MS, or is it XCopy
> lite, or what? And what's the difference between xcopy lite and LID4 XCOPY?

Andy,
XCOPY is huge and keeps getting bigger (apart from the recent
trimming of the "LID1" generation mentioned above). Most SCSI
commands are sent to the LU involved, but for EXTENDED COPY
command itself, should it be sent to the source or destination
of the copy, or another LU which acts as a third party copy
manager? Also when you examine the parameter block associated
with the EXTENDED COPY command, one starts thinking there
must be a cleaner and simpler way.

In the real world VMware have been using the basic disk->disk
XCOPY for around 10 years. Their usage seems to be dominant in
the market place.

While the LID4 rewrite/additions were occurring at T10 in 2011,
a proposal was made to simplify things with a two step,
disk-to-disk copy. The glue between those two steps is a token.
The first step (POPULATE TOKEN) is sent to the copy source **,
the second step is sent to the copy destination (WRITE USING
TOKEN). The document is called:
        XCOPYv2: Extended Copy Plus & Lite
[11-077R1 at www.t10.org] and is worth looking at. The
"Bucket Of Bytes" (BOB) in that document has become the
"Representation Of Data" (ROD) found in T10 documents today.

Commercially MS and NetApp (seemingly) were behind this
new approach. And MS christened it ODX which is found in their
recent OS products.


I have written this page about the ddpt utility's
implementation of these two disk-to-disk copy approaches:
   http://sg.danny.cz/sg/ddpt_xcopy_odx.html

Doug Gilbert


** and as NAB pointed out a "point-in-time" copy (snapshot)
    is done in this step, with the copy output going to the
    ROD. A ROD token can be thought of as a 512 byte long
    "pointer" to a ROD. That snapshot is a complex and
    resource consuming action for an implementation and may
    explain why ODX isn't high on NAB's todo list.
    FreeNAS's implementation cuts a corner here and just
    holds the gather-list in the ROD. That seems to violate
    SBC-3's description but is allowed by SPC-4. Any language
    lawyers like to comment?

Re: [LSF/MM ATTEND] iSCSI/iSER MQ + EXTENDED_COPY host support

[Andy Grover]

On 01/08/2015 03:11 PM, Douglas Gilbert wrote:
> Andy,
> XCOPY is huge and keeps getting bigger (apart from the recent
> trimming of the "LID1" generation mentioned above). Most SCSI
> commands are sent to the LU involved, but for EXTENDED COPY
> command itself, should it be sent to the source or destination
> of the copy, or another LU which acts as a third party copy
> manager? Also when you examine the parameter block associated
> with the EXTENDED COPY command, one starts thinking there
> must be a cleaner and simpler way.
>
> In the real world VMware have been using the basic disk->disk
> XCOPY for around 10 years. Their usage seems to be dominant in
> the market place.
>
> While the LID4 rewrite/additions were occurring at T10 in 2011,
> a proposal was made to simplify things with a two step,
> disk-to-disk copy. The glue between those two steps is a token.
> The first step (POPULATE TOKEN) is sent to the copy source **,
> the second step is sent to the copy destination (WRITE USING
> TOKEN). The document is called:
>         XCOPYv2: Extended Copy Plus & Lite
> [11-077R1 at www.t10.org] and is worth looking at. The
> "Bucket Of Bytes" (BOB) in that document has become the
> "Representation Of Data" (ROD) found in T10 documents today.
>
> Commercially MS and NetApp (seemingly) were behind this
> new approach. And MS christened it ODX which is found in their
> recent OS products.
>
>
> I have written this page about the ddpt utility's
> implementation of these two disk-to-disk copy approaches:
>    http://sg.danny.cz/sg/ddpt_xcopy_odx.html

Thanks for your and nab's reply and this web page, they answer all my 
questions. I was failing to find anything on MS or vendor sites with 
this kind of info.

Thanks again -- Andy