Re: T10 PI offloading seems to be broken in iser/isert in 4.5/4.6-rc

["Nicholas A. Bellinger" <nab@linux-iscsi.org>]

On Fri, 2016-04-08 at 15:15 -0700, Christoph Hellwig wrote:
> Hi all,
> 
> I've recently started looking into adding support for signature MRs
> in the RDMA R/W API, but it seems like it's already broken upstream.
> 
> I suspect it's the target that's at fault, but I'm not sure.  I've
> tested 4.5 and todays 4.6-rc tree on both the target and the client
> side.
> 
> The setup is pretty simple: on the target I've loaded scsi_debug
> with dif=1 and dix=1, and set I've set fabric_prot_type=1, t10_pi=1
> and ImmediateData to No on the tpg, and then did a simple mkfs.ext4.
> 
> Just doing the login I get the following error on the target:
> 
> [  369.890182] Bad block size given: 0
> [  369.894131] mlx5_0:mlx5_ib_post_send:3296:(pid 331): 
> [  369.894228] isert: isert_reg_sig_mr: fast registration failed, ret:-22
> [  369.901580] isert: isert_handle_prot_cmd: conn ffff880904d54000 failed to fast reg mr
> [  369.906348] iSCSI/iqn.1993-08.org.debian:01:197dad5d1f17: Unsupported SCSI Opcode 0xa3, sending CHECK_CONDITION.
> [  369.910395] isert: isert_put_datain: Cmd: ffff880902b26b00 failed to prepare RDMA res
> 
> If I then do an actual I/O think go downhill quickly.

It looks like the isert_set_dif_domain() assignments of
ib_sig_domain->sig.dif.* from dev->dev_attrib.block_size + friends is
being zeroed before making it into mlx5_ib_post_send() ->
IB_WR_REG_SIG_MR -> set_sig_umr_wr() -> set_sig_data_segment().