WRITE BUFFER commands through SG_IO getting rounded up to sector past 32k

[Aravind Parchuri <aravind.parchuri@gmail.com>]

While trying to fire WRITE BUFFER commands to an iscsi target, we found 
that if the command is larger than 32kB, it gets rounded up to the next 
sector before it reaches the LLD (in this case, open-iscsi). If this is 
expected behavior, please disregard the rest of the message. 
Unfortunately, I haven't found any documentation that mandates 
sector-multiple sizes for SG_IO ioctls, and besides,  the above behavior 
seems to have been introduced recently (post 2.6.12). The above scenario 
was produced in 2.6.19.2 (a Fedora update kernel, actually).

Here is the relevant log section for a command with dxfer_len 32896 
bytes, with SCSI_LOG_TIMEOUT=5 and DEBUG_SCSI enabled in open-iscsi:

Mar  5 12:39:12 ITX003048865470 kernel: sg_ioctl: sg4, cmd=0x2285
Mar  5 12:39:12 ITX003048865470 kernel: sg_common_write:  scsi 
opcode=0x3b, cmd_size=10
Mar  5 12:39:12 ITX003048865470 kernel: sg_start_req: dxfer_len=32896
Mar  5 12:39:12 ITX003048865470 kernel: sg_build_indirect: 
buff_size=32896, blk_size=33280
Mar  5 12:39:12 ITX003048865470 kernel: sg_write_xfer: num_xfer=32896, 
iovec_count=0, k_use_sg=2
Mar  5 12:39:12 ITX003048865470 kernel: iscsi: cmd [itt 0x23 total 33280 
imm_data 33280 unsol count 0, unsol offset 33280]
Mar  5 12:39:12 ITX003048865470 kernel: iscsi: ctask enq [write cid 0 sc 
e5ffcb00 cdb 0x3b itt 0x23 len 33280 cmdsn 36 win 257]
Mar  5 12:39:12 ITX003048865470 kernel: iscsi: ctask deq [cid 0 xmstate 
2 itt 0x23]
Mar  5 12:39:12 ITX003048865470 kernel:  connection0:0: iscsi: detected 
conn error (1011)
Mar  5 12:39:12 ITX003048865470 kernel: iscsi: blocking session
Mar  5 12:39:12 ITX003048865470 kernel: iscsi: failing in progress sc 
e5ffcb00 itt 0x23
Mar  5 12:39:12 ITX003048865470 kernel: sg_cmd_done: sg4, pack_id=0, 
res=0x20000Mar  5 12:39:12 ITX003048865470 kernel: sg_finish_rem_req: 
res_used=0
Mar  5 12:39:12 ITX003048865470 kernel: sg_remove_scat: k_use_sg=2

The first print from open-iscsi is from inside its queuecommand, and it 
shows that the scsi_cmnd->request_bufflen is 33280 ("total 33280"). When 
I tried the same experiment with 2.6.12, the number matched the dxfer_len.

In fact, even with the newer kernel, all commands smaller than 32k make 
it through with the right size, which leads me to think it's something 
to do with the scatter-gather list. I'm not particularly familiar with 
the scsi code, but inside scsi_execute_async, in scsi_map_req_sg, should 
the request's data_len be set to the dxfer_len instead of summing up the 
scatter-gather list lengths? I must mention again that I haven't really 
gone through the code thoroughly.

Anyway, the above behavior leads to a problem with open-iscsi - it fills 
in a pdu with the Data Segment Length equal to the sector-rounded 
number, while the cdb contains the original dxfer_len. The target uses 
the cdb's value to allocate buffers for the command, so it fails to 
receive the whole packet. Any help would be appreciated. Thanks,


Aravind.