From: Aravind Parchuri <aravind.parchuri@gmail.com>
To: michaelc@cs.wisc.edu
Cc: linux-scsi@vger.kernel.org
Subject: Re: WRITE BUFFER commands through SG_IO getting rounded up to sector past 32k
Date: Fri, 09 Mar 2007 20:56:42 -0800 [thread overview]
Message-ID: <45F23A8A.5020703@gmail.com> (raw)
In-Reply-To: <45F1EFA2.9070800@gmail.com>
FYI, commenting out the sector roundup in sg_build_indirect ( blk_size =
(blk_size + SG_SECTOR_MSK) & (~SG_SECTOR_MSK); ) made everything work
fine with my test and open-iscsi in general. I'm sure there's a
glaringly obvious reason for that roundup (one that I'm unaware of), so
I presume this isn't an acceptable solution,
Aravind.
michaelc@cs.wisc.edu wrote:
> My log messages were getting all mixed up, so I cleaned up my little
> test to send just one command at a time. It actually looks like the mid
> layer passes the command through to open-iscsi with the right size the
> first time, but then it sends a second command with request_bufflen = 0.
>
> I can verify that the command completed on the target just like the
> regular ones did, so there should be no reason for a retry of any sort.
>
> Here's the log for a 32896 byte command:
> Mar 9 11:27:43 ITX000c292c3c8d kernel: sg_open: dev=0, flags=0x802
> Mar 8 11:27:43 ITX000c292c3c8d kernel: sg_add_sfp: sfp=0xcbadc000
> Mar 8 11:27:43 ITX000c292c3c8d kernel: sg_build_reserve: req_size=32768
> Mar 8 11:27:43 ITX000c292c3c8d kernel: sg_build_indirect:
> buff_size=32768, blk_size=32768
> Mar 8 11:27:43 ITX000c292c3c8d kernel: sg_add_sfp: bufflen=32768,
> k_use_sg=1
> Mar 8 11:27:43 ITX000c292c3c8d kernel: sg_ioctl: sg0, cmd=0x2285
> Mar 8 11:27:43 ITX000c292c3c8d kernel: sg_common_write: scsi
> opcode=0x3b, cmd_size=10
> Mar 8 11:27:43 ITX000c292c3c8d kernel: sg_start_req: dxfer_len=32896
> Mar 8 11:27:43 ITX000c292c3c8d kernel: sg_build_indirect:
> buff_size=32896, blk_size=33280
> Mar 8 11:27:43 ITX000c292c3c8d kernel: sg_write_xfer: num_xfer=32896,
> iovec_count=0, k_use_sg=2
> Mar 8 11:27:43 ITX000c292c3c8d kernel: iscsi_queuecommand: opcode 3b
> request_bufflen 32896 transfersize 32896
> Mar 8 11:27:43 ITX000c292c3c8d kernel: iscsi_queuecommand: opcode 3b
> request_bufflen 0 transfersize 0
> Mar 8 11:27:44 ITX000c292c3c8d last message repeated 289 times
> Mar 8 11:27:44 ITX000c292c3c8d kernel: sg_release: sg0
> Mar 8 11:27:44 ITX000c292c3c8d kernel: sg_fasync: sg0, mode=0
> Mar 8 11:27:44 ITX000c292c3c8d kernel: sg_remove_sfp: worrisome, 1
> writes pending
> Mar 8 11:27:44 ITX000c292c3c8d kernel: iscsi_queuecommand: opcode 3b
> request_bufflen 0 transfersize 0
> Mar 8 11:27:57 ITX000c292c3c8d last message repeated 3197 times
> Mar 8 11:27:57 ITX000c292c3c8d kernel: SysRq : Changing Loglevel
> Mar 8 11:27:57 ITX000c292c3c8d kernel: Loglevel set to 0
> Mar 8 11:27:57 ITX000c292c3c8d kernel: iscsi_queuecommand: opcode 3b
> request_bufflen 0 transfersize 0
>
> For comparison, here's a 30 kB command:
> Mar 8 11:24:20 ITX000c292c3c8d kernel: sg_open: dev=0, flags=0x802
> Mar 8 11:24:20 ITX000c292c3c8d kernel: sg_add_sfp: sfp=0xcb826000
> Mar 8 11:24:20 ITX000c292c3c8d kernel: sg_build_reserve: req_size=32768
> Mar 8 11:24:20 ITX000c292c3c8d kernel: sg_build_indirect:
> buff_size=32768, blk_size=32768
> Mar 8 11:24:20 ITX000c292c3c8d kernel: sg_add_sfp: bufflen=32768,
> k_use_sg=1
> Mar 8 11:24:20 ITX000c292c3c8d kernel: sg_ioctl: sg0, cmd=0x2285
> Mar 8 11:24:20 ITX000c292c3c8d kernel: sg_common_write: scsi
> opcode=0x3b, cmd_size=10
> Mar 8 11:24:20 ITX000c292c3c8d kernel: sg_start_req: dxfer_len=30848
> Mar 8 11:24:20 ITX000c292c3c8d kernel: sg_link_reserve: size=30848
> Mar 8 11:24:20 ITX000c292c3c8d kernel: sg_write_xfer: num_xfer=30848,
> iovec_count=0, k_use_sg=1
> Mar 8 11:24:20 ITX000c292c3c8d kernel: iscsi_queuecommand: opcode 3b
> request_bufflen 30848 transfersize 30848
> Mar 8 11:24:20 ITX000c292c3c8d kernel: sg_cmd_done: sg0, pack_id=0,
> res=0x0
> Mar 8 11:24:20 ITX000c292c3c8d kernel: sg_finish_rem_req: res_used=1
> Mar 8 11:24:20 ITX000c292c3c8d kernel: sg_unlink_reserve: req->k_use_sg=1
> Mar 8 11:24:20 ITX000c292c3c8d kernel: sg_release: sg0
> Mar 8 11:24:20 ITX000c292c3c8d kernel: sg_fasync: sg0, mode=0
> Mar 8 11:24:20 ITX000c292c3c8d kernel: sg_remove_scat: k_use_sg=1
>
>
> Here's a 33 kB command, which also goes through fine:
> Mar 8 12:21:53 ITX000c292c3c8d kernel: Kernel logging (proc) stopped.
> Mar 8 12:21:53 ITX000c292c3c8d kernel: Kernel log daemon terminating.
> Mar 8 12:21:54 ITX000c292c3c8d exiting on signal 15
> Mar 8 12:21:54 ITX000c292c3c8d syslogd 1.4.1: restart.
> Mar 8 12:21:55 ITX000c292c3c8d kernel: klogd 1.4.1, log source =
> /proc/kmsg started.
> Mar 8 12:22:43 ITX000c292c3c8d kernel: sg_open: dev=0, flags=0x802
> Mar 8 12:22:43 ITX000c292c3c8d kernel: sg_add_sfp: sfp=0xcb9ef000
> Mar 8 12:22:43 ITX000c292c3c8d kernel: sg_build_reserve: req_size=32768
> Mar 8 12:22:43 ITX000c292c3c8d kernel: sg_build_indirect:
> buff_size=32768, blk_size=32768
> Mar 8 12:22:43 ITX000c292c3c8d kernel: sg_add_sfp: bufflen=32768,
> k_use_sg=1
> Mar 8 12:22:43 ITX000c292c3c8d kernel: sg_ioctl: sg0, cmd=0x2285
> Mar 8 12:22:43 ITX000c292c3c8d kernel: sg_common_write: scsi
> opcode=0x3b, cmd_size=10
> Mar 8 12:22:43 ITX000c292c3c8d kernel: sg_start_req: dxfer_len=33792
> Mar 8 12:22:43 ITX000c292c3c8d kernel: sg_build_indirect:
> buff_size=33792, blk_size=33792
> Mar 8 12:22:43 ITX000c292c3c8d kernel: sg_write_xfer: num_xfer=33792,
> iovec_count=0, k_use_sg=2
> Mar 8 12:22:43 ITX000c292c3c8d kernel: iscsi_queuecommand: opcode 3b
> request_bufflen 33792 transfersize 33792
> Mar 8 12:22:43 ITX000c292c3c8d kernel: sg_cmd_done: sg0, pack_id=0,
> res=0x0
> Mar 8 12:22:43 ITX000c292c3c8d kernel: sg_finish_rem_req: res_used=0
> Mar 8 12:22:43 ITX000c292c3c8d kernel: sg_remove_scat: k_use_sg=2
> Mar 8 12:22:43 ITX000c292c3c8d kernel: sg_release: sg0
> Mar 8 12:22:43 ITX000c292c3c8d kernel: sg_fasync: sg0, mode=0
> Mar 8 12:22:43 ITX000c292c3c8d kernel: sg_remove_scat: k_use_sg=1
>
> It's still got something to do with the sector-aligned scatter-gather
> buffer allocation in sg_build_indirect. sg_cmd_done isn't ever called,
> so is there a check somewhere in the completion path that's still using
> the wrong size?
>
> Aravind.
>
>
>
>
>
> michaelc@cs.wisc.edu wrote:
>> Aravind Parchuri wrote:
>>> The patch has some problem. While ioctls with dxfer_len < 32k still make
>>> it through properly, the problematic ones now show up in open-iscsi's
>>> queuecommand with request_bufflen = 0. I'm not sure what the problem is
>>> now.
>>>
>>
>> Could you send the sg and iscsi log output with the patch?
>
>
>
next prev parent reply other threads:[~2007-03-10 4:56 UTC|newest]
Thread overview: 11+ messages / expand[flat|nested] mbox.gz Atom feed top
2007-03-07 0:01 WRITE BUFFER commands through SG_IO getting rounded up to sector past 32k Aravind Parchuri
2007-03-07 0:39 ` Mike Christie
2007-03-08 21:59 ` Aravind Parchuri
2007-03-08 22:53 ` Mike Christie
2007-03-09 23:37 ` Aravind Parchuri
2007-03-10 4:56 ` Aravind Parchuri [this message]
2007-03-10 20:59 ` Mike Christie
2007-03-11 1:11 ` Aravind Parchuri
2007-03-12 18:06 ` Mike Christie
2007-03-12 23:58 ` Aravind Parchuri
2007-03-13 17:52 ` Mike Christie
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=45F23A8A.5020703@gmail.com \
--to=aravind.parchuri@gmail.com \
--cc=linux-scsi@vger.kernel.org \
--cc=michaelc@cs.wisc.edu \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox