From: Mike Snitzer <snitzer@redhat.com>
To: "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: linux-scsi@vger.kernel.org,
James.Bottomley@hansenpartnership.com, jaxboe@fusionio.com,
dm-devel@redhat.com, michaelc@cs.wisc.edu,
Vivek Goyal <vgoyal@redhat.com>
Subject: Re: dm-io async WRITE_SAME results in iSCSI NULL pointer [was: Re: Write same support]
Date: Mon, 20 Feb 2012 18:44:10 -0500 [thread overview]
Message-ID: <20120220234410.GC31853@redhat.com> (raw)
In-Reply-To: <20120220184623.GA29931@redhat.com>
On Mon, Feb 20 2012 at 1:46pm -0500,
Mike Snitzer <snitzer@redhat.com> wrote:
> I'm adding more debug printks to blk_rq_map_sg() to try to understand
> what is going on... will share more once I have it.
The REQ_WRITE_SAME request, that SCSI is processing on behalf of the
dm_kcopyd_zero() generated bio, has multiple bios (as if merging
occurred).
Curiously, using the dm_kcopyd_zero() interface, I'll see repeat calls
to elv_rq_merge_ok() for a bio with a given bi_sector:
<...>-10 [000] 6047.137941: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214336
<...>-10 [000] 6047.137942: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214336
<...>-10 [000] 6047.137942: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214336
<...>-10 [000] 6047.137943: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214336
<...>-10 [000] 6047.137943: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214336
<...>-10 [000] 6047.137944: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214336
<...>-10 [000] 6047.137944: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214336
<...>-10 [000] 6047.137945: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214336
<...>-10 [000] 6047.137958: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214464
<...>-10 [000] 6047.137959: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214464
<...>-10 [000] 6047.137959: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214464
<...>-10 [000] 6047.137960: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214464
<...>-10 [000] 6047.137960: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214464
<...>-10 [000] 6047.137961: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214464
<...>-10 [000] 6047.137961: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214464
<...>-10 [000] 6047.137962: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214464
<...>-10 [000] 6047.137963: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214464
So something in the dm-kcopyd and dm-io bio submission path is causing
multiple calls to elv_rq_merge_ok() for the _same_ bio, really quite
bizarre!
If I use the bdev_write_same() interface I only get one elv_rq_merge_ok
for a given bi_sector:
<...>-2088 [001] 10430.160868: elv_rq_merge_ok: WRITE_SAME bio bi_sector=4652928
<...>-1990 [001] 10432.565862: elv_rq_merge_ok: WRITE_SAME bio bi_sector=6659456
<...>-1990 [001] 10434.050269: elv_rq_merge_ok: WRITE_SAME bio bi_sector=6667904
<...>-1990 [001] 10434.238763: elv_rq_merge_ok: WRITE_SAME bio bi_sector=6668032
<...>-1990 [001] 10435.852311: elv_rq_merge_ok: WRITE_SAME bio bi_sector=6668160
<...>-1990 [001] 10437.730371: elv_rq_merge_ok: WRITE_SAME bio bi_sector=6668288
<...>-1990 [001] 10438.275437: elv_rq_merge_ok: WRITE_SAME bio bi_sector=6668416
<...>-1990 [001] 10439.737049: elv_rq_merge_ok: WRITE_SAME bio bi_sector=6668544
<...>-1990 [001] 10440.100221: elv_rq_merge_ok: WRITE_SAME bio bi_sector=6668672
<...>-1990 [001] 10441.987857: elv_rq_merge_ok: WRITE_SAME bio bi_sector=6668800
<...>-1990 [001] 10443.875244: elv_rq_merge_ok: WRITE_SAME bio bi_sector=6668928
And as if the above wasn't weird enough, I can avoid the scatter-gather
NULL pointer (in libiscsi_tcp when using the dm_kcopyd_zero() interface)
if I switch elv_rq_merge_ok() to checking the rq->cmd_flags for
REQ_WRITE_SAME, rather than checking the request's first bio's bi_rw:
/*
* Don't merge write same requests
*/
- if ((bio->bi_rw & REQ_WRITE_SAME) || (rq->bio->bi_rw & REQ_WRITE_SAME))
+ if ((bio->bi_rw & REQ_WRITE_SAME) || (rq->cmd_flags & REQ_WRITE_SAME))
return 0;
That would seem to imply to me that some WRITE SAME bios are losing the
REQ_WRITE_SAME flag in bio->bi_rw!? (or there is some other un-guarded
merge point -- also associated with the issue above).
(It at least starts to explain why I was seeing 512b sg segments at the
end of a WRITE's sg list with 4096b segments... but I still have
unanswered questions I need to sort out).
next prev parent reply other threads:[~2012-02-20 23:44 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-01-31 0:31 Write same support Martin K. Petersen
2012-01-31 0:31 ` [PATCH 1/5] block: Implement support for WRITE SAME Martin K. Petersen
2012-02-07 21:40 ` Vivek Goyal
2012-02-13 22:19 ` Martin K. Petersen
2012-02-14 8:05 ` Rolf Eike Beer
2012-02-15 15:33 ` Vivek Goyal
2012-02-16 3:29 ` Martin K. Petersen
2012-02-16 17:16 ` Vivek Goyal
2012-02-16 19:12 ` Martin K. Petersen
2012-02-08 22:50 ` Mike Snitzer
2012-02-08 23:12 ` Martin K. Petersen
2012-02-09 3:33 ` Mike Snitzer
2012-02-09 3:40 ` Mike Snitzer
2012-01-31 0:31 ` [PATCH 2/5] block: Make blkdev_issue_zeroout use " Martin K. Petersen
2012-01-31 0:31 ` [PATCH 3/5] block: ioctl to zero block ranges Martin K. Petersen
2012-01-31 0:31 ` [PATCH 4/5] scsi: Add a report opcode helper Martin K. Petersen
2012-01-31 19:53 ` Jeff Garzik
2012-01-31 20:16 ` Martin K. Petersen
2012-01-31 0:31 ` [PATCH 5/5] sd: Implement support for WRITE SAME Martin K. Petersen
2012-02-20 16:16 ` Mike Snitzer
2012-02-20 17:36 ` Martin K. Petersen
2012-02-20 18:28 ` Mike Snitzer
2012-02-03 19:15 ` Write same support Mike Snitzer
2012-02-03 19:20 ` Roland Dreier
2012-02-16 20:02 ` Mike Snitzer
2012-02-16 20:46 ` Martin K. Petersen
2012-02-16 21:09 ` Mike Snitzer
2012-02-16 21:03 ` dm-io async WRITE_SAME results in iSCSI NULL pointer [was: Re: Write same support] Mike Snitzer
2012-02-16 21:25 ` Mike Christie
2012-02-16 21:35 ` Mike Snitzer
2012-02-20 17:44 ` Martin K. Petersen
2012-02-20 18:46 ` Mike Snitzer
2012-02-20 23:44 ` Mike Snitzer [this message]
2012-02-21 0:07 ` Martin K. Petersen
2012-02-21 3:18 ` Mike Snitzer
2012-02-21 3:57 ` Martin K. Petersen
2012-02-21 6:55 ` Mike Snitzer
2012-02-21 12:31 ` Martin K. Petersen
2012-02-21 14:42 ` Mike Snitzer
2012-02-21 19:33 ` Mike Snitzer
2012-02-21 21:31 ` Martin K. Petersen
2012-02-21 23:36 ` Mike Snitzer
2012-02-21 19:47 ` Vivek Goyal
2012-02-21 19:56 ` Martin K. Petersen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20120220234410.GC31853@redhat.com \
--to=snitzer@redhat.com \
--cc=James.Bottomley@hansenpartnership.com \
--cc=dm-devel@redhat.com \
--cc=jaxboe@fusionio.com \
--cc=linux-scsi@vger.kernel.org \
--cc=martin.petersen@oracle.com \
--cc=michaelc@cs.wisc.edu \
--cc=vgoyal@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).