From: Mike Snitzer <snitzer@redhat.com>
To: "Martin K. Petersen" <martin.petersen@oracle.com>
Cc: linux-scsi@vger.kernel.org,
James.Bottomley@hansenpartnership.com, jaxboe@fusionio.com,
dm-devel@redhat.com, michaelc@cs.wisc.edu,
Vivek Goyal <vgoyal@redhat.com>
Subject: Re: dm-io async WRITE_SAME results in iSCSI NULL pointer [was: Re: Write same support]
Date: Mon, 20 Feb 2012 18:44:10 -0500 [thread overview]
Message-ID: <20120220234410.GC31853@redhat.com> (raw)
In-Reply-To: <20120220184623.GA29931@redhat.com>
On Mon, Feb 20 2012 at 1:46pm -0500,
Mike Snitzer <snitzer@redhat.com> wrote:
> I'm adding more debug printks to blk_rq_map_sg() to try to understand
> what is going on... will share more once I have it.
The REQ_WRITE_SAME request, that SCSI is processing on behalf of the
dm_kcopyd_zero() generated bio, has multiple bios (as if merging
occurred).
Curiously, using the dm_kcopyd_zero() interface, I'll see repeat calls
to elv_rq_merge_ok() for a bio with a given bi_sector:
<...>-10 [000] 6047.137941: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214336
<...>-10 [000] 6047.137942: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214336
<...>-10 [000] 6047.137942: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214336
<...>-10 [000] 6047.137943: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214336
<...>-10 [000] 6047.137943: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214336
<...>-10 [000] 6047.137944: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214336
<...>-10 [000] 6047.137944: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214336
<...>-10 [000] 6047.137945: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214336
<...>-10 [000] 6047.137958: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214464
<...>-10 [000] 6047.137959: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214464
<...>-10 [000] 6047.137959: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214464
<...>-10 [000] 6047.137960: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214464
<...>-10 [000] 6047.137960: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214464
<...>-10 [000] 6047.137961: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214464
<...>-10 [000] 6047.137961: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214464
<...>-10 [000] 6047.137962: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214464
<...>-10 [000] 6047.137963: elv_rq_merge_ok: WRITE_SAME bio bi_sector=5214464
So something in the dm-kcopyd and dm-io bio submission path is causing
multiple calls to elv_rq_merge_ok() for the _same_ bio, really quite
bizarre!
If I use the bdev_write_same() interface I only get one elv_rq_merge_ok
for a given bi_sector:
<...>-2088 [001] 10430.160868: elv_rq_merge_ok: WRITE_SAME bio bi_sector=4652928
<...>-1990 [001] 10432.565862: elv_rq_merge_ok: WRITE_SAME bio bi_sector=6659456
<...>-1990 [001] 10434.050269: elv_rq_merge_ok: WRITE_SAME bio bi_sector=6667904
<...>-1990 [001] 10434.238763: elv_rq_merge_ok: WRITE_SAME bio bi_sector=6668032
<...>-1990 [001] 10435.852311: elv_rq_merge_ok: WRITE_SAME bio bi_sector=6668160
<...>-1990 [001] 10437.730371: elv_rq_merge_ok: WRITE_SAME bio bi_sector=6668288
<...>-1990 [001] 10438.275437: elv_rq_merge_ok: WRITE_SAME bio bi_sector=6668416
<...>-1990 [001] 10439.737049: elv_rq_merge_ok: WRITE_SAME bio bi_sector=6668544
<...>-1990 [001] 10440.100221: elv_rq_merge_ok: WRITE_SAME bio bi_sector=6668672
<...>-1990 [001] 10441.987857: elv_rq_merge_ok: WRITE_SAME bio bi_sector=6668800
<...>-1990 [001] 10443.875244: elv_rq_merge_ok: WRITE_SAME bio bi_sector=6668928
And as if the above wasn't weird enough, I can avoid the scatter-gather
NULL pointer (in libiscsi_tcp when using the dm_kcopyd_zero() interface)
if I switch elv_rq_merge_ok() to checking the rq->cmd_flags for
REQ_WRITE_SAME, rather than checking the request's first bio's bi_rw:
/*
* Don't merge write same requests
*/
- if ((bio->bi_rw & REQ_WRITE_SAME) || (rq->bio->bi_rw & REQ_WRITE_SAME))
+ if ((bio->bi_rw & REQ_WRITE_SAME) || (rq->cmd_flags & REQ_WRITE_SAME))
return 0;
That would seem to imply to me that some WRITE SAME bios are losing the
REQ_WRITE_SAME flag in bio->bi_rw!? (or there is some other un-guarded
merge point -- also associated with the issue above).
(It at least starts to explain why I was seeing 512b sg segments at the
end of a WRITE's sg list with 4096b segments... but I still have
unanswered questions I need to sort out).
next prev parent reply other threads:[~2012-02-20 23:44 UTC|newest]
Thread overview: 44+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-01-31 0:31 Write same support Martin K. Petersen
2012-01-31 0:31 ` [PATCH 1/5] block: Implement support for WRITE SAME Martin K. Petersen
2012-02-07 21:40 ` Vivek Goyal
2012-02-13 22:19 ` Martin K. Petersen
2012-02-14 8:05 ` Rolf Eike Beer
2012-02-15 15:33 ` Vivek Goyal
2012-02-16 3:29 ` Martin K. Petersen
2012-02-16 17:16 ` Vivek Goyal
2012-02-16 19:12 ` Martin K. Petersen
2012-02-08 22:50 ` Mike Snitzer
2012-02-08 23:12 ` Martin K. Petersen
2012-02-09 3:33 ` Mike Snitzer
2012-02-09 3:40 ` Mike Snitzer
2012-01-31 0:31 ` [PATCH 2/5] block: Make blkdev_issue_zeroout use " Martin K. Petersen
2012-01-31 0:31 ` [PATCH 3/5] block: ioctl to zero block ranges Martin K. Petersen
2012-01-31 0:31 ` [PATCH 4/5] scsi: Add a report opcode helper Martin K. Petersen
2012-01-31 19:53 ` Jeff Garzik
2012-01-31 20:16 ` Martin K. Petersen
2012-01-31 0:31 ` [PATCH 5/5] sd: Implement support for WRITE SAME Martin K. Petersen
2012-02-20 16:16 ` Mike Snitzer
2012-02-20 17:36 ` Martin K. Petersen
2012-02-20 18:28 ` Mike Snitzer
2012-02-03 19:15 ` Write same support Mike Snitzer
2012-02-03 19:20 ` Roland Dreier
2012-02-16 20:02 ` Mike Snitzer
2012-02-16 20:46 ` Martin K. Petersen
2012-02-16 21:09 ` Mike Snitzer
2012-02-16 21:03 ` dm-io async WRITE_SAME results in iSCSI NULL pointer [was: Re: Write same support] Mike Snitzer
2012-02-16 21:25 ` Mike Christie
2012-02-16 21:35 ` Mike Snitzer
2012-02-20 17:44 ` Martin K. Petersen
2012-02-20 18:46 ` Mike Snitzer
2012-02-20 23:44 ` Mike Snitzer [this message]
2012-02-21 0:07 ` Martin K. Petersen
2012-02-21 3:18 ` Mike Snitzer
2012-02-21 3:57 ` Martin K. Petersen
2012-02-21 6:55 ` Mike Snitzer
2012-02-21 12:31 ` Martin K. Petersen
2012-02-21 14:42 ` Mike Snitzer
2012-02-21 19:33 ` Mike Snitzer
2012-02-21 21:31 ` Martin K. Petersen
2012-02-21 23:36 ` Mike Snitzer
2012-02-21 19:47 ` Vivek Goyal
2012-02-21 19:56 ` Martin K. Petersen
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20120220234410.GC31853@redhat.com \
--to=snitzer@redhat.com \
--cc=James.Bottomley@hansenpartnership.com \
--cc=dm-devel@redhat.com \
--cc=jaxboe@fusionio.com \
--cc=linux-scsi@vger.kernel.org \
--cc=martin.petersen@oracle.com \
--cc=michaelc@cs.wisc.edu \
--cc=vgoyal@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.