[PATCH v1 0/4] block: fix bio_will_gap()

[PATCH v1 0/4] block: fix bio_will_gap()

[Ming Lei]

Hi Guys,

The bio passed to bio_will_gap() may be fast cloned from upper
layer(dm, md, bcache, fs, ...), or from bio splitting in block
core. Unfortunately bio_will_gap() just figures out the last
bvec via 'bi_io_vec[prev->bi_vcnt - 1]' directly, and this way
is obviously wrong in case of fast-cloned bio.

It is observed that lots of BIOs are still merged even if
the virt boundary limit is violated by the merge, and the issue
was reported from Sagi Grimberg.
    
This patch introduces two helpers for getting the first and last
bvec of one bio and applys them to fix the issue. Sagi tested
the last patchset and confirmed the fix.

V1:
	- get bvec directly for non-cloned bio
	- implement bio_get_last_bvec() with single bio_advance_iter(),
	and avoid to use bio_for_each_segment() which looks a bit inefficient
	- avoid to double check queue_virt_boundary() in bio_will_gap()


 block/blk-merge.c      |  8 ++------
 include/linux/bio.h    | 41 +++++++++++++++++++++++++++++++++++++++++
 include/linux/blkdev.h | 21 ++++++++++++++++-----
 3 files changed, 59 insertions(+), 11 deletions(-)

Thanks,
Ming

[PATCH v1 1/4] block: bio: introduce helpers to get the 1st and last bvec

[Ming Lei]

The bio passed to bio_will_gap() may be fast cloned from upper
layer(dm, md, bcache, fs, ...), or from bio splitting in block
core.

Unfortunately bio_will_gap() just figures out the last bvec via
'bi_io_vec[prev->bi_vcnt - 1]' directly, and this way is obviously
wrong.

This patch introduces two helpers for getting the first and last
bvec of one bio for fixing the issue.

Cc: stable@vger.kernel.org
Reported-by: Sagi Grimberg <sagig@dev.mellanox.co.il>
Cc: Christoph Hellwig <hch@infradead.org>
Signed-off-by: Ming Lei <ming.lei@canonical.com>
---
 include/linux/bio.h | 41 +++++++++++++++++++++++++++++++++++++++++
 1 file changed, 41 insertions(+)

diff --git a/include/linux/bio.h b/include/linux/bio.h
index 5349e68..3ce2e45 100644
--- a/include/linux/bio.h
+++ b/include/linux/bio.h
@@ -310,6 +310,47 @@ static inline void bio_clear_flag(struct bio *bio, unsigned int bit)
 	bio->bi_flags &= ~(1U << bit);
 }
 
+static inline void bio_get_first_bvec(struct bio *bio, struct bio_vec *bv)
+{
+	*bv = bio_iovec(bio);
+}
+
+/*
+ * bio_get_last_bvec() is introduced to get the last bvec of one
+ * bio for bio_will_gap().
+ */
+static inline void bio_get_last_bvec(struct bio *bio, struct bio_vec *bv)
+{
+	struct bvec_iter iter = bio->bi_iter;
+	int idx;
+
+	if (!bio_flagged(bio, BIO_CLONED)) {
+		*bv = bio->bi_io_vec[bio->bi_vcnt - 1];
+		return;
+	}
+
+	if (unlikely(!bio_multiple_segments(bio))) {
+		*bv = bio_iovec(bio);
+		return;
+	}
+
+	bio_advance_iter(bio, &iter, iter.bi_size);
+
+	if (!iter.bi_bvec_done)
+		idx = iter.bi_idx - 1;
+	else	/* in the middle of bvec */
+		idx = iter.bi_idx;
+
+	*bv = bio->bi_io_vec[idx];
+
+	/*
+	 * iter.bi_bvec_done records actual length of the last bvec
+	 * if this bio ends in the middle of one io vector
+	 */
+	if (iter.bi_bvec_done)
+		bv->bv_len = iter.bi_bvec_done;
+}
+
 enum bip_flags {
 	BIP_BLOCK_INTEGRITY	= 1 << 0, /* block layer owns integrity data */
 	BIP_MAPPED_INTEGRITY	= 1 << 1, /* ref tag has been remapped */
-- 
1.9.1

Re: [PATCH v1 1/4] block: bio: introduce helpers to get the 1st and last bvec

[Sagi Grimberg]

Looks fine,

Reviewed-by: Sagi Grimberg <sagig@mellanox.com>

Re: [PATCH v1 1/4] block: bio: introduce helpers to get the 1st and last bvec

[Christoph Hellwig]

> +/*
> + * bio_get_last_bvec() is introduced to get the last bvec of one
> + * bio for bio_will_gap().
> + */

I don't think this comment adds any value.

Otherwise looks fine:

Reviewed-by: Christoph Hellwig <hch@lst.de>

[PATCH v1 2/4] block: check virt boundary in bio_will_gap()

[Ming Lei]

In the following patch, the way for figuring out
the last bvec will be changed with a bit cost introduced,
so return immediately if the queue doesn't have virt
boundary limit. Actually most of devices have not
this limit.

Cc: Sagi Grimberg <sagig@dev.mellanox.co.il>
Cc: Christoph Hellwig <hch@infradead.org>
Signed-off-by: Ming Lei <ming.lei@canonical.com>
---
 include/linux/blkdev.h | 12 +++++++++---
 1 file changed, 9 insertions(+), 3 deletions(-)

diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 4571ef1..5023401 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -1372,6 +1372,13 @@ static inline void put_dev_sector(Sector p)
 	page_cache_release(p.v);
 }
 
+static inline bool __bvec_gap_to_prev(struct request_queue *q,
+				struct bio_vec *bprv, unsigned int offset)
+{
+	return offset ||
+		((bprv->bv_offset + bprv->bv_len) & queue_virt_boundary(q));
+}
+
 /*
  * Check if adding a bio_vec after bprv with offset would create a gap in
  * the SG list. Most drivers don't care about this, but some do.
@@ -1381,14 +1388,13 @@ static inline bool bvec_gap_to_prev(struct request_queue *q,
 {
 	if (!queue_virt_boundary(q))
 		return false;
-	return offset ||
-		((bprv->bv_offset + bprv->bv_len) & queue_virt_boundary(q));
+	return __bvec_gap_to_prev(q, bprv, offset);
 }
 
 static inline bool bio_will_gap(struct request_queue *q, struct bio *prev,
 			 struct bio *next)
 {
-	if (!bio_has_data(prev))
+	if (!bio_has_data(prev) || !queue_virt_boundary(q))
 		return false;
 
 	return bvec_gap_to_prev(q, &prev->bi_io_vec[prev->bi_vcnt - 1],
-- 
1.9.1

Re: [PATCH v1 2/4] block: check virt boundary in bio_will_gap()

[Sagi Grimberg]

Looks fine,

Reviewed-by: Sagi Grimberg <sagig@mellanox.com>

Re: [PATCH v1 2/4] block: check virt boundary in bio_will_gap()

[Christoph Hellwig]

Looks fine,

Reviewed-by: Christoph Hellwig <hch@lst.de>

[PATCH v1 3/4] block: get the 1st and last bvec via helpers

[Ming Lei]

This patch applies the two introduced helpers to
figure out the 1st and last bvec, and fixes the
original way after bio splitting.

Cc: stable@vger.kernel.org
Reported-by: Sagi Grimberg <sagig@dev.mellanox.co.il>
Cc: Christoph Hellwig <hch@infradead.org>
Signed-off-by: Ming Lei <ming.lei@canonical.com>
---
 include/linux/blkdev.h | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)

diff --git a/include/linux/blkdev.h b/include/linux/blkdev.h
index 5023401..de4fc002 100644
--- a/include/linux/blkdev.h
+++ b/include/linux/blkdev.h
@@ -1394,11 +1394,16 @@ static inline bool bvec_gap_to_prev(struct request_queue *q,
 static inline bool bio_will_gap(struct request_queue *q, struct bio *prev,
 			 struct bio *next)
 {
-	if (!bio_has_data(prev) || !queue_virt_boundary(q))
+	if (!bio_has_data(prev) || !queue_virt_boundary(q)) {
 		return false;
+	} else {
+		struct bio_vec pb, nb;
 
-	return bvec_gap_to_prev(q, &prev->bi_io_vec[prev->bi_vcnt - 1],
-				next->bi_io_vec[0].bv_offset);
+		bio_get_last_bvec(prev, &pb);
+		bio_get_first_bvec(next, &nb);
+
+		return __bvec_gap_to_prev(q, &pb, nb.bv_offset);
+	}
 }
 
 static inline bool req_gap_back_merge(struct request *req, struct bio *bio)
-- 
1.9.1

Re: [PATCH v1 3/4] block: get the 1st and last bvec via helpers

[Sagi Grimberg]

Looks fine,

Reviewed-by: Sagi Grimberg <sagig@mellanox.com>

Re: [PATCH v1 3/4] block: get the 1st and last bvec via helpers

[Christoph Hellwig]

On Fri, Feb 19, 2016 at 11:20:21AM +0800, Ming Lei wrote:
> This patch applies the two introduced helpers to
> figure out the 1st and last bvec, and fixes the
> original way after bio splitting.

Looks fine,

Reviewed-by: Christoph Hellwig <hch@lst.de>

[PATCH v1 4/4] block: merge: get the 1st and last bvec via helpers

[Ming Lei]

This patch applies the two introduced helpers to
figure out the 1st and last bvec.

Cc: Sagi Grimberg <sagig@dev.mellanox.co.il>
Cc: Christoph Hellwig <hch@infradead.org>
Signed-off-by: Ming Lei <ming.lei@canonical.com>
---
 block/blk-merge.c | 8 ++------
 1 file changed, 2 insertions(+), 6 deletions(-)

diff --git a/block/blk-merge.c b/block/blk-merge.c
index 888a7fe..2613531 100644
--- a/block/blk-merge.c
+++ b/block/blk-merge.c
@@ -304,7 +304,6 @@ static int blk_phys_contig_segment(struct request_queue *q, struct bio *bio,
 				   struct bio *nxt)
 {
 	struct bio_vec end_bv = { NULL }, nxt_bv;
-	struct bvec_iter iter;
 
 	if (!blk_queue_cluster(q))
 		return 0;
@@ -316,11 +315,8 @@ static int blk_phys_contig_segment(struct request_queue *q, struct bio *bio,
 	if (!bio_has_data(bio))
 		return 1;
 
-	bio_for_each_segment(end_bv, bio, iter)
-		if (end_bv.bv_len == iter.bi_size)
-			break;
-
-	nxt_bv = bio_iovec(nxt);
+	bio_get_last_bvec(bio, &end_bv);
+	bio_get_first_bvec(nxt, &nxt_bv);
 
 	if (!BIOVEC_PHYS_MERGEABLE(&end_bv, &nxt_bv))
 		return 0;
-- 
1.9.1

Re: [PATCH v1 4/4] block: merge: get the 1st and last bvec via helpers

[Sagi Grimberg]

Looks fine,

Reviewed-by: Sagi Grimberg <sagig@mellanox.com>

Re: [PATCH v1 4/4] block: merge: get the 1st and last bvec via helpers

[Christoph Hellwig]

Looks fine,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH v1 0/4] block: fix bio_will_gap()

[Sagi Grimberg]

> Hi Guys,
>
> The bio passed to bio_will_gap() may be fast cloned from upper
> layer(dm, md, bcache, fs, ...), or from bio splitting in block
> core. Unfortunately bio_will_gap() just figures out the last
> bvec via 'bi_io_vec[prev->bi_vcnt - 1]' directly, and this way
> is obviously wrong in case of fast-cloned bio.
>
> It is observed that lots of BIOs are still merged even if
> the virt boundary limit is violated by the merge, and the issue
> was reported from Sagi Grimberg.
>
> This patch introduces two helpers for getting the first and last
> bvec of one bio and applys them to fix the issue. Sagi tested
> the last patchset and confirmed the fix.
>
> V1:
> 	- get bvec directly for non-cloned bio
> 	- implement bio_get_last_bvec() with single bio_advance_iter(),
> 	and avoid to use bio_for_each_segment() which looks a bit inefficient
> 	- avoid to double check queue_virt_boundary() in bio_will_gap()

Thanks Ming,

Jens, can this make the next 4.5-rc since this regression was detected
in 4.5?

Thanks,
Sagi.