[PATCH 7/8] blk-throttle: Do not throttle WRITEs at device level except direct IO

[Vivek Goyal <vgoyal@redhat.com>]

Now WRITES are not throttled at device level except any WRITES
which are marked as DIRECT. This patch I have lifted from Andrea
Righi's previously posted series on lkml.

This will mean that any filesystem internally generated WRITES
also all not be throttled. If need be, down the line situation can
be improved if file system can give a hint to throttling logic that
it is ok to throttle certain WRITES without risk of being
serialized.

Signed-off-by: Vivek Goyal <vgoyal@redhat.com>
---
 block/blk-throttle.c      |    8 ++++++++
 fs/direct-io.c            |    1 +
 include/linux/blk_types.h |    2 ++
 3 files changed, 11 insertions(+), 0 deletions(-)

diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index c8bdc59..0d63e3b 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -1293,6 +1293,14 @@ int blk_throtl_bio(struct request_queue *q, struct bio **biop)
 	}
 
 	/*
+	 * WRITES bio are throttled here only if they are directIO. Otherwise
+	 * these are coming from page cache and these must have been subjected
+	 * to throttling rules while process was writting to page cache.
+	 */
+	if (rw == WRITE && !(bio->bi_rw & REQ_DIRECT_IO))
+		return 0;
+
+	/*
 	 * A throtl_grp pointer retrieved under rcu can be used to access
 	 * basic fields like stats and io rates. If a group has no rules,
 	 * just update the dispatch stats in lockless manner and return.
diff --git a/fs/direct-io.c b/fs/direct-io.c
index ac5f164..7858828 100644
--- a/fs/direct-io.c
+++ b/fs/direct-io.c
@@ -361,6 +361,7 @@ static void dio_bio_submit(struct dio *dio)
 	unsigned long flags;
 
 	bio->bi_private = dio;
+	bio->bi_rw |= REQ_DIRECT_IO;
 
 	spin_lock_irqsave(&dio->bio_lock, flags);
 	dio->refcount++;
diff --git a/include/linux/blk_types.h b/include/linux/blk_types.h
index 2a7cea5..f46e8a4 100644
--- a/include/linux/blk_types.h
+++ b/include/linux/blk_types.h
@@ -131,6 +131,7 @@ enum rq_flag_bits {
 	__REQ_RAHEAD,		/* read ahead, can fail anytime */
 	__REQ_THROTTLED,	/* This bio has already been subjected to
 				 * throttling rules. Don't do it again. */
+	__REQ_DIRECT_IO,	/* It is a direct IO operation */
 
 	/* request only flags */
 	__REQ_SORTED,		/* elevator knows about this request */
@@ -172,6 +173,7 @@ enum rq_flag_bits {
 
 #define REQ_RAHEAD		(1 << __REQ_RAHEAD)
 #define REQ_THROTTLED		(1 << __REQ_THROTTLED)
+#define REQ_DIRECT_IO		(1 << __REQ_DIRECT_IO)
 
 #define REQ_SORTED		(1 << __REQ_SORTED)
 #define REQ_SOFTBARRIER		(1 << __REQ_SOFTBARRIER)
-- 
1.7.4.4