[PATCH] block: Reserve only one queue tag for sync IO if only 3 tags are available

[PATCH] block: Reserve only one queue tag for sync IO if only 3 tags are available

[Jan Kara]

In case a device has three tags available we still reserve two of them
for sync IO. That leaves only a single tag for async IO such as
writeback from flusher thread which results in poor performance.

Allow async IO to consume two tags in case queue has three tag availabe
to get a decent async write performance.

This patch improves streaming write performance on a machine with such disk
from ~21 MB/s to ~52 MB/s. Also postmark throughput in presence of
streaming writer improves from 8 to 12 transactions per second so sync
IO doesn't seem to be harmed in presence of heavy async writer.

Signed-off-by: Jan Kara <jack@suse.cz>
---
 block/blk-tag.c | 11 +++++++++--
 1 file changed, 9 insertions(+), 2 deletions(-)

diff --git a/block/blk-tag.c b/block/blk-tag.c
index cc345e1..3f33d86 100644
--- a/block/blk-tag.c
+++ b/block/blk-tag.c
@@ -348,9 +348,16 @@ int blk_queue_start_tag(struct request_queue *q, struct request *rq)
 	 */
 	max_depth = bqt->max_depth;
 	if (!rq_is_sync(rq) && max_depth > 1) {
-		max_depth -= 2;
-		if (!max_depth)
+		switch (max_depth) {
+		case 2:
 			max_depth = 1;
+			break;
+		case 3:
+			max_depth = 2;
+			break;
+		default:
+			max_depth -= 2;
+		}
 		if (q->in_flight[BLK_RW_ASYNC] > max_depth)
 			return 1;
 	}
-- 
1.8.1.4

Re: [PATCH] block: Reserve only one queue tag for sync IO if only 3 tags are available

[Jens Axboe]

On Fri, Jun 28 2013, Jan Kara wrote:
> In case a device has three tags available we still reserve two of them
> for sync IO. That leaves only a single tag for async IO such as
> writeback from flusher thread which results in poor performance.
> 
> Allow async IO to consume two tags in case queue has three tag availabe
> to get a decent async write performance.
> 
> This patch improves streaming write performance on a machine with such disk
> from ~21 MB/s to ~52 MB/s. Also postmark throughput in presence of
> streaming writer improves from 8 to 12 transactions per second so sync
> IO doesn't seem to be harmed in presence of heavy async writer.

That's pretty crazy! Never seen a device like that. But yes, it
obviously exists, and the 2 reserved tags doesn't work well for that. So
it's an improvement.

Thanks, will queue up for 3.11.

-- 
Jens Axboe

Re: [PATCH] block: Reserve only one queue tag for sync IO if only 3 tags are available

[Jan Kara]

On Fri 28-06-13 21:31:52, Jens Axboe wrote:
> On Fri, Jun 28 2013, Jan Kara wrote:
> > In case a device has three tags available we still reserve two of them
> > for sync IO. That leaves only a single tag for async IO such as
> > writeback from flusher thread which results in poor performance.
> > 
> > Allow async IO to consume two tags in case queue has three tag availabe
> > to get a decent async write performance.
> > 
> > This patch improves streaming write performance on a machine with such disk
> > from ~21 MB/s to ~52 MB/s. Also postmark throughput in presence of
> > streaming writer improves from 8 to 12 transactions per second so sync
> > IO doesn't seem to be harmed in presence of heavy async writer.
> 
> That's pretty crazy! Never seen a device like that. But yes, it
> obviously exists, and the 2 reserved tags doesn't work well for that. So
> it's an improvement.
  Yeah, I agree it's a weird hardware. It's a ppc64 machine where storage
identifies as:
scsi2 : IBM 573E Storage Adapter
scsi 2:0:4:0: Direct-Access     IBM      ST373453LC       C51C PQ: 0 ANSI: 3
scsi 2:0:5:0: Direct-Access     IBM   H0 ST373207LC       C70D PQ: 0 ANSI: 4
scsi 2:0:8:0: Direct-Access     IBM   H0 ST373207LC       C70D PQ: 0 ANSI: 4

  It took me a while to figure out why this machine observes performance
characteristics noone else sees.

								Honza
-- 
Jan Kara <jack@suse.cz>
SUSE Labs, CR