[PATCH] block: forbid to re-enable I/O stat accounting

[PATCH] block: forbid to re-enable I/O stat accounting

[Jerome Marchand]

When we stop I/O stat accounting we stop to update the in-flight
requests counter and we need this counter to be reliable for
accounting I/O stats. Unfortunately updating in_flight field may
affect performance. So, until we have a better solution, just forbid
to re-enable I/O stat accounting after it has been disabled.

Signed-off-by: Jerome Marchand <jmarchan@redhat.com>
---
 block/blk-sysfs.c |   13 ++++++++-----
 1 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/block/blk-sysfs.c b/block/blk-sysfs.c
index e29ddfc..2903874 100644
--- a/block/blk-sysfs.c
+++ b/block/blk-sysfs.c
@@ -208,12 +208,15 @@ static ssize_t queue_iostats_store(struct request_queue *q, const char *page,
 	unsigned long stats;
 	ssize_t ret = queue_var_store(&stats, page, count);
 
-	spin_lock_irq(q->queue_lock);
-	if (stats)
-		queue_flag_set(QUEUE_FLAG_IO_STAT, q);
-	else
+	/*
+	 * When we disable io stat accounting we stop tracking essential data,
+	 * so don't allow to reenable it as it would lead to inconsistency.
+	 */
+	if (!stats) {
+		spin_lock_irq(q->queue_lock);
 		queue_flag_clear(QUEUE_FLAG_IO_STAT, q);
-	spin_unlock_irq(q->queue_lock);
+		spin_unlock_irq(q->queue_lock);
+	}
 
 	return ret;
 }
-- 
1.5.5.1

Re: [PATCH] block: forbid to re-enable I/O stat accounting

[Andrew Morton]

Please fix your email headers - what I received was

From: Jerome Marchand <jmarchan@redhat.com>
To: unlisted-recipients:;;@imap1.linux-foundation.org (no To-header on input)
Cc: Jens Axboe <axboe@kernel.dk>


On Thu, 19 Mar 2009 11:36:50 +0100 Jerome Marchand <jmarchan@redhat.com> wrote:

> When we stop I/O stat accounting we stop to update the in-flight
> requests counter and we need this counter to be reliable for
> accounting I/O stats. Unfortunately updating in_flight field may
> affect performance. So, until we have a better solution, just forbid
> to re-enable I/O stat accounting after it has been disabled.

hm.  Is it really so hard to just quiesce the device until all in-flight
requests have drained?  freeze_bdev() might be a suitable starting point?

Re: [PATCH] block: forbid to re-enable I/O stat accounting

[Jens Axboe]

On Sat, Mar 21 2009, Andrew Morton wrote:
> 
> Please fix your email headers - what I received was
> 
> From: Jerome Marchand <jmarchan@redhat.com>
> To: unlisted-recipients:;;@imap1.linux-foundation.org (no To-header on input)
> Cc: Jens Axboe <axboe@kernel.dk>

Ditto, I didn't think anyone else got it. At least noone else got my
reply :-)

> On Thu, 19 Mar 2009 11:36:50 +0100 Jerome Marchand <jmarchan@redhat.com> wrote:
> 
> > When we stop I/O stat accounting we stop to update the in-flight
> > requests counter and we need this counter to be reliable for
> > accounting I/O stats. Unfortunately updating in_flight field may
> > affect performance. So, until we have a better solution, just forbid
> > to re-enable I/O stat accounting after it has been disabled.
> 
> hm.  Is it really so hard to just quiesce the device until all in-flight
> requests have drained?  freeze_bdev() might be a suitable starting point?

That was my suggestion as well, we really need to be able to turn it
back on again. Ideally, I'd like iostat and friends to turn it on/off
when somebody does the monitoring. A one-way street option is not very
nice.

-- 
Jens Axboe

Re: [PATCH] block: forbid to re-enable I/O stat accounting

[Jens Axboe]

On Sat, Mar 21 2009, Andrew Morton wrote:
> 
> Please fix your email headers - what I received was
> 
> From: Jerome Marchand <jmarchan@redhat.com>
> To: unlisted-recipients:;;@imap1.linux-foundation.org (no To-header on input)
> Cc: Jens Axboe <axboe@kernel.dk>
> 
> 
> On Thu, 19 Mar 2009 11:36:50 +0100 Jerome Marchand <jmarchan@redhat.com> wrote:
> 
> > When we stop I/O stat accounting we stop to update the in-flight
> > requests counter and we need this counter to be reliable for
> > accounting I/O stats. Unfortunately updating in_flight field may
> > affect performance. So, until we have a better solution, just forbid
> > to re-enable I/O stat accounting after it has been disabled.
> 
> hm.  Is it really so hard to just quiesce the device until all in-flight
> requests have drained?  freeze_bdev() might be a suitable starting point?

freeze_bdev() is too far up, I think. It would be better to just reuse
the quiscing code we use for switching io schedulers on-the-fly.
Basically something ala:

        spin_lock_irq(q->queue_lock);
        queue_flag_set(QUEUE_FLAG_ELVSWITCH, q);
        elv_drain_elevator(q);
        while (q->rq.elvpriv) {
                blk_start_queueing(q);
                spin_unlock_irq(q->queue_lock);
                msleep(10);
                spin_lock_irq(q->queue_lock);
                elv_drain_elevator(q);
        }

        re-enable IO accounting;
        spin_unlock_irq(q->queue_lock);

and then don't account requests that don't have REQ_ELVPRIV set. I think
that should be acceptable. I was pretty sure I had a patch for something
else that yanked this logic into start/stop helpers... Ah, found it:

http://git.kernel.dk/?p=linux-2.6-block.git;a=commitdiff;h=8cb9f793b82024cd43ed5d4583a5287d002abdf8

-- 
Jens Axboe