Re: [PATCH V2] block/throttle: Add IO throttled information in blkio.throttle.

[Vivek Goyal <vgoyal@redhat.com>]

On Tue, Sep 04, 2012 at 10:12:49PM +0800, Tao Ma wrote:
> On 09/04/2012 09:35 PM, Vivek Goyal wrote:
> > On Fri, Aug 31, 2012 at 01:15:09PM +0800, Tao Ma wrote:
> > 
> > [..]
> >> diff --git a/block/blk-throttle.c b/block/blk-throttle.c
> >> index 1588c2d..9317d71 100644
> >> --- a/block/blk-throttle.c
> >> +++ b/block/blk-throttle.c
> >> @@ -46,6 +46,8 @@ struct tg_stats_cpu {
> >>  	struct blkg_rwstat		service_bytes;
> >>  	/* total IOs serviced, post merge */
> >>  	struct blkg_rwstat		serviced;
> >> +	/* total IOs queued, not submitted to the underlying device. */
> >> +	struct blkg_rwstat		io_queued;
> >>  };
> > 
> > Couple of questions.
> > 
> > - blkg_rwstat is "unsigned" and io_queued can go negative too (Because
> >   throttled bio can very well be dispatched from other cpu from a worker
> >   thread). So is it a good idea to represent a negative number with 
> >   unsingned type?
> > 
> > - As this stat is per cpu, a reader might very well see negative (or a
> >   huge unsigned value) as number of io_queued. Not sure if that is acceptable.
> >   How would user space come to know whether it is a valid value or not. I
> >   thought per cpu stats are good for continuously increasing values but
> >   not necessarily for values which can increase as well as decrease.
> You are right. So I should just use throtl_grp->nr_queued to display the
> total numbers of ios being throttled and I guess a rcu_read_lock should
> be enough for me to access that data.

Not sure how rcu_read_lock() is going to help. Can you explain a bit more.

For 64bit, we should not require any locking as updation always happens
under queue_lock(for io_queued). And lockless reading should be just fine
as updates to 64bit values will be atomic.

Only problem is reading of 64bit io_queued on 32bit platforms.

As updation always happens under queue_lock, we don't gain anything by making
this stat per cpu. And for 32bit, we probably can updation/reading using
sequence counter to make sure we don't get invalid values and read will 
still be lockless.

Thanks
Vivek