Re: [PATCH v2] writeback: avoid race when update bandwidth

[Fengguang Wu <fengguang.wu@intel.com>]

On Wed, Jun 13, 2012 at 01:59:20PM +1000, Dave Chinner wrote:
> On Tue, Jun 12, 2012 at 07:52:19PM +0800, Fengguang Wu wrote:
> > On Tue, Jun 12, 2012 at 07:46:01PM +0800, Wanpeng Li wrote:
> > > From: Wanpeng Li <liwp@linux.vnet.ibm.com>
> > > 
> > > "V1 -> V2"
> > > * remove dirty_lock
> > > 
> > > Since bdi->wb.list_lock is used to protect the b_* lists,
> > > so the flushers who call wb_writeback to writeback pages will
> > > stuck when bandwidth update policy holds this lock. In order
> > > to avoid this race we can introduce a new bandwidth_lock who
> > > is responsible for protecting bandwidth update policy.
> > > 
> > > Signed-off-by: Wanpeng Li <liwp.linux@gmail.com>
> > 
> > Applied with a new title "writeback: use a standalone lock for
> > updating write bandwidth". "race" is sensitive because it often
> > refers to some locking error.
> 
> Fengguang - can we get some evidence that this is a contended lock
> before changing the scope of it? All of the previous "breaking up
> global locks" have been done based on lock contention data, so
> moving back to a global lock for this needs to have the same
> analysis provided...

Good point. Attached is the lockstat for the case "10 disks each runs
100 dd dirtier tasks":

        lkp-ne02/JBOD-10HDD-thresh=4G/xfs-100dd-1-3.2.0-rc5

The wb->list_lock contention is much better than I expected, which is
good.  What stand out are
                                                        waittime-total
- &rq->lock             by double_rq_lock()             6738952.13
- clockevents_lock      by clockevents_notify()         2155554.37
- mapping->tree_lock    by test_clear_page_writeback()   931550.13
- sb_lock               by grab_super_passive()          918815.87
- &zone->lru_lock       by pagevec_lru_move_fn()         912681.05

- sysfs_mutex           by sysfs_permission()           24029975.20 # mutex
- ip->i_lock            by xfs_ilock()                  18428284.10 # mrlock

Thanks,
Fengguang