Re: [PATCH v2] writeback: avoid race when update bandwidth

Fengguang Wu
2012-06-13T12:14:41Z

On Wed, Jun 13, 2012 at 01:59:20PM +1000, Dave Chinner wrote:
> On Tue, Jun 12, 2012 at 07:52:19PM +0800, Fengguang Wu wrote:
> > On Tue, Jun 12, 2012 at 07:46:01PM +0800, Wanpeng Li wrote:
> > > From: Wanpeng Li <liwp@linux.vnet.ibm.com>
> > >
> > > "V1 -> V2"
> > > * remove dirty_lock
> > >
> > > Since bdi->wb.list_lock is used to protect the b_* lists,
> > > so the flushers who call wb_writeback to writeback pages will
> > > stuck when bandwidth update policy holds this lock. In order
> > > to avoid this race we can introduce a new bandwidth_lock who
> > > is responsible for protecting bandwidth update policy.
> > >
> > > Signed-off-by: Wanpeng Li <liwp.linux@gmail.com>
> >
> > Applied with a new title "writeback: use a standalone lock for
> > updating write bandwidth". "race" is sensitive because it often
> > refers to some locking error.
>
> Fengguang - can we get some evidence that this is a contended lock
> before changing the scope of it? All of the previous "breaking up
> global locks" have been done based on lock contention data, so
> moving back to a global lock for this needs to have the same
> analysis provided...

Good point. Attached is the lockstat for the case "10 disks each runs
100 dd dirtier tasks":

lkp-ne02/JBOD-10HDD-thresh=4G/xfs-100dd-1-3.2.0-rc5

The wb->list_lock contention is much better than I expected, which is
good. What stand out are
waittime-total
- &rq->lock by double_rq_lock() 6738952.13
- clockevents_lock by clockevents_notify() 2155554.37
- mapping->tree_lock by test_clear_page_writeback() 931550.13
- sb_lock by grab_super_passive() 918815.87
- &zone->lru_lock by pagevec_lru_move_fn() 912681.05

- sysfs_mutex by sysfs_permission() 24029975.20 # mutex
- ip->i_lock by xfs_ilock() 18428284.10 # mrlock

Thanks,
Fengguang