Re: [2.6.36-rc3] Workqueues, XFS, dependencies and deadlocks

[Tejun Heo <tj@kernel.org>]

Hello,

On 09/08/2010 10:22 AM, Dave Chinner wrote:
> Ok, it looks as if the WQ_HIGHPRI is all that was required to avoid
> the log IO completion starvation livelocks. I haven't yet pulled
> the tree below, but I've now created about a billion inodes without
> seeing any evidence of the livelock occurring.
> 
> Hence it looks like I've been seeing two livelocks - one caused by
> the VM that Mel's patches fix, and one caused by the workqueue
> changeover that is fixed by the WQ_HIGHPRI change.
> 
> Thanks for you insights, Tejun - I'll push the workqueue change
> through the XFS tree to Linus.

Great, BTW, I have several questions regarding wq usage in xfs.

* Do you think @max_active > 1 could be useful for xfs?  If most works
  queued on the wq are gonna contend for the same (blocking) set of
  resources, it would just make more threads sleeping on those
  resources but otherwise it would help reducing execution latency a
  lot.

* xfs_mru_cache is a singlethread workqueue.  Do you specifically need
  singlethreadedness (strict ordering of works) or is it just to avoid
  creating dedicated per-cpu workers?  If the latter, there's no need
  to use singlethread one anymore.

* Are all four workqueues in xfs used during memory allocation?  With
  the new implementation, the reasons to have dedicated wqs are,

  - Forward progress guarantee in the memory allocation path.  Each
    workqueue w/ WQ_RESCUER has _one_ rescuer thread reserved for
    execution of works on the specific wq, which will be used under
    memory pressure to make forward progress.

  - A wq is a flush domain.  You can flush works on it as a group.

  - A wq is also a attribute domain.  If certain work items need to be
    handled differently (highpri, cpu intensive, execution ordering,
    etc...), they can be queued to a wq w/ those attributes specified.

  Maybe some of those workqueues can drop WQ_RESCUER or merged or just
  use the system workqueue?

Thanks.

-- 
tejun