Re: [RFC PATCHSET] workqueue: implement concurrency managed workqueue

[Tejun Heo <tj@kernel.org>]

David Howells wrote:
> Tejun Heo <tj@kernel.org> wrote:
> 
>> Given that slow-work isn't being used too extensively yet, I was
>> thinking whether that part could be pushed down to the caller.  Or, we
>> can also wrap work and export an interface which supports the get/put
>> reference.
> 
> The caller of what?

The user of the API.  It can be implemented there too, right?

> I found the refcounting much easier to manage in slow-work when slow-work
> actively got/put refs on the work items it was queueing.  The reason for that
> is that slow-work can handle the queue/queue races and the requeue/execute
> races much more efficiently.
> 
> Part of this was due to the fact I wanted to prevent re-entry into the work
> executor, and to do that I had maintenance flags in the work item struct - but
> that meant that slow-work had to modify the work item after execution.
> 
> So I should adjust point 1 on my list.
> 
>   (1) Work items can be requeued whilst they are executing, but the execution
>       function will not be re-entered until after the current execution
>       completes, but rather the execution will be deferred.

This is already guaranteed on a single cpu, so unless a work ends up
being scheduled on a different cpu, it will be okay.  This actually is
about the same problem as how to support singlethread workqueue.  I'm
not entirely sure how to choose the cpu for such works yet.

> One possible problem with assuming that you can no longer access the work item
> after you call the execution function, is that it's slightly dodgy to retain
> the pointer to it to prevent reentry as the item can be destroyed, reallocated
> and queued before the execution function returns.

All the above is already implemented to avoid running the same work
parallelly on the same cpu and flushing.

>> Binding is usually beneficial and doesn't matter for IO intensive
>> ones, so...
> 
> The scenario I'm thinking of is this: someone who has an NFS volume cached
> through FS-Cache does a tar of a large tree of files (say a kernel source
> tree).  FS-Cache adds a long duration work item for each of those files
> (~32000) to create structure in the cache.  Will all of those wind up bound to
> the same CPU as was running tar?

Yeap, something to think about.  I considered adding a cpu workqueue
which isn't bound to any cpu to serve that type of workload but it
seemed too complex for the problem.  Maybe simple round robin with
per-cpu throttling should do the trick?

Thanks.

-- 
tejun