cmwq and dm-crypt devices? (was: Re: md: dm-crypt: Add option to re-use a new global work-queue.)

[Mike Snitzer <snitzer@redhat.com>]

Hi,

On Tue, Apr 27 2010 at  4:58pm -0400,
San Mehat <san@google.com> wrote:

> *ping* Any word on my previous counter-proposal? Shall I prepare
> another patch for consideration?

The new concurrency managed workqueues (cmwq) that went in to 2.6.36
_should_ hopefully obviate the need for the patch you proposed:
https://patchwork.kernel.org/patch/94179/

Some background on cmwq:
Documentation/workqueue.txt
http://lwn.net/Articles/403891/

We on the DM team haven't explored the impact cmwq has on dm-crypt
devices yet so more research and testing is needed here.  But it'd be
nice to have a hypothesis on how much cmwq will help us solve the
our dm-crypt goals "for free".

[Cc'ing Tejun]

Tejun,

Your insight on how dm-crypt should be using cmwq to achieve the
following conflicting goals would be appreciated:

1) scale down the number of workqueue threads associated with N devices
   (w/ 2 workqueue threads per device) so that the number of threads is
   reasonable ("reasonable" is TBD but I'd imagine it doesn't buy us a
   lot to have 2 single thread workqueues dedicated to each dm-crypt
   device).

   [seems dm-crypt will already get this "for free" using
    create_singlethread_workqueue's WQ_UNBOUND?]

2) scale up the number of workqueue threads used for a single dm-crypt
   device so that a device can realize per-cpu concurrency (to address
   Andi's scalability concerns: https://patchwork.kernel.org/patch/244031/)

   [the desired locality is currently missing due to dm-crypt's current
    use of WQ_UNBOUND; so it is clear the way the workqueues are created
    will be important]

Regards,
Mike