Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

[Suparna Bhattacharya <suparna@in.ibm.com>]

On Thu, Feb 22, 2007 at 03:36:58PM +0100, Ingo Molnar wrote:
> 
> * Suparna Bhattacharya <suparna@in.ibm.com> wrote:
> 
> > > maybe it will, maybe it wont. Lets try? There is no true difference 
> > > between having a 'request structure' that represents the current 
> > > state of the HTTP connection plus a statemachine that moves that 
> > > request between various queues, and a 'kernel stack' that goes in 
> > > and out of runnable state and carries its processing state in its 
> > > stack - other than the amount of RAM they take. (the kernel stack is 
> > > 4K at a minimum - so with a million outstanding requests they would 
> > > use up 4 GB of RAM. With 20k outstanding requests it's 80 MB of RAM 
> > > - that's acceptable.)
> > 
> > At what point are the cachemiss threads destroyed ? In other words how 
> > well does this adapt to load variations ? For example, would this 80MB 
> > of RAM continue to be locked down even during periods of lighter loads 
> > thereafter ?
> 
> you can destroy them at will from user-space too - just start a slow 
> timer that zaps them if load goes down. I can add a 
> sys_async_thread_exit(nr_threads) API to be able to drive this without 
> knowing the TIDs of those threads, and/or i can add a kernel-internal 
> mechanism to zap inactive threads. It would be rather easy and 
> low-overhead - the v2 code already had a max_nr_threads tunable, i can 
> reintroduce it. So the size of the pool of contexts does not have to be 
> permanent at all.

If you can find a way to do this without additional tunables burden on
the administrator that would certainly help ! IIRC, performance problems
linked to having too many or too few AIO kernel threads has been a commonly
reported issue elsewhere - it would be nice to be able to avoid repeating
the crux of that (mistake) in Linux. To me, any need to manually tune the
number has always seemed to defeat the very benefit of adaptability of varying
loads that AIO intrinsically provides. 

Regards
Suparna

> 
> 	Ingo

-- 
Suparna Bhattacharya (suparna@in.ibm.com)
Linux Technology Center
IBM Software Lab, India