Re: Syslets, Threadlets, generic AIO support, v6

[Ingo Molnar <mingo@elte.hu>]

* Eric Dumazet <dada1@cosmosbay.com> wrote:

> > speedup: i suggested O_ANY 6 years ago as a speedup to Apache - 
> > non-linear fds are cheaper to allocate/map:
> > 
> >   http://www.mail-archive.com/linux-kernel@vger.kernel.org/msg23820.html
> > 
> > (i definitely remember having written code for that too, but i 
> > cannot find that in the archives. hm.) In theory we could avoid 
> > _all_ fd-bitmap overhead as well and use a per-process list/pool of 
> > struct file buffers plus a maximum-fd field as the 'non-linear fd 
> > allocator' (at the price of only deallocating them at process exit 
> > time).
> 
> Only very few apps need to open more than 100.000 files.

yes. I did not list it as a primary reason for private fds, it's just a 
nice side-effect. As long as the other apps are not hurt, i see no 
problem in improving the >100K open files case.

> As these files are likely sockets, O_ANY is not a solution.

why not? It would be a natural thing to extend sys_socket() with a 
'flags' parameter and pass in O_ANY (along with any other possible fd 
parameter like O_NDELAY, which could be inherited over connect()).

> A trick is to try to keep first 64 handles freed, so that kernel wont 
> consume too much cpu time and cache in get_unused_fd()
> 
> http://lkml.org/lkml/2005/9/15/307

this is basically a user-space front-end cache to fd allocation - which 
duplicates data needlessly. I dont see any problem with doing this in 
the kernel. (Also, obviously 'first 64 handles' could easily break with 
certain types of apps so glibc cannot do this.)

	Ingo