Re: [patch 00/13] Syslets, "Threadlets", generic AIO support, v3

Ingo Molnar
2007-02-28T23:20:23Z

* Linus Torvalds <torvalds@linux-foundation.org> wrote:

> So I would repeat my call for getting rid of the atoms, and instead
> just do a "single submission" at a time. Do the linking by running a
> threadlet that has user space code (and the stack overhead), which is
> MUCH more flexible. And do nonlinked single system calls without
> *either* atoms *or* a user-space stack footprint.

I agree that threadlets are much more flexible - and they might in fact
win in the long run due to that.

i'll add a one-shot syscall API in v6 and then we'll be able to see them
side by side. (wanted to do that in v5 but it got delayed by x86_64
issues, x86_64's entry code is certainly ... tricky wrt. ptregs saving)

wrt. one-shot syscalls, the user-space stack footprint would still
probably be there, because even async contexts that only do single-shot
processing need to drop out of kernel mode to handle signals. We could
probably hack the signal routing code to never deliver to such threads
(but bounce it over to the head context, which is always available) but
i think that would be a bit messy. (i dont exclude it though)

I think syslets might also act as a prototyping platform for new system
calls. If any particular syslet atom string comes up more frequently
(and we could even automate the profiling of that within the kernel),
then it's a good candidate for a standalone syscall. Currently we dont
have such information in any structured way: the connection between
streams of syscalls done by applications is totally opaque to the
kernel.

Also, i genuinely believe that to be competitive (performance-wise) with
fully in-kernel queueing solutions, we need syslets - the syslet NULL
overhead is 20 cycles (this includes copying, engine overhead, etc.),
the syscall NULL overhead is 280-300 cycles. It could probably be made
more capable by providing more special system calls like sys_upcall() to
execute a user-space function. (that way a syslet could still execute
user-space code without having to exit out of kernel mode too
frequently) Or perhaps a sys_x86_bytecode() call, that would execute a
pre-verified, kernel-stored sequence of simplified x86 bytecode, using
the kernel stack.

My fear is that if we force all these things over to one-shot syscalls
or threadlets then this will become another second-tier mechanism. By
providing syslets we give the message: "sure, come on and play within
the kernel if you want to, but it's not easy".

Ingo