re: kqueue, kevent - kernel event notification mechanism

re: kqueue, kevent - kernel event notification mechanism

[Dan Kegel]

Carlo Wood <carlo@alinoe.com> wrote:
> are there any plans to implement the kqueue, kevent system calls
> in the linux kernel?
> 
> There is no substitute for them currently; select() and poll()
> both suffer from dramatic cpu usage when a daemon is loaded with
> a few thousand clients.

There is already an efficient mechanism - although with a very
different interface - in the 2.4.x kernel: edge triggered readiness
notification via realtime signals.  This matches one style of
kqueue()/kevent() usage.  For those who prefer level-triggered
readiness notification (more like what poll() gives you),
I have written a wrapper class that papers over the difference
efficiently, at the cost of making the app tell my class about
all the EWOULDBLOCKs it gets.
See http://www.kegel.com/c10k.html#nb.sigio for details.
I don't particularly like signals for this, as they're a
bit heavyweight, and the global signal queue is a bit precious,
but it does benchmark well, even for thousands of clients.

(That page also describes kqueue etc. and the experimental
/dev/poll drivers.)
- Dan

Re: kqueue, kevent - kernel event notification mechanism

[Carlo Wood]

On Wed, Dec 05, 2001 at 11:33:06PM -0800, Dan Kegel wrote:
> See http://www.kegel.com/c10k.html#nb.sigio for details.

Wow!  Thanks for writing that *wonderful* page!!!  That is
exactly the documentation I was looking for to improve my
'An Object Oriented Core library for Event Driven Applications'
library (http://libcw.sourceforge.net/), a library intended
for Heavy Duty networking applications that I've been working
on for the past few years.

Thanks!

-- 
Carlo Wood <carlo@alinoe.com>

kqueue, kevent - kernel event notification mechanism

[Carlo Wood]

Hiya,

are there any plans to implement the kqueue, kevent system calls
in the linux kernel?

There is no substitute for them currently; select() and poll()
both suffer from dramatic cpu usage when a daemon is loaded with
a few thousand clients.

It would be good for linux to implement event-driven i/o, if not
by means of kqueue/kevent (BSD-ish interface) then at least in
some other way.

It has been shown that a heavily loaded networking daemon like
an IRC daemon uses 35% less cpu on freeBSD by using event-driven
i/o instead of poll().

For those who are not into this material, allow me to explain
the difference.  The current i/o calls, select and poll, scan all
open file descriptors for events, every call.  This is clearly
not scalable.  Event driven means that the kernel never scans
all open file descriptors but queues events as they happen on
the network card - the latter would in principle be scalable
to any number of open file descriptors, as long as there is
enough memory and as long as there is a limited number of
file descriptors active per time unit.

-- 
Carlo Wood <carlo@alinoe.com>

Re: kqueue, kevent - kernel event notification mechanism

[Alan Cox]

> are there any plans to implement the kqueue, kevent system calls
> in the linux kernel?

Not that I know of

> It would be good for linux to implement event-driven i/o, if not
> by means of kqueue/kevent (BSD-ish interface) then at least in
> some other way.

The standards way is asynchronous io (aio_* interfaces). That is being
worked on

> the difference.  The current i/o calls, select and poll, scan all
> open file descriptors for events, every call.  This is clearly
> not scalable.  Event driven means that the kernel never scans

The API isnt directly the problem. In fact you can make a fine scalable select
by implementing

	poll_setup(..............)
	poll_add/poll_remove
	poll_wait

as multiple calls giving basically the same interface that apps expected
anyway.  Take a look at the various /dev/poll experimental interfaces and
bits of code.

Alan

Re: kqueue, kevent - kernel event notification mechanism

[Benjamin LaHaise]

On Thu, Dec 06, 2001 at 12:57:21AM +0000, Alan Cox wrote:
> The API isnt directly the problem. In fact you can make a fine scalable select
> by implementing
> 
> 	poll_setup(..............)
> 	poll_add/poll_remove
> 	poll_wait
> 
> as multiple calls giving basically the same interface that apps expected
> anyway.  Take a look at the various /dev/poll experimental interfaces and
> bits of code.

My aio patches already have a poll operation, however it acts as a one shot 
operation: an async poll does not complete until the state indicates 
readiness or it is cancelled.  That's needed as there is a 1-1 relationship 
between submitted aio operations and the space in the completion ring.  Still, 
it looks like it will work quite nicely as a means of accelerating exiting 
poll() based servers.

		-ben

		-ben