Slow signal delivery to server process with heavy I/O

[Dallas Clement <dallas.a.clement@gmail.com>]

Hi All,

I've noticed that asynchronous signals such as SIGINT, SIGTERM etc are
delivered to my process long after the signal is sent if the receiving
process is handling lots of I/O.  My process is a multi-threaded web
server.  It's got one thread waiting on 'select' to accept incoming
connections and a thread pool which reads the data with 'recv'.

When I batter the web server with incoming traffic and I try to
shutdown the server by sending a SIGINT or SIGTERM, I have observed
that the web server finishes handling the incoming traffic before the
kernel dispatches the signal to the process.  It appears that the
'select' and 'recv' calls are getting highest priority with regard to
scheduling.

I realize this test may appear unnatural and is perhaps unrealistic,
but I would like to be able to shutdown my server gracefully within a
reasonable amount of time, no matter what kind of load it is handling.
 Don't want to have to wait several minutes for my signals to get
handled under heavy load.  Could someone please explain why signal
delivery is slow under these conditions?

Thanks in advance,

Dallas