Re: [PATCH v3 0/3] epoll: introduce round robin wakeup mode

[Ingo Molnar <mingo@kernel.org>]

* Jason Baron <jbaron@akamai.com> wrote:

> 2) We are using the wakeup in this case to 'assign' work more 
> permanently to the thread. That is, in the case of a listen socket 
> we then add the connected socket to the woken up threads local set 
> of epoll events. So the load persists past the wake up. And in this 
> case, doing the round robin wakeups, simply allows us to access more 
> cpu bandwidth. (I'm also looking into potentially using cpu affinity 
> to do the wakeups as well as you suggested.)

So this is the part that I still don't understand.

What difference does LIFO versus FIFO wakeups make to CPU utilization: 
a thread waiting for work is idle, no matter whether it ran most 
recently or least recently.

Once an idle worker thread is woken it will compute its own work, for 
whatever time it needs to, and won't be bothered by epoll again until 
it finished its work and starts waiting again.

So regardless the wakeup order it's the same principal bandwidth 
utilization, modulo caching artifacts [*] and modulo scheduling 
artifacts [**]:

[*]  Caching artifacts: in that sense Andrew's point stands: given 
     multiple equivalent choices it's more beneficial to pick a thread 
     that was most recently used (and is thus most cache-hot - i.e. 
     the current wakeup behavior), versus a thread that was least 
     recently used (and is thus the most cache-cold - i.e. the 
     round-robin wakeup you introduce).

[**] The hack patch I posted in my previous reply.

Thanks,

	Ingo