Re: [RFC] Userspace RCU: (ab)using futexes to save cpu cycles and energy

Paul E. McKenney
2009-10-04T01:54:02Z

On Wed, Sep 23, 2009 at 01:48:20PM -0400, Mathieu Desnoyers wrote:
> Hi,
>
> When implementing the call_rcu() "worker thread" in userspace, I ran
> into the problem that it had to be woken up periodically to check if
> there are any callbacks to execute. However, I easily imagine that this
> does not fit well with the "green computing" definition.
>
> Therefore, I've looked at ways to have the call_rcu() callers waking up
> this worker thread when callbacks are enqueued. However, I don't want to
> take any lock and the fast path (when no wake up is required) should not
> cause any cache-line exchange.
>
> Here are the primitives I've created. I'd like to have feedback on my
> futex use, just to make sure I did not do any incorrect assumptions.
>
> This could also be eventually used in the QSBR Userspace RCU quiescent
> state and in mb/signal userspace RCU when exiting RCU read-side C.S. to
> ensure synchronize_rcu() does not busy-wait for too long.
>
> /*
> * Wake-up any waiting defer thread. Called from many concurrent threads.
> */
> static void wake_up_defer(void)
> {
> if (unlikely(atomic_read(&defer_thread_futex) == -1))
> atomic_set(&defer_thread_futex, 0);
> futex(&defer_thread_futex, FUTEX_WAKE,
> 0, NULL, NULL, 0);
> }
>
> /*
> * Defer thread waiting. Single thread.
> */
> static void wait_defer(void)
> {
> atomic_dec(&defer_thread_futex);
> if (atomic_read(&defer_thread_futex) == -1)
> futex(&defer_thread_futex, FUTEX_WAIT, -1,
> NULL, NULL, 0);
> }

The standard approach would be to use pthread_cond_wait() and
pthread_cond_broadcast(). Unfortunately, this would require holding a
pthread_mutex_lock across both operations, which would not necessarily
be so good for wake-up-side scalability.

That said, without this sort of heavy-locking approach, wakeup races
are quite difficult to avoid.

Thanx, Paul