Re: Futexes III : performance numbers

[Hubertus Franke <frankeh@watson.ibm.com>]

On Wednesday 06 March 2002 02:54 am, Rusty Russell wrote:
> On Tue, 5 Mar 2002 16:23:14 -0500
>
> Hubertus Franke <frankeh@watson.ibm.com> wrote:
> > Interesting... the strict FIFO ordering of my fast semaphores limits
> > performance as seen by 99.71% contention, so we always ditch
> > into the kernel. Convoy Avoidance locks 2.5 times better.
> > Wohh futex rock, BUT... with 0.29% contention it basically tells
> > me that we are exhausting our entire quantum getting the lock
> > without contention. So their is some serious fairness issue here
> > at least for the tightly scheduled locks. Compare the M numbers
> > for 2 and 3 children.
>
> Fairness <sigh>.  This patch should be much more FIFO: it works by handing
> the mutex straight to the first one on the queue if there is one, and only
> actually "freeing" it if there's noone waiting.
>
> Unfortunately, it seems to hurt performance by 50% on tdbtorture (although
> there are weird scheduler things happening too).
>
> Here's the "fair" patch:
> Rusty.

Rusty why not provide something along the line <snipped all over the place>.

#define FUTEX_DOWN    (0)
#define FUTEX_UP          (1)
#define FUTEX_FAIR_UP (2)

static int futex_up(struct list_head *head, atomic_t *count)
{
       atomic_set(count, 1);
       smp_wmb();
       wake_one_waiter(head, count);
       return 0;
}  

static int futex_fair_up(truct list_head *head, atomic_t *count)
{
       spin_lock(&futex_lock);
       if (!pass_futex(head, count))
               /* Noone to receive: set to one and leave it free. */
               atomic_set(count, 1);
       spin_unlock(&futex_lock);
       return 0;
}


asmlinkage int sys_futex(void *uaddr, int op) 
{
       <..... snip ....>

       head = hash_futex(page, pos_in_page);
       switch (op) {

       case FUTEX_DOWN:
               ret = futex_down(head, page_address(page) + pos_in_page);
               break;

       case FUTEX_UP:
               ret = futex_up(head, page_address(page) + pos_in_page);
               break;
 
       case FUTEX_FAIR_UP:
               ret = futex_fair_up(head, page_address(page) + pos_in_page);
               break;
 
       default :

       <..... snip ....>
}


This would satisfy the fair vs. calock issue, you let the app decide what to 
use. Best of all, it seems to me you can even mix it.
Imagine, if a process knows it will soon reaquire the lock, then it would
use FUTEX_UP to avoid being tagged back to the end of wait queue
avoiding costly scheduling events. At the same time, if the process knows
that its done for a while with that lock, then it issues FUTEX_FAIR_UP.
The best of two worlds..

It also shows how cleanly the code can be expanded in the future.
The more I look at the hash queues the more I like it. 
Will look at the rwlock now. Let me know what you think.

-- 
-- Hubertus Franke  (frankeh@watson.ibm.com)