Re: [RFC] NUMA futex hashing

Eric Dumazet
2006-08-08T17:08:21Z

On Tuesday 08 August 2006 18:58, Ulrich Drepper wrote:
> On 8/8/06, Eric Dumazet <dada1@cosmosbay.com> wrote:
> > So we really can... but for 'private futexes' which are the vast majority
> > of futexes needed by typical program (using POSIX pshared thread mutex
> > attribute PTHREAD_PROCESS_PRIVATE, currently not used by NPTL glibc)
>
> Nonsense. Mutexes are by default always private. They explicitly
> have to be marked as sharable. This happens using the
> pthread_mutexattr_setpshared function which takes
> PTHREAD_PROCESS_PRIVATE or PTHREAD_PROCESS_SHARED in the second
> parameter. So the former _is_ clearly used.
>

I was saying that PTHREAD_PROCESS_PRIVATE or PTHREAD_PROCESS_SHARED info is
not provided to the kernel (because futex api/implementation dont need to).
It was not an attack on glibc.

> > Of course we would need a new syscall, and to change glibc to be able to
> > actually use this new private_futex syscall.
>
> No, why? The kernel already does recognize private mutexes. It just
> checks whether the pages used to store it are private or mapped. This
> requires some interaction with the memory subsystem but as long as no
> crashes happen the data can change underneath. It's the program's
> fault if it does.

But if you let futex code doing the vma walk to check the private/shared
status, you still need the mmap_sem locking.

Moreover, a program can mmap() a file (shared in terms of VMA), and continue
to use a PTHREAD_PROCESS_PRIVATE mutex lying in this shared zone
(Example : shmem or hugetlb mapping, wich API might always give a 'shared'
vma)

>
> On the waker side you would search the local futex hash table/tree
> first and if this doesn't yield a match, search the global table.
> Wakeup calls without any waiters are usually rare.

If the two searches touch two different cache lines in the hash table, we
might have a performance regression.
Of course we might chose a hash function so that the same slot is accessed.

Eric