first little problem with private futexes

first little problem with private futexes

[Ulrich Drepper]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Here's a first little issue with private futex I came across.  But a
real bug but a hole.

When we use clone() with CLONE_CHILD_CLEARTID possible waiters are woken
upon termination of the thread.  This operation uses FUTEX_WAKE so far.
 But it in almost all cases local memory and I would even be in favor of
setting this into stone.   It wouldn't break anything I know of.

The problem is we cannot just go over to using
FUTEX_WAIT|FUTEX_PRIVATE_FLAG since this would break binaries using any
glibc out there so far.

There are three ways out of this I can see:

1.  do nothing, always use the shared futexes.  Not very attractive IMO

2.  try private futexes first, then shared one.  This is even less
    attractive since in the many cases there is no waiter and we cannot
    determine whether the private futex notification succeeded and we're
    doing the expensive work as well

3.  tell the kernel whether we want the new or the old notification.
    This can be done using a number of ways

    a) using some prctl().  Another unconditional syscall, not nice.

    b) using a new CLONE_* flag.  We have currently 5 bits left and can
       recover two more (CLONE_DETACHED, CLONE_STOPPED).  And we can
       invent ways to add more bits.


I'm in favor of 3b but if somebody argues the costs are not justified
because the effects of using the shared futex notification isn't high
enough I can accept that, too.

- --
➧ Ulrich Drepper ➧ Red Hat, Inc. ➧ 444 Castro St ➧ Mountain View, CA ❖
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (GNU/Linux)
Comment: Using GnuPG with Fedora - http://enigmail.mozdev.org

iD8DBQFGUJXl2ijCOnn/RHQRAvkOAJsEmm+TiWlWRJvT5nbk0lXrpvpTaQCgo/5j
FPWYxtgUIZwrdFk/K79dIi8=
=kbfx
-----END PGP SIGNATURE-----

Re: first little problem with private futexes

[Eric Dumazet]

Ulrich Drepper a écrit :
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> Here's a first little issue with private futex I came across.  But a
> real bug but a hole.
> 
> When we use clone() with CLONE_CHILD_CLEARTID possible waiters are woken
> upon termination of the thread.  This operation uses FUTEX_WAKE so far.
>  But it in almost all cases local memory and I would even be in favor of
> setting this into stone.   It wouldn't break anything I know of.
> 
> The problem is we cannot just go over to using
> FUTEX_WAIT|FUTEX_PRIVATE_FLAG since this would break binaries using any
> glibc out there so far.
> 
> There are three ways out of this I can see:
> 
> 1.  do nothing, always use the shared futexes.  Not very attractive IMO

Why do you find this non attractive ?

How is it performance critical ?

If a program is stupid enough to create/destroy many threads per second, I 
doubt it relies on a faster thread termination :)

> 
> 2.  try private futexes first, then shared one.  This is even less
>     attractive since in the many cases there is no waiter and we cannot
>     determine whether the private futex notification succeeded and we're
>     doing the expensive work as well
> 
> 3.  tell the kernel whether we want the new or the old notification.
>     This can be done using a number of ways
> 
>     a) using some prctl().  Another unconditional syscall, not nice.
> 
>     b) using a new CLONE_* flag.  We have currently 5 bits left and can
>        recover two more (CLONE_DETACHED, CLONE_STOPPED).  And we can
>        invent ways to add more bits.
> 
> 
> I'm in favor of 3b but if somebody argues the costs are not justified
> because the effects of using the shared futex notification isn't high
> enough I can accept that, too.

Re: first little problem with private futexes

[Ulrich Drepper]

On 5/20/07, Eric Dumazet <dada1@cosmosbay.com> wrote:
> > 1.  do nothing, always use the shared futexes.  Not very attractive IMO
>
> Why do you find this non attractive ?
>
> How is it performance critical ?

You should know better than any other that the problem is not that the
problem itself is the only one affected.  If threads terminate all
other programs and threads are affected since the global locks for the
shared futexes are needed.  That's the case I'm concerned about.  It's
not really about a single app creating many many threads over and over
again.  It's about many apps which do use threads (and that number
will have to rise) starts and stop threads at a reasonable rate.  It's
just one more unnecessary point of contact between concurrently
running apps.

Re: first little problem with private futexes

[Eric Dumazet]

Ulrich Drepper a écrit :
> On 5/20/07, Eric Dumazet <dada1@cosmosbay.com> wrote:
>> > 1.  do nothing, always use the shared futexes.  Not very attractive IMO
>>
>> Why do you find this non attractive ?
>>
>> How is it performance critical ?
> 
> You should know better than any other that the problem is not that the
> problem itself is the only one affected.  If threads terminate all
> other programs and threads are affected since the global locks for the
> shared futexes are needed.  That's the case I'm concerned about.  It's
> not really about a single app creating many many threads over and over
> again.  It's about many apps which do use threads (and that number
> will have to rise) starts and stop threads at a reasonable rate.  It's
> just one more unnecessary point of contact between concurrently
> running apps.

Well, current private futex code still use global locks (one common hash table 
were all waited futexes are queued, private or shared)

'Only' mmap_sem and inode/mm refcounter inc/dec are avoided.

My proposal of having separate namespace was hold, in order to get the 
'private futexes' accepted in kernel.

So for the moment, I am not sure glibc should try to optimize CLEARTID operation.