Re: [PATCH 0/6] ipc/sem.c: performance improvements, FIFO

[Manfred Spraul <manfred@colorfullife.com>]

On 06/14/2013 09:05 PM, Mike Galbraith wrote:
> 32 of 64 cores DL980 without the -rt killing goto again loop removal I
> showed you.  Unstable, not wonderful throughput.
Unfortunately the -rt approach is defintively unstable:
> @@ -285,9 +274,29 @@ static inline int sem_lock(struct sem_ar
>                  * but have to wait for the global lock to be released.
>                  */
>                 if (unlikely(spin_is_locked(&sma->sem_perm.lock))) {
> -                       spin_unlock(&sem->lock);
> - spin_unlock_wait(&sma->sem_perm.lock);
> -                       goto again;
> +                       spin_lock(&sma->sem_perm.lock);
> +                       if (sma->complex_count)
> +                               goto wait_array;
> +
> +                       /*
> +                        * Acquiring our sem->lock under the global lock
> +                        * forces new complex operations to wait for us
> +                        * to exit our critical section.
> +                        */
> +                       spin_lock(&sem->lock);
> +                       spin_unlock(&sma->sem_perm.lock);

Assume there is one op (semctl(), whatever) that acquires the global 
lock - and a continuous stream of simple ops.
- spin_is_locked() returns true due to the semctl().
- then simple ops will switch to spin_lock(&sma->sem_perm.lock).
- since the spinlock is acquired, the next operation will get true from 
spin_is_locked().

It will stay that way around - as long as there is at least one op 
waiting for sma->sem_perm.lock.
With enough cpus, it will stay like this forever.

--
     Manfred