spinlock which can morph into a mutex

spinlock which can morph into a mutex

[Miquel van Smoorenburg]

I'm trying to implement a dynamically resizable hashtable, and
I have found that after resizing the table I need to call
synchronize_rcu() and finish up before letting other writers
(inserts, deletes) access the table.

Ofcourse during the hashtable update a spinlock is held to
exclude the other writers. But I cannot hold this spinlock over
synchronize_rcu(), yet the other writers still need to be excluded.

So I probably need a mutex instead of a spinlock, but I want to
keep minimal overhead for the common case (when no resizing is in
progress).  I think I need a spinlock that can morph into a mutex ..

I was thinking about using something like the code below.
It is sortof like a spinlock, but it's ofcourse less fair
than actual ticketed spinlocks.

I'm working off 2.6.27 at the moment, but I noticed that in
2.6.28 adaptive spinning was introduced for mutexes. Is the
approach below still worth it with adaptive spinning or could
I just convert the spinlocks to mutexes with minimal extra overhead ?

Example code:

int real_mutex_lock = 0; // can use int since mutex ops are barriers
struct mutex mutex;

// 1. used instead of spinlock() [common case]
while (mutex_trylock(&mutex) == 0) {
	if (real_mutex_lock) {
		mutex_lock(&mutex);
		break;
	}
}
.. have lock, do work
mutex_unlock(&mutex);


// 2. When we want to lock and be able to sleep [seldomly used]
mutex_lock(&mutex);
real_mutex_lock = 1;
smp_wmb();

.. do work ..
real_mutex_lock = 0;
mutex_unlock(&mutex);

Mike.

Re: spinlock which can morph into a mutex

[Andi Kleen]

Miquel van Smoorenburg <miquels@cistron.nl> writes:
>
> So I probably need a mutex instead of a spinlock, but I want to
> keep minimal overhead for the common case (when no resizing is in
> progress).  I think I need a spinlock that can morph into a mutex ..

The standed mutexes already do that by themselves.

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.

Re: spinlock which can morph into a mutex

[Peter Zijlstra]

On Fri, 2009-12-18 at 15:30 +0100, Miquel van Smoorenburg wrote:

> I just convert the spinlocks to mutexes with minimal extra overhead ?

Yep.

Re: spinlock which can morph into a mutex

[Thomas Gleixner]

On Fri, 18 Dec 2009, Miquel van Smoorenburg wrote:

> I'm trying to implement a dynamically resizable hashtable, and
> I have found that after resizing the table I need to call
> synchronize_rcu() and finish up before letting other writers
> (inserts, deletes) access the table.
> 
> Ofcourse during the hashtable update a spinlock is held to
> exclude the other writers. But I cannot hold this spinlock over
> synchronize_rcu(), yet the other writers still need to be excluded.
> 
> So I probably need a mutex instead of a spinlock, but I want to
> keep minimal overhead for the common case (when no resizing is in
> progress).  I think I need a spinlock that can morph into a mutex ..

Is the writer frequency and the possible contention so high that you
need a spinlock at all ?
 
> I was thinking about using something like the code below.
> It is sortof like a spinlock, but it's ofcourse less fair
> than actual ticketed spinlocks.
> 
> I'm working off 2.6.27 at the moment, but I noticed that in
> 2.6.28 adaptive spinning was introduced for mutexes. Is the
> approach below still worth it with adaptive spinning or could
> I just convert the spinlocks to mutexes with minimal extra overhead ?

Test it :)

If the mutex is still to heavy weight for you, then you can solve it
without implementing another weird concurrency control:

writer side:

       spin_lock(&hash_lock);

       if (unlikely(hash_update_active)) {
       	  spin_unlock(&hash_lock);
       	  wait_event_(un)interruptible(&hash_wq, !hash_update_active);
	  spin_lock(&hash_lock);
       }

resize side:

       spin_lock(&hash_lock);
       hash_update_active = 1;
       ....
       spin_unlock(&hash_lock);
       synchronize_rcu();
       hash_update_active = 0;
       wake_up(&hash_wq);

Thanks,

	tglx

Re: spinlock which can morph into a mutex

[Miquel van Smoorenburg]

On Fri, 2009-12-18 at 18:14 +0100, Thomas Gleixner wrote:
> On Fri, 18 Dec 2009, Miquel van Smoorenburg wrote:
>  I think I need a spinlock that can morph into a mutex ..
> 
> Is the writer frequency and the possible contention so high that you
> need a spinlock at all ?

Possibly - I don't want to degrade the performance of existing code
(which uses a spinlock).
 
> Test it :)

Good point.

> If the mutex is still to heavy weight for you, then you can solve it
> without implementing another weird concurrency control:
> 
>        	  wait_event_(un)interruptible(&hash_wq, !hash_update_active);
>
>        hash_update_active = 1;
>        ....
>        hash_update_active = 0;
>        wake_up(&hash_wq);

Ah, ofcourse. Thanks for pointing that out!

Thanks everybody for your input. I gained quite a bit of insight.

Mike.