Re: [patch] timer/hrtimer: take per cpu locks in sane order

[Andrew Morton <akpm@linux-foundation.org>]

On Fri, 2 Mar 2007 15:23:08 +0100 Heiko Carstens <heiko.carstens@de.ibm.com> wrote:

> On Fri, Mar 02, 2007 at 02:04:33PM +0100, Ingo Molnar wrote:
> > 
> > * Heiko Carstens <heiko.carstens@de.ibm.com> wrote:
> > 
> > > -	spin_lock(&new_base->lock);
> > > -	spin_lock(&old_base->lock);
> > > +	/*
> > > +	 * If we take a lock from a different cpu, make sure we have always
> > > +	 * the same locking order. That is the lock that belongs to the cpu
> > > +	 * with the lowest number is taken first.
> > > +	 */
> > > +	lock1 = smp_processor_id() < cpu ? &new_base->lock : &old_base->lock;
> > > +	lock2 = smp_processor_id() < cpu ? &old_base->lock : &new_base->lock;
> > > +	spin_lock(lock1);
> > > +	spin_lock(lock2);
> > 
> > looks good to me. Wouldnt this be cleaner via double_lock_timer() - 
> > similar to how double_rq_lock() works in kernel/sched.c - instead of 
> > open-coding it?
> 
> Something like the stuff below? Exploits the knowledge that the two
> tvec_base_t's are in a per_cpu array. Otherwise I would end up passing
> a lot of redundant stuff. But still I think that isn't a good solution
> but rather a hack...?
> I'd go for the patch above.

Yeah, it'd be nicer to pass in the CPU number(s), use that to make the
ordering decision.  Perhaps (smp_processor_id() - cpu).

> ---
> Index: linux-2.6/kernel/timer.c
> ===================================================================
> --- linux-2.6.orig/kernel/timer.c
> +++ linux-2.6/kernel/timer.c
> @@ -1640,6 +1640,28 @@ static void migrate_timer_list(tvec_base
>  	}
>  }
>  
> +static void __devinit double_tvec_lock(tvec_base_t *base1, tvec_base_t *base2)
> +{
> +	if (base1 < base2) {
> +		spin_lock(&base1->lock);
> +		spin_lock(&base2->lock);
> +	} else {
> +		spin_lock(&base2->lock);
> +		spin_lock(&base1->lock);
> +	}
> +}
> +
> +static void __devinit double_tvec_unlock(tvec_base_t *base1, tvec_base_t *base2)
> +{
> +	if (base1 < base2) {
> +		spin_unlock(&base1->lock);
> +		spin_unlock(&base2->lock);
> +	} else {
> +		spin_unlock(&base2->lock);
> +		spin_unlock(&base1->lock);
> +	}
> +}

And to undo the locks in the reverse order from that in which they were
taken.