Re: deadlocks if use htb

Peter Zijlstra
2009-01-15T10:47:15Z

On Thu, 2009-01-15 at 09:01 +0000, Jarek Poplawski wrote:
> On Wed, Jan 14, 2009 at 03:28:03PM +0100, Peter Zijlstra wrote:
> ....
> > Right, found all that...
> >
> > Can't spot anything obviously wrong though.. hrtimer_start*() does
> > remove_hrtimer() which checks STATE_ENQUEUED, STATE_PENDING and pulls it
> > off the relevant list before it continues the enqueue.
> >
> > However a loop in enqueue_hrtimer() would suggest a corrupted RB-tree,
> > but I cannot find an RB-op that doesn't hold base-lock.
> >
>
> I've revisited it yesterday, and if I don't miss something, there is
> possible a scenario similar to this:
>
> cpu1: cpu2:
>
> run_hrtimer_pending
> spin_unlock
> restart = fn(timer)
>
> hrtimer_start
> enqueue_hrtimer
>
> hrtimer_start
> remove_hrtimer
> (the HRTIMER_STATE_CALLBACK is removed)
>
> switch_hrtimer_base
> spin_lock
> (not this hrtimer's anymore)
> __remove_hrtimer
> list_add_tail enqueue_hrtimer
>

(looking at .28 code)

run_hrtimer_pending() reads like:

while (pending timers) {
__remove_hrtimer(timer, HRTIMER_STATE_CALLBACK);
spin_unlock(&cpu_base->lock);

fn(timer);

spin_lock(&cpu_base->lock);
timer->state &= ~HRTIMER_STATE_CALLBACK; // _should_ result in HRTIMER_STATE_INACTIVE
if (HRTIMER_RESTART)
re-queue
else if (timer->state != INACTIVE) {
// so another cpu re-queued this timer _while_ we were executing it.
if (timer is first && !reprogramm) {
__remove_hrtimer(timer, HRTIMER_STATE_PENDING);
list_add_tail(timer, &cb_pending);
}
}
}

So in the window where we drop the lock, one can, as you said, have
another cpu requeue the timer, but the rb_entry and list_entry are free,
so it should not cause the data corruption we're seeing.