Re: [PATCH] make cancel_rearming_delayed_work() reliable

Oleg Nesterov
2007-05-14T19:45:30Z

On 05/13, Tejun Heo wrote:
>
> Oleg Nesterov wrote:
> > Heh. I thought about another bit-in-pointer too. I can't explain this,
> > but I personally hate these bits even more than barriers.
>
> I'm the other way around but it's like saying "I like donkey poo better
> than horse poo". Let's accept that we have different tastes in poos.

Yes sure :)

> >> * run_workqueue() clears the cwq pointer and VALID bit while holding
> >> lock before executing the work.
> >
> > We should not clear cwq. I guess you meant "does list_del() and clears
> > VALID bit".
>
> Yeap, I've used the term 'clearing' as more of a logical term - making
> the pointer invalid, but is there any reason why we can't clear the
> pointer itself?

Because this breaks cancel_work_sync(work), it needs get_wq_data(work) for
wait_on_work(work).

> >> * try_to_grab_pending() checks VALID && pointers equal after grabbing
> >> cwq->lock.
> >
> > We don't even need to check the pointers. VALID bit is always changed
> > under cwq->lock. So, if we see VALID under cwq->lock, we have a right
> > pointer.
>
> But there are multiple cwq->lock's. VALID can be set with one of other
> cwq->lock's locked.

Yes, you are right, thanks.

So, try_to_grab_pending() should check "VALID && pointers equal" atomically.
We can't do "if (VALID && cwq == get_wq_data(work))". We should do something
like this

(((long)cwq) | VALID | PENDING) == atomic_long_read(&work->data)

Yes? I need to think more about this.

> I didn't really get the smp_mb__before_spinlock() thing. How are you
> planning to use it? spinlock() is already a read barrier. What do you
> gain by making it also a write barrier?

As I said, we can shift set_wq_data() from insert_work() to __queue_work(),

static void insert_work(struct cpu_workqueue_struct *cwq,
struct work_struct *work, int tail)
{
if (tail)
list_add_tail(&work->entry, &cwq->worklist);
else
list_add(&work->entry, &cwq->worklist);
wake_up(&cwq->more_work);
}

static void __queue_work(struct cpu_workqueue_struct *cwq,
struct work_struct *work)
{
unsigned long flags;

set_wq_data(work, cwq);

smp_mb_before_spinlock();
spin_lock_irqsave(&cwq->lock, flags);
insert_work(cwq, work, 1);
spin_unlock_irqrestore(&cwq->lock, flags);
}

this needs very minor changes.

BTW, in _theory_, spinlock() is not a read barrier, yes?
>From Documentation/memory-barriers.txt

Memory operations that occur before a LOCK operation may appear to happen
after it completes.

Oleg.