Re: [PATCH] make cancel_rearming_delayed_work() reliable

[Oleg Nesterov <oleg@tv-sign.ru>]

On 05/11, Tejun Heo wrote:
>
> Oleg Nesterov wrote:
> >
> > However, I agree, this smp_wmb() in insert_work() should die. We can
> > introduce "smp_mb__before_spinlock()" (no-op on x86 at least) to kill it.
> 
> Yeah, right, we allow cwq pointer to change without holding the lock.
> Although I still think that is where we should fix the problem.  Taking
> down CPU is a cold cold path.  We can afford a lot of overhead there.
> IMHO, if we can do that, it would be far better than memory barrier
> dance which tends to be difficult to understand and thus prove/maintain
> correctness.  I'll think about it more.

Yes I hate this barrier too. That is why changelog explicitly mentions it.

With some trivial code modifications we can move set_wq_data() from insert_work()
to __queue_work(), then

	void set_wq_data(work, cwq)
	{
		struct cpu_workqueue_struct *old = get_wq_data(work);

		if (likely(cwq == old))
			return;

		if (old)
			spin_lock(old->lock);

		atomic_long_set(&work->data, ...);

		if (old)
			spin_lock(old->lock);
	}

I can't say I like this very much, though. I'd prefer use smp_mb__before_spinlock().
Probably we can do something else.

But first I'd like to kill cwq_should_stop(). (Gautham, Srivatsa, you were
right, but I was blind, deaf, and stupid).

Oleg.