Re: [patch] cpufreq: mark cpufreq_tsc() as core_initcall_sync

["Paul E. McKenney" <paulmck@us.ibm.com>]

On Thu, Nov 23, 2006 at 05:59:10PM +0300, Oleg Nesterov wrote:
> (Sorry, responding to the wrong message)
> 
> Paul E. McKenney wrote:
> >
> > I am concerned about this as well, and am beginning to suspect that I
> > need to make a special-purpose primitive specifically for Jens that he
> > can include with his code.
> 
> How about this?

For Jens, it might be OK.  For general use, I believe that this has
difficulties with the sequence of events I sent out on November 20th, see:

http://marc.theaimsgroup.com/?l=linux-kernel&m=116397154808901&w=2

Might also be missing a few memory barriers, see below.

> 	struct xxx_struct {
> 		int completed;
> 		atomic_t ctr[2];
> 		struct mutex mutex;
> 		wait_queue_head_t wq;
> 	};
> 
> 	void init_xxx_struct(struct xxx_struct *sp)
> 	{
> 		sp->completed = 0;
> 		atomic_set(sp->ctr + 0, 1);	// active
> 		atomic_set(sp->ctr + 1, 0);	// inactive
> 		mutex_init(&sp->mutex);
> 		init_waitqueue_head(&sp->wq);
> 	}
> 
> 	int xxx_read_lock(struct xxx_struct *sp)
> 	{
> 		for (;;) {
> 			int idx = sp->completed & 0x1;
> 			if (likely(atomic_inc_not_zero(sp->ctr + idx)))

Need an after-atomic-inc memory barrier here?

> 				return idx;
> 		}
> 	}
> 
> 	void xxx_read_unlock(struct xxx_struct *sp, int idx)
> 	{

Need a before-atomic-dec memory barrier here?

> 		if (unlikely(atomic_dec_and_test(sp->ctr + idx)))
> 			wake_up(&sp->wq);
> 	}
> 
> 	void synchronize_xxx(struct xxx_struct *sp)
> 	{
> 		int idx;
> 
> 		mutex_lock(&sp->mutex);
> 
> 		idx = ++sp->completed & 0x1;
> 		smp_mb__before_atomic_inc();
> 		atomic_inc(&sp->ctr + idx);
> 
> 		idx = !idx;
> 		if (!atomic_dec_and_test(&sp->ctr + idx))
> 			__wait_event(&sp->wq, !atomic_read(&sp->ctr + idx));

I don't understand why an unlucky sequence of events mightn't be able
to hang this __wait_event().  Suppose we did the atomic_dec_and_test(),
then some other CPU executed xxx_read_unlock(), finding no one to awaken,
then we execute the __wait_event()?  What am I missing here?

> 
> 		mutex_unlock(&sp->mutex);
> 	}
> 
> Yes, cache thrashing... But I think this is hard to avoid if we want writer
> to be fast.
> 
> I do not claim this is the best solution, but for some reason I'd like to
> suggest something that doesn't need synchronize_sched(). What do you think
> about correctness at least?

The general approach seems reasonable, but I do have the concerns above.

						Thanx, Paul