Re: [patch] cpufreq: mark cpufreq_tsc() as core_initcall_sync

["Paul E. McKenney" <paulmck@us.ibm.com>]

On Mon, Nov 20, 2006 at 12:17:31AM +0300, Oleg Nesterov wrote:
> On 11/19, Alan Stern wrote:
> > On Sun, 19 Nov 2006, Oleg Nesterov wrote:
> >
> > > > What happens if synchronize_xxx manages to execute inbetween
> > > > xxx_read_lock's
> > > >
> > > >  		idx = sp->completed & 0x1;
> > > >  		atomic_inc(sp->ctr + idx);
> > > >
> > > > statements?
> > >
> > > Oops. I forgot about explicit mb() before sp->completed++ in synchronize_xxx().
> > >
> > > So synchronize_xxx() should do
> > >
> > > 	smp_mb();
> > > 	idx = sp->completed++ & 0x1;
> > >
> > > 	for (;;) { ... }
> > >
> > > >               You see, there's no way around using synchronize_sched().
> > >
> > > With this change I think we are safe.
> > >
> > > If synchronize_xxx() increments ->completed in between, the caller of
> > > xxx_read_lock() will see all memory ops (started before synchronize_xxx())
> > > completed. It is ok that synchronize_xxx() returns immediately.
> >
> > Yes, the reader will see a consistent picture, but it will have
> > incremented the wrong element of sp->ctr[].  What happens if another
> > synchronize_xxx() occurs while the reader is still running?
> 
> It will wait for xxx_read_unlock() on reader's side. And for this reason
> this idx in fact is not exactly wrong :)

I am not seeing this.

Let's assume sp->completed starts out zero.

o	CPU 0 starts executing xxx_read_lock(), but is interrupted
	(or whatever) just before the atomic_inc().  Upon return,
	it will increment sp->ctr[0].

o	CPU 1 executes synchronize_xxx() to completion, which it
	can because CPU 0 has not yet incremented the counter.
	It waited on sp->ctr[0], and incremented sp->completed to 1.

o	CPU 0 returns from interrupt and completes xxx_read_lock(),
	but has incremented sp->ctr[0].

o	CPU 0 continues into its critical section, picking up a
	pointer to an xxx-protected data structure (or, in Jens's
	case starting an xxx-protected I/O).

o	CPU 1 executes another synchronize_xxx().  This completes
	immediately because it is waiting for sp->ctr[1] to go
	to zero, but CPU 0 incremented sp->ctr[0].  (Right?)

o	CPU 1 continues, either freeing a data structure while
	CPU 0 is still referencing it, or, in Jens's case, completing
	an I/O barrier while there is still outstanding I/O.

Or am I missing something?

						Thanx, Paul