Re: [patch] cpufreq: mark cpufreq_tsc() as core_initcall_sync

[Jens Axboe <jens.axboe@oracle.com>]

On Sun, Nov 19 2006, Paul E. McKenney wrote:
> On Sat, Nov 18, 2006 at 09:46:24PM +0300, Oleg Nesterov wrote:
> > On 11/17, Paul E. McKenney wrote:
> > >
> > > Oleg, any thoughts about Jens's optimization?  He would code something
> > > like:
> > >
> > > 	if (srcu_readers_active(&my_srcu))
> > > 		synchronize_srcu();
> > > 	else
> > > 		smp_mb();
> > 
> > Well, this is clearly racy, no? I am not sure, but may be we can do
> > 
> > 	smp_mb();
> > 	if (srcu_readers_active(&my_srcu))
> > 		synchronize_srcu();
> > 
> > in this case we also need to add 'smp_mb()' into srcu_read_lock() after
> > 'atomic_inc(&sp->hardluckref)'.
> > 
> > > However, he is doing ordered I/O requests rather than protecting data
> > > structures.
> > 
> > Probably this makes a difference, but I don't understand this.
> 
> OK, one hypothesis here...
> 
> 	The I/Os must be somehow explicitly ordered to qualify
> 	for I/O-barrier separation.  If two independent processes
> 	issue I/Os concurrently with a third process doing an
> 	I/O barrier, the I/O barrier is free to separate the
> 	two concurrent I/Os or not, on its whim.
> 
> Jens, is the above correct?  If so, what would the two processes

That's completely correct, hence my somewhat relaxed approach with SRCU.

> need to do in order to ensure that their I/O was considered to be
> ordered with respect to the I/O barrier?  Here are some possibilities:

If we consider the barrier a barrier in a certain stream of requests,
it is the responsibility of the issuer of that barrier to ensure that
the queueing is ordered. So if two "unrelated" streams of requests with
barriers hit __make_request() at the same time, we don't go to great
lengths to ensure who gets there firt.

> 1.	I/O barriers apply only to preceding and following I/Os from
> 	the process issuing the I/O barrier.
> 
> 2.	As for #1 above, but restricted to task rather than process.
> 
> 3.	I/O system calls that have completed are ordered by the
> 	barrier to precede I/O system calls that have not yet
> 	started, but I/O system calls still in flight could legally
> 	land on either side of the concurrently executing I/O
> 	barrier.
> 
> 4.	Something else entirely?
> 
> Given some restriction like one of the above, it is entirely possible
> that we don't even need the memory barrier...

3 is the closest. The request queue doesn't really know the scope of the
barrier, it has to rely on the issuer getting it right. If you have two
competing processes issuing io and process A relies on process B issuing
a barrier, they have to synchronize that between them. Normally that is
not a problem, since that's how the file systems always did io before
barriers on items that need to be on disk (it was a serialization point
anyway, it's just a stronger one now).

That said, I think the

        smp_mb();
        if (srcu_readers_active(sp))
                synchronize_srcu();

makes the most sense.

-- 
Jens Axboe