Re: [patch 1/2] Linux Kernel Markers - Support Multiple Probes

["Paul E. McKenney" <paulmck@linux.vnet.ibm.com>]

On Thu, Dec 20, 2007 at 09:25:40AM -0500, Mathieu Desnoyers wrote:
> * Paul E. McKenney (paulmck@linux.vnet.ibm.com) wrote:
> > On Tue, Dec 04, 2007 at 02:45:06PM -0500, Mathieu Desnoyers wrote:
> > > * Andrew Morton (akpm@linux-foundation.org) wrote:
> > > > On Tue, 4 Dec 2007 14:21:00 -0500
> > > > Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> wrote:
> > > > 
> > > > > > > + */
> > > > > > > +void marker_probe_cb(const struct marker *mdata, void *call_private,
> > > > > > > +	const char *fmt, ...)
> > > > > > > +{
> > > > > > > +	va_list args;
> > > > > > > +	char ptype;
> > > > > > > +
> > > > > > > +	preempt_disable();
> > > > > > 
> > > > > > What are the preempt_disable()s doing in here?
> > > > > > 
> > > > > > Unless I missed something obvious, a comment is needed here (at least).
> > > > > > 
> > > > > 
> > > > > They make sure the teardown of the callbacks can be done correctly when
> > > > > they are in modules and they insure RCU read coherency. Will add
> > > > > comment.
> > > > 
> > > > So shouldn't it be using rcu_read_lock()?  If that does not suit, should we
> > > > be adding new rcu primitives rather than open-coding and adding dependencies?
> > > 
> > > Hrm, yes, good point. Since there seems to be extra magic under
> > > __acquire(RCU);  and  rcu_read_acquire();, the the fact that I use
> > > rcu_barrier() for synchronization, we should. I'll change it.
> > 
> > (Sorry to show up so late...  It has been a bit crazy of late...)
> > 
> > The __acquire(RCU) and rcu_read_acquire() are strictly for the benefit
> > of sparse -- they allow it to detect mismatched rcu_read_lock() and
> > rcu_read_unlock() pairs.  (Restricted to a single function, but so
> > it goes.)
> > 
> > I don't claim to fully understand this code, so may be way off base.
> > However, it looks like you are relying on stop_machine(), which in
> > turn interacts with preempt_disable(), but -not- necessarily with
> > rcu_read_lock().  Now, your rcu_barrier() call -does- interact with
> > rcu_read_lock() correctly, but either you need the preempt_disable()s
> > to interact correctly with stop_machine(), or you need to update the
> > comments calling out dependency on stop_machine().
> > 
> > Or it might be that the RCU API needs a bit of expanding.  For example,
> > if you absolutely must use call_rcu(), and you also must absolutely
> > rely on stop_machine(), this might indicate that we need to add a
> > call_rcu_sched() as an asynchronous counterpart to synchronize_sched().
> > This would also require an rcu_sched_barrier() as well, to allow safe
> > unloading of modules using call_rcu_sched().
> > 
> > Or am I missing something?
> > 
> 
> Hi Paul,
> 
> Sorry about the late response; I was away for small vacation :)
> 
> Yes, I need both :
> 
> - disabling preemption at marker site is required to protect against
>   deletion of probe code when modules are unloaded.
> - I use the call_rcu() to execute delayed free of my data structures. I
>   could do all that synchronously with synchronize_sched(), but batch
>   registration/unregistration would be just too slow. I don't want to
>   take a few minutes to activate ~100 probes, that would be insane.
> 
> So yes, adding the new piece of API sounds like a good idea. Meanwhile,
> I guess I could just do this in the code executed around probe call,
> although it has a performance impact :
> 
>   rcu_read_lock();
>   preempt_disable();
> 
>   probe_call();
> 
>   preempt_enable();
>   rcu_read_unlock();

This will work -- and I will see about getting you a call_rcu_sched().
Trivial in non-CONFIG_PREEMPT and CONFIG_PREEMPT, will require a bit
more effort for -rt.  ;-)

						Thanx, Paul

> Thanks very much for the review,
> 
> Mathieu
> 
> 
> > 							Thanx, Paul
> 
> -- 
> Mathieu Desnoyers
> Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
> OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68