Re: Markers: multi-probe locking fun (was: Re: [PATCH 0/2] Markers Implementation for RCU Tracing - Ver II)

[Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>]

* Paul E. McKenney (paulmck@linux.vnet.ibm.com) wrote:
> On Tue, Feb 19, 2008 at 05:27:40PM +0100, Jan Kiszka wrote:
> > Paul E. McKenney wrote:
> > > On Mon, Feb 18, 2008 at 01:47:31PM +0100, Jan Kiszka wrote:
> > >> K. Prasad wrote:
> > >>> Hi Ingo,
> > >>> 	Please accept these patches into the rt tree which convert the
> > >>> existing RCU tracing mechanism for Preempt RCU and RCU Boost into
> > >>> markers.
> > >>>  
> > >>> These patches are based upon the 2.6.24-rc5-rt1 kernel tree.
> > >>>  
> > >>> Along with marker transition, the RCU Tracing infrastructure has also
> > >>> been modularised to be built as a kernel module, thereby enabling
> > >>> runtime changes to the RCU Tracing infrastructure.
> > >>>  
> > >>> Patch [1/2] - Patch that converts the Preempt RCU tracing in
> > >>> rcupreempt.c into markers.
> > >>>  
> > >>> Patch [1/2] - Patch that converts the Preempt RCU Boost tracing in
> > >>> rcupreempt-boost.c into markers.
> > >>>  
> > >> I have a technical problem with marker-based RCU tracing: It causes
> > >> nasty recursions with latest multi-probe marker patches (sorry, no link
> > >> at hand, can be found in latest LTTng, maybe also already in -mm). Those
> > >> patches introduce a marker probe trampoline like this:
> > >>
> > >> void marker_probe_cb(const struct marker *mdata, void *call_private,
> > >> 	const char *fmt, ...)
> > >> {
> > >> 	va_list args;
> > >> 	char ptype;
> > >>
> > >> 	/*
> > >> 	 * rcu_read_lock does two things : disabling preemption to make sure the
> > >> 	 * teardown of the callbacks can be done correctly when they are in
> > >> 	 * modules and they insure RCU read coherency.
> > >> 	 */
> > >> 	rcu_read_lock();
> > >> 	preempt_disable();
> > >> 	...
> > >>
> > >> Can we do multi-probe with pure preempt_disable/enable protection? I
> > >> guess it's fine with classic RCU, but what about preemptible RCU? Any
> > >> suggestion appreciated!
> > > 
> > > If you substitute synchronize_sched() for synchronize_rcu(), this should
> > > work fine.  Of course, this approach would cause RCU tracing to degrade
> > > latencies somewhat in -rt.
> > > 
> > > If tracing is using call_rcu(), we will need to add a call_sched()
> > > or some such.
> > 
> > You mean something like "#define call_sched call_rcu_classic"?
> 
> This would work for Classic RCU.  For preemptible RCU, a bit more
> work is needed.
> 
> > I just learned that there is another reason for killing
> > rcu_read_lock&friends from the marker probes: It can deadlock on -rt
> > with PREEMPT_RCU_BOOST (hit probe inside rq-lock protected region =>
> > rcu_read_unlock triggers unboost => stuck on rq_lock :( ).
> 
> OK, good to know.  Guess we need a call_sched() for -rt and for
> preemptible RCU sooner rather than later...
> 

Yup, I would need call_sched() and sched_barrier().

Thanks,

Mathieu

> 						Thanx, Paul

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68