Re: [patch 1/2] Linux Kernel Markers - Support Multiple Probes

[Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca>]

* Paul E. McKenney (paulmck@linux.vnet.ibm.com) wrote:
> On Tue, Dec 04, 2007 at 02:45:06PM -0500, Mathieu Desnoyers wrote:
> > * Andrew Morton (akpm@linux-foundation.org) wrote:
> > > On Tue, 4 Dec 2007 14:21:00 -0500
> > > Mathieu Desnoyers <mathieu.desnoyers@polymtl.ca> wrote:
> > > 
> > > > > > + */
> > > > > > +void marker_probe_cb(const struct marker *mdata, void *call_private,
> > > > > > +	const char *fmt, ...)
> > > > > > +{
> > > > > > +	va_list args;
> > > > > > +	char ptype;
> > > > > > +
> > > > > > +	preempt_disable();
> > > > > 
> > > > > What are the preempt_disable()s doing in here?
> > > > > 
> > > > > Unless I missed something obvious, a comment is needed here (at least).
> > > > > 
> > > > 
> > > > They make sure the teardown of the callbacks can be done correctly when
> > > > they are in modules and they insure RCU read coherency. Will add
> > > > comment.
> > > 
> > > So shouldn't it be using rcu_read_lock()?  If that does not suit, should we
> > > be adding new rcu primitives rather than open-coding and adding dependencies?
> > 
> > Hrm, yes, good point. Since there seems to be extra magic under
> > __acquire(RCU);  and  rcu_read_acquire();, the the fact that I use
> > rcu_barrier() for synchronization, we should. I'll change it.
> 
> (Sorry to show up so late...  It has been a bit crazy of late...)
> 
> The __acquire(RCU) and rcu_read_acquire() are strictly for the benefit
> of sparse -- they allow it to detect mismatched rcu_read_lock() and
> rcu_read_unlock() pairs.  (Restricted to a single function, but so
> it goes.)
> 
> I don't claim to fully understand this code, so may be way off base.
> However, it looks like you are relying on stop_machine(), which in
> turn interacts with preempt_disable(), but -not- necessarily with
> rcu_read_lock().  Now, your rcu_barrier() call -does- interact with
> rcu_read_lock() correctly, but either you need the preempt_disable()s
> to interact correctly with stop_machine(), or you need to update the
> comments calling out dependency on stop_machine().
> 
> Or it might be that the RCU API needs a bit of expanding.  For example,
> if you absolutely must use call_rcu(), and you also must absolutely
> rely on stop_machine(), this might indicate that we need to add a
> call_rcu_sched() as an asynchronous counterpart to synchronize_sched().
> This would also require an rcu_sched_barrier() as well, to allow safe
> unloading of modules using call_rcu_sched().
> 
> Or am I missing something?
> 

Hi Paul,

Sorry about the late response; I was away for small vacation :)

Yes, I need both :

- disabling preemption at marker site is required to protect against
  deletion of probe code when modules are unloaded.
- I use the call_rcu() to execute delayed free of my data structures. I
  could do all that synchronously with synchronize_sched(), but batch
  registration/unregistration would be just too slow. I don't want to
  take a few minutes to activate ~100 probes, that would be insane.

So yes, adding the new piece of API sounds like a good idea. Meanwhile,
I guess I could just do this in the code executed around probe call,
although it has a performance impact :

  rcu_read_lock();
  preempt_disable();

  probe_call();

  preempt_enable();
  rcu_read_unlock();

Thanks very much for the review,

Mathieu


> 							Thanx, Paul

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68