Re: Djprobes questions

[Mathieu Desnoyers <compudj@krystal.dyndns.org>]

* Masami Hiramatsu (masami.hiramatsu.pt@hitachi.com) wrote:
> Hi Mathieu,
> 
> Mathieu Desnoyers wrote:
> > Hi Masami,
> > 
> > I recently had to add support for inline code patching on i386 to my
> > marker infrastructure. Clearly, it looks like what is done in djprobes,
> > with the main difference that I only patch the immediate value of a 2
> > bytes "load immediate" instruction.
> 
> That's interesting.
> 
> > I think I found a solution to one of the main issues with djprobes : it
> > currently has to wait for each CPU to hit the probe before being sure
> > that it's safe to patch the code with something else than an int3. This
> > is due to PIII errata 49, which says that a CPU much execute a
> > serializing instruction before executing cross-modified code.
> 
> Hmm, djprobe already might not wait for each CPU to hit the probe
> point. It just wait scheduler synchronization instead of that.

Oh, I see. You can do it because you don't support fully preemptible
kernels. It makes sense it that scenario. I could not use this scheme
for my markers though, since I support fully preemptible kernels.

> And after that, it issues cpuid for cache serialization before
> executing cross-modified code.
> 

Ok, so you already used the IPI serialization I am talking about. Great :)

> The most difficult point of the djprobe is that it has to replace
> "live" instructions. So we must check other processors not to run
> those instructions carefully.
> 
> > Here is what I do : While I use a breakpoint to fall in a trap for the
> > CPUs that hit the site currently being modified, I also send an IPI to
> > all CPUs so they execute cpuid. Once it returns, I am sure that every
> > CPU has executed a serializing instruction, which enables me to go on
> > with the complete code modification, therefore removing the initial
> > breakpoint.
> 
> I think its OK. That is the same way which I've done in djprobe.
> 
> > Here is my code :
> > 
> > http://ltt.polymtl.ca/cgi-bin/gitweb.cgi?p=linux-2.6-lttng.git;a=blob;f=arch/i386/kernel/marker.c;h=89b06f02f0966685be260d6364a0dd94c3d14456;hb=v2.6.20-lttng
> > 
> > (Comments are welcome)
> > 
> > On a second note, looking at the djprobes code triggered some question 
> > in my mind about the safety of using a worker thread to "make sure"
> > every interrupt context has returned (so there is no IP pointing into
> > the modified code). The following scenario might be possible : an
> > interrupt handler (or trap handler) reenables interrupts, does irq_exit()
> > or nmi_exit() (which reenables preemption) but does not do iret yet. My
> > understanding is that it could be scheduled and have a return IP
> > pointing to the code that is being modified. Am I right ?
> 
> Same idea was already discussed. It might work on normal kernel,
> but, unfortunately, it doesn't work on preemptive kernel. And actually,
> that idea is same as synchronize_sched(). So, I've used it on normal
> kernel. In the case of preemptive kernel, currently, I'm using
> freeze_processes() suggested by Ingo.
> 

It looks like a good way to know where the processes are stopped.

> Anyway, I and Satoshi are developing a static analysis tool to
> check whether target instructions can be replaced by long jump.
> I'd like to release djprobe patch against latest kernel after
> developed it.
> 

Great, continue your good work!

Regards,

Mathieu


> -- 
> Masami HIRAMATSU
> Linux Technology Center
> Hitachi, Ltd., Systems Development Laboratory
> E-mail: masami.hiramatsu.pt@hitachi.com
> 
> 

-- 
Mathieu Desnoyers
Computer Engineering Ph.D. Student, Ecole Polytechnique de Montreal
OpenPGP key fingerprint: 8CD5 52C3 8E3C 4140 715F  BA06 3F25 A8FE 3BAE 9A68