Re: [PATCH v7 00/16] tracing: probeevent: Improve fetcharg features

["Naveen N. Rao" <naveen.n.rao@linux.vnet.ibm.com>]

Steven Rostedt wrote:
> On Sat, 5 May 2018 00:48:28 +0900
> Masami Hiramatsu <mhiramat@kernel.org> wrote:
>> > 
>> > Also, when looking at the kprobe code, I was looking at this function:
>> >   
>> > > /* Ftrace callback handler for kprobes -- called under preepmt disabed */
>> > > void kprobe_ftrace_handler(unsigned long ip, unsigned long parent_ip,
>> > > 			   struct ftrace_ops *ops, struct pt_regs *regs)
>> > > {
>> > > 	struct kprobe *p;
>> > > 	struct kprobe_ctlblk *kcb;
>> > > 
>> > > 	/* Preempt is disabled by ftrace */
>> > > 	p = get_kprobe((kprobe_opcode_t *)ip);
>> > > 	if (unlikely(!p) || kprobe_disabled(p))
>> > > 		return;
>> > > 
>> > > 	kcb = get_kprobe_ctlblk();
>> > > 	if (kprobe_running()) {
>> > > 		kprobes_inc_nmissed_count(p);
>> > > 	} else {
>> > > 		unsigned long orig_ip = regs->ip;
>> > > 		/* Kprobe handler expects regs->ip = ip + 1 as breakpoint hit */
>> > > 		regs->ip = ip + sizeof(kprobe_opcode_t);
>> > > 
>> > > 		/* To emulate trap based kprobes, preempt_disable here */
>> > > 		preempt_disable();
>> > > 		__this_cpu_write(current_kprobe, p);
>> > > 		kcb->kprobe_status = KPROBE_HIT_ACTIVE;
>> > > 		if (!p->pre_handler || !p->pre_handler(p, regs)) {
>> > > 			__skip_singlestep(p, regs, kcb, orig_ip);
>> > > 			preempt_enable_no_resched();  
>> > 
>> > This preemption disabling and enabling looks rather strange. Looking at
>> > git blame, it appears this was added for jprobes. Can we remove it now
>> > that jprobes is going away?  
>> 
>> No, that is not for jprobes but for compatibility with kprobe's user
>> handler. Since this transformation is done silently, user can not
>> change their handler for ftrace case. So we need to keep this condition
>> same as original kprobes.
>> 
>> And anyway, for using smp_processor_id() for accessing per-cpu,
>> we should disable preemption, correct?
> 
> But as stated at the start of the function:
> 
>  /* Preempt is disabled by ftrace */
> 
> 
> The reason I ask, is that we have for this function:
> 
> 		/* To emulate trap based kprobes, preempt_disable here */
> 		preempt_disable();
> 		__this_cpu_write(current_kprobe, p);
> 		kcb->kprobe_status = KPROBE_HIT_ACTIVE;
> 		if (!p->pre_handler || !p->pre_handler(p, regs)) {
> 			__skip_singlestep(p, regs, kcb, orig_ip);
> 			preempt_enable_no_resched();
> 		}
> 
> And in arch/x86/kernel/kprobes/core.c we have:
> 
> 	preempt_disable();
> 
> 	kcb = get_kprobe_ctlblk();
> 	p = get_kprobe(addr);
> 
> 	if (p) {
> 		if (kprobe_running()) {
> 			if (reenter_kprobe(p, regs, kcb))
> 				return 1;
> 		} else {
> 			set_current_kprobe(p, regs, kcb);
> 			kcb->kprobe_status = KPROBE_HIT_ACTIVE;
> 
> 			/*
> 			 * If we have no pre-handler or it returned 0, we
> 			 * continue with normal processing.  If we have a
> 			 * pre-handler and it returned non-zero, it prepped
> 			 * for calling the break_handler below on re-entry
> 			 * for jprobe processing, so get out doing nothing
> 			 * more here.
> 			 */
> 			if (!p->pre_handler || !p->pre_handler(p, regs))
> 				setup_singlestep(p, regs, kcb, 0);
> 			return 1;
> 
> 
> Which is why I thought it was for jprobes. I'm a bit confused about
> where preemption is enabled again.

Jprobes was the in-kernel user for that. However, users can write custom 
kprobe [pre-]handlers that return a non-zero value if they want to 
suppress normal processing of the probe (single stepping the instruction 
where the probe was installed). In this case, the custom handler is 
expected to deal with re-enabling preemption before returning from the 
pre handler. Or, there must be some other way to re-enable preemption 
later on like with jprobes -- where the hook would cause a trap to 
complete jprobe handling.

For optprobes, we actually break this and do not disable preemption.  
But, the expectation there is that the user set a post-handler to force 
optprobes to be disabled, *if* they want to do custom handling by 
returning a non-zero return value from the pre handler.

For KPROBES_ON_FTRACE, we need to emulate the behavior of the normal, 
trap-based kprobes. This is the reason preemption needs to be disabled 
again, so as to balance it with the user's custom handler re-enabling 
it.

Of course, with the in-kernel user (jprobes) now gone, it is anybody's 
guess as to who is still depending on this custom pre-handler behavior 
;)


- Naveen