Re: Re: kprobing "hash_64.constprop.26" crashes the system, recursion through get_kprobe?

[Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com>]

(2013/03/01 17:43), Timo Juhani Lindfors wrote:
> Masami Hiramatsu <masami.hiramatsu.pt@hitachi.com> writes:
>>> I am unable to recreate this problem on a fedora system; hash_64 is
>>> inlined AFAICS.
> 
> Thanks testing. How does the disassembly of get_kprobes look for you?

Actually, there is no "call" instruction in get_kprobe.

>> I also tried and couldn't recreate hash_64 problem on my ubuntu 12.10.
>> Could you tell us your kconfig?
> 
> Sure, see
> 
> http://lindi.iki.fi/lindi/config-3.7-trunk-amd64_3.7.3-1~experimental.1

Thanks! I'll check that.

> 
> Note that I am doing the tests under xen. Should I redo the tests under
> kvm?

No, I think the issue should be reproduced again even on kvm.

>>> This is a clear case of recursion. Either of the two options should fix
>>> the problem.
>>
>> I think __kprobes is better if it works, because it causes kprobes
>> issue (no problem in other parts).
>> (Again, since I can't reproduce this hash_64.constprop,
>> I'm not sure that __kprobes can fix that clearly.)

OK, I've ensured that __kprobes on an inline function (I used kzalloc
for test) moves all constprop instances into .kprobes.text section.
So I think it can fix your case too.

> Ok. My worry is that hash_64 is not the only function that is called
> from a __kprobes marked function. For example kprobes_module_callback
> calls mutex_lock. Should we also mark mutex_lock with __kprobes?

No, you don't need to do it, because kprobes_module_callback
is never called from the probe handler (unless user doesn't do)

This recursive call issue happens when the probe point(function)
is executed inside the kprobe int3 handler. In your case,
it happened on the kprobe_exceptions_notify(inlined kprobe_handler code)
which is called from do_int3 via notify_die.

> 
> Based on the disassembly of code between __kprobes_text_start and
> __kprobes_text_end I made a function call graph that shows functions
> that get called from this region:
> 
> http://lindi.iki.fi/lindi/kprobes/function-call-graph1.png
> 
> The __kprobes functions are marked with red color. I had to omit
> mutex_lock, mutex_unlock and printk from the graph to make it readable
> (I also omited functions that don't call other functions). As you can
> see, this is quite a minefield :)

Actually, __kprobes is used much more than we need, because that
is safer (and also, that is used just for marking functions
related to kprobes).

Only what functions we need to mark __kprobes is the functions
called from do_int3, and optimized_callback, until processing
recursive call (which is done by kprobe_running() and
reenter_kprobe()).

Thank you,

-- 
Masami HIRAMATSU
IT Management Research Dept. Linux Technology Center
Hitachi, Ltd., Yokohama Research Laboratory
E-mail: masami.hiramatsu.pt@hitachi.com