Re: 2.6.33: ftrace triggers soft lockup

[Li Zefan <lizf@cn.fujitsu.com>]

>>>>>> I am not sure if this is ftrace's fault, but it is ftrace who triggers
>>>>>> the soft lockup. On my machine, it is pretty easy, just run:
>>>>>>
>>>>>> echo function_graph > current_tracer
>>>>>>
>>>> I can't say that because I didn't try -rc6.
>>>>
>>> Sigh, 2.6.33-rc6 doesn't work, even 2.6.32 doesn't work...
>> So basically you are saying that the function_graph tracer, when enabled
>> has a high overhead? Well, unfortunately, that's expected.
>>
>> The function_graph tracer traces the start and end of every function. It
>> uses the same mechanism as function tracer to trace the start of the
>> function (mcount), but to trace the exit of a function, in the enter of
>> the function it hijacks the return address and replaces it to call a
>> trampoline. This trampoline will do the trace and then jump back to the
>> original return address.
>>
>> Doing this breaks branch prediction in the CPU, as the CPU uses call/ret
>> as part of its branch prediction analysis. So function graph tracing is
>> not just twice as slow as function tracing, it actually has a bigger
>> impact than that.
> 
> 
> It's true it has a high overhead, but not to the point of
> making the whole system unusable. We are supposed to be even
> far from that. I'm currently able to turn on the function graph
> tracer and use firefox without problems. It's just a bit slower
> but it's far from a visible starvation.
> 
> And Li seems to see the same thing.

Yes, and more than this. I can see segmentation fault while testing,
both my testing threads and kernel building threads that are running
at the same time can get segfault.

> For now I can not test, but I will try this week-end.
>