Re: MIPS Function Tracer question

[David Daney <ddaney.cavm@gmail.com>]

On 11/30/2012 06:53 AM, Alan Cooper wrote:
> I needed to be more specific. The issues I'm seeing are only on 32BIT
> platforms with dynamic tracing.
> Let me explain the issue. When the compiler flag "-pg" is specified
> for 32BIT platforms to enable tracing, the compiler adds "addiu
> sp,sp,-8" in the delay slot after every call to _mcount (some legacy
> thing where 2 arguments used to be pushed on the stack). The _mcount
> routine is expected to adjust the sp by 8 before returning.

If the kernel's tracer infrastructure is broken for 32-bit kernels, then 
it should be fixed.  I think it was only really tested on 64-bit kernels.

It seems that the sp adjustment should be replaced by NOP as well if 
tracing is disabled.

> The
> problem is when tracing is disabled, all calls to _mcount are
> dynamically converted from "jal _mcount" to "nop" but the following
> "addiu sp,sp,-8" instruction is unchanged and when executed leaves the
> sp off by -8 for the remainder of the function. This bug is hidden
> when the compiler is told to use frame pointers because all offsets
> into the stack use the fp instead of the sp and the sp is restored
> from the frame pointer instead of just adding a constant to sp. When
> frame pointers are not enabled the code crashes. I don't think the
> toolchain version makes any difference because the _mcount assembly
> language routine adjusts the stack pointer by 8 if not CONFIG_64BIT
> regardless of the toolchain version.
>
> On Thu, Nov 29, 2012 at 6:00 PM, David Daney <ddaney.cavm@gmail.com> wrote:
>> On 11/29/2012 01:04 PM, Alan Cooper wrote:
>>>
>>> I've been doing some testing of the MIPS Function Tracer functionality
>>> on the 3.3 kernel. I was surprised to find that the option to generate
>>> frame pointers was required for tracing.
>>
>>
>> It is not really required for MIPS function tracing, but the Kconfigs for
>> some reason set it.
>>
>>
>>>   When I don't enable
>>> FRAME_POINTER along with FUNCTION_TRACER, the kernel hangs on boot. I
>>> also noticed that a checkin to the 3.4 kernel
>>> (b732d439cb43336cd6d7e804ecb2c81193ef63b0) no longer forces on
>>> FRAME_POINTER when FUNCTION_TRACER is selected. I was wondering how it
>>> works in 3.4 and beyond, so I built a Malta kernel from the latest
>>> MIPS tree with FUNCTION_TRACING enabled and tested it with QEMU. The
>>> kernel hung the same way. I can think of 2 reasons for this:
>>> 1. Function tracing is broken for MIPS in 3.4 and beyond.
>>> 2. The 4.5.3 GNU C compiler I'm using is generating different code for
>>> function tracing.
>>
>>
>> Function tracing works best with recent versions of GCC (those that support
>> -mmcount-ra-address).
>>
>>
>>> I was wondering if anyone has MIPS function tracing working in 3.4 or
>>> later?
>>
>>
>> Yes.  Using GCC 4.7.0 on an octeon kernel (based on 3.4.14):
>>
>> # tracer: function_graph
>> #
>> # CPU  DURATION                  FUNCTION CALLS
>> # |     |   |                     |   |   |   |
>>    1)               |  __fsnotify_parent() {
>>    1)   7.154 us    |  } /* __fsnotify_parent */
>>    1)               |  fsnotify() {
>>    1)               |    __srcu_read_lock() {
>>    1)               |      add_preempt_count() {
>>    1)   1.356 us    |      } /* add_preempt_count */
>>    1)               |      sub_preempt_count() {
>>    1)   1.385 us    |      } /* sub_preempt_count */
>>    1)   6.747 us    |    } /* __srcu_read_lock */
>>    1)               |    __srcu_read_unlock() {
>>    1)               |      add_preempt_count() {
>>    1)   1.383 us    |      } /* add_preempt_count */
>>    1)               |      sub_preempt_count() {
>>    1)   1.358 us    |      } /* sub_preempt_count */
>>    1)   6.642 us    |    } /* __srcu_read_unlock */
>>    1) + 17.861 us   |  } /* fsnotify */
>> .
>> .
>> .
>>
>>
>>
>>>
>>> I did figure out why it's hanging and I have some changes that will
>>> allow the function tracer to run without frame pointers, but before I
>>> proceed I want to rule out compiler differences.
>>>
>>> Thanks
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
>>> the body of a message to majordomo@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>> Please read the FAQ at  http://www.tux.org/lkml/
>>>
>>>
>>