Oprofile callgraph support on the MIPS architecture

Oprofile callgraph support on the MIPS architecture

[Gergely Kis]

Dear List Members,

We would like to announce the initial version of oprofile callgraph
support on the MIPS architecture.

Our implementation requires no user space changes to oprofile, only a
few kernel patches need to be applied. In this first version we
provide kernel patches for the 2.6.32 version, but we intend to update
the patches to the latest kernel version to submit the patches into
the mainline kernel. We also intend to provide a version for the
Honeycomb (3.0) Android kernels.

The current version was mostly tested with MIPS Android, running the
Froyo release (2.2). We tested the implementation on Sigma Designs
8654 and ViXS XCode 4210 platforms, running in little endian mode.

One interesting property of our callgraph implementation is that it
does not require the presence of frame pointers. This way there is no
need to recompile production binaries or kernels with frame pointers
enabled. One may simply compile oprofile as a module, and enable it on
demand, even on production builds.

We also added a way to restrict the callgraph generation to kernel
space / user space, if desired. This way the overhead of callgraph
generation may be reduced.

The home page also provides a few sample callgraphs created using
kcachegrind from the oprofile data.

You may download the code and access the documentation on the following URL:
http://oss.homejinni.com/redmine/projects/mips-oprofile

Best Regards,
Gergely

Re: Oprofile callgraph support on the MIPS architecture

[David Daney]

On 03/26/2011 07:02 PM, Gergely Kis wrote:
> Dear List Members,
>
> We would like to announce the initial version of oprofile callgraph
> support on the MIPS architecture.
>
[...]

> You may download the code and access the documentation on the following URL:
> http://oss.homejinni.com/redmine/projects/mips-oprofile
>

Instructions for submitting patches may be found here:

http://git.linux-mips.org/?p=linux.git;a=tree;f=Documentation/development-process;hb=f70c04ff7ad61bca1ed06390558568c6c8074139

That said, how do you handle the case of getting a fault while reading 
code/data while unwinding?

Also I don't see how you handle these cases:

o Leaf functions where neither the $ra is saved to the stack, nor the 
stack pointer adjusted.

o Functions where $sp is adjusted several times (use of alloca() or VLAs).

o Functions with multiple return points (i.e. 'jr $ra' in the middle of 
a function).

o Functions with more than 1024 instructions.


Thanks,
David Daney

Re: Oprofile callgraph support on the MIPS architecture

[Gergely Kis]

Dear David,

On Sun, Mar 27, 2011 at 8:04 AM, David Daney <david.s.daney@gmail.com> wrote:
[...]
>
>> You may download the code and access the documentation on the following
>> URL:
>> http://oss.homejinni.com/redmine/projects/mips-oprofile
>>
>
> Instructions for submitting patches may be found here:
>
> http://git.linux-mips.org/?p=linux.git;a=tree;f=Documentation/development-process;hb=f70c04ff7ad61bca1ed06390558568c6c8074139

Thank you for the pointer, we of course plan to submit the patch using
the proper process. We only wanted to announce that we are working on
this.

>
> That said, how do you handle the case of getting a fault while reading
> code/data while unwinding?
>
In case there is a fault, we basically return to the caller, so the
building of the callgraph is stopped. We looked at the ARM version and
they handled this case in a similar way.

> Also I don't see how you handle these cases:
>
> o Leaf functions where neither the $ra is saved to the stack, nor the stack
> pointer adjusted.
>
We currently don't have a special handling for this, but we plan to
try to detect the prologue of leaf functions as well, if possible.
This detection process will probably never be
100% accurate, but we have found the call graph outputs even in their
current form useful.

Oprofile call graphs are not always accurate anyways, because of the
statistical nature of oprofile.

> o Functions where $sp is adjusted several times (use of alloca() or VLAs).
>
> o Functions with multiple return points (i.e. 'jr $ra' in the middle of a
> function).
Yes, this is a shortcoming in the current implementation, we are
already working on changing the prologue detection to detect the exact
combination of the prologue instructions.

We are also looking at the stack unwinding function used to display
the kernel stacktraces when an oops or other error condition occurs,
to see if we can refactor it to suit our needs as well. This way a
single solution for stack walking could be included in the kernel.

>
> o Functions with more than 1024 instructions.
Currently we set this (arbitrary) limit. We can probably change it, or
make it configurable, but until we are using heuristics to detect the
function boundaries, I think we should have a maximum number of
allowed steps for the stack walking functions.

What do you think?

Thank you,
Gergely Kis

Re: Oprofile callgraph support on the MIPS architecture

[David Daney]

On 03/29/2011 03:57 PM, Gergely Kis wrote:

[...]
>> That said, how do you handle the case of getting a fault while reading
>> code/data while unwinding?
>>
> In case there is a fault, we basically return to the caller, so the
> building of the callgraph is stopped. We looked at the ARM version and
> they handled this case in a similar way.
>

But it looks like you are invoking get_user() from interrupt context?  
As far as I know that is not allowed.  Have you tested it?

I don't see where you handle faults when trying to read kernel memory.


>> Also I don't see how you handle these cases:
>>
>> o Leaf functions where neither the $ra is saved to the stack, nor the stack
>> pointer adjusted.
>>
> We currently don't have a special handling for this, but we plan to
> try to detect the prologue of leaf functions as well, if possible.
> This detection process will probably never be
> 100% accurate, but we have found the call graph outputs even in their
> current form useful.
>
> Oprofile call graphs are not always accurate anyways, because of the
> statistical nature of oprofile.
>
>> o Functions where $sp is adjusted several times (use of alloca() or VLAs).
>>
>> o Functions with multiple return points (i.e. 'jr $ra' in the middle of a
>> function).
> Yes, this is a shortcoming in the current implementation, we are
> already working on changing the prologue detection to detect the exact
> combination of the prologue instructions.
>
> We are also looking at the stack unwinding function used to display
> the kernel stacktraces when an oops or other error condition occurs,
> to see if we can refactor it to suit our needs as well. This way a
> single solution for stack walking could be included in the kernel.
>
>> o Functions with more than 1024 instructions.
> Currently we set this (arbitrary) limit. We can probably change it, or
> make it configurable, but until we are using heuristics to detect the
> function boundaries, I think we should have a maximum number of
> allowed steps for the stack walking functions.
>
> What do you think?
>

My questions about your unwinding algorithm were really rhetorical in 
nature.

It is not possible to do robust unwinding by code examination, precisely 
because there is no way to identify the start of a function.

David Daney