other platforms interested in in-kernel unwinder?

other platforms interested in in-kernel unwinder?

[David Mosberger]

Soon I'd like to start work on replacing the existing ia64
stack-unwinder with something based on libunwind [1].  I can obviously
do this for ia64 only, but given the per-annual bug reports about
broken backtraces on various platforms, I thought perhaps other arch
maintainers would be interested in having an in-kernel unwinder, too.

libunwind already has a DWARF2 unwinder and works, in addition to
ia64, on x86 and x86-64.  I hope to add PA-RISC and Alpha soon, too.

Given this, it might make sense to add a libunwind-based unwinder not
just for ia64, but other platforms, too.  IIRC, there may be
plans/code for in-kernel unwinder for PPC{,64} already?

The upside of using an in-kernel unwinder would primarily be reliable
stack-traces, at the expense of a somewhat increased kernel-size (for
x86-64, the core of the user-level unwinder adds up to about 60KB of
code (compiled with -O) and about 32KB of data.  Most of the code-size
is due to the actual DWARF2 expression interpreter, so there is
probably not much hope of getting a _really_ tiny DWARF2 unwinder, but
certainly for workstation/server-type machines, it's probably worth
spending <128KB of memory in return for reliable stack traces.

So, how does this sound?  Any interest from say, x86{-64}, PPC, SPARC,
or any other arch maintainer?

Thanks,

	--david

[1] http://www.hpl.hp.com/research/linux/libunwind/

Re: other platforms interested in in-kernel unwinder?

[Arnd Bergmann]

[-- Attachment #1: Type: text/plain, Size: 787 bytes --]

On Dunnersdag 02 Dezember 2004 18:49, David Mosberger wrote:

> So, how does this sound?  Any interest from say, x86{-64}, PPC, SPARC,
> or any other arch maintainer?

I think it doesn't make much sense for unwinding s390 kernel stacks,
since they already have reliable back traces trough the backchain
pointers. In particular, those pointers are needed anyway for easy
unwinding by hand when debugging in z/VM.

My limited experience on ppc64 suggests that the situation there
is very similar.

OTOH, I've heard that oprofile is unwinding user stacks to get call
graph information. For that purpose, a kernel unwinder might turn
out very valuable on many architectures, including those that use
a backchain pointer in kernel but not in user space.

	Arnd <><





[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: other platforms interested in in-kernel unwinder?

[David Mosberger]

>>>>> On Fri, 3 Dec 2004 10:37:22 +0100, Arnd Bergmann <arnd@arndb.de> said:

  Arnd> I think it doesn't make much sense for unwinding s390 kernel
  Arnd> stacks, since they already have reliable back traces trough
  Arnd> the backchain pointers. In particular, those pointers are
  Arnd> needed anyway for easy unwinding by hand when debugging in
  Arnd> z/VM.

Yes, that certainly makes sense.

On s390, do you get just the frame-chain or do the frames contain
enough information to figure out which callee-saved registers where
saved?

  Arnd> My limited experience on ppc64 suggests that the situation
  Arnd> there is very similar.

Oh, that's a surprise.  I didn't realize ppc was maintaining a
frame-chain.  Can somebody confirm?

  Arnd> OTOH, I've heard that oprofile is unwinding user stacks to get
  Arnd> call graph information.

That's correct.

  Arnd> For that purpose, a kernel unwinder might turn out very
  Arnd> valuable on many architectures, including those that use a
  Arnd> backchain pointer in kernel but not in user space.

I should point out that, at the moment, my plan is to only support
unwinding within the kernel itself, so in that sense, it may or may
not help oprofile.

A corretion to yesterday's message: I forgot that the x86-64 libunwind
that I was looking at had debugging enabled, which causes lots of
replicated code because inlining is disabled.  I don't have x86-64
figures handy for the no-debug version, but for x86, the entire
libunwind library, some of which won't be needed in the kernel
version, is just 35KB of text and ~16KB of data.

	--david

Re: other platforms interested in in-kernel unwinder?

[Arnd Bergmann]

[-- Attachment #1: Type: text/plain, Size: 481 bytes --]

On Freedag 03 Dezember 2004 22:09, David Mosberger wrote:

> On s390, do you get just the frame-chain or do the frames contain
> enough information to figure out which callee-saved registers where
> saved?

IIRC, compilers up to gcc 3.4 store all callee-saved registers in a fixed
offset, while with gcc-4.0 and the patched gcc-3.4 used in fedora core 3
only the back chain pointer and the return address can be read directly
without looking at the code or dwarf2 data.

 Arnd <><

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: other platforms interested in in-kernel unwinder?

[David Mosberger]

>>>>> On Fri, 3 Dec 2004 23:56:18 +0100, Arnd Bergmann <arnd@arndb.de> said:

  Arnd> IIRC, compilers up to gcc 3.4 store all callee-saved registers
  Arnd> in a fixed offset, while with gcc-4.0 and the patched gcc-3.4
  Arnd> used in fedora core 3 only the back chain pointer and the
  Arnd> return address can be read directly without looking at the
  Arnd> code or dwarf2 data.

OK, thanks for the clarification.  So it sounds like libunwind _might_
be useful if you ever wanted to play a similar trick we do on ia64,
where we construct the (preserved) user-level state on the fly.  But
then again, s390 probably has few enough registers that this may not
be much of a win.

Thanks,

	--david

Re: other platforms interested in in-kernel unwinder?

[Paul Mackerras]

David Mosberger writes:

> Oh, that's a surprise.  I didn't realize ppc was maintaining a
> frame-chain.  Can somebody confirm?

Yes, both ppc and ppc64 maintain a series of stack frames linked
together, so getting the call chain is pretty straightforward.  (This
is a convention and part of the ABI, not something required by the
hardware.)

There are two things an unwinder could help with.  First, because of
the conventions about leaf functions, the first return address can be
in the link register or on the stack, or both, and the link register
value may or may not need to be used as part of the call chain.  An
unwinder could presumably get that right (at the moment we have some
heuristics that work pretty well but not perfectly).

Secondly, an unwinder could get us the contents of the callee-saved
registers in each stack frame, which may be useful for debugging.

I think an unwinder would be of some help to those of us doing kernel
debugging, but I don't know that it would make a dramatic difference.

Regards,
Paul.

Re: other platforms interested in in-kernel unwinder?

[David Mosberger]

>>>>> On Sat, 4 Dec 2004 14:59:40 +1100, Paul Mackerras <paulus@samba.org> said:

  Paul> David Mosberger writes:
  >> Oh, that's a surprise.  I didn't realize ppc was maintaining a
  >> frame-chain.  Can somebody confirm?

  Paul> Yes, both ppc and ppc64 maintain a series of stack frames
  Paul> linked together, so getting the call chain is pretty
  Paul> straightforward.  (This is a convention and part of the ABI,
  Paul> not something required by the hardware.)

OK, thanks for the info.  There is always something new to learn... ;-)

  Paul> There are two things an unwinder could help with.  First,
  Paul> because of the conventions about leaf functions, the first
  Paul> return address can be in the link register or on the stack, or
  Paul> both, and the link register value may or may not need to be
  Paul> used as part of the call chain.  An unwinder could presumably
  Paul> get that right (at the moment we have some heuristics that
  Paul> work pretty well but not perfectly).

  Paul> Secondly, an unwinder could get us the contents of the
  Paul> callee-saved registers in each stack frame, which may be
  Paul> useful for debugging.

  Paul> I think an unwinder would be of some help to those of us doing
  Paul> kernel debugging, but I don't know that it would make a
  Paul> dramatic difference.

Nobody so far has said they badly want an in-kernel unwinder for
anything other than ia64, so I'll probably go ahead and make it
ia64-specific for now, while keeping an eye open for not doing
anything that would make it more difficult to add in the DWARF2
unwinder should some day the need arise.

Thanks,

	--david