Re: [PATCH v4 0/3] Compile-time stack frame pointer validation

Ingo Molnar
2015-05-21T10:16:21Z

* Borislav Petkov <bp@alien8.de> wrote:

> On Wed, May 20, 2015 at 11:25:37AM -0500, Josh Poimboeuf wrote:
> > > I've never quite understood what the '?' means.
> >
> > It basically means "here's a function address we found on the stack,
> > which may or may not have been called." It's needed because stack
> > walking isn't currently 100% reliable.
>
> Yeah, that was not that trivial to figure out at the time:
>
> unsigned long
> print_context_stack(struct thread_info *tinfo,
> ...
>
> if (__kernel_text_address(addr)) {
> if ((unsigned long) stack == bp + sizeof(long)) {
> ops->address(data, addr, 1);
> frame = frame->next_frame;
> bp = (unsigned long) frame;
> } else {
> ops->address(data, addr, 0);
> }
>
> and that ops->address is
>
> print_trace_address()
> |-> printk_stack_address()
>
> So if I'm understanding this correctly, if rBP+8 is equal to rSP, i.e.
> return address is on the stack, then this frame got called.
>
> Otherwise -> "?".
>
> I might be missing something though...

So this is how we are printing backtraces on x86:

We always scan the full kernel stack for return addresses stored on
the kernel stack(s) [*], from stack top to stack bottom, and print out
anything that 'looks like' a kernel text address.

If it fits into the frame pointer chain, we print it without a
question mark, knowing that it's part of the real backtrace.

If the address does not fit into our expected frame pointer chain we
still print it, but we print a '?'. It can mean two things:

- either the address is not part of the call chain: it's just stale
values on the kernel stack, from earlier function calls. This is
the common case.

- or it is part of the call chain, but the frame pointer was not set
up properly within the function, so we don't recognize it. See the
200+ assembly functions that Josh's build time validation found.

This way we will always print out the real call chain (plus a few more
entries), regardless of whether the frame pointer was set up correctly
or not - but in most cases we'll get the call chain right as well. The
entries printed are strictly in stack order, so you can deduce more
information from that as well.

The most important property of this method is that we _never_ lose
information: we always strive to print _all_ addresses on the stack(s)
that look like kernel text addresses, so if debug information is
wrong, we still print out the real call chain as well - just with more
question marks than ideal.

Thanks,

Ingo

[*] For things like IRQ stacks and ISTs we also scan those stacks, in
the right order, and try to cross from one stack into another
reconstructing the call chain. This works most of the time.