How to detect STACKOVEFLOW on mips

How to detect STACKOVEFLOW on mips

[Adam Jiang]

Hello, list.

I'm having a problem with kernel mode stack on my box. It seems that
STACKOVERFLOW happened to Linux kernel. However, I can't prove it
because the lack of any detection in __do_IRQ() function just like on
the other architectures. If you know something about, please help me
on following two questions.
- Is there any possible to do this on MIPS?
- or, more simple question, how could I get the address $sp pointed by
asm() notation in C?

Any suggestion from you will be appreciated.

Best regards,
/Adam

Re: How to detect STACKOVEFLOW on mips

[Phil Staub]

On 06/29/2010 10:59 PM, Adam Jiang wrote:
> Hello, list.
>
> I'm having a problem with kernel mode stack on my box. It seems that
> STACKOVERFLOW happened to Linux kernel. However, I can't prove it
> because the lack of any detection in __do_IRQ() function just like on
> the other architectures. If you know something about, please help me
> on following two questions.
> - Is there any possible to do this on MIPS?

The mechanisms I know about for detecting stack overflow include:

1. Use of the MMU - stack ends at a page boundary, adjacent page is
either unmapped or mapped read-only and causes an exception if violated.

2. Hooks inserted into toolchain to cause any stack decrement to be
first tested against a limit.

3. Fill entire stack with a recognizable pattern before first
use. After suspected stack overflow, check to see if the pattern has
been disturbed in the area of the stack limit.

(Disclaimer: I've used all of these in some form on other OSes, but
not on Linux. Someone else may have a more directly relevant answer.)

> - or, more simple question, how could I get the address $sp pointed by
> asm() notation in C?

How about something like:

{
	long x;
	...
	asm("move %0,$29":"=g"(x));
	...
}

Phil

>
> Any suggestion from you will be appreciated.
>
> Best regards,
> /Adam
>
>


-- 
Phil Staub, Senior Member of Technical Staff, Wind River
Direct: 702.290.0470 Fax: 702.982.0085

Re: How to detect STACKOVEFLOW on mips

[Ralf Baechle]

On Wed, Jun 30, 2010 at 02:59:42PM +0900, Adam Jiang wrote:

> I'm having a problem with kernel mode stack on my box. It seems that
> STACKOVERFLOW happened to Linux kernel. However, I can't prove it
> because the lack of any detection in __do_IRQ() function just like on
> the other architectures. If you know something about, please help me
> on following two questions.
> - Is there any possible to do this on MIPS?
> - or, more simple question, how could I get the address $sp pointed by
> asm() notation in C?

Due to the large register frame on MIPS the stack is 8kB on 32-bit, 16kB
on 64-bit or PAGE_SIZE, whatever is larger.  This is should be hard to
overflow by accident unless doing something outrageously stupid.

To access the stackpointer include <linux/thread_info.h>.  The function
current_thread_info() will return the pointer to the struct thread_info
of the current thread.  This structure is located at the bottom of the
stack.  With something like

  register void *stackp __asm__("$29");

you then can access the stack pointer as the stackp variable.  You
obviously need to maintain the relation

  current_thread_info() + 1 < stackp

at all times - and you better have a bit of extra space available just for
peace of mind.

There used to be some code for other architectures that zeros the stack
page and counts how much of that has been overwritten by the stack.  That
was never ported to MIPS.

Another helper to find functions that do excessive static allocations is
"make checkstack".

  Ralf

Re: How to detect STACKOVEFLOW on mips

[Ralf Baechle]

On Wed, Jun 30, 2010 at 07:27:10AM -0700, Phil Staub wrote:

> >I'm having a problem with kernel mode stack on my box. It seems that
> >STACKOVERFLOW happened to Linux kernel. However, I can't prove it
> >because the lack of any detection in __do_IRQ() function just like on
> >the other architectures. If you know something about, please help me
> >on following two questions.
> >- Is there any possible to do this on MIPS?
> 
> The mechanisms I know about for detecting stack overflow include:
> 
> 1. Use of the MMU - stack ends at a page boundary, adjacent page is
> either unmapped or mapped read-only and causes an exception if violated.

Won't easily work on MIPS as the stack is allocated in KSEG0 / XKPHYS
which are unmapped segments.  It would be necessary to relocate the stack
into a mapped space.

Ultra-ancient Linux/MIPS kernels actually used to do that but that code
may well even predate everything that still exists on linux-mips.org.

> 2. Hooks inserted into toolchain to cause any stack decrement to be
> first tested against a limit.
> 
> 3. Fill entire stack with a recognizable pattern before first
> use. After suspected stack overflow, check to see if the pattern has
> been disturbed in the area of the stack limit.

This was afaik never ported to MIPS though that'd be easy.

> (Disclaimer: I've used all of these in some form on other OSes, but
> not on Linux. Someone else may have a more directly relevant answer.)
> 
> >- or, more simple question, how could I get the address $sp pointed by
> >asm() notation in C?
> 
> How about something like:
> 
> {
> 	long x;
> 	...
> 	asm("move %0,$29":"=g"(x));
> 	...
> }

That will do.  Or even something portable like:

{
	unsigned long foo;

	return &foo;
}

which used to work (GNU alloca and others were using this) but I'm sure
GCC has learned how to optimize this to shreds.

  Ralf

Re: How to detect STACKOVEFLOW on mips

[David VomLehn]

On Wed, Jun 30, 2010 at 09:27:10AM -0500, Phil Staub wrote:
> On 06/29/2010 10:59 PM, Adam Jiang wrote:
> > Hello, list.
> >
> > I'm having a problem with kernel mode stack on my box. It seems that
> > STACKOVERFLOW happened to Linux kernel. However, I can't prove it
> > because the lack of any detection in __do_IRQ() function just like on
> > the other architectures. If you know something about, please help me
> > on following two questions.
> > - Is there any possible to do this on MIPS?
> 
> The mechanisms I know about for detecting stack overflow include:
> 
> 1. Use of the MMU - stack ends at a page boundary, adjacent page is
> either unmapped or mapped read-only and causes an exception if violated.

Kernel stacks are allocated in kseg0, which is direct mapped. This means
conversions from virtual to physical addresses don't use the TLBs, so
this strategy doesn't work on MIPS.

> 2. Hooks inserted into toolchain to cause any stack decrement to be
> first tested against a limit.

I don't know of MIPS hooks for this but there may be some. It would have
to be tuned specifically for the kernel because of the way you need to get
the stack bounds (see below).

> 3. Fill entire stack with a recognizable pattern before first
> use. After suspected stack overflow, check to see if the pattern has
> been disturbed in the area of the stack limit.

This one pretty much always works. The only exception is if you allocate
a big array on the stack and don't use enough of it to alter memory in
the area at which you are looking.  Allocating big arrays on the stack
is a kernel no-no, so you should be in good shape here.

To implement this, you have to know the stack bounds. Start by getting
getting current_thread_info(), which is a pointer to a struct
thread_info. The struct thread_info is at the low address part of the
memory allocated for a task stack, which is given by union thread_union.
So, you can cast current_thread_info to a union thread_union* and put
the result in, say, p. Then the valid stack space is from
(char*)(&p->thread_info + 1) to ((char *)p + THREAD_SIZE/sizeof(long)).

> (Disclaimer: I've used all of these in some form on other OSes, but
> not on Linux. Someone else may have a more directly relevant answer.)
> 
> > - or, more simple question, how could I get the address $sp pointed by
> > asm() notation in C?
> 
> How about something like:
> 
> {
> 	long x;
> 	...
> 	asm("move %0,$29":"=g"(x));
> 	...
> }

Depending on the version of gcc you are using, you should also be able
to use __builtin_frame_address(), though I haven't tried this in the kernel.

> Phil
> 
> > Best regards,
> > /Adam

-- 
David VL

Re: How to detect STACKOVEFLOW on mips

[Matt Fleming]

On Wed, 30 Jun 2010 15:50:06 +0100, Ralf Baechle <ralf@linux-mips.org> wrote:
> 
> There used to be some code for other architectures that zeros the stack
> page and counts how much of that has been overwritten by the stack.  That
> was never ported to MIPS.
> 
> Another helper to find functions that do excessive static allocations is
> "make checkstack".

Both SH and sparc use the mcount function (enabled with the -pg switch
to gcc) to check the stack has not overflowed. The relevant code is in
arch/{sh,sparc}/lib/mcount.S. This checks the stack pointer value on
every function call. Yeah, it's heavy-weight, but an implementation for
MIPS should be able to catch almost the exact point at which stack
overflow occurs.

Re: How to detect STACKOVEFLOW on mips

[Maciej W. Rozycki]

On Wed, 30 Jun 2010, David VomLehn wrote:

> > How about something like:
> > 
> > {
> > 	long x;
> > 	...
> > 	asm("move %0,$29":"=g"(x));
> > 	...
> > }
> 
> Depending on the version of gcc you are using, you should also be able
> to use __builtin_frame_address(), though I haven't tried this in the kernel.

 It works just fine with a zero argument; the usual restrictions apply to 
non-zero arguments.  Or alternatively an explicit register variable will 
do as suggested by Ralf.  Both approaches result in a bit better code than 
an explicit move; the former is portable to other platforms even. :)

  Maciej

Re: How to detect STACKOVEFLOW on mips

[Ralf Baechle]

On Wed, Jun 30, 2010 at 10:57:41PM +0100, Matt Fleming wrote:

> On Wed, 30 Jun 2010 15:50:06 +0100, Ralf Baechle <ralf@linux-mips.org> wrote:
> > 
> > There used to be some code for other architectures that zeros the stack
> > page and counts how much of that has been overwritten by the stack.  That
> > was never ported to MIPS.
> > 
> > Another helper to find functions that do excessive static allocations is
> > "make checkstack".
> 
> Both SH and sparc use the mcount function (enabled with the -pg switch
> to gcc) to check the stack has not overflowed. The relevant code is in
> arch/{sh,sparc}/lib/mcount.S. This checks the stack pointer value on
> every function call. Yeah, it's heavy-weight, but an implementation for
> MIPS should be able to catch almost the exact point at which stack
> overflow occurs.

Which often isn't so helpful.  The alarm gets triggered on the last stack
pointer decrement but according to murphy the overflow has happened 10
levels up in the callchain.

  Ralf

Re: How to detect STACKOVEFLOW on mips

[Matt Fleming]

On Mon, Jul 05, 2010 at 11:56:27AM +0100, Ralf Baechle wrote:
> On Wed, Jun 30, 2010 at 10:57:41PM +0100, Matt Fleming wrote:
> 
> > On Wed, 30 Jun 2010 15:50:06 +0100, Ralf Baechle <ralf@linux-mips.org> wrote:
> > > 
> > > There used to be some code for other architectures that zeros the stack
> > > page and counts how much of that has been overwritten by the stack.  That
> > > was never ported to MIPS.
> > > 
> > > Another helper to find functions that do excessive static allocations is
> > > "make checkstack".
> > 
> > Both SH and sparc use the mcount function (enabled with the -pg switch
> > to gcc) to check the stack has not overflowed. The relevant code is in
> > arch/{sh,sparc}/lib/mcount.S. This checks the stack pointer value on
> > every function call. Yeah, it's heavy-weight, but an implementation for
> > MIPS should be able to catch almost the exact point at which stack
> > overflow occurs.
> 
> Which often isn't so helpful.  The alarm gets triggered on the last stack
> pointer decrement but according to murphy the overflow has happened 10
> levels up in the callchain.

Last decrement? The alarm should be triggered the next time the
function in which the overflow occurs makes a function call. I don't
see how you could go down a level of the callchain and not trigger the
alarm if the overflow has happened?

Re: How to detect STACKOVEFLOW on mips

[Ralf Baechle]

On Mon, Jul 05, 2010 at 02:09:31PM +0100, Matt Fleming wrote:

> > Which often isn't so helpful.  The alarm gets triggered on the last stack
> > pointer decrement but according to murphy the overflow has happened 10
> > levels up in the callchain.
> 
> Last decrement? The alarm should be triggered the next time the
> function in which the overflow occurs makes a function call. I don't
> see how you could go down a level of the callchain and not trigger the
> alarm if the overflow has happened?

guilt()
{
	char array[6000];

	blurb(&array);
}

blurb(void *p)
{
	frob(p);
}

With the deep nesting of the current kernel there is a good chance a
check in mcount will not be triggered in blurb() but possibly in frob
or even further down the callchain.

  Ralf

Re: How to detect STACKOVEFLOW on mips

[Matt Fleming]

On Mon, Jul 05, 2010 at 02:35:33PM +0100, Ralf Baechle wrote:
> On Mon, Jul 05, 2010 at 02:09:31PM +0100, Matt Fleming wrote:
> 
> > > Which often isn't so helpful.  The alarm gets triggered on the last stack
> > > pointer decrement but according to murphy the overflow has happened 10
> > > levels up in the callchain.
> > 
> > Last decrement? The alarm should be triggered the next time the
> > function in which the overflow occurs makes a function call. I don't
> > see how you could go down a level of the callchain and not trigger the
> > alarm if the overflow has happened?
> 
> guilt()
> {
> 	char array[6000];
> 
> 	blurb(&array);
> }
> 
> blurb(void *p)
> {
> 	frob(p);
> }
> 
> With the deep nesting of the current kernel there is a good chance a
> check in mcount will not be triggered in blurb() but possibly in frob
> or even further down the callchain.

Ah, I think I see what you mean. You're saying that you may not find
the culprit function using the massive amount of stack spac, which
eventually leads to the overflow? Yeah, that's a fair point. I think
if the mcount technique was used in conjunction with
CONFIG_DEBUG_STACK_USAGE it might be more helpful in that situation.