Re: i386 ABI and the stack

["H. Peter Anvin" <hpa@zytor.com>]

Albert Cahalan wrote:
> I just saw git commit 21528454f6dd18231ae20102f98aa8f51b6ec1b9
> go in with this:
> 
> + * Accessing the stack below %esp is always a bug.
> + * The large cushion allows instructions like enter
> + * and pusha to work. ("enter $65535,$31" pushes
> + * 32 pointers and then decrements %esp by 65535.)
> 
> Exactly how is an access below %esp a bug if we just added support?
> It looks like we now have a 65664-byte red zone on i386, and probably
> on x86-64 once the matching patch goes in. (the space reserved by
> signal handlers may differ, though perhaps it should not)

No, we don't.  The enter instruction is special because it *atomically* 
drops the stack and probes the stack pointer; if the instruction fails, 
then the stack pointer is rolled back, which is why the kernel needs to 
be aware of it.

We could add a redzone to i386 (and then get compilers to know about 
it), but we haven't already done so.  The difference is that we'd have 
to adjust the stack pointer before writing a signal stack frame. 
However, libc probably needs to be aware of this, because this zone 
needs to also be reserved for every stack in a threaded program.

> This is water under the bridge anyway, because of gcc 2.xx.x bugs.
> 
> It seems that we're throwing away performance if we discourage
> the compiler from taking advantage of this area to optimize
> leaf functions and perhaps improve instruction scheduling.

Probably, although likely not much; x86 processors tend to need to 
optimize push/pop anyway.  However, as x86-64 shows, having a small 
redzone might be worth it.

	-hpa