Re: Segfault on the i386 enter instruction

[Bart Hartgers <bart@etpmod.phys.tue.nl>]

linux-os (Dick Johnson) wrote:
> On Mon, 15 May 2006, Bart Hartgers wrote:
> 
>> Tomasz Malesinski wrote:
>>> On Fri, May 12, 2006 at 03:50:20PM +0200, Andi Kleen wrote:
>>>> Handling it like you expect would require to disassemble
>>>> the function in the page fault handler and it's probably not
>>>> worth doing that for this weird case.
>>> Does it mean that the ENTER instruction should not be used to create
>>> stack frames in Linux programs?
>>>
>> Basically, yes. Here is a link to a relevant discussion in the 2.2.7 era:
>>
>> http://groups.google.co.nz/groups?selm=7i86ni%24b7n%241%40palladium.transmeta.com
>>
>> And perhaps x86-64 is handled different because of the red zone (some
>> memory below the stack-pointer that can be accessed legally)?
>>
>> Groeten,
>> Bart
> 
> The enter instruction works perfectly fine. The processors were
> designed to use both enter and leave. There are no prohibitions
> against their use. It's just that if you play games with assembly
> so you create a stack-pointer wrap situation, you can get a
> bounds error.

No. The assembly is fine. Also enter does what it is supposed to do. The
problem is that enter can cause a pagefault on an address (far) below
the %esp, and Linu[xs] considers that an error (for good reasons).

Groeten,
Bart
-- 
Bart Hartgers - TUE Eindhoven - http://plasimo.phys.tue.nl/bart/contact/