Re: Segfault on the i386 enter instruction

[Andi Kleen <ak@suse.de>]

Tomasz Malesinski <tmal@mimuw.edu.pl> writes:

> The code attached below segfaults on the enter instruction. It works
> when a stack frame is created by the three commented out
> instructions and also when the first operand of the enter instruction
> is small (less than about 6500 on my system).

The difference is the value of the stack pointer when the page fault
of extending the stack downwards occurs. For the long sequence 
ESP is already changed when it happens. For ENTER the CPU undoes
the change before raising the fault. The page fault handler
checks the page fault against ESP to catch invalid references below
the stack.

I don't think the 64bit kernel does anything different here than the 
32bit kernel. I tested it on a 32bit box and it faulted there too.

Handling it like you expect would require to disassemble 
the function in the page fault handler and it's probably not 
worth doing that for this weird case.

-Andi