Re: Segfault on the i386 enter instruction

[Denis Vlasenko <vda@ilport.com.ua>]

On Friday 12 May 2006 17:42, linux-os (Dick Johnson) wrote:
> >>> 	enter $10008, $0
> >>> #	pushl %ebp
> >>> #	movl %esp,%ebp
> >>> #	subl $10008,%esp
> >>> 	addl $-12,%esp
> >>          ^^^^^^^^^^^^^^____________ WTF
> >>          adding a negative number is subtracting that positive value.
> >>          You just subtracted 0xfffffff3 (on a 32-bit machine) from
> >>          the stack pointer. It damn-well better seg-fault!
> >
> > No. Try it yourself.
> > --
> > vda
> 
> It doesn't matter. It means that you still own the space there
> (it's mapped into your process). The code is bogus, broken beyond
> all repair. It has nothing to do with 'enter' it has to do with
> putting the stack pointer (wrapping it) to somewhere it shouldn't
> be. The stack pointer is normally around 0xafff0000. It just got
> wrapped down past zero up to fafff00d, then stuff got pushed
> onto it for the call.

Obviously you

(a) Don't want to actually try to compile and run it.
It will run. For Tomasz, it runs ok with 3-insn instruction sequence
instead of enter. For me, it works just fine with enter. But it works.
Why do you think it is not enough?

(b) can't do 32-bit math. You made two mistakes.
    -12 is 0xfffffff4, not 0xfffffff3.
    0xafff0000 + 0xfffffff4 = 0xaffefff4, not 0xfafff00d

and
(c) do not realize that 32bit i386+ CPUs check segment limits
    AFTER performing 32bit math (i.e. overflow into 33th bit
    is truncated instead of triggering limit violation)
--
vda