Segfault on the i386 enter instruction

Segfault on the i386 enter instruction

[Tomasz Malesinski]

The code attached below segfaults on the enter instruction. It works
when a stack frame is created by the three commented out
instructions and also when the first operand of the enter instruction
is small (less than about 6500 on my system).

AFAIK, the only difference between creating a stack frame with the
enter instruction or push/mov/sub is that enter checks if the new
value of esp is inside the stack segment limit.

I tested it on a vanilla kernel 2.4.26 on Intel Celeron and also on
probably non-vanilla 2.6.16.13 running on 3 dual core AMD Opteron,
quite busy, server. It is working in 32-bit mode. Interestingly, on
the second machine sometimes the program worked correctly.

I am not subscribed to the list. Please cc replies to me.


	.file	"a.c"
	.version	"01.01"
gcc2_compiled.:
.section	.rodata
.LC0:
	.string	"asdf\n"
.text
	.align 4
.globl main
	.type	 main,@function
main:
	enter $10008, $0
#	pushl %ebp
#	movl %esp,%ebp
#	subl $10008,%esp
	addl $-12,%esp
	pushl $.LC0
	call printf
	addl $16,%esp
.L2:
	leave
	ret
.Lfe1:
	.size	 main,.Lfe1-main
	.ident	"GCC: (GNU) 2.95.4 20011002 (Debian prerelease)"

-- 
Tomek Malesinski

Re: Segfault on the i386 enter instruction

[Denis Vlasenko]

On Friday 12 May 2006 16:16, Tomasz Malesinski wrote:
> The code attached below segfaults on the enter instruction. It works
> when a stack frame is created by the three commented out
> instructions and also when the first operand of the enter instruction
> is small (less than about 6500 on my system).
> 
> AFAIK, the only difference between creating a stack frame with the
> enter instruction or push/mov/sub is that enter checks if the new
> value of esp is inside the stack segment limit.
> 
> I tested it on a vanilla kernel 2.4.26 on Intel Celeron and also on
> probably non-vanilla 2.6.16.13 running on 3 dual core AMD Opteron,
> quite busy, server. It is working in 32-bit mode. Interestingly, on
> the second machine sometimes the program worked correctly.

Does not segfault for me:

# gcc Segfault.S

# ./a.out
asdf

# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 11
model name      : Intel(R) Celeron(TM) CPU                1200MHz
stepping        : 1
cpu MHz         : 1196.201
cache size      : 256 KB
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 2
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr sse
bogomips        : 2395.77

# gcc -v 2>&1 | tail -1
gcc version 3.4.3

--
vda
--
vda

Re: Segfault on the i386 enter instruction

[Andi Kleen]

Tomasz Malesinski <tmal@mimuw.edu.pl> writes:

> The code attached below segfaults on the enter instruction. It works
> when a stack frame is created by the three commented out
> instructions and also when the first operand of the enter instruction
> is small (less than about 6500 on my system).

The difference is the value of the stack pointer when the page fault
of extending the stack downwards occurs. For the long sequence 
ESP is already changed when it happens. For ENTER the CPU undoes
the change before raising the fault. The page fault handler
checks the page fault against ESP to catch invalid references below
the stack.

I don't think the 64bit kernel does anything different here than the 
32bit kernel. I tested it on a 32bit box and it faulted there too.

Handling it like you expect would require to disassemble 
the function in the page fault handler and it's probably not 
worth doing that for this weird case.

-Andi

Re: Segfault on the i386 enter instruction

[Denis Vlasenko]

On Friday 12 May 2006 16:50, Andi Kleen wrote:
> Tomasz Malesinski <tmal@mimuw.edu.pl> writes:
> 
> > The code attached below segfaults on the enter instruction. It works
> > when a stack frame is created by the three commented out
> > instructions and also when the first operand of the enter instruction
> > is small (less than about 6500 on my system).
> 
> The difference is the value of the stack pointer when the page fault
> of extending the stack downwards occurs. For the long sequence 
> ESP is already changed when it happens. For ENTER the CPU undoes
> the change before raising the fault. The page fault handler
> checks the page fault against ESP to catch invalid references below
> the stack.
> 
> I don't think the 64bit kernel does anything different here than the 
> 32bit kernel. I tested it on a 32bit box and it faulted there too.

For me, it doesn't fault. I looked with strace. It doesn't fault even with
enter $64008, $0

Is it something in newest kernels? Mine is 2.6.16-rc5.
--
vda

Re: Segfault on the i386 enter instruction

[Tomasz Malesinski]

On Fri, May 12, 2006 at 03:50:20PM +0200, Andi Kleen wrote:
> Handling it like you expect would require to disassemble 
> the function in the page fault handler and it's probably not 
> worth doing that for this weird case.

Does it mean that the ENTER instruction should not be used to create
stack frames in Linux programs?

-- 
Tomek Malesinski

Re: Segfault on the i386 enter instruction

[Bart Hartgers]

Tomasz Malesinski wrote:
> On Fri, May 12, 2006 at 03:50:20PM +0200, Andi Kleen wrote:
>> Handling it like you expect would require to disassemble 
>> the function in the page fault handler and it's probably not 
>> worth doing that for this weird case.
> 
> Does it mean that the ENTER instruction should not be used to create
> stack frames in Linux programs?
> 

Basically, yes. Here is a link to a relevant discussion in the 2.2.7 era:

http://groups.google.co.nz/groups?selm=7i86ni%24b7n%241%40palladium.transmeta.com

And perhaps x86-64 is handled different because of the red zone (some
memory below the stack-pointer that can be accessed legally)?

Groeten,
Bart

-- 
Bart Hartgers - TUE Eindhoven - http://plasimo.phys.tue.nl/bart/contact/

Re: Segfault on the i386 enter instruction

[Andi Kleen]

> http://groups.google.co.nz/groups?selm=7i86ni%24b7n%241%40palladium.transmeta.com

Linus' reasoning is actually outdated - on most modern x86s it is not
slower than the expanded code sequence because it will generate the
same number of macro/u-ops. But it would be still extremly ugly to 
implement, which is a good reason not to.

> And perhaps x86-64 is handled different because of the red zone (some
> memory below the stack-pointer that can be accessed legally)?

Yes, but it's only 128 bytes so it won't help for larger frames.
It will also work if you preextended the stack before, but i wouldn't
rely on it.

-Andi

Re: Segfault on the i386 enter instruction

[linux-os (Dick Johnson)]

On Mon, 15 May 2006, Bart Hartgers wrote:

> Tomasz Malesinski wrote:
>> On Fri, May 12, 2006 at 03:50:20PM +0200, Andi Kleen wrote:
>>> Handling it like you expect would require to disassemble
>>> the function in the page fault handler and it's probably not
>>> worth doing that for this weird case.
>>
>> Does it mean that the ENTER instruction should not be used to create
>> stack frames in Linux programs?
>>
>
> Basically, yes. Here is a link to a relevant discussion in the 2.2.7 era:
>
> http://groups.google.co.nz/groups?selm=7i86ni%24b7n%241%40palladium.transmeta.com
>
> And perhaps x86-64 is handled different because of the red zone (some
> memory below the stack-pointer that can be accessed legally)?
>
> Groeten,
> Bart

The enter instruction works perfectly fine. The processors were
designed to use both enter and leave. There are no prohibitions
against their use. It's just that if you play games with assembly
so you create a stack-pointer wrap situation, you can get a
bounds error.


Cheers,
Dick Johnson
Penguin : Linux version 2.6.16.4 on an i686 machine (5592.89 BogoMips).
New book: http://www.lymanschool.com
_
\x1a\x04

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Re: Segfault on the i386 enter instruction

[Bart Hartgers]

linux-os (Dick Johnson) wrote:
> On Mon, 15 May 2006, Bart Hartgers wrote:
> 
>> Tomasz Malesinski wrote:
>>> On Fri, May 12, 2006 at 03:50:20PM +0200, Andi Kleen wrote:
>>>> Handling it like you expect would require to disassemble
>>>> the function in the page fault handler and it's probably not
>>>> worth doing that for this weird case.
>>> Does it mean that the ENTER instruction should not be used to create
>>> stack frames in Linux programs?
>>>
>> Basically, yes. Here is a link to a relevant discussion in the 2.2.7 era:
>>
>> http://groups.google.co.nz/groups?selm=7i86ni%24b7n%241%40palladium.transmeta.com
>>
>> And perhaps x86-64 is handled different because of the red zone (some
>> memory below the stack-pointer that can be accessed legally)?
>>
>> Groeten,
>> Bart
> 
> The enter instruction works perfectly fine. The processors were
> designed to use both enter and leave. There are no prohibitions
> against their use. It's just that if you play games with assembly
> so you create a stack-pointer wrap situation, you can get a
> bounds error.

No. The assembly is fine. Also enter does what it is supposed to do. The
problem is that enter can cause a pagefault on an address (far) below
the %esp, and Linu[xs] considers that an error (for good reasons).

Groeten,
Bart
-- 
Bart Hartgers - TUE Eindhoven - http://plasimo.phys.tue.nl/bart/contact/

Re: Segfault on the i386 enter instruction

[linux-os (Dick Johnson)]

On Fri, 12 May 2006, Tomasz Malesinski wrote:

> The code attached below segfaults on the enter instruction. It works
> when a stack frame is created by the three commented out
> instructions and also when the first operand of the enter instruction
> is small (less than about 6500 on my system).
>
> AFAIK, the only difference between creating a stack frame with the
> enter instruction or push/mov/sub is that enter checks if the new
> value of esp is inside the stack segment limit.
>
> I tested it on a vanilla kernel 2.4.26 on Intel Celeron and also on
> probably non-vanilla 2.6.16.13 running on 3 dual core AMD Opteron,
> quite busy, server. It is working in 32-bit mode. Interestingly, on
> the second machine sometimes the program worked correctly.
>
> I am not subscribed to the list. Please cc replies to me.
>
>
> 	.file	"a.c"
> 	.version	"01.01"
> gcc2_compiled.:
> .section	.rodata
> .LC0:
> 	.string	"asdf\n"
> .text
> 	.align 4
> .globl main
> 	.type	 main,@function
> main:
> 	enter $10008, $0
> #	pushl %ebp
> #	movl %esp,%ebp
> #	subl $10008,%esp
> 	addl $-12,%esp
         ^^^^^^^^^^^^^^____________ WTF
         adding a negative number is subtracting that positive value.
         You just subtracted 0xfffffff3 (on a 32-bit machine) from
         the stack pointer. It damn-well better seg-fault!


> 	pushl $.LC0
> 	call printf
> 	addl $16,%esp
> .L2:
> 	leave
> 	ret
> .Lfe1:
> 	.size	 main,.Lfe1-main
> 	.ident	"GCC: (GNU) 2.95.4 20011002 (Debian prerelease)"
>
> --
> Tomek Malesinski
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Cheers,
Dick Johnson
Penguin : Linux version 2.6.16.4 on an i686 machine (5592.89 BogoMips).
New book: http://www.lymanschool.com
_
\x1a\x04

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Re: Segfault on the i386 enter instruction

[Denis Vlasenko]

On Friday 12 May 2006 17:07, linux-os (Dick Johnson) wrote:
> > 	.file	"a.c"
> > 	.version	"01.01"
> > gcc2_compiled.:
> > .section	.rodata
> > .LC0:
> > 	.string	"asdf\n"
> > .text
> > 	.align 4
> > .globl main
> > 	.type	 main,@function
> > main:
> > 	enter $10008, $0
> > #	pushl %ebp
> > #	movl %esp,%ebp
> > #	subl $10008,%esp
> > 	addl $-12,%esp
>          ^^^^^^^^^^^^^^____________ WTF
>          adding a negative number is subtracting that positive value.
>          You just subtracted 0xfffffff3 (on a 32-bit machine) from
>          the stack pointer. It damn-well better seg-fault!

No. Try it yourself.
--
vda

Re: Segfault on the i386 enter instruction

[linux-os (Dick Johnson)]

On Fri, 12 May 2006, Denis Vlasenko wrote:

> On Friday 12 May 2006 17:07, linux-os (Dick Johnson) wrote:
>>> 	.file	"a.c"
>>> 	.version	"01.01"
>>> gcc2_compiled.:
>>> .section	.rodata
>>> .LC0:
>>> 	.string	"asdf\n"
>>> .text
>>> 	.align 4
>>> .globl main
>>> 	.type	 main,@function
>>> main:
>>> 	enter $10008, $0
>>> #	pushl %ebp
>>> #	movl %esp,%ebp
>>> #	subl $10008,%esp
>>> 	addl $-12,%esp
>>          ^^^^^^^^^^^^^^____________ WTF
>>          adding a negative number is subtracting that positive value.
>>          You just subtracted 0xfffffff3 (on a 32-bit machine) from
>>          the stack pointer. It damn-well better seg-fault!
>
> No. Try it yourself.
> --
> vda

It doesn't matter. It means that you still own the space there
(it's mapped into your process). The code is bogus, broken beyond
all repair. It has nothing to do with 'enter' it has to do with
putting the stack pointer (wrapping it) to somewhere it shouldn't
be. The stack pointer is normally around 0xafff0000. It just got
wrapped down past zero up to fafff00d, then stuff got pushed
onto it for the call.

>

Cheers,
Dick Johnson
Penguin : Linux version 2.6.16.4 on an i686 machine (5592.89 BogoMips).
New book: http://www.lymanschool.com
_
\x1a\x04

****************************************************************
The information transmitted in this message is confidential and may be privileged.  Any review, retransmission, dissemination, or other use of this information by persons or entities other than the intended recipient is prohibited.  If you are not the intended recipient, please notify Analogic Corporation immediately - by replying to this message or by sending an email to DeliveryErrors@analogic.com - and destroy all copies of this information, including any attachments, without reading or disclosing them.

Thank you.

Re: Segfault on the i386 enter instruction

[Denis Vlasenko]

On Friday 12 May 2006 17:42, linux-os (Dick Johnson) wrote:
> >>> 	enter $10008, $0
> >>> #	pushl %ebp
> >>> #	movl %esp,%ebp
> >>> #	subl $10008,%esp
> >>> 	addl $-12,%esp
> >>          ^^^^^^^^^^^^^^____________ WTF
> >>          adding a negative number is subtracting that positive value.
> >>          You just subtracted 0xfffffff3 (on a 32-bit machine) from
> >>          the stack pointer. It damn-well better seg-fault!
> >
> > No. Try it yourself.
> > --
> > vda
> 
> It doesn't matter. It means that you still own the space there
> (it's mapped into your process). The code is bogus, broken beyond
> all repair. It has nothing to do with 'enter' it has to do with
> putting the stack pointer (wrapping it) to somewhere it shouldn't
> be. The stack pointer is normally around 0xafff0000. It just got
> wrapped down past zero up to fafff00d, then stuff got pushed
> onto it for the call.

Obviously you

(a) Don't want to actually try to compile and run it.
It will run. For Tomasz, it runs ok with 3-insn instruction sequence
instead of enter. For me, it works just fine with enter. But it works.
Why do you think it is not enough?

(b) can't do 32-bit math. You made two mistakes.
    -12 is 0xfffffff4, not 0xfffffff3.
    0xafff0000 + 0xfffffff4 = 0xaffefff4, not 0xfafff00d

and
(c) do not realize that 32bit i386+ CPUs check segment limits
    AFTER performing 32bit math (i.e. overflow into 33th bit
    is truncated instead of triggering limit violation)
--
vda

Re: Segfault on the i386 enter instruction

[Bill Davidsen]

linux-os (Dick Johnson) wrote:
> On Fri, 12 May 2006, Tomasz Malesinski wrote:
> 
>> The code attached below segfaults on the enter instruction. It works
>> when a stack frame is created by the three commented out
>> instructions and also when the first operand of the enter instruction
>> is small (less than about 6500 on my system).
>>
>> AFAIK, the only difference between creating a stack frame with the
>> enter instruction or push/mov/sub is that enter checks if the new
>> value of esp is inside the stack segment limit.
>>
>> I tested it on a vanilla kernel 2.4.26 on Intel Celeron and also on
>> probably non-vanilla 2.6.16.13 running on 3 dual core AMD Opteron,
>> quite busy, server. It is working in 32-bit mode. Interestingly, on
>> the second machine sometimes the program worked correctly.
>>
>> I am not subscribed to the list. Please cc replies to me.
>>
>>
>> 	.file	"a.c"
>> 	.version	"01.01"
>> gcc2_compiled.:
>> .section	.rodata
>> .LC0:
>> 	.string	"asdf\n"
>> .text
>> 	.align 4
>> .globl main
>> 	.type	 main,@function
>> main:
>> 	enter $10008, $0
>> #	pushl %ebp
>> #	movl %esp,%ebp
>> #	subl $10008,%esp
>> 	addl $-12,%esp
>          ^^^^^^^^^^^^^^____________ WTF
>          adding a negative number is subtracting that positive value.
Right, adding -12 is the same as subtracting 12. I have no idea what 
you're getting at with the next two lines.
>          You just subtracted 0xfffffff3 (on a 32-bit machine) from
>          the stack pointer. It damn-well better seg-fault!
No, we subtracted 12. I'm not sure where that number came from, it's the 
1's complement of 12 but I'm dead sure Linux code isn't running on any 
1's comp machines.

-- 
    -bill davidsen (davidsen@tmr.com)
"The secret to procrastination is to put things off until the
  last possible moment - but no longer"  -me

Re: Segfault on the i386 enter instruction

[Stas Sergeev]

Hi.

Andi Kleen wrote:
> Handling it like you expect would require to disassemble 
> the function in the page fault handler and it's probably not 
> worth doing that for this weird case.
Just wondering, is this case really that weird?
In fact, the check against %esp that the kernel
does, looks strange. I realize that it can catch a
(very rare) user-space bug of accessing below %esp, but
other than that it looks redundant (IMHO) and as soon as
it triggers the false-positives, what is it really good for?
Aren't the rlimit and the other checks of acct_stack_growth()
not enough, or am I missing something obvious?

Re: Segfault on the i386 enter instruction

[Andi Kleen]

> Aren't the rlimit and the other checks of acct_stack_growth()
> not enough, or am I missing something obvious?

Traditionally Linux doesn't have a stack ulimit.

-Andi

Re: Segfault on the i386 enter instruction

[Stas Sergeev]

Hi.

Andi Kleen wrote:
>> Aren't the rlimit and the other checks of acct_stack_growth()
>> not enough, or am I missing something obvious?
> Traditionally Linux doesn't have a stack ulimit.
That clarifies the roots of this %esp check, as without
the stack ulimit and without the proper memory accounting
(the case of 2.4?) such a check is the "last hope" - I've
got the point. But are there the reasons to still keep it
in 2.6, considering also the false-positives? It seems to
have the STACK_RLIMIT and it seems to get the memory accounting
right, and not too many arches seem to have such a check even.

Re: Segfault on the i386 enter instruction

[Andi Kleen]

On Mon, May 15, 2006 at 09:15:31PM +0400, Stas Sergeev wrote:
> Hi.
> 
> Andi Kleen wrote:
> >>Aren't the rlimit and the other checks of acct_stack_growth()
> >>not enough, or am I missing something obvious?
> >Traditionally Linux doesn't have a stack ulimit.
> That clarifies the roots of this %esp check, as without
> the stack ulimit and without the proper memory accounting
> (the case of 2.4?) such a check is the "last hope" - I've
> got the point. But are there the reasons to still keep it
> in 2.6, considering also the false-positives? It seems to
> have the STACK_RLIMIT and it seems to get the memory accounting
> right, and not too many arches seem to have such a check even.

Linux doesn't have a STACK_RLIMIT by default no.
It is set by a few distributions (for use with flexmmap) in PAM, but
not by all. The kernel defaults don't have it.

-Andi

Re: Segfault on the i386 enter instruction

[Stas Sergeev]

Hello.

Andi Kleen wrote:
> Linux doesn't have a STACK_RLIMIT by default no.
By default - OK, I thought you mean that it doesn't have
the STACK_RLIMIT at all (doesn't implement it).

> It is set by a few distributions (for use with flexmmap) in PAM, but
> not by all. The kernel defaults don't have it.
That might explain why I get
$ ulimit -s
8192
on fedora core.
Thanks for explanations.

Re: Segfault on the i386 enter instruction

[Lee Revell]

On Mon, 2006-05-15 at 23:38 +0400, Stas Sergeev wrote:

> > It is set by a few distributions (for use with flexmmap) in PAM, but
> > not by all. The kernel defaults don't have it.
> That might explain why I get
> $ ulimit -s
> 8192
> on fedora core.
> Thanks for explanations.

Just FYI, this does actually have an important effect on multithreaded
programs - glibc will allocate RLIMIT_STACK for each thread stack.  If
mlockall() is used this can eat quite a bit of memory.  It's a real
world problem for some pro audio apps.

Lee

Re: Segfault on the i386 enter instruction

[Ingo Oeser]

Hi Lee,

On Monday, 15. May 2006 21:56, Lee Revell wrote:
> Just FYI, this does actually have an important effect on multithreaded
> programs - glibc will allocate RLIMIT_STACK for each thread stack.  If
> mlockall() is used this can eat quite a bit of memory.  It's a real
> world problem for some pro audio apps.

If it is: pthread_attr_setstacksize() is your friend.
If you like to use the big hammer: just lower RLIMIT_STACK.

So no unsolvable real world problem here :-)


Regards

Ingo Oeser

Re: Segfault on the i386 enter instruction

[Lee Revell]

On Tue, 2006-05-16 at 00:49 +0200, Ingo Oeser wrote:
> Hi Lee,
> 
> On Monday, 15. May 2006 21:56, Lee Revell wrote:
> > Just FYI, this does actually have an important effect on multithreaded
> > programs - glibc will allocate RLIMIT_STACK for each thread stack.  If
> > mlockall() is used this can eat quite a bit of memory.  It's a real
> > world problem for some pro audio apps.
> 
> If it is: pthread_attr_setstacksize() is your friend.
> If you like to use the big hammer: just lower RLIMIT_STACK.
> 
> So no unsolvable real world problem here :-)
> 

Yep, that's exactly what we do in JACK.  POSIX makes it quite explicit
that one should not assume the default thread stack size is sane...

Lee

Re: Segfault on the i386 enter instruction

[Chuck Ebbert]

In-Reply-To: <44676F42.7080907@aknet.ru>

On Sun, 14 May 2006 21:56:18 +0400, Stas Sergeev wrote:

> Andi Kleen wrote:
> > Handling it like you expect would require to disassemble 
> > the function in the page fault handler and it's probably not 
> > worth doing that for this weird case.
> Just wondering, is this case really that weird?
> In fact, the check against %esp that the kernel
> does, looks strange. I realize that it can catch a
> (very rare) user-space bug of accessing below %esp, but
> other than that it looks redundant (IMHO) and as soon as
> it triggers the false-positives, what is it really good for?

I can't get a SIGSEGV on any native i386 kernel, not even when
running on AMD64.  It only happens on native x86_64 kernels.

Looking at the AMD instruction manual, the pseudo-code for 'enter'
ends with:

RSP.s = RSP - temp_ALLOC_SPACE // Leave "temp_ALLOC_SPACE" free bytes on
                               // the stack
WRITE_MEM.v [SS:RSP.s] = temp_unused // ENTER finishes with a memory write
                                     // check on the final stack pointer,
                                     // but no write actually occurs.
RBP.v = temp_RBP
EXIT


And the Intel manual says:

IF 64-Bit Mode (StackSize = 64)
THEN
        RBP = FrameTemp;
        RSP = RSP - Size;
ELSE IF StackSize = 32
THEN
        EBP = FrameTemp;
        ESP = ESP - Size;
ELSE (* StackSize = 16 *)
        BP = FrameTemp;
        SP = SP - Size;
FI;
END;


Intel says nothing about a write check.  Is that a mistake in the manual
or is that something only AMD64 does, and then only in long mode?


-- 
Chuck

"The x86 isn't all that complex -- it just doesn't make a lot of sense."
                                                        -- Mike Johnson

Re: Segfault on the i386 enter instruction

[Andi Kleen]

On Tuesday 16 May 2006 04:29, Chuck Ebbert wrote:
> In-Reply-To: <44676F42.7080907@aknet.ru>
> 
> On Sun, 14 May 2006 21:56:18 +0400, Stas Sergeev wrote:
> 
> > Andi Kleen wrote:
> > > Handling it like you expect would require to disassemble 
> > > the function in the page fault handler and it's probably not 
> > > worth doing that for this weird case.
> > Just wondering, is this case really that weird?
> > In fact, the check against %esp that the kernel
> > does, looks strange. I realize that it can catch a
> > (very rare) user-space bug of accessing below %esp, but
> > other than that it looks redundant (IMHO) and as soon as
> > it triggers the false-positives, what is it really good for?
> 
> I can't get a SIGSEGV on any native i386 kernel, not even when
> running on AMD64.  It only happens on native x86_64 kernels.

I reproduced the original SIGSEGV on several i386 kernels.

> Intel says nothing about a write check.  Is that a mistake in the manual
> or is that something only AMD64 does, and then only in long mode?

In 98+% of all cases when Intel and AMD documentation differ
in subtle detail it's a documentation bug.

-Andi
 

Re: Segfault on the i386 enter instruction

[Chuck Ebbert]

In-Reply-To: <200605161132.18610.ak@suse.de>

On Tue, 16 May 2006 11:32:18 +0200, Andi Kleen wrote:

> On Tuesday 16 May 2006 04:29, Chuck Ebbert wrote:
> > In-Reply-To: <44676F42.7080907@aknet.ru>
> > 
> > On Sun, 14 May 2006 21:56:18 +0400, Stas Sergeev wrote:
> > 
> > > Andi Kleen wrote:
> > > > Handling it like you expect would require to disassemble 
> > > > the function in the page fault handler and it's probably not 
> > > > worth doing that for this weird case.
> > > Just wondering, is this case really that weird?
> > > In fact, the check against %esp that the kernel
> > > does, looks strange. I realize that it can catch a
> > > (very rare) user-space bug of accessing below %esp, but
> > > other than that it looks redundant (IMHO) and as soon as
> > > it triggers the false-positives, what is it really good for?
> > 
> > I can't get a SIGSEGV on any native i386 kernel, not even when
> > running on AMD64.  It only happens on native x86_64 kernels.
> 
> I reproduced the original SIGSEGV on several i386 kernels.

OK, I got SIGSEGV on a 2.6.9 i386 kernel in addition to ia32 mode on x86_64.
But it doesn't happen on any recent 2.6, even with "enter $65535,$0".
Digging, I found the stack vma is 22 pages (88k) on recent i386
kernels while it's only 8k on 2.6.9 and 12k in x86_64 ia32 emulation.
So you have to go deeper into the stack before you will hit this on
recent i386 kernels.
 
> > Intel says nothing about a write check.  Is that a mistake in the manual
> > or is that something only AMD64 does, and then only in long mode?
> 
> In 98+% of all cases when Intel and AMD documentation differ
> in subtle detail it's a documentation bug.

Yeah, that's why it's good to have both on hand.  But sometimes it can be
hard to tell which one is wrong. :)


BTW one easy way to fix this bug would be to enlarge the window for
access below the stack pointer to allow for the largest possible enter
instruction, i.e. "enter $65535,$31".  On x86_64 that would be 65536+256
instead of the current 128 bytes.

-- 
Chuck
"The x86 isn't all that complex -- it just doesn't make a lot of sense."
                                                        -- Mike Johnson