Hello.

Denis Vlasenko wrote:
>> This is what happens: CPU is doing IRET in 32bit mode.
>> Stack layout:
> +20 (empty) <--- SS:ESP
> +16 old_CS
> +12 old_EIP
>  +8 old_EFLAGS
>  +4 old_SS
>  +0 old_ESP
JFYI, looks like you swapped CS<-->EIP and SS<-->ESP
on your stack frame layout.
 
> If old_SS references 'small' data segment (16-bit one),
> processor does not restore ESP from old_ESP.
> It restores lower 16 bits only. Upper 16 bits are filled
> with garbage (or just not modified at all?).
Not modified at all, yes. That's why it is always
greater than TASK_SIZE (0xc0000000) - it is still
in a kernel space.

> This works fine because processor uses SP, not ESP,
> for subsequent push/pop/call/ret operations.
> But if code uses full ESP, thinking that upper 16 bits are zero,
> it will crash badly. Correct me if I'm wrong.
That's correct. But you should note that the
program not just "thinks" that the upper 16 bits
are zero. It writes zero there itself, and a few
instructions later - oops, it is no longer zero...
My test-case does "hlt" to force the context switch,
but in fact it is not necessary. I tried an
empty loop instead of HLT. If the loop is long
enough and some IRQ being handled by the kernel
in a mean time, the ESP is corrupted. So the
user-space never knows when exactly it gets corrupted.
And, as I mentioned already, since the code is 32bit,
it just does the things like "mov ebp,esp" and
uses ebp to access the function params and locals,
which crashes, or uses ESP directly to access
something, etc.

> Hmm. I think you need to *emulate* this IRET.
> Is this IRET happens in kernel or in dosemu code?
The problem happens only when iret is used to
switch to another priv level. So it is the kernel's
iret of course, something to the effect of entry.S:128
I think. The control is transferred from kernel code
to the DOS code directly. dosemu code is completely
bypassed. dosemu have no chances to intercept that.
The DOS program is running on its own (as long as
it doesn't trigger an exception, or SIGALRM comes),
and the CPU is corrupting its ESP in *any* place,
since it happens when an external IRQ arrives.

>> That's strange indeed! Apparently your ESP value,
>> 0xccf1fb10, is greater than 0xc0000000, in which
>> case my program should just write "BUG!", but for
>> you it doesn't. Haven't you altered the code
>> somehow?
> No, I didn't alter anything. Strange indeed.
So that proves that writing the bug-free code, esp.
when the asm is involved, is not always as easy as
I assume...

> I am thoroughly confused now. Two back-to-back
> 'printf("Now sp=%08lx\n", new_esp);'
> gave different result.  How this can happen??
My code did the assumption that the ESP should
not change within the execution of main(). This
is true only if you don't enable the optimization
(as I did), or use -fno-defer-pop option of gcc.
Fixed code is attached (just for the overall
completeness:) Should survive the optimization now.