Troubles with JIT compiler

Troubles with JIT compiler

[Scott Sibley]

I'm debugging a script engine. The engine compiles expressions into
asm instructions, assigns that data to a function pointer, and
executes the function, passing one argument.

I'm new to assembly, and pretty much stuck on the first issue I ran into.

Here are the function's instructions for a basic assignment operation:

0x8067990:    push   %ebp
0x8067991:    mov    %esp,%ebp
0x8067993:    sub    $0x8,%esp
0x8067999:    fnstcw (%esp)
0x806799c:    mov    (%esp),%eax
0x806799f:    or     $0xc00,%eax
0x80679a4:    mov    %eax,0x4(%esp)
0x80679a8:    fldcw  0x4(%esp)
0x80679ac:    flds   0x806793c
0x80679b2:    fsts   0x805f014
0x80679b8:    fstps  0x8067954
0x80679be:    fldcw  (%esp)
0x80679c1:    add    $0x8,%esp
0x80679c7:    emms
0x80679c9:    leave
0x80679ca:    ret

Well, it appears to be crashing at the first instruction. Here are the
values of ebp and esp.

(gdb) x/x $ebp
0xbffff168:    0xbffff188
(gdb) x/x $esp
0xbffff14c:    0x0804e481

Any clue why this would cause problems? Let me know if I need to
provide more info.
--
To unsubscribe from this list: send the line "unsubscribe linux-assembly" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Troubles with JIT compiler

[Robert Plantz]

On Thu, 2010-01-21 at 22:12 -0600, Scott Sibley wrote:
> I'm debugging a script engine. The engine compiles expressions into
> asm instructions, assigns that data to a function pointer, and
> executes the function, passing one argument.
> 
> I'm new to assembly, and pretty much stuck on the first issue I ran into.
> 
> Here are the function's instructions for a basic assignment operation:
> 
> 0x8067990:    push   %ebp
> 0x8067991:    mov    %esp,%ebp
> 0x8067993:    sub    $0x8,%esp
> 0x8067999:    fnstcw (%esp)
> 0x806799c:    mov    (%esp),%eax
> 0x806799f:    or     $0xc00,%eax
> 0x80679a4:    mov    %eax,0x4(%esp)
> 0x80679a8:    fldcw  0x4(%esp)
> 0x80679ac:    flds   0x806793c
> 0x80679b2:    fsts   0x805f014
> 0x80679b8:    fstps  0x8067954
> 0x80679be:    fldcw  (%esp)
> 0x80679c1:    add    $0x8,%esp
> 0x80679c7:    emms
> 0x80679c9:    leave
> 0x80679ca:    ret
> 
> Well, it appears to be crashing at the first instruction. Here are the
> values of ebp and esp.
> 
> (gdb) x/x $ebp
> 0xbffff168:    0xbffff188
> (gdb) x/x $esp
> 0xbffff14c:    0x0804e481
> 

An immediate problem I see is that the stack pointer is not properly
aligned. This is 32-bit code, and the Intel manual says that the stack
should be aligned at 32-bit addresses. That is, the least significant
digit in esp should be 0, 4, 8, or c.

I also note that the values in ebp and esp are very far apart.
Typically, they contain similar values -- addresses somewhere in the
stack.

I would look at how the stack was set up in this program.

--Bob

Re: Troubles with JIT compiler

[Scott Sibley]

On Thu, Jan 21, 2010 at 11:26 PM, Robert Plantz <plantz@sonoma.edu> wrote:
> On Thu, 2010-01-21 at 22:12 -0600, Scott Sibley wrote:
>> I'm debugging a script engine. The engine compiles expressions into
>> asm instructions, assigns that data to a function pointer, and
>> executes the function, passing one argument.
>>
>> I'm new to assembly, and pretty much stuck on the first issue I ran into.
>>
>> Here are the function's instructions for a basic assignment operation:
>>
>> 0x8067990:    push   %ebp
>> 0x8067991:    mov    %esp,%ebp
>> 0x8067993:    sub    $0x8,%esp
>> 0x8067999:    fnstcw (%esp)
>> 0x806799c:    mov    (%esp),%eax
>> 0x806799f:    or     $0xc00,%eax
>> 0x80679a4:    mov    %eax,0x4(%esp)
>> 0x80679a8:    fldcw  0x4(%esp)
>> 0x80679ac:    flds   0x806793c
>> 0x80679b2:    fsts   0x805f014
>> 0x80679b8:    fstps  0x8067954
>> 0x80679be:    fldcw  (%esp)
>> 0x80679c1:    add    $0x8,%esp
>> 0x80679c7:    emms
>> 0x80679c9:    leave
>> 0x80679ca:    ret
>>
>> Well, it appears to be crashing at the first instruction. Here are the
>> values of ebp and esp.
>>
>> (gdb) x/x $ebp
>> 0xbffff168:    0xbffff188
>> (gdb) x/x $esp
>> 0xbffff14c:    0x0804e481
>>
>
> An immediate problem I see is that the stack pointer is not properly
> aligned. This is 32-bit code, and the Intel manual says that the stack
> should be aligned at 32-bit addresses. That is, the least significant
> digit in esp should be 0, 4, 8, or c.
>
> I also note that the values in ebp and esp are very far apart.
> Typically, they contain similar values -- addresses somewhere in the
> stack.
>
> I would look at how the stack was set up in this program.
>
> --Bob
>
>
>

Hey, Robert. Thanks for replying.

How can I look into how the stack's being setup? This is a C program
that's compiling
data as instruction code into a pointer, and casting that pointer to a
function pointer, then
calling that function pointer. So the C code is managing the stack if
I'm not mistaken.
Correct me if I'm wrong.

Here's where the instructions are compiled:

IL_CORE_COMPILE(avs_x86_compiler_compile)
{
    X86GlobalData *gd = X86_GLOBALDATA(ctx);
    ILInstruction *insn;

    avs_debug(print("X86: Compiling started..."));
    /* Initialize X86 Assembler opcode context */
    x86_context_init(&gd->ctx, 4096, 1024*1024);

    /* Compile function entrance, setup stack frame*/
    x86_emit1(&gd->ctx, pushl, ebp);
    x86_emit2(&gd->ctx, movl, esp, ebp);

    /* Setup floating point rounding mode to integer truncation */
    x86_emit2(&gd->ctx, subl, imm(8), esp);
    x86_emit1(&gd->ctx, fstcw, disp(0, esp));
    x86_emit2(&gd->ctx, movl, disp(0, esp), eax);
    x86_emit2(&gd->ctx, orl, imm(0xc00), eax);
    x86_emit2(&gd->ctx, movl, eax, disp(4, esp));
    x86_emit1(&gd->ctx, fldcw, disp(4, esp));

    for (insn=avs_il_tree_base(tree); insn != NULL; insn = insn->next) {
        avs_debug(print("X86: Compiling instruction: %p", insn));
        compile_opcode(gd, obj, insn);
    }

    /* Restore floating point rounding mode */
    x86_emit1(&gd->ctx, fldcw, disp(0, esp));
    x86_emit2(&gd->ctx, addl, imm(8), esp);

    /* Cleanup stack frame */
    x86_emit0(&gd->ctx, emms);
    x86_emit0(&gd->ctx, leave);
    x86_emit0(&gd->ctx, ret);

    /* Link machine */
    obj->run = (AvsRunnableExecuteCall) gd->ctx.buf;
    avs_debug(print("X86: Compiling finished..."));
    avs_debug(print("X86: Function: %p", obj->run));
    return 0;
}
--
To unsubscribe from this list: send the line "unsubscribe linux-assembly" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Troubles with JIT compiler

[Robert Plantz]

Very big oops here! See below.


> >>
> >> Well, it appears to be crashing at the first instruction. Here are the
> >> values of ebp and esp.
> >>
> >> (gdb) x/x $ebp
> >> 0xbffff168:    0xbffff188
> >> (gdb) x/x $esp
> >> 0xbffff14c:    0x0804e481
> >>

I misread your gdb display. I'm used to using the i r command to look in
the registers.

You used the x/x command, which shows the contents of the register, then
what it is pointing to.

So the values in ebp and esp DO look reasonable. That's the range I
expect in 32-bit program, and they are reasonably close to each other.

I apologize for my error.

From what you have posted, I don't see any problems with the first few
instructions. So I have no idea why the program is crashing there.

Sorry...

--Bob

Re: Troubles with JIT compiler

[Scott Sibley]

On Fri, Jan 22, 2010 at 1:01 AM, Robert Plantz <plantz@sonoma.edu> wrote:
> Very big oops here! See below.
>
>
>> >>
>> >> Well, it appears to be crashing at the first instruction. Here are the
>> >> values of ebp and esp.
>> >>
>> >> (gdb) x/x $ebp
>> >> 0xbffff168:    0xbffff188
>> >> (gdb) x/x $esp
>> >> 0xbffff14c:    0x0804e481
>> >>
>
> I misread your gdb display. I'm used to using the i r command to look in
> the registers.
>
> You used the x/x command, which shows the contents of the register, then
> what it is pointing to.
>
> So the values in ebp and esp DO look reasonable. That's the range I
> expect in 32-bit program, and they are reasonably close to each other.
>
> I apologize for my error.
>
> >From what you have posted, I don't see any problems with the first few
> instructions. So I have no idea why the program is crashing there.
>
> Sorry...
>
> --Bob
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-assembly" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Ah I see. Just for giggles here's the output of 'i r'

(gdb) i r
eax            0x8067888	134641800
ecx            0xbffff07c	-1073745796
edx            0x8067990	134642064
ebx            0x970ff4	9900020
esp            0xbffff14c	0xbffff14c
ebp            0xbffff168	0xbffff168
esi            0x0	0
edi            0x0	0
eip            0x8067990	0x8067990
eflags         0x206	[ PF IF ]
cs             0x73	115
ss             0x7b	123
ds             0x7b	123
es             0x7b	123
fs             0x0	0
gs             0x33	51
--
To unsubscribe from this list: send the line "unsubscribe linux-assembly" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Troubles with JIT compiler

[Scott Sibley]

On Fri, Jan 22, 2010 at 1:16 AM, Scott Sibley <sisibley@gmail.com> wrote:
> On Fri, Jan 22, 2010 at 1:01 AM, Robert Plantz <plantz@sonoma.edu> wrote:
>> Very big oops here! See below.
>>
>>
>>> >>
>>> >> Well, it appears to be crashing at the first instruction. Here are the
>>> >> values of ebp and esp.
>>> >>
>>> >> (gdb) x/x $ebp
>>> >> 0xbffff168:    0xbffff188
>>> >> (gdb) x/x $esp
>>> >> 0xbffff14c:    0x0804e481
>>> >>
>>
>> I misread your gdb display. I'm used to using the i r command to look in
>> the registers.
>>
>> You used the x/x command, which shows the contents of the register, then
>> what it is pointing to.
>>
>> So the values in ebp and esp DO look reasonable. That's the range I
>> expect in 32-bit program, and they are reasonably close to each other.
>>
>> I apologize for my error.
>>
>> >From what you have posted, I don't see any problems with the first few
>> instructions. So I have no idea why the program is crashing there.
>>
>> Sorry...
>>
>> --Bob
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-assembly" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
> Ah I see. Just for giggles here's the output of 'i r'
>
> (gdb) i r
> eax            0x8067888        134641800
> ecx            0xbffff07c       -1073745796
> edx            0x8067990        134642064
> ebx            0x970ff4 9900020
> esp            0xbffff14c       0xbffff14c
> ebp            0xbffff168       0xbffff168
> esi            0x0      0
> edi            0x0      0
> eip            0x8067990        0x8067990
> eflags         0x206    [ PF IF ]
> cs             0x73     115
> ss             0x7b     123
> ds             0x7b     123
> es             0x7b     123
> fs             0x0      0
> gs             0x33     51
>

After someone's tip, I found a solution to this. Passing the linker
'-z execstack' made it run, no problem. It's funny that Linux didn't
complain about it, but rather just seg-faulted.
--
To unsubscribe from this list: send the line "unsubscribe linux-assembly" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Troubles with JIT compiler

[Robert Plantz]

On Fri, 2010-01-22 at 04:45 -0600, Scott Sibley wrote:

> After someone's tip, I found a solution to this. Passing the linker
> '-z execstack' made it run, no problem. It's funny that Linux didn't
> complain about it, but rather just seg-faulted.
> --

Thank you, Scott, for sharing the solution with us.

According to the man page for ld, that option "Marks the object as
requiring executable stack."

I don't know much about jit compilers, so I can't comment on the GNU
development environment not complaining about it. I do know that if
there is something we don't like about it, we can get double our money
back. :-)

--Bob

Re: Troubles with JIT compiler

[Brian Raiter]

> After someone's tip, I found a solution to this. Passing the linker
> '-z execstack' made it run, no problem. It's funny that Linux didn't
> complain about it, but rather just seg-faulted.

But the segfault is the complaint. What else would you hope for? In
general neither the compiler nor the linker has enough information to
reliably warn you of the issue before runtime.

It may also be worth noting that making stack memory non-executable by
default is a relatively recent change to Linux, done to ameliorate the
effectiveness of stack-based code exploits.

b