x86_64 system call entry points

x86_64 system call entry points

[Martin Bisson]

Hi, (first post)

I am trying to trace system calls (entry + exit) on x86_64 architecture, 
so I am basically working on arch/x86_64/kernel/entry.S and 
arch/x86_64/ia32/ia32entry.S (for 32-bit compatibility).  I want to test 
that my code works for all possible ways to enter a system call, which 
are (to my knowledge) the three following instructions:
- int $0x80
- sysenter
- syscall

So these three ways in, used in a 32 or 64 bits executable, makes 6 
possible ways to enter a system call that I need to test.  However, I 
have problems:

- sysenter/32 bits, executed on a 32 bit machine: I get a segfault on 
the sysenter instruction.  I use the following code to enter the system 
call:
pid_t getpid32()
{
    pid_t resultvar;

    asm volatile (
    "push    %%ebp\n\t"
    "push    %%ecx\n\t"
    "push    %%edx\n\t"
    "mov     %%esp,%%ebp\n\t"
    "sysenter\n\t"
    ".space 20,0x90\n\t"
    "pop     %%edx\n\t"
    "pop     %%ecx\n\t"
    "pop     %%ebp\n\t"
    : "=a" (resultvar)   
    : "0" (__NR_getpid)
    : "memory");

    return resultvar;
}

Is there something wrong in the way I pass the parameters?  I know this 
instruction can be tricky because of the way it messes different 
registers...


- int $0x80/64 bits: All system calls return -1 (EINTR).  Is there 
something wrong in the way I call it:
pid_t getpid64()
{
    pid_t resultvar;

    asm volatile (
    "int $0x80\n\t"
    : "=a" (resultvar)   
    : "0" (__NR_getpid)
    : "memory");

    return resultvar;
}


- sysenter/64 bits: I get an illegal instruction.  I've read that it's 
not implemented on AMD-64 (which is what I have).  Is there ANY x86_64 
machine on which this instruction is implemented?  Does this mean that 
the code that handles this case in entry.S has never been run?

- syscall/64 bits: works fine
- int $0x80/32 bits: works fine
- syscall/32 bits: illegal instruction, but I guess that's all right 
because of the machine I use.

Can someone help with any of my issues (entering a system call with 
sysenter/32 bits, int $0x80/64 bits or sysenter/64 bits)?

Thanks in advance,

Mart

P.S.  First post on this list, please reply in private if I did 
something wrong so that I won't do it again.

Re: x86_64 system call entry points

[Andi Kleen]

Martin Bisson <bissonm@discreet.com> writes:
 
> - sysenter/32 bits, executed on a 32 bit machine: I get a segfault on
> the sysenter instruction.  I use the following code to enter the
> system call:
> pid_t getpid32()
> {
>     pid_t resultvar;
> 
>     asm volatile (
>     "push    %%ebp\n\t"
>     "push    %%ecx\n\t"
>     "push    %%edx\n\t"
>     "mov     %%esp,%%ebp\n\t"
>     "sysenter\n\t"

You can't use your own SYSENTER. Since sysenter doesn't pass the original
address and the kernel always returns to a fixed address - which is in the vsyscall
page. The fixed address is used because there are not enough registers to pass
both syscall arguments and return address.

The instruction has a few other quirks and isn't exactly the best one Intel
ever designed.

> - int $0x80/64 bits: All system calls return -1 (EINTR).  Is there
> something wrong in the way I call it:

Hmm, it should do an 32bit syscall even from 64bit. I can take a look.

> pid_t getpid64()
> {
>     pid_t resultvar;
> 
>     asm volatile (
>     "int $0x80\n\t"
>     : "=a" (resultvar)     : "0" (__NR_getpid)
>     : "memory");
> 
>     return resultvar;
> }
> 
> 
> - sysenter/64 bits: I get an illegal instruction.  I've read that it's
> not implemented on AMD-64 (which is what I have).  Is there ANY x86_64
> machine on which this instruction is implemented?  Does this mean that
> the code that handles this case in entry.S has never been run?

Intel machines have it, but they don't have SYSCALL from compat mode. That
is why both are implemented. It should end at the code in ia32entry.S 

Again you can't use your own - so likely it is returning to the vsyscall page
and then jumping to a bogus address on the stack. You can verify that by
single stepping through it with a debugger.


> - syscall/64 bits: works fine
> - int $0x80/32 bits: works fine
> - syscall/32 bits: illegal instruction, but I guess that's all right
> because of the machine I use.

On AMD it should work. The kernel vsyscall page uses it.

-Andi

Re: x86_64 system call entry points II

[Andi Kleen]

Martin Bisson <bissonm@discreet.com> writes:
> 
> - int $0x80/64 bits: All system calls return -1 (EINTR).  Is there
> something wrong in the way I call it:
> pid_t getpid64()
> {
>     pid_t resultvar;
> 
>     asm volatile (
>     "int $0x80\n\t"
>     : "=a" (resultvar)     : "0" (__NR_getpid)
>     : "memory");
> 
>     return resultvar;
> }

I tested it now. Since it ends up as a 32bit syscall you're not 
actually calling 64bit getpid() - but 32bit mkdir(). That is because 32bit and 64bit
have different syscall numbers. For me it returns -14, which is EFAULT.
Expected for a random argument to mkdir()

-Andi

Re: x86_64 system call entry points II

[Martin Bisson]

Andi Kleen wrote:

>Martin Bisson <bissonm@discreet.com> writes:
>  
>
>>- int $0x80/64 bits: All system calls return -1 (EINTR).  Is there
>>something wrong in the way I call it:
>>pid_t getpid64()
>>{
>>    pid_t resultvar;
>>
>>    asm volatile (
>>    "int $0x80\n\t"
>>    : "=a" (resultvar)     : "0" (__NR_getpid)
>>    : "memory");
>>
>>    return resultvar;
>>}
>>    
>>
>
>I tested it now. Since it ends up as a 32bit syscall you're not 
>actually calling 64bit getpid() - but 32bit mkdir(). That is because 32bit and 64bit
>have different syscall numbers. For me it returns -14, which is EFAULT.
>Expected for a random argument to mkdir()
>  
>
It makes a lot of sense...

I think the bottom line for me is simply that I cannot enter system 
calls any way I want with any kernel, among other things because the 
kernel expects that we got into the system call using vsyscall stuff and 
that the correct way to return to user space is also found in the 
vsyscall page.

Thanks for your input.

Mart

Re: x86_64 system call entry points II

[Andi Kleen]

> I think the bottom line for me is simply that I cannot enter system 
> calls any way I want with any kernel,

Only for SYSENTER. int 0x80 and SYSCALL don't have this problem.

For SYSENTER you can just call the trampoline in the vsyscall page.
glibc does that too. The address can be found in the ELF aux vector.

-Andi