Is compat_sys_ioctl called when both kernel and userland are 64bit ?

Is compat_sys_ioctl called when both kernel and userland are 64bit ?

[april]

Hi all:

I am doing something to support 32bit userland and 64bit kernel.
I know compat_sys_ioctl will be called if 32bit userland send ioctl to
64bit kernel.

The compat ioctl do some conversion(suppose I provide those ioctls).
For example:
if a struct has a field  type is long or a pointer,which the size is
different between userland and kernel

I notice in the x86_64_defconfig file:
CONFIG_X86_64=y
CONFIG_IA32_EMULATION=y
and in "trap_init" function,
#ifdef CONFIG_IA32_EMULATION
        set_system_intr_gate(IA32_SYSCALL_VECTOR, ia32_syscall);
        set_bit(IA32_SYSCALL_VECTOR, used_vectors);
#endif

it seems when a kernel is built to x86_64, whenever a ioctl called
,the call flow will be ia32_syscall -->compat_sys_ioctl->compat
ioctl(if provided)
but I have some doubt:

My question is:
1.  Is compat_sys_ioctl called when both kernel and userland are 64bit
(suppose I provide those compat ioctl functions)?
     if so, it may have problem, for the compat ioctl suppose userland
is 32bit, and do the conversion, if a field is unsinged long or a
pointer, it will do the        wrong things .

     If not, how kernel knows the driver is 32bit or 64bit?

2. when using mmap, the 64bit kernel will return a 64bit address, and
a userland(32bit) application can only get the lower 32bit,
it can work when the memory is not large.

Is kernel do some conversion? or fortunately the higher 32bit is 00000000 ?

Thanks

Re: Is compat_sys_ioctl called when both kernel and userland are 64bit ?

[Andi Kleen]

april <aapril03@gmail.com> writes:
>
> it seems when a kernel is built to x86_64, whenever a ioctl called
> ,the call flow will be ia32_syscall -->compat_sys_ioctl->compat
> ioctl(if provided)

Only for syscalls throught int 0x80 (or 32bit SYSCALL/SYSENTER), which normally 
only come from 32bit processes. In theory 64bit processes
could use them, but they normally don't.

> but I have some doubt:
>
> My question is:
> 1.  Is compat_sys_ioctl called when both kernel and userland are 64bit
> (suppose I provide those compat ioctl functions)?

No.

>      If not, how kernel knows the driver is 32bit or 64bit?

There are no 32bit drivers on a 64bit kernel, all drivers are 64bit.

For 32bit processes running on a 64bit kernel it depends on the 
entry points (int 0x80 or SYSENTER/SYSCALL)

> 2. when using mmap, the 64bit kernel will return a 64bit address, and
> a userland(32bit) application can only get the lower 32bit,
> it can work when the memory is not large.

The kernel makes sure to only hand out 32bit addresses to 32bit
processes.

-Andi

-- 
ak@linux.intel.com -- Speaking for myself only.

Re: Is compat_sys_ioctl called when both kernel and userland are 64bit ?

[april]

2009/8/12 Andi Kleen <andi@firstfloor.org>:
> april <aapril03@gmail.com> writes:
>>
>> it seems when a kernel is built to x86_64, whenever a ioctl called
>> ,the call flow will be ia32_syscall -->compat_sys_ioctl->compat
>> ioctl(if provided)
>
> Only for syscalls throught int 0x80 (or 32bit SYSCALL/SYSENTER), which normally
> only come from 32bit processes. In theory 64bit processes
> could use them, but they normally don't.
>

so you mean in 64bit process with a 64bit Linux kernel, system call is
not through int 0x80 ?
I know little about x86_64 arch, so I suppose it still through INT 0x80
can you tell me more about this? or where to get more information abort it?

>> but I have some doubt:
>>
>> My question is:
>> 1.  Is compat_sys_ioctl called when both kernel and userland are 64bit
>> (suppose I provide those compat ioctl functions)?
>
> No.
>
>>      If not, how kernel knows the driver is 32bit or 64bit?
>
> There are no 32bit drivers on a 64bit kernel, all drivers are 64bit.
>
> For 32bit processes running on a 64bit kernel it depends on the
> entry points (int 0x80 or SYSENTER/SYSCALL)
>
>> 2. when using mmap, the 64bit kernel will return a 64bit address, and
>> a userland(32bit) application can only get the lower 32bit,
>> it can work when the memory is not large.
>
> The kernel makes sure to only hand out 32bit addresses to 32bit
> processes.
>
I am using a 64bit kernel ,32bit Ubuntu9.04 which kernel was replaced
by a 64bit one,
and add some compat ioctl to make drm module work(on VIA platform),X
can work well(only 512M ram)
I am not test but guess it will has problem is memory is large (for
example ,over 4G)

> -Andi
>
> --
> ak@linux.intel.com -- Speaking for myself only.
>