linux-2.6.1 x86_64 : STACK_TOP and text/data

linux-2.6.1 x86_64 : STACK_TOP and text/data

[dada1]

Hi all

Anybody knows why STACK_TOP is defined to 0xc0000000 in x86_64 ?

This means that stack allocated variables are all in the first 4GB 
quadrant in memory.
As the default virtual addresses of text/data of a programm are in this 
same quadrant, some programming errors could be undetected.
(Some programmers could still cast some pointers to 'unsigned int' for 
example, and this could 'work')

Tru64 has a different strategy :
Program text starts at 0x120000000
Program data starts at 0x140000000
Stack is just under text, but still not in the first 4GB quadrant.

This way, programmers errors are likely to be detected at dev time.

Another point is that BSS zone (heap) cannot exceed 3GB in x86_64 mode, 
since the brk hit the stack.
libc malloc then fallback to use a lot of arenas... suboptimal in terms 
of  vmas.

Strangely, in ia32 emulation mode, the stack is placed at the 4GB limit !

Thank you
Eric Dumazet

Re: linux-2.6.1 x86_64 : STACK_TOP and text/data

[Andi Kleen]

dada1 <dada1@cosmosbay.com> writes:

> Anybody knows why STACK_TOP is defined to 0xc0000000 in x86_64 ?

STACK_TOP is only for 32bit a.out executables running on x86-64
ELF 32bit and 64bit programs use different defaults.

-Andi

Re: linux-2.6.1 x86_64 : STACK_TOP and text/data

[dada1]

Andi Kleen wrote:

> STACK_TOP is only for 32bit a.out executables running on x86-64
>
>ELF 32bit and 64bit programs use different defaults.
>
>-Andi
>
>
>  
>
Hi Andi

I'm afraid not Andi

I changed include/asm-x86_64/a.out.h

#define STACK_TOP  0x10c0000000   /* instead of 0xc0000000 */

then, after reboot :

file /sbin/init
/sbin/init: ELF 64-bit LSB executable, AMD x86-64, version 1 (SYSV), for 
GNU/Linux 2.4.0, dynamically linked (uses shared libs), stripped


cat /proc/1/maps

00400000-00408000 r-xp 00000000 03:01 556032                             
/sbin/init
00508000-00509000 rw-p 00008000 03:01 556032                             
/sbin/init
00509000-0052a000 rwxp 00000000 00:00 0
10bfffe000-10c0000000 rwxp fffffffffffff000 00:00 0
2a95556000-2a95569000 r-xp 00000000 03:01 637734                         
/lib64/ld-2.3.2.so
2a95569000-2a9556a000 rw-p 00000000 00:00 0
2a95669000-2a9566a000 rw-p 00013000 03:01 637734                         
/lib64/ld-2.3.2.so
2a9566a000-2a957a2000 r-xp 00000000 03:01 637741                         
/lib64/libc-2.3.2.so
2a957a2000-2a9586a000 ---p 00138000 03:01 637741                         
/lib64/libc-2.3.2.so
2a9586a000-2a958a7000 rw-p 00100000 03:01 637741                         
/lib64/libc-2.3.2.so
2a958a7000-2a958ac000 rw-p 00000000 00:00 0

See you

Re: linux-2.6.1 x86_64 : STACK_TOP and text/data

[Andi Kleen]

On Tue, 27 Jan 2004 19:57:23 +0100
dada1 <dada1@cosmosbay.com> wrote:

> Andi Kleen wrote:
> 
> > STACK_TOP is only for 32bit a.out executables running on x86-64
> >
> >ELF 32bit and 64bit programs use different defaults.
> >
> >-Andi
> >
> >
> >  
> >
> Hi Andi
> 
> I'm afraid not Andi

You're right. Thanks for reporting this. This seems to be a 2.6 
specific bug, it didn't happen in 2.4.

I will fix it. It should definitely use PAGE_OFFSET for 64bit 
processes and 4GB for !3GB 32bit processes.

-Andi

Re: linux-2.6.1 x86_64 : STACK_TOP and text/data

[dada1]

Andi Kleen wrote:

>
>You're right. Thanks for reporting this. This seems to be a 2.6 
>specific bug, it didn't happen in 2.4.
>
>I will fix it. It should definitely use PAGE_OFFSET for 64bit 
>processes and 4GB for !3GB 32bit processes.
>
>-Andi
>
>
>  
>
Another thing I noticed in last glibc CVS (nptl)

Thread stacks are also allocated in the 1GB quadrant :

nptl/sysdeps/x86_64/pthreaddef.h
/* We prefer to have the stack allocated in the low 4GB since this
   allows faster context switches.  */      
#define ARCH_MAP_FLAGS MAP_32BIT 

Is this really true ?
Is memory allocated in the low 4GB is faster on x86_64  (64bit kernel, 
64 bit user prog ?)

Thank you

Eric Dumazet

Re: linux-2.6.1 x86_64 : STACK_TOP and text/data

[Andi Kleen]

On Tue, 27 Jan 2004 20:45:24 +0100
dada1 <dada1@cosmosbay.com> wrote:

> Another thing I noticed in last glibc CVS (nptl)
> 
> Thread stacks are also allocated in the 1GB quadrant :
> 
> nptl/sysdeps/x86_64/pthreaddef.h
> /* We prefer to have the stack allocated in the low 4GB since this
>    allows faster context switches.  */      
> #define ARCH_MAP_FLAGS MAP_32BIT 
> 
> Is this really true ?
> Is memory allocated in the low 4GB is faster on x86_64  (64bit kernel, 
> 64 bit user prog ?)

That only applies to areas referenced set by set_thread_area() and
referenced by segment registers.  For pointers <4GB it can use a faster method at 
context switch.

They probably do that because they put the thread local data at the 
bottom of the stack and it has to be referenced using %gs.
They should use a fallback if the MAP_32BIT allocation fails.

I suspect they would be better off if they allocated the thread local
data separately. The 2.4 kernel used to do the same, but switched to 
separate allocation because this gives better cache colouring
(stacks tend to be aligned too much and use only parts of the cache) 

MAP_32BIT only allocates in the first 2GB BTW, it's really MAP_31BIT.

-Andi