RE: >3G Memory support

RE: >3G Memory support

[marc.miller]

Hi

Support of 4G of RAM is a configuration option when you compile the kernel.  Is that setting turned on?  

I think it's in "General Options" when you do a "make menuconfig" (I don't have a machine up at the moment that I can check).  There are three options:  Less than 1G, 1G to less than 4G, and 4G or more.  That last option is the one you would want.  

If that's already enabled, hopefully one of the memory guys can pitch in...

Marc J. Miller
Server Software Alliance Manager
Software & Infrastructure Initiatives Team
AMD, Inc.

marc.miller@amd.com
1-800-538-8450 x43325

 -----Original Message-----
From: 	devnull@adc.idt.com [mailto:devnull@adc.idt.com] 
Sent:	Wednesday, June 19, 2002 3:30 PM
To:	linux-kernel@vger.kernel.org
Subject:	>3G Memory support

Hello All.

I have a PC with 4G of Memory and would like a process to be able to
address all 4G of memory.

I am running 2.4.13-ac8

The way i understand it is that linux shares the top 1G of process address
space with all processes on the system(so on systems with 4G is physical
addressability, it leaves 3G for each process).

>From the archives, i learnt that i need to modify __PAGE_OFFSET and change
it from the default  (0xC0000000).

Looking at /usr/src/linux/include/asm-i386/page.h

<<SNIP>>
/*
 * This handles the memory map.. We could make this a config
 * option, but too many people screw it up, and too few need
 * it.
 *
 * A __PAGE_OFFSET of 0xC0000000 means that the kernel has
 * a virtual address space of one gigabyte, which limits the
 * amount of physical memory you can use to about 950MB.
 *
 * If you want more physical memory than this then see the
 *   CONFIG_HIGHMEM4G
 * and CONFIG_HIGHMEM64G options in the kernel configuration.
 */

<<END_OF_SNIP>>

When i compiled my kernel, i set CONFIG_HIGHMEM4G.

Does this mean that all my programs should be able to address 4G ?

If yes, i dont think it is working very well.

#include<stdio.h>
#include<sys/resource.h>
#include<unistd.h>
#include<sys/wait.h>
#include<stdlib.h>

main(int argc, char **argv)
{

        char    *n;
  int   size = 1073741000;
        unsigned int totalsize = 0;

        while(size > 1000)      {
                while(n=(char *) malloc(size))  {
                        bzero(n, size);
                        totalsize += size;
                }
//              usleep(999999) ;
                size = size / 2;
        }
        printf("Total size = %lu\n", totalsize);
}

The variable "totalsize" goes upto 3G only.

If seting CONFIG_HIGHMEM4G is not the way to proceed, could someone please
point me in the right direction.

Thanks for taking the time to read my email, and sorry about the long
post.

Best Regards,


/dev/null

devnull@adc.idt.com


-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html
Please read the FAQ at  http://www.tux.org/lkml/

Re: >3G Memory support

[William Lee Irwin III]

On Wed, Jun 19, 2002 at 07:01:08PM -0700, marc.miller@amd.com wrote:
> Support of 4G of RAM is a configuration option when you compile the
> kernel.  Is that setting turned on?  
> I think it's in "General Options" when you do a "make menuconfig"
> (I don't have a machine up at the moment that I can check).  There
> are three options:  Less than 1G, 1G to less than 4G, and 4G or more.
> That last option is the one you would want.  
> If that's already enabled, hopefully one of the memory guys can pitch in...

This is actually yet another "32-bit virtualspace sucks" issue. You can't
get at all your RAM from userspace because the virtualspace set aside for
the kernel prevents you from using it to map physical memory. 64-bit
virtualspace is too vast to be easily exhausted this way.

The original form of highmem was "BIGMEM" which used (almost) disjoint
user and kernel virtual address spaces, but this is not terribly
efficient with respect to lazy TLB entry invalidation.


Cheers,
Bill

Re: >3G Memory support

[Eric W. Biederman]

William Lee Irwin III <wli@holomorphy.com> writes:

> On Wed, Jun 19, 2002 at 07:01:08PM -0700, marc.miller@amd.com wrote:
> > Support of 4G of RAM is a configuration option when you compile the
> > kernel.  Is that setting turned on?  
> > I think it's in "General Options" when you do a "make menuconfig"
> > (I don't have a machine up at the moment that I can check).  There
> > are three options:  Less than 1G, 1G to less than 4G, and 4G or more.
> > That last option is the one you would want.  
> > If that's already enabled, hopefully one of the memory guys can pitch in...
> 
> This is actually yet another "32-bit virtualspace sucks" issue. You can't
> get at all your RAM from userspace because the virtualspace set aside for
> the kernel prevents you from using it to map physical memory. 64-bit
> virtualspace is too vast to be easily exhausted this way.

Note.  Getting at all of the memory isn't impossible but you have
to allocate a very large posix shared memory segment, and page it
in and out of your address space.

But the only easy solution to this problem is a 64bit machine.  At an
address bit consumed every year or so we have 2 or 3 decades before we
will need to move to 128bit machines to resolve this issue yet again.

At least the situation now is better than with ems.sys and xmm.sys and
their kin the last time x86 hit an address space wall.  Though we are
a little bit out from the point where all machines are configured with
more memory than they can practically use.

Eric

>3G Memory support

[devnull]

Hello All.

I have a PC with 4G of Memory and would like a process to be able to
address all 4G of memory.

I am running 2.4.13-ac8

The way i understand it is that linux shares the top 1G of process address
space with all processes on the system(so on systems with 4G is physical
addressability, it leaves 3G for each process).

>From the archives, i learnt that i need to modify __PAGE_OFFSET and change
it from the default  (0xC0000000).

Looking at /usr/src/linux/include/asm-i386/page.h

<<SNIP>>
/*
 * This handles the memory map.. We could make this a config
 * option, but too many people screw it up, and too few need
 * it.
 *
 * A __PAGE_OFFSET of 0xC0000000 means that the kernel has
 * a virtual address space of one gigabyte, which limits the
 * amount of physical memory you can use to about 950MB.
 *
 * If you want more physical memory than this then see the
 *   CONFIG_HIGHMEM4G
 * and CONFIG_HIGHMEM64G options in the kernel configuration.
 */

<<END_OF_SNIP>>

When i compiled my kernel, i set CONFIG_HIGHMEM4G.

Does this mean that all my programs should be able to address 4G ?

If yes, i dont think it is working very well.

#include<stdio.h>
#include<sys/resource.h>
#include<unistd.h>
#include<sys/wait.h>
#include<stdlib.h>

main(int argc, char **argv)
{

        char    *n;
  int   size = 1073741000;
        unsigned int totalsize = 0;

        while(size > 1000)      {
                while(n=(char *) malloc(size))  {
                        bzero(n, size);
                        totalsize += size;
                }
//              usleep(999999) ;
                size = size / 2;
        }
        printf("Total size = %lu\n", totalsize);
}

The variable "totalsize" goes upto 3G only.

If seting CONFIG_HIGHMEM4G is not the way to proceed, could someone please
point me in the right direction.

Thanks for taking the time to read my email, and sorry about the long
post.

Best Regards,


/dev/null

devnull@adc.idt.com

Re: >3G Memory support

[Brian Gerst]

devnull@adc.idt.com wrote:
> Hello All.
> 
> I have a PC with 4G of Memory and would like a process to be able to
> address all 4G of memory.
> 
> I am running 2.4.13-ac8
> 
> The way i understand it is that linux shares the top 1G of process address
> space with all processes on the system(so on systems with 4G is physical
> addressability, it leaves 3G for each process).
> 
>>From the archives, i learnt that i need to modify __PAGE_OFFSET and change
> it from the default  (0xC0000000).
> 
> Looking at /usr/src/linux/include/asm-i386/page.h
> 
> <<SNIP>>
> /*
>  * This handles the memory map.. We could make this a config
>  * option, but too many people screw it up, and too few need
>  * it.
>  *
>  * A __PAGE_OFFSET of 0xC0000000 means that the kernel has
>  * a virtual address space of one gigabyte, which limits the
>  * amount of physical memory you can use to about 950MB.
>  *
>  * If you want more physical memory than this then see the
>  *   CONFIG_HIGHMEM4G
>  * and CONFIG_HIGHMEM64G options in the kernel configuration.
>  */
> 
> <<END_OF_SNIP>>
> 
> When i compiled my kernel, i set CONFIG_HIGHMEM4G.
> 
> Does this mean that all my programs should be able to address 4G ?

No.  It means the kernel can access all 4GB of memory.  For memory above 
the 950MB that it can directly map, it needs to use dynamic mappings 
(kmap).  User space is always 3GB virtual space per process, regardless 
of the highmem setting.

--
				Brian Gerst

Re: >3G Memory support

[devnull]

> >
> > When i compiled my kernel, i set CONFIG_HIGHMEM4G.
> >
> > Does this mean that all my programs should be able to address 4G ?
>
> No.  It means the kernel can access all 4GB of memory.  For memory above
> the 950MB that it can directly map, it needs to use dynamic mappings
> (kmap).  User space is always 3GB virtual space per process, regardless
> of the highmem setting.

Is there a way to make a process in the user space to able to access 4GB
at all. What limits user space to 3GB.

If not in current 2.4.x / 2.5.x, is this something planned in the future
releases ?

Thanks for your time.

Regards,

/dev/null

devnull@adc.idt.com

Re: >3G Memory support

[Brian Gerst]

devnull@adc.idt.com wrote:
>>>When i compiled my kernel, i set CONFIG_HIGHMEM4G.
>>>
>>>Does this mean that all my programs should be able to address 4G ?
>>
>>No.  It means the kernel can access all 4GB of memory.  For memory above
>>the 950MB that it can directly map, it needs to use dynamic mappings
>>(kmap).  User space is always 3GB virtual space per process, regardless
>>of the highmem setting.
> 
> 
> Is there a way to make a process in the user space to able to access 4GB
> at all. 

Not all at one time, but you can map/unmap shared memory segments to 
access more memory.

> What limits user space to 3GB.

Hardware limitations imposed by the x86 architecture.  The x86 only has 
_one_ virtual address space, which has to be shared by user space and 
kernel space.  It is not possible to give user space more virtual 
address space without taking it away from the kernel.

> If not in current 2.4.x / 2.5.x, is this something planned in the future
> releases ?

Support for 64-bit processors like Intel Itanium and AMD Hammer 
processors which do not have such limitations.

--
				Brian Gerst

Re: >3G Memory support

[Jirka Kosina]

On Thu, 20 Jun 2002, Brian Gerst wrote:

> > What limits user space to 3GB.
> Hardware limitations imposed by the x86 architecture.  The x86 only has 
> _one_ virtual address space, which has to be shared by user space and 
> kernel space.  It is not possible to give user space more virtual 
> address space without taking it away from the kernel.

You can theoretically run kernel in one task (I mean "task" in the 
Intel-processor-meaning of the word ;) ) and userspace programs in another 
task, which will result in having 4GB of memory for both of them, won't 
it?

I know, there would be big compilcations - for example copying data 
betweeen userspace and kernelspace should be problem, and if implemented, 
I can't imagine how to do it considerably fast...which is IMHO the main 
reason why it isn't done such way.

-- 
JiKos.

Re: >3G Memory support

[William Lee Irwin III]

On Thu, 20 Jun 2002, Brian Gerst wrote:
>> Hardware limitations imposed by the x86 architecture.  The x86 only has 
>> _one_ virtual address space, which has to be shared by user space and 
>> kernel space.  It is not possible to give user space more virtual 
>> address space without taking it away from the kernel.

On Thu, Jun 20, 2002 at 04:50:26PM +0200, Jirka Kosina wrote:
> You can theoretically run kernel in one task (I mean "task" in the 
> Intel-processor-meaning of the word ;) ) and userspace programs in another 
> task, which will result in having 4GB of memory for both of them, won't 
> it?

This is what BIGMEM did. It predated the current highmem implementation.


Cheers,
Bill

Re: >3G Memory support

[Richard B. Johnson]

On Thu, 20 Jun 2002 devnull@adc.idt.com wrote:

> > >
> > > When i compiled my kernel, i set CONFIG_HIGHMEM4G.
> > >
> > > Does this mean that all my programs should be able to address 4G ?
> >
> > No.  It means the kernel can access all 4GB of memory.  For memory above
> > the 950MB that it can directly map, it needs to use dynamic mappings
> > (kmap).  User space is always 3GB virtual space per process, regardless
> > of the highmem setting.
> 
> Is there a way to make a process in the user space to able to access 4GB
> at all. What limits user space to 3GB.
> 
> If not in current 2.4.x / 2.5.x, is this something planned in the future
> releases ?
> 
> Thanks for your time.

The Intel 32-bit processors provide 32-bit virtual address space which
is, in Unix/Linux machines, shared between the kernel and the process.

If you add address space to the process, you need to take it away from
the kernel and vice-versa. The result is that each process gets its own
data and (usually) code, but shares the kernel. Each process also gets
to share, via memory-mapping, major portions of runtime-libraries like
the 'C' runtime library. Even though code is shared, it takes up
address-space.

The kernel can use RAM that does not exist in the 'lower 32-bits' to
satisfy virtual RAM. This is the "High memory" option. Nevertheless,
a task ends up with 32-bits of address space.

The "fix" is to either wait for 64-bit machines, at which time users
will complain that they can't address all 64 bits, or re-do the kernel
as a VAX/VMS kind of virtual memory system.

It is, in principle, possible for user virtual address space to range
from 0 to 0xffffffff. The kernel could have its own virtual address
space which is not shared at all. The problem is there is only one
set of page tables on Intel machines. You would have to make two, or
more, in an area where there is a 1:1 virtual to physical address-
translation, you would have to transition to that area, disable paging,
flush the cache, load the kernel page-tables, enable paging, then
call/execute kernel code every time you made a kernel system call.
Oh yes, you have to do the reverse to return control to the user who
called the kernel code. Copying data to/from kernel space would be
a bear. The physical address of any user buffer would have to be
mapped into the kernel's address space (like a mailbox), on the off-
chance that the kernel might need to access it. Nasty.

This would make system calls take milliseconds instead of microseconds.

FYI, you can explore user-address possibilities with simple 'C' code:

#include <stdio.h>
char data;
int main()
{
  char stack;
  unsigned long avail;

  avail = 0xffffffff - (long)&stack;
  printf("Address available above stack %08lx\n", avail);
  avail = 0xffffffff - (long)&data;
  printf("Address available above data %08lx\n", avail);
  avail = 0xffffffff - (long)main;
  printf("Address available above code %08lx\n", avail);
}




Cheers,
Dick Johnson

Penguin : Linux version 2.4.18 on an i686 machine (797.90 BogoMips).

                 Windows-2000/Professional isn't.