Using more than 256 MB of memory on SB1250 in 32-bit mode, revisited

Using more than 256 MB of memory on SB1250 in 32-bit mode, revisited

[Matthew Starzewski]

[-- Attachment #1: Type: text/plain, Size: 6020 bytes --]

I've tried to enable HIGHMEM to access all 512MB of
SDRAM on a BCM1125 based board as per this previous
thread:

Using more than 256 MB of memory on SB1250 in 32-bit mode :
http://www.spinics.net/lists/mips/msg14396.html
BCM1125 Board: XPedite3000 PrPMC
http://www.xes-inc.com/Products/XPedite/XPedite3000/XPedite3000.html

In MIPS32 mode, the memory map comes out to the following:

Physical Memory Map:
0x00000000 - 0x0FFFFFFF   256MB
0x80000000 - 0x8FFFFFFF   256MB
Virtual Memory Map:
0x80000000 - 0x8FFFFFFF   256MB
<<<< INACCESSIBLE >>>> 256MB

My goal in enabling HIGHMEM was to get at the upper 256MB
much as described here.  Access to the upper 256MB through
HIGHMEM may incur a performance hit, but it's certainly better
going without.

http://home.earthlink.net/~jknapka/linux-mm/vminit.html#PAGING_INIT

However, what I get is a stall as per the log pasted below.  Enabling
CONFIG_64BIT_PHYS_ADDR does not make a difference.  Results
were cross-verified between CFE and another bootloader.

When I hand the physical memory described above off to add_memory_region,
I noticed a few odd things. One thing that looks suspicious is that the variable
void *high_memory ends up being set to 0xa0000000, or right at KSEG1,
in mm/init.c.

Also, num_physpages became huge, 0x90000, because the init code in
kernel/setup.c and mm/init.c want a page for every memory location, even
highmem.  Is this appropriate when the memory is not directly accessible
via __va and virt_to_phys?  

This may be an ancillary effect of what's mentioned above, but when num_physpages
grows in size, nr_free_pages doesn't track with it, so in void vfs_caches_init(unsigned long
mempages) and the like, you get a horrible underflow condition:

/* code */
        printk("MJS - nr_free_pages():0x%X\n", nr_free_pages());
        printk("MJS - OLD mempages:0x%X\n", mempages);
        reserve = (mempages - nr_free_pages()) * 3/2;
        mempages -= reserve;
        printk("MJS - NEW reserve:0x%X mempages:0x%X\n",
               reserve, mempages);

/* printout */
MJS - nr_free_pages():0x1E9A0
MJS - OLD mempages:0x90000
MJS - NEW reserve:0xAA190 mempages:0xFFFE5E70


Let me know what you think of this issue.  In anticipation of the
"Why not use MIPS64 build?" question, we'd prefer to and will, but the 
MIPS64 build has underperformed or had bugs (SATA seek time,
networking signals, etc) that MIPS32 doesn't.  For these issues
or any case where the MIPS64 build isn't there yet, it makes
sense to have the MIPS32 path open.

Regards,
Matt


============= Kernel Boot Log ==================

CFE version 1.0.37 for BCM91125E (64bit,SP,BE)
Build Date: Mon Jun 28 18:33:29 CDT 2004 (mstarzewski@lsys1)
Copyright (C) 2000,2001,2002,2003 Broadcom Corporation.

Initializing Arena.
Initializing Devices.
BCM91125E board revision 1
Config switch: 0
CPU: 1125H A2
L2 Cache: 256KB
SysCfg: 00800000004A0600 [PLL_DIV: 12, IOB0_DIV: CPUCLK/4, IOB1_DIV: CPUCLK/3]
CPU type 0x40103: 600MHz
Total memory: 0x20000000 bytes (512MB)

Total memory used by CFE:  0x8FE8AFC0 - 0x90000000 (1527872)
Initialized Data:          0x8FE8AFC0 - 0x8FE94D40 (40320)
BSS Area:                  0x8FE94D40 - 0x8FE95430 (1776)
Local Heap:                0x8FE95430 - 0x8FF95430 (1048576)
Stack Area:                0x8FF95430 - 0x8FF97430 (8192)
Text (code) segment:       0x8FF97440 - 0x8FFFFFB8 (428920)
Boot area (physical):      0x0FE49000 - 0x0FE89000
Relocation Factor:         I:F0397440 - D:0DF8AFC0

CFE> ifconfig eth0 -addr=10.52.33.67 -mask=255.255.0.0;boot -elf 10.52.0.4:/home/mstarzewski/li
nux/kernels/mips26_works_cfe/vmlinux
eth0: Link speed: 100BaseT FDX
Device eth0:  hwaddr 40-00-10-06-40-00, ipaddr 10.52.33.67, mask 255.255.0.0
        gateway not set, nameserver not set
Loader:elf Filesys:tftp Dev:eth0 File:10.52.0.4:/home/mstarzewski/linux/kernels/mips26_works_cf
e/vmlinux Options:(null)
Loading: 0xffffffff80100000/2558312 0xffffffff80370968/146032 Entry at 0xffffffff8032f018
Variable Name        Value
-------------------- --------------------------------------------------
BOOT_CONSOLE         uart0
CPU_TYPE             1125H
CPU_REVISION         A2
CPU_NUM_CORES        1
CPU_SPEED            600
CFE_VERSION          1.0.37
CFE_BOARDNAME        BCM91125E
CFE_MEMORYSIZE       512
NET_DEVICE           eth0
NET_IPADDR           10.52.33.67
NET_NETMASK          255.255.0.0
NET_GATEWAY          0.0.0.0
NET_NAMESERVER       0.0.0.0
BOOT_DEVICE          eth0
BOOT_FILE            10.52.0.4:/home/mstarzewski/linux/kernels/mips26_works_cfe/vmlinux
DELETING BOOT_CONSOLE
DELETING CPU_REVISION
DELETING CPU_SPEED
DELETING CFE_BOARDNAME
DELETING NET_DEVICE
DELETING NET_NETMASK
DELETING NET_NAMESERVER
DELETING BOOT_FILE
Variable Name        Value
-------------------- --------------------------------------------------
Closing network.
Starting program at 0xffffffff8032f018
CFE addr:0x000000000FE8A000 size:0x0000000000000000, type:0
CFE addr:0x0000000010000000 size:0x0000000000000000, type:-2147483648
Broadcom SiByte BCM1125H A2 @ 600 MHz (SB1 rev 3)
Board type: SiByte BCM91250A (SWARM)
Linux version 2.6.6-rc3 (mstarzewski@lsys1) (gcc version 3.2.3 with SiByte modifications) #17 S
MP Thu Dec 9 11:08:23 CST 2004
CPU revision is: 00040103
FPU revision is: 000f0103
This kernel optimized for board runs with CFE
Determined physical RAM map:
 memory: 0fe89e00 @ 00000000 (usable)
 memory: 0ffffe00 @ 80000000 (usable)
1791MB HIGHMEM available.
On node 0 totalpages: 589823
  DMA zone: 131072 pages, LIFO batch:16
  Normal zone: 0 pages, LIFO batch:1
  HighMem zone: 458751 pages, LIFO batch:16
Built 1 zonelists
Kernel command line: root=/dev/nfs ip=auto rw
PID hash table entries: 4096 (order 12: 32768 bytes)
Memory: 495040k/260644k available (1850k kernel code, 27020k reserved, 382k data, 264k init, 26
2140k highmem)
Calibrating delay loop... 399.36 BogoMIPS

<< STALL >>

[-- Attachment #2: Type: text/html, Size: 12088 bytes --]

Using more than 256 MB of memory on SB1250 in 32-bit mode, revisited

[Matthew Starzewski]

[-- Attachment #1: Type: text/plain, Size: 6020 bytes --]

I've tried to enable HIGHMEM to access all 512MB of
SDRAM on a BCM1125 based board as per this previous
thread:

Using more than 256 MB of memory on SB1250 in 32-bit mode :
http://www.spinics.net/lists/mips/msg14396.html
BCM1125 Board: XPedite3000 PrPMC
http://www.xes-inc.com/Products/XPedite/XPedite3000/XPedite3000.html

In MIPS32 mode, the memory map comes out to the following:

Physical Memory Map:
0x00000000 - 0x0FFFFFFF   256MB
0x80000000 - 0x8FFFFFFF   256MB
Virtual Memory Map:
0x80000000 - 0x8FFFFFFF   256MB
<<<< INACCESSIBLE >>>> 256MB

My goal in enabling HIGHMEM was to get at the upper 256MB
much as described here.  Access to the upper 256MB through
HIGHMEM may incur a performance hit, but it's certainly better
going without.

http://home.earthlink.net/~jknapka/linux-mm/vminit.html#PAGING_INIT

However, what I get is a stall as per the log pasted below.  Enabling
CONFIG_64BIT_PHYS_ADDR does not make a difference.  Results
were cross-verified between CFE and another bootloader.

When I hand the physical memory described above off to add_memory_region,
I noticed a few odd things. One thing that looks suspicious is that the variable
void *high_memory ends up being set to 0xa0000000, or right at KSEG1,
in mm/init.c.

Also, num_physpages became huge, 0x90000, because the init code in
kernel/setup.c and mm/init.c want a page for every memory location, even
highmem.  Is this appropriate when the memory is not directly accessible
via __va and virt_to_phys?  

This may be an ancillary effect of what's mentioned above, but when num_physpages
grows in size, nr_free_pages doesn't track with it, so in void vfs_caches_init(unsigned long
mempages) and the like, you get a horrible underflow condition:

/* code */
        printk("MJS - nr_free_pages():0x%X\n", nr_free_pages());
        printk("MJS - OLD mempages:0x%X\n", mempages);
        reserve = (mempages - nr_free_pages()) * 3/2;
        mempages -= reserve;
        printk("MJS - NEW reserve:0x%X mempages:0x%X\n",
               reserve, mempages);

/* printout */
MJS - nr_free_pages():0x1E9A0
MJS - OLD mempages:0x90000
MJS - NEW reserve:0xAA190 mempages:0xFFFE5E70


Let me know what you think of this issue.  In anticipation of the
"Why not use MIPS64 build?" question, we'd prefer to and will, but the 
MIPS64 build has underperformed or had bugs (SATA seek time,
networking signals, etc) that MIPS32 doesn't.  For these issues
or any case where the MIPS64 build isn't there yet, it makes
sense to have the MIPS32 path open.

Regards,
Matt


============= Kernel Boot Log ==================

CFE version 1.0.37 for BCM91125E (64bit,SP,BE)
Build Date: Mon Jun 28 18:33:29 CDT 2004 (mstarzewski@lsys1)
Copyright (C) 2000,2001,2002,2003 Broadcom Corporation.

Initializing Arena.
Initializing Devices.
BCM91125E board revision 1
Config switch: 0
CPU: 1125H A2
L2 Cache: 256KB
SysCfg: 00800000004A0600 [PLL_DIV: 12, IOB0_DIV: CPUCLK/4, IOB1_DIV: CPUCLK/3]
CPU type 0x40103: 600MHz
Total memory: 0x20000000 bytes (512MB)

Total memory used by CFE:  0x8FE8AFC0 - 0x90000000 (1527872)
Initialized Data:          0x8FE8AFC0 - 0x8FE94D40 (40320)
BSS Area:                  0x8FE94D40 - 0x8FE95430 (1776)
Local Heap:                0x8FE95430 - 0x8FF95430 (1048576)
Stack Area:                0x8FF95430 - 0x8FF97430 (8192)
Text (code) segment:       0x8FF97440 - 0x8FFFFFB8 (428920)
Boot area (physical):      0x0FE49000 - 0x0FE89000
Relocation Factor:         I:F0397440 - D:0DF8AFC0

CFE> ifconfig eth0 -addr=10.52.33.67 -mask=255.255.0.0;boot -elf 10.52.0.4:/home/mstarzewski/li
nux/kernels/mips26_works_cfe/vmlinux
eth0: Link speed: 100BaseT FDX
Device eth0:  hwaddr 40-00-10-06-40-00, ipaddr 10.52.33.67, mask 255.255.0.0
        gateway not set, nameserver not set
Loader:elf Filesys:tftp Dev:eth0 File:10.52.0.4:/home/mstarzewski/linux/kernels/mips26_works_cf
e/vmlinux Options:(null)
Loading: 0xffffffff80100000/2558312 0xffffffff80370968/146032 Entry at 0xffffffff8032f018
Variable Name        Value
-------------------- --------------------------------------------------
BOOT_CONSOLE         uart0
CPU_TYPE             1125H
CPU_REVISION         A2
CPU_NUM_CORES        1
CPU_SPEED            600
CFE_VERSION          1.0.37
CFE_BOARDNAME        BCM91125E
CFE_MEMORYSIZE       512
NET_DEVICE           eth0
NET_IPADDR           10.52.33.67
NET_NETMASK          255.255.0.0
NET_GATEWAY          0.0.0.0
NET_NAMESERVER       0.0.0.0
BOOT_DEVICE          eth0
BOOT_FILE            10.52.0.4:/home/mstarzewski/linux/kernels/mips26_works_cfe/vmlinux
DELETING BOOT_CONSOLE
DELETING CPU_REVISION
DELETING CPU_SPEED
DELETING CFE_BOARDNAME
DELETING NET_DEVICE
DELETING NET_NETMASK
DELETING NET_NAMESERVER
DELETING BOOT_FILE
Variable Name        Value
-------------------- --------------------------------------------------
Closing network.
Starting program at 0xffffffff8032f018
CFE addr:0x000000000FE8A000 size:0x0000000000000000, type:0
CFE addr:0x0000000010000000 size:0x0000000000000000, type:-2147483648
Broadcom SiByte BCM1125H A2 @ 600 MHz (SB1 rev 3)
Board type: SiByte BCM91250A (SWARM)
Linux version 2.6.6-rc3 (mstarzewski@lsys1) (gcc version 3.2.3 with SiByte modifications) #17 S
MP Thu Dec 9 11:08:23 CST 2004
CPU revision is: 00040103
FPU revision is: 000f0103
This kernel optimized for board runs with CFE
Determined physical RAM map:
 memory: 0fe89e00 @ 00000000 (usable)
 memory: 0ffffe00 @ 80000000 (usable)
1791MB HIGHMEM available.
On node 0 totalpages: 589823
  DMA zone: 131072 pages, LIFO batch:16
  Normal zone: 0 pages, LIFO batch:1
  HighMem zone: 458751 pages, LIFO batch:16
Built 1 zonelists
Kernel command line: root=/dev/nfs ip=auto rw
PID hash table entries: 4096 (order 12: 32768 bytes)
Memory: 495040k/260644k available (1850k kernel code, 27020k reserved, 382k data, 264k init, 26
2140k highmem)
Calibrating delay loop... 399.36 BogoMIPS

<< STALL >>

[-- Attachment #2: Type: text/html, Size: 12088 bytes --]

Re: Using more than 256 MB of memory on SB1250 in 32-bit mode, revisited

[Thomas Petazzoni]

[-- Attachment #1: Type: text/plain, Size: 1226 bytes --]

Hello,

Matthew Starzewski a écrit :
> I've tried to enable HIGHMEM to access all 512MB of
> SDRAM on a BCM1125 based board as per this previous
> thread:
>  
> Using more than 256 MB of memory on SB1250 in 32-bit mode :
> http://www.spinics.net/lists/mips/msg14396.html
> BCM1125 Board: XPedite3000 PrPMC
> http://www.xes-inc.com/Products/XPedite/XPedite3000/XPedite3000.html

I'm really unsure of what I'll say, but I've seen people on this list 
talking about CONFIG_DISCONTIGMEM, an option for the kernel, which is :

"Say Y to upport efficient handling of discontiguous physical memory,
  for architectures which are either NUMA (Non-Uniform Memory Access)
  or have huge holes in the physical address space for other reasons.
  See <file:Documentation/vm/numa> for more."

Maybe it's what you're looking for, maybe not.

I'm still very surprised that Linux cannot handle strange physical 
memory configuration simply (holes in physical memory, DMA memory at 
higher addresses than normal memory).

Thomas
-- 
PETAZZONI Thomas - thomas.petazzoni@enix.org
http://thomas.enix.org - Jabber: thomas.petazzoni@jabber.dk
http://kos.enix.org, http://sos.enix.org
Fingerprint : 0BE1 4CF3 CEA4 AC9D CC6E  1624 F653 CB30 98D3 F7A7

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 256 bytes --]

Re: Using more than 256 MB of memory on SB1250 in 32-bit mode, revisited

[Maciej W. Rozycki]

On Fri, 10 Dec 2004, Thomas Petazzoni wrote:

> I'm still very surprised that Linux cannot handle strange physical 
> memory configuration simply (holes in physical memory, DMA memory at 
> higher addresses than normal memory).

 That's i386 legacy, later supported by other platforms using a similar
memory model -- starting at 0, mostly contiguous and only two DMA zones
for ISA and 32-bit PCI respectively, both starting at 0.  Remember, most
people working on Linux only have an i386 PC.  If you have something very
different, you need to write code to support it yourself.  If you do it
well enough, it'll be gladly accepted.

  Maciej

Re: Using more than 256 MB of memory on SB1250 in 32-bit mode, revisited

[Matthew Starzewski]

> I'm really unsure of what I'll say, but I've seen people on this list
> talking about CONFIG_DISCONTIGMEM, an option for the kernel, which is :
>
> "Say Y to upport efficient handling of discontiguous physical memory,
>  for architectures which are either NUMA (Non-Uniform Memory Access)
>   or have huge holes in the physical address space for other reasons.
>  See <file:Documentation/vm/numa> for more."

I've tried using DISCONTIGMEM on the MIPS32 build, but it yields the
following build error.  Maybe someone familiar w/ the SGI IP27 (Origin 200
and
2000) code could tell us whether simply turning on DISCONTIGMEM is
a good idea.

include/linux/mmzone.h:364:2: #error NODES_SHIFT > MAX_NODES_SHIFT

Also, even if it compiled with DISCONTIGMEM, a look at
sgi-ip27/ip27-memory.c
shows no sign of HIGHMEM support, what I was depending on to use the upper
256MB of memory as per my original email.  Now this may be OK; depending on
how
flexible the DISCONTIGMEM option is, perhaps I could map a wired TLB or
KSEG3
to the 2nd 256MB.

Thoughts?

Matt


----- Original Message -----
From: "Thomas Petazzoni" <thomas.petazzoni@enix.org>
To: "Matthew Starzewski" <mstarzewski@xes-inc.com>
Cc: <linux-mips@linux-mips.org>; <Steve.Finney@SpirentCom.COM>
Sent: Friday, December 10, 2004 2:46 AM
Subject: Re: Using more than 256 MB of memory on SB1250 in 32-bit mode,
revisited

Re: Using more than 256 MB of memory on SB1250 in 32-bit mode, revisited

[Matthew Starzewski]

> I'm really unsure of what I'll say, but I've seen people on this list
> talking about CONFIG_DISCONTIGMEM, an option for the kernel, which is :
>
> "Say Y to upport efficient handling of discontiguous physical memory,
>  for architectures which are either NUMA (Non-Uniform Memory Access)
>   or have huge holes in the physical address space for other reasons.
>  See <file:Documentation/vm/numa> for more."

I've tried using DISCONTIGMEM on the MIPS32 build, but it yields the
following build error.  Maybe someone familiar w/ the SGI IP27 (Origin 200
and
2000) code could tell us whether simply turning on DISCONTIGMEM is
a good idea.

include/linux/mmzone.h:364:2: #error NODES_SHIFT > MAX_NODES_SHIFT

Also, even if it compiled with DISCONTIGMEM, a look at
sgi-ip27/ip27-memory.c
shows no sign of HIGHMEM support, what I was depending on to use the upper
256MB of memory as per my original email.  Now this may be OK; depending on
how
flexible the DISCONTIGMEM option is, perhaps I could map a wired TLB or
KSEG3
to the 2nd 256MB.

Thoughts?

Matt


----- Original Message -----
From: "Thomas Petazzoni" <thomas.petazzoni@enix.org>
To: "Matthew Starzewski" <mstarzewski@xes-inc.com>
Cc: <linux-mips@linux-mips.org>; <Steve.Finney@SpirentCom.COM>
Sent: Friday, December 10, 2004 2:46 AM
Subject: Re: Using more than 256 MB of memory on SB1250 in 32-bit mode,
revisited

Re: Using more than 256 MB of memory on SB1250 in 32-bit mode, revisited

[Thomas Petazzoni]

[-- Attachment #1: Type: text/plain, Size: 453 bytes --]

Hello,

Matthew Starzewski a écrit :

> Thoughts?

Did you look at 
http://www.linux-mips.org/archives/linux-mips/2004-12/msg00053.html and 
more particularly 
http://www.linux-mips.org/archives/linux-mips/2004-12/msg00053.html ?

Thomas
-- 
PETAZZONI Thomas - thomas.petazzoni@enix.org
http://thomas.enix.org - Jabber: thomas.petazzoni@jabber.dk
http://kos.enix.org, http://sos.enix.org
Fingerprint : 0BE1 4CF3 CEA4 AC9D CC6E  1624 F653 CB30 98D3 F7A7

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 256 bytes --]

Re: Using more than 256 MB of memory on SB1250 in 32-bit mode, revisited

[Ralf Baechle]

On Thu, Dec 09, 2004 at 04:49:35PM -0600, Matthew Starzewski wrote:

> I've tried to enable HIGHMEM to access all 512MB of
> SDRAM on a BCM1125 based board as per this previous
> thread:
> 
> Using more than 256 MB of memory on SB1250 in 32-bit mode :
> http://www.spinics.net/lists/mips/msg14396.html
> BCM1125 Board: XPedite3000 PrPMC
> http://www.xes-inc.com/Products/XPedite/XPedite3000/XPedite3000.html
> 
> In MIPS32 mode, the memory map comes out to the following:
> 
> Physical Memory Map:
> 0x00000000 - 0x0FFFFFFF   256MB
> 0x80000000 - 0x8FFFFFFF   256MB
> Virtual Memory Map:
> 0x80000000 - 0x8FFFFFFF   256MB
> <<<< INACCESSIBLE >>>> 256MB
> 
> My goal in enabling HIGHMEM was to get at the upper 256MB
> much as described here.  Access to the upper 256MB through
> HIGHMEM may incur a performance hit, but it's certainly better
> going without.
> 
> http://home.earthlink.net/~jknapka/linux-mm/vminit.html#PAGING_INIT
> 
> However, what I get is a stall as per the log pasted below.  Enabling
> CONFIG_64BIT_PHYS_ADDR does not make a difference.  Results
> were cross-verified between CFE and another bootloader.

You need that if you want to address more than 1GB of memory on a BCM1250.

> When I hand the physical memory described above off to add_memory_region,
> I noticed a few odd things. One thing that looks suspicious is that the variable
> void *high_memory ends up being set to 0xa0000000, or right at KSEG1,
> in mm/init.c.
> 
> Also, num_physpages became huge, 0x90000, because the init code in
> kernel/setup.c and mm/init.c want a page for every memory location, even
> highmem.  Is this appropriate when the memory is not directly accessible
> via __va and virt_to_phys?  

You mean a struct page, not a page, I assume.  Yes, that's correct and
means that mem_map[] is a huge memory consumer in particular on systems
such as BCM1250 where RAM was scattered over the address space with a
shotgun.

The critical point would be where mem_map[] fills up the entire low-mem
which at 64 bytes per struct page and 256MB low-mem would happen at
4194304 pages or 16GB total memory at a page size at 4kB page size.
That's a suprenum, so guaranteed to not be exceeded ;-)  In reality
for reasonable performance a 1:4 ratio between lowmem and highmem should
not be exceeded.

What aggrevates the situation without CONFIG_DISCONTIG on the BCM1250 is
the large gap in it's address space from 0x10000000 - 0x8000000, that's
1.75GB which will eat 28MB of lowmem for mem_map - for nothing.  That's
not lethal but certainly deserves optimization.

> Let me know what you think of this issue.  In anticipation of the
> "Why not use MIPS64 build?" question, we'd prefer to and will, but the 
> MIPS64 build has underperformed or had bugs (SATA seek time,

Why would SATA seek times have any relation to the underlying kernel ...

> networking signals, etc) that MIPS32 doesn't.  For these issues
> or any case where the MIPS64 build isn't there yet, it makes
> sense to have the MIPS32 path open.

You could try to map some memory to CKSEG2/3 at the price of reducing the
amount of address space available, see also the current thread with
subject "HIGHMEM".  That approach will only work well upto 1GB RAM on
the BCM1250.  At that point you'll run into the limit of it's 32-bit
PCI bus, so will have to deal with bounce buffers or make use of it's
somewhat limited I/O MMU.

  Ralf

Re: Using more than 256 MB of memory on SB1250 in 32-bit mode, revisited

[Matthew Starzewski]

> This may be an ancillary effect of what's mentioned above, but when
num_physpages
> grows in size, nr_free_pages doesn't track with it, so in void
vfs_caches_init(unsigned long
> mempages) and the like, you get a horrible underflow condition:
>
> /* code */
>        printk("MJS - nr_free_pages():0x%X\n", nr_free_pages());
>        printk("MJS - OLD mempages:0x%X\n", mempages);
>        reserve = (mempages - nr_free_pages()) * 3/2;
>        mempages -= reserve;
>        printk("MJS - NEW reserve:0x%X mempages:0x%X\n",
>               reserve, mempages);
>
> /* printout */
> MJS - nr_free_pages():0x1E9A0
> MJS - OLD mempages:0x90000
> MJS - NEW reserve:0xAA190 mempages:0xFFFE5E70

I thought I'd wrap up this thread, especially with the HIGHMEM thread still
going around:

http://www.linux-mips.org/archives/linux-mips/2004-12/msg00141.html

The above underflow *was* my problem.  I was working in 2.6.6-rc3 before;
with the patch below from 2.6.7-rc1 everything works fine.

#  VFS cache sizing fix for small machines
http://linux.bkbits.net:8080/linux-2.6/diffs/fs/dcache.c@1.81?nav=index.html
|src/|src/fs|hist/fs/dcache.c
# BitKeeper ChangeSet
http://linux.bkbits.net:8080/linux-2.6/cset@1.1717.23.50?nav=index.html|src/
|src/fs|related/fs/dcache.c

In fs/dcache.c, vfs_caches_init():
<  reserve = (mempages - nr_free_pages()) * 3/2;
> reserve = min((mempages - nr_free_pages()) * 3/2, mempages - 1);

Regards,
Matt


----- Original Message -----
From: "Thomas Petazzoni" <thomas.petazzoni@enix.org>
To: "Matthew Starzewski" <mstarzewski@xes-inc.com>
Cc: <linux-mips@linux-mips.org>
Sent: Friday, December 10, 2004 10:10 AM
Subject: Re: Using more than 256 MB of memory on SB1250 in 32-bit mode,
revisited

Re: Using more than 256 MB of memory on SB1250 in 32-bit mode, revisited

[Matthew Starzewski]

> This may be an ancillary effect of what's mentioned above, but when
num_physpages
> grows in size, nr_free_pages doesn't track with it, so in void
vfs_caches_init(unsigned long
> mempages) and the like, you get a horrible underflow condition:
>
> /* code */
>        printk("MJS - nr_free_pages():0x%X\n", nr_free_pages());
>        printk("MJS - OLD mempages:0x%X\n", mempages);
>        reserve = (mempages - nr_free_pages()) * 3/2;
>        mempages -= reserve;
>        printk("MJS - NEW reserve:0x%X mempages:0x%X\n",
>               reserve, mempages);
>
> /* printout */
> MJS - nr_free_pages():0x1E9A0
> MJS - OLD mempages:0x90000
> MJS - NEW reserve:0xAA190 mempages:0xFFFE5E70

I thought I'd wrap up this thread, especially with the HIGHMEM thread still
going around:

http://www.linux-mips.org/archives/linux-mips/2004-12/msg00141.html

The above underflow *was* my problem.  I was working in 2.6.6-rc3 before;
with the patch below from 2.6.7-rc1 everything works fine.

#  VFS cache sizing fix for small machines
http://linux.bkbits.net:8080/linux-2.6/diffs/fs/dcache.c@1.81?nav=index.html
|src/|src/fs|hist/fs/dcache.c
# BitKeeper ChangeSet
http://linux.bkbits.net:8080/linux-2.6/cset@1.1717.23.50?nav=index.html|src/
|src/fs|related/fs/dcache.c

In fs/dcache.c, vfs_caches_init():
<  reserve = (mempages - nr_free_pages()) * 3/2;
> reserve = min((mempages - nr_free_pages()) * 3/2, mempages - 1);

Regards,
Matt


----- Original Message -----
From: "Thomas Petazzoni" <thomas.petazzoni@enix.org>
To: "Matthew Starzewski" <mstarzewski@xes-inc.com>
Cc: <linux-mips@linux-mips.org>
Sent: Friday, December 10, 2004 10:10 AM
Subject: Re: Using more than 256 MB of memory on SB1250 in 32-bit mode,
revisited