Re: HIGHMEM

Re: HIGHMEM

[Jon Anders Haugum]

> In 2.4 the support for CONFIG_DISCONTIG and CONFIG_NUMA are a bit tangled
> with each other because IP27 is the only platform to uses these features and
> it needs both.  Other than that you can also just setup your system as 0x0 -
> 0x10000000 being RAM, 0x10000000 - 0x20000000 being reserved memory and
> 0x20000000 - 0x30000000 being highmem.  Which works but is a bit wasteful.
>
> Issue #2 is that we don't support the combination of CONFIG_DISCONTIG and
> CONFIG_HIGHMEM.  And highmem is a lobotomized solution for lobotomized
> silicon anyway.  You have a 64-bit processor - use it's capabilities :-)
>
> Issue #3 - As I recall the TX4937's H3 core is suffering from cache aliases.
> Handling those efficiently for highmem is not easily possible and so we
> don't even try.  More recent kernels will refuse to enable highmem on such
> cache configurations but something like 2.4.18 which by now is an almost 3
> year old antique doesn't know about that and will happily crash.
>
> I recommend you should go for a 64-bit kernel instead.  And 64-bit support
> is certainly better in 2.6 than in 2.4.  Especially the area of 32-bit
> binary compatibility has been improved significantly.

What about on a processor like the AMD au1550?

I've tried the latest from 2.4 branch in cvs, and I haven't been 
successful in geting past INIT either...

Can I place all the memory from address 0x20000000 to get more than 
256Meg?


-- 
Jon Anders Haugum

Re: HIGHMEM

[Ralf Baechle]

On Tue, Dec 07, 2004 at 11:29:04AM +0100, Jon Anders Haugum wrote:

> What about on a processor like the AMD au1550?
> 
> I've tried the latest from 2.4 branch in cvs, and I haven't been 
> successful in geting past INIT either...
> 
> Can I place all the memory from address 0x20000000 to get more than 
> 256Meg?

Yes, that should just work on Alchemy.

  Ralf

HIGHMEM

[Hdei Nunoe]

Hi there,

Has anyone succeeded the HIGHMEM with discontiguous physical memory?
I am using kernel 2.4.18 on TX4937 with two chunks of 256Mbyte memory.
There is 256Mbyte gap in between the physical memory blocks - lower 
memory is 0x00000000 to 0x10000000, upper memory is 0x20000000 to
0x30000000.  System hungs when it create INIT process.

Cheers,
-hdei

HIGHMEM

[Hdei Nunoe]

Hi there,

Has anyone succeeded the HIGHMEM with discontiguous physical memory?
I am using kernel 2.4.18 on TX4937 with two chunks of 256Mbyte memory.
There is 256Mbyte gap in between the physical memory blocks - lower 
memory is 0x00000000 to 0x10000000, upper memory is 0x20000000 to
0x30000000.  System hungs when it create INIT process.

Cheers,
-hdei

Re: HIGHMEM

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 642 bytes --]

On Tue, 2004-12-07 10:07:26 +0900, Hdei Nunoe <nunoe@co-nss.co.jp>
wrote in message <001101c4dbf9$1da02270$3ca06096@NUNOE>:

> Has anyone succeeded the HIGHMEM with discontiguous physical memory?
> I am using kernel 2.4.18 on TX4937 with two chunks of 256Mbyte memory.

2.4.18 is obsolete...

MfG, JBG

-- 
Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481             _ O _
"Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg  _ _ O
 fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!   O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: HIGHMEM

[Hdei Nunoe]

Hi Jan,

Will it work on the newer version?

Cheers,
-hdei

----- Original Message ----- 
From: "Jan-Benedict Glaw" <jbglaw@lug-owl.de>
To: "Hdei Nunoe" <nunoe@co-nss.co.jp>
Cc: <linux-mips@linux-mips.org>
Sent: Tuesday, December 07, 2004 6:17 PM
Subject: Re: HIGHMEM

> Has anyone succeeded the HIGHMEM with discontiguous physical memory?
> I am using kernel 2.4.18 on TX4937 with two chunks of 256Mbyte memory.

2.4.18 is obsolete...

MfG, JBG

-- 

Re: HIGHMEM

[Hdei Nunoe]

Hi Jan,

Will it work on the newer version?

Cheers,
-hdei

----- Original Message ----- 
From: "Jan-Benedict Glaw" <jbglaw@lug-owl.de>
To: "Hdei Nunoe" <nunoe@co-nss.co.jp>
Cc: <linux-mips@linux-mips.org>
Sent: Tuesday, December 07, 2004 6:17 PM
Subject: Re: HIGHMEM

> Has anyone succeeded the HIGHMEM with discontiguous physical memory?
> I am using kernel 2.4.18 on TX4937 with two chunks of 256Mbyte memory.

2.4.18 is obsolete...

MfG, JBG

-- 

Re: HIGHMEM

[Jan-Benedict Glaw]

[-- Attachment #1: Type: text/plain, Size: 547 bytes --]

On Tue, 2004-12-07 18:56:33 +0900, Hdei Nunoe <nunoe@co-nss.co.jp>
wrote in message <002101c4dc43$08c4c7d0$3ca06096@NUNOE>:

> Will it work on the newer version?

Did you try recent 2.6.x versions?

MfG, JBG

-- 
Jan-Benedict Glaw       jbglaw@lug-owl.de    . +49-172-7608481             _ O _
"Eine Freie Meinung in  einem Freien Kopf    | Gegen Zensur | Gegen Krieg  _ _ O
 fuer einen Freien Staat voll Freier Bürger" | im Internet! |   im Irak!   O O O
ret = do_actions((curr | FREE_SPEECH) & ~(NEW_COPYRIGHT_LAW | DRM | TCPA));

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: HIGHMEM

[Ralf Baechle]

On Tue, Dec 07, 2004 at 10:07:26AM +0900, Hdei Nunoe wrote:

> Has anyone succeeded the HIGHMEM with discontiguous physical memory?
> I am using kernel 2.4.18 on TX4937 with two chunks of 256Mbyte memory.
> There is 256Mbyte gap in between the physical memory blocks - lower 
> memory is 0x00000000 to 0x10000000, upper memory is 0x20000000 to
> 0x30000000.  System hungs when it create INIT process.

In 2.4 the support for CONFIG_DISCONTIG and CONFIG_NUMA are a bit tangled
with each other because IP27 is the only platform to uses these features
and it needs both.  Other than that you can also just setup your system
as 0x0 - 0x10000000 being RAM, 0x10000000 - 0x20000000 being reserved
memory and 0x20000000 - 0x30000000 being highmem.  Which works but is a
bit wasteful.

Issue #2 is that we don't support the combination of CONFIG_DISCONTIG and
CONFIG_HIGHMEM.  And highmem is a lobotomized solution for lobotomized
silicon anyway.  You have a 64-bit processor - use it's capabilities :-)

Issue #3 - As I recall the TX4937's H3 core is suffering from cache
aliases.  Handling those efficiently for highmem is not easily possible
and so we don't even try.  More recent kernels will refuse to enable
highmem on such cache configurations but something like 2.4.18 which by
now is an almost 3 year old antique doesn't know about that and will
happily crash.

I recommend you should go for a 64-bit kernel instead.  And 64-bit support
is certainly better in 2.6 than in 2.4.  Especially the area of 32-bit
binary compatibility has been improved significantly.

  Ralf

Re: HIGHMEM

[Atsushi Nemoto]

>>>>> On Tue, 7 Dec 2004 10:58:37 +0100, Ralf Baechle <ralf@linux-mips.org> said:
ralf> Issue #3 - As I recall the TX4937's H3 core is suffering from
ralf> cache aliases.  Handling those efficiently for highmem is not
ralf> easily possible and so we don't even try.  More recent kernels
ralf> will refuse to enable highmem on such cache configurations but
ralf> something like 2.4.18 which by now is an almost 3 year old
ralf> antique doesn't know about that and will happily crash.

ralf> I recommend you should go for a 64-bit kernel instead.  And
ralf> 64-bit support is certainly better in 2.6 than in 2.4.
ralf> Especially the area of 32-bit binary compatibility has been
ralf> improved significantly.

And this is a small step to a 64-bit kernel for TX49XX.  Could you
apply?

--- linux-mips/arch/mips/mm/Makefile	2004-12-13 09:39:09.000000000 +0900
+++ linux/arch/mips/mm/Makefile	2004-12-13 09:52:54.000000000 +0900
@@ -49,6 +49,7 @@
 endif
 ifdef CONFIG_MIPS64
 obj-$(CONFIG_CPU_R4300)		+= tlbex64.o
+obj-$(CONFIG_CPU_TX49XX)	+= tlbex64.o
 obj-$(CONFIG_CPU_R4X00)		+= tlbex64.o
 obj-$(CONFIG_CPU_R5000)		+= tlbex64.o
 obj-$(CONFIG_CPU_NEVADA)	+= tlbex64.o

Re: HIGHMEM

[Ralf Baechle]

On Mon, Dec 13, 2004 at 01:34:09PM +0900, Atsushi Nemoto wrote:

> And this is a small step to a 64-bit kernel for TX49XX.  Could you
> apply?

Sure, done.

  Ralf

Re: HIGHMEM

[Atsushi Nemoto]

>>>>> On Wed, 15 Dec 2004 15:16:13 +0100, Ralf Baechle <ralf@linux-mips.org> said:

>> And this is a small step to a 64-bit kernel for TX49XX.  Could you
>> apply?

ralf> Sure, done.

Thanks.  And this is next step.  Could you apply too?

diff -u linux-mips/include/asm-mips/addrspace.h linux/include/asm-mips/addrspace.h 
--- linux-mips/include/asm-mips/addrspace.h	Sun Sep 26 21:31:45 2004
+++ linux/include/asm-mips/addrspace.h	Sat Dec 18 21:21:01 2004
@@ -126,6 +126,7 @@
     || defined (CONFIG_CPU_R4X00)					\
     || defined (CONFIG_CPU_R5000)					\
     || defined (CONFIG_CPU_NEVADA)					\
+    || defined (CONFIG_CPU_TX49XX)					\
     || defined (CONFIG_CPU_MIPS64)
 #define	KUSIZE			0x0000010000000000	/* 2^^40 */
 #define	KUSIZE_64		0x0000010000000000	/* 2^^40 */

Re: HIGHMEM

[Hdei Nunoe]

Ralf,

Thanks for the info!  I still have a ocuple of question, hope you do not 
mind.

> In 2.4 the support for CONFIG_DISCONTIG and CONFIG_NUMA are a bit tangled
> with each other because IP27 is the only platform to uses these features
> and it needs both.

Is it named "sgi-ip27"?

> Other than that you can also just setup your system
> as 0x0 - 0x10000000 being RAM, 0x10000000 - 0x20000000 being reserved
> memory and 0x20000000 - 0x30000000 being highmem.  Which works but is a
> bit wasteful.

The gap in physical memory is 0x10000000 - 0x20000000, but it is 
0x90000000 -
0xC0000000 in virtual memory because there is K1 segment.  So the macros 
such
as __pa() or __va() does not work, I think.  Started to wonder it might not 
be easy
as just changing the PAGE_OFFSET value.  Do you see?

cheers,
-hdei

Re: HIGHMEM

[Hdei Nunoe]

Ralf,

Thanks for the info!  I still have a ocuple of question, hope you do not 
mind.

> In 2.4 the support for CONFIG_DISCONTIG and CONFIG_NUMA are a bit tangled
> with each other because IP27 is the only platform to uses these features
> and it needs both.

Is it named "sgi-ip27"?

> Other than that you can also just setup your system
> as 0x0 - 0x10000000 being RAM, 0x10000000 - 0x20000000 being reserved
> memory and 0x20000000 - 0x30000000 being highmem.  Which works but is a
> bit wasteful.

The gap in physical memory is 0x10000000 - 0x20000000, but it is 
0x90000000 -
0xC0000000 in virtual memory because there is K1 segment.  So the macros 
such
as __pa() or __va() does not work, I think.  Started to wonder it might not 
be easy
as just changing the PAGE_OFFSET value.  Do you see?

cheers,
-hdei

Re: HIGHMEM

[Ralf Baechle]

On Tue, Dec 14, 2004 at 01:26:55PM +0900, Hdei Nunoe wrote:

> >In 2.4 the support for CONFIG_DISCONTIG and CONFIG_NUMA are a bit tangled
> >with each other because IP27 is the only platform to uses these features
> >and it needs both.
> 
> Is it named "sgi-ip27"?

Yes, obviously :-)

> >Other than that you can also just setup your system
> >as 0x0 - 0x10000000 being RAM, 0x10000000 - 0x20000000 being reserved
> >memory and 0x20000000 - 0x30000000 being highmem.  Which works but is a
> >bit wasteful.
> 
> The gap in physical memory is 0x10000000 - 0x20000000, but it is 
> 0x90000000 -
> 0xC0000000 in virtual memory because there is K1 segment.  So the macros 
> such
> as __pa() or __va() does not work, I think.  Started to wonder it might not 
> be easy
> as just changing the PAGE_OFFSET value.  Do you see?

PAGE_OFFSET is the difference of a ZONE_NORMAL's virtual address and it's
physical address.  Once there is no more 1:1 mapping between physical and
virtual addresses such as in your suggestion PAGE_OFFSET can no longer be
used, that is you need to rewrite all users of this function.

  Ralf

Re: HIGHMEM

[Hdei Nunoe]

Ralf,

It worked fine with small changes!!   Comment and Criticism are welcome.

>> >Other than that you can also just setup your system
>> >as 0x0 - 0x10000000 being RAM, 0x10000000 - 0x20000000 being reserved
>> >memory and 0x20000000 - 0x30000000 being highmem.  Which works but is a
>> >bit wasteful.

I have set up my system as following :
- 0x00000000 - 0x10000000 RAM
- 0x10000000 - 0x40000000 RESERVED
- 0x40000000 - 0x50000000 RAM
So that the gap in physical address space and one in virtual address space 
becomes equal.
It is 0x40000000 in this case (0x10000000 - 0x40000000 vs 0x90000000 - 
0xc0000000).
That means no physical <-> virtual macro change are needed.  hehe...
And the MMU mapping to the upper 256MB is fixed with the CP0 wired register.

Changes are pretty small as follows :
--- tlb-r4k.c ---
381c381
<       write_32bit_cp0_register(CP0_WIRED, 0);
---
>       write_32bit_cp0_register(CP0_WIRED, 8 /* XXX 0 */);

--- pgtable.h ---
123c123
< #define VMALLOC_START     KSEG2
---
> #define VMALLOC_START     KSEG3               /* XXX KSEG2 */

--- page.h ---
148c148
< #define HIGHMEM_START 0x20000000UL
---
> #define HIGHMEM_START 0x50000000UL    /* XXX 0x20000000UL */

--- prom.c ---
>       add_memory_region(0, (256*1024*1024), BOOT_MEM_RAM);
>       add_memory_region(0x10000000, (256*1024*1024), BOOT_MEM_RESERVED);
>       add_memory_region(0x20000000, (512*1024*1024), BOOT_MEM_RESERVED);
>       add_memory_region(0x40000000, (256*1024*1024), BOOT_MEM_RAM);
>
>       {
>           TLB_TBL32 tlb;
>           unsigned int nTlb = 48;             /* max TLB entry */
>           unsigned int size = 0x02000000;     /* 32MB boundary */
>           unsigned int vadr = 0xc0000000;     /* virtual address */
>           unsigned int padr = 0x40000000;     /* physical address */
>
>           printk ("memory map started.\n");
>             for (i=0; i<8; i++)
>               {
>               tlb.mask = 0x01ffe000;          /* 16M bit[24-13] */
>               tlb.hi  = vadr + (i*size);
>               tlb.lo1 = (padr >> 6) | ((i*size + (size/2)) >> 6) | 0x1f;
>               tlb.lo0 = (padr >> 6) | ((i*size) >> 6) | 0x1f;
>               setTLB32 (i, &tlb);
>               }
>             for (i=8; i<nTlb; i++)
>               {
>               tlb.mask = 0x0;
>               tlb.hi  = 0x80000000;
>               tlb.lo1 = 0;
>               tlb.lo0 = 0;
>               setTLB32 (i, &tlb);
>               }
>             for (i=0; i<nTlb; i++)
>               {
>               getTLB32 (i, &tlb);
>               printk ("get: mask=0x%08x hi=0x%08x lo0=0x%08x 
> lo1=0x%08x.\n",
>                       tlb.mask, tlb.hi, tlb.lo0, tlb.lo1);
>               }
>       }

Cheers,
-hdei

Re: HIGHMEM

[Hdei Nunoe]

Ralf,

It worked fine with small changes!!   Comment and Criticism are welcome.

>> >Other than that you can also just setup your system
>> >as 0x0 - 0x10000000 being RAM, 0x10000000 - 0x20000000 being reserved
>> >memory and 0x20000000 - 0x30000000 being highmem.  Which works but is a
>> >bit wasteful.

I have set up my system as following :
- 0x00000000 - 0x10000000 RAM
- 0x10000000 - 0x40000000 RESERVED
- 0x40000000 - 0x50000000 RAM
So that the gap in physical address space and one in virtual address space 
becomes equal.
It is 0x40000000 in this case (0x10000000 - 0x40000000 vs 0x90000000 - 
0xc0000000).
That means no physical <-> virtual macro change are needed.  hehe...
And the MMU mapping to the upper 256MB is fixed with the CP0 wired register.

Changes are pretty small as follows :
--- tlb-r4k.c ---
381c381
<       write_32bit_cp0_register(CP0_WIRED, 0);
---
>       write_32bit_cp0_register(CP0_WIRED, 8 /* XXX 0 */);

--- pgtable.h ---
123c123
< #define VMALLOC_START     KSEG2
---
> #define VMALLOC_START     KSEG3               /* XXX KSEG2 */

--- page.h ---
148c148
< #define HIGHMEM_START 0x20000000UL
---
> #define HIGHMEM_START 0x50000000UL    /* XXX 0x20000000UL */

--- prom.c ---
>       add_memory_region(0, (256*1024*1024), BOOT_MEM_RAM);
>       add_memory_region(0x10000000, (256*1024*1024), BOOT_MEM_RESERVED);
>       add_memory_region(0x20000000, (512*1024*1024), BOOT_MEM_RESERVED);
>       add_memory_region(0x40000000, (256*1024*1024), BOOT_MEM_RAM);
>
>       {
>           TLB_TBL32 tlb;
>           unsigned int nTlb = 48;             /* max TLB entry */
>           unsigned int size = 0x02000000;     /* 32MB boundary */
>           unsigned int vadr = 0xc0000000;     /* virtual address */
>           unsigned int padr = 0x40000000;     /* physical address */
>
>           printk ("memory map started.\n");
>             for (i=0; i<8; i++)
>               {
>               tlb.mask = 0x01ffe000;          /* 16M bit[24-13] */
>               tlb.hi  = vadr + (i*size);
>               tlb.lo1 = (padr >> 6) | ((i*size + (size/2)) >> 6) | 0x1f;
>               tlb.lo0 = (padr >> 6) | ((i*size) >> 6) | 0x1f;
>               setTLB32 (i, &tlb);
>               }
>             for (i=8; i<nTlb; i++)
>               {
>               tlb.mask = 0x0;
>               tlb.hi  = 0x80000000;
>               tlb.lo1 = 0;
>               tlb.lo0 = 0;
>               setTLB32 (i, &tlb);
>               }
>             for (i=0; i<nTlb; i++)
>               {
>               getTLB32 (i, &tlb);
>               printk ("get: mask=0x%08x hi=0x%08x lo0=0x%08x 
> lo1=0x%08x.\n",
>                       tlb.mask, tlb.hi, tlb.lo0, tlb.lo1);
>               }
>       }

Cheers,
-hdei

Re: HIGHMEM

[Ralf Baechle]

On Tue, Jan 11, 2005 at 11:33:51AM +0900, Hdei Nunoe wrote:

> I have set up my system as following :
> - 0x00000000 - 0x10000000 RAM
> - 0x10000000 - 0x40000000 RESERVED
> - 0x40000000 - 0x50000000 RAM

Your setup may work it's pretty wasteful; you're burning 12MB memory for
unused kernel data structures.

  Ralf

Re: HIGHMEM

[Geert Uytterhoeven]

On Tue, 11 Jan 2005, Ralf Baechle wrote:
> On Tue, Jan 11, 2005 at 11:33:51AM +0900, Hdei Nunoe wrote:
> > I have set up my system as following :
> > - 0x00000000 - 0x10000000 RAM
> > - 0x10000000 - 0x40000000 RESERVED
> > - 0x40000000 - 0x50000000 RAM
> 
> Your setup may work it's pretty wasteful; you're burning 12MB memory for
> unused kernel data structures.

Time to start using virtually contiguous memory, like m68k does? ;-)
(no, it's not in mainline)

Gr{oetje,eeting}s,

						Geert

--
Geert Uytterhoeven -- There's lots of Linux beyond ia32 -- geert@linux-m68k.org

In personal conversations with technical people, I call myself a hacker. But
when I'm talking to journalists I just say "programmer" or something like that.
							    -- Linus Torvalds

Re: HIGHMEM

[Ralf Baechle]

On Tue, Jan 11, 2005 at 11:38:59AM +0100, Geert Uytterhoeven wrote:

> Time to start using virtually contiguous memory, like m68k does? ;-)
> (no, it's not in mainline)

Won't work for MIPS; there's the KSEG1 address gap.

  Ralf

HIGHMEM

[Nagy Tibor]

Hi,

I am sorry, I have found your e-mail address in 
./arch/i386/kernel/setup.c. I have the problem below since a year, and 
there is no solution yet. I guess, the problem is about e820. The 
problem exists in 2.4.x and also in 2.6.1.

We have two Dell Poweredge servers, an older one (PowerEdge 6300) and a
newer one (PowerEdge 6400). Both servers have 4GB RAM, but the Linux
kernel uses about 500MB less memory in the newer machine.

----------------------- 2.4.20 ---------------------------------

See /var/log/boot.msg on the old one:

<4>Linux version 2.4.20 (root@dell632) (gcc version 2.95.3 20010315
(SuSE)) #4 SMP Fri Jan 10 12:07:00 CET 2003
<6>BIOS-provided physical RAM map:
<4> BIOS-e820: 0000000000000000 - 00000000000a0000 (usable)
<4> BIOS-e820: 0000000000100000 - 00000000fbffe000 (usable)
<4> BIOS-e820: 00000000fbffe000 - 00000000fc000000 (reserved)
<4> BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved)
<4> BIOS-e820: 00000000fee00000 - 00000000fee10000 (reserved)
<4> BIOS-e820: 00000000fff80000 - 0000000100000000 (reserved)
<5>3135MB HIGHMEM available.
<5>896MB LOWMEM available.

and on the new one:

<4>Linux version 2.4.20 (root@alfa) (gcc version 2.95.3 20010315 (SuSE))
#10 SMP Fri Mar 28 15:40:45 CET 2003
<6>BIOS-provided physical RAM map:
<4> BIOS-e820: 0000000000000000 - 00000000000a0000 (usable)
<4> BIOS-e820: 0000000000100000 - 00000000dfff0000 (usable)
<4> BIOS-e820: 00000000dfff0000 - 00000000dfffec00 (ACPI data)
<4> BIOS-e820: 00000000dfffec00 - 00000000dffff000 (reserved)
<4> BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved)
<4> BIOS-e820: 00000000fee00000 - 00000000fee10000 (reserved)
<4> BIOS-e820: 00000000fff80000 - 0000000100000000 (reserved)
<5>2687MB HIGHMEM available.
<5>896MB LOWMEM available.

There is a big hole between 00000000dffff000 and 00000000fec00000, which
is not used on the new machine. What can I do?

------------------------------ 2.6.1 ------------------------------
I do not see in boot.msg such a detailed memory map, but the next line 
in boot.msg indicates, that the situation is the same:

<6>Memory: 3627908k/3669952k available (2918k kernel code, 40908k 
reserved, 1049k data, 180k init, 2752448k highmem)

--------------------------------------------------------------------

ACPI can not be disabled in BIOS. If ACPI is disbled in the kernel, the 
memory mapping remains the same, but CPU hyperthreading does not work also.


Thanks for your help.

Tibor


------------------------------------------------------------------------
Tibor Nagy
E-mail: nagyt@otpbank.hu
------------------------------------------------------------------------

Re: HIGHMEM

[Sean Neakums]

Nagy Tibor <nagyt@otpbank.hu> writes:

> Hi,
>
> I am sorry, I have found your e-mail address in
> ./arch/i386/kernel/setup.c. I have the problem below since a year, and
> there is no solution yet. I guess, the problem is about e820. The
> problem exists in 2.4.x and also in 2.6.1.
>
> We have two Dell Poweredge servers, an older one (PowerEdge 6300) and a
> newer one (PowerEdge 6400). Both servers have 4GB RAM, but the Linux
> kernel uses about 500MB less memory in the newer machine.

I may be talking through my hat, but I think that in this case you
need to select the option for support of 64G highmem.  If I recall,
"4G highmem" refers not to the total amount to the memory, but to the
highest physical address that can be accessed.

Re: HIGHMEM

[Martin J. Bligh]

>> I am sorry, I have found your e-mail address in
>> ./arch/i386/kernel/setup.c. I have the problem below since a year, and
>> there is no solution yet. I guess, the problem is about e820. The
>> problem exists in 2.4.x and also in 2.6.1.
>> 
>> We have two Dell Poweredge servers, an older one (PowerEdge 6300) and a
>> newer one (PowerEdge 6400). Both servers have 4GB RAM, but the Linux
>> kernel uses about 500MB less memory in the newer machine.
> 
> I may be talking through my hat, but I think that in this case you
> need to select the option for support of 64G highmem.  If I recall,
> "4G highmem" refers not to the total amount to the memory, but to the
> highest physical address that can be accessed.

That's exactly correct. Whether the gain of 500MB of RAM is worth the
overhead of PAE is another question ... but that's how to do it ;-)

M.

Re: HIGHMEM

[Matt Domsch]

On Fri, Feb 13, 2004 at 08:05:14AM -0800, Martin J. Bligh wrote:
> >> We have two Dell Poweredge servers, an older one (PowerEdge 6300) and a
> >> newer one (PowerEdge 6400). Both servers have 4GB RAM, but the Linux
> >> kernel uses about 500MB less memory in the newer machine.
> > 
> > I may be talking through my hat, but I think that in this case you
> > need to select the option for support of 64G highmem.  If I recall,
> > "4G highmem" refers not to the total amount to the memory, but to the
> > highest physical address that can be accessed.
> 
> That's exactly correct. Whether the gain of 500MB of RAM is worth the
> overhead of PAE is another question ... but that's how to do it ;-)

If the chipset and BIOS can't remap the physical RAM out of the
address space needed by the PCI devices and into PAE space, then PAE
doesn't buy you anything.  You need chipset support for RAM remapping,
which doesn't exist on the servers mentioned.

Thanks,
Matt

-- 
Matt Domsch
Sr. Software Engineer, Lead Engineer
Dell Linux Solutions linux.dell.com & www.dell.com/linux
Linux on Dell mailing lists @ http://lists.us.dell.com

Re: HIGHMEM

[Martin J. Bligh]

> On Fri, Feb 13, 2004 at 08:05:14AM -0800, Martin J. Bligh wrote:
>> >> We have two Dell Poweredge servers, an older one (PowerEdge 6300) and a
>> >> newer one (PowerEdge 6400). Both servers have 4GB RAM, but the Linux
>> >> kernel uses about 500MB less memory in the newer machine.
>> > 
>> > I may be talking through my hat, but I think that in this case you
>> > need to select the option for support of 64G highmem.  If I recall,
>> > "4G highmem" refers not to the total amount to the memory, but to the
>> > highest physical address that can be accessed.
>> 
>> That's exactly correct. Whether the gain of 500MB of RAM is worth the
>> overhead of PAE is another question ... but that's how to do it ;-)
> 
> If the chipset and BIOS can't remap the physical RAM out of the
> address space needed by the PCI devices and into PAE space, then PAE
> doesn't buy you anything.  You need chipset support for RAM remapping,
> which doesn't exist on the servers mentioned.

Ah, didn't realise you had a hardware problem as well ;-)

M.

Re: HIGHMEM

[david parsons]

On Fri, Feb 13, 2004 at 01:12:48PM +0000, Sean Neakums wrote:
> Nagy Tibor <nagyt@otpbank.hu> writes:
> 
> > Hi,
> >
> > I am sorry, I have found your e-mail address in
> > ./arch/i386/kernel/setup.c. I have the problem below since a year, and
> > there is no solution yet. I guess, the problem is about e820. The
> > problem exists in 2.4.x and also in 2.6.1.
> >
> > We have two Dell Poweredge servers, an older one (PowerEdge 6300) and a
> > newer one (PowerEdge 6400). Both servers have 4GB RAM, but the Linux
> > kernel uses about 500MB less memory in the newer machine.
> 
> I may be talking through my hat, but I think that in this case you
> need to select the option for support of 64G highmem.  If I recall,
> "4G highmem" refers not to the total amount to the memory, but to the
> highest physical address that can be accessed.


   Its been several years since I did anything with the 820 code,
   but it all uses long longs so it should be fine with up to 4gb
   and a casual look at the now much hacked code doesn't seem to
   show anything that would leap out and start laughing at you.


                 ____
   david parsons \bi/ ``Sanitize the BIOS e820 map''?  
                  \/

Re: HIGHMEM

[Matt Domsch]

On Fri, Feb 13, 2004 at 01:20:36PM +0100, Nagy Tibor wrote:
> We have two Dell Poweredge servers, an older one (PowerEdge 6300) and a
> newer one (PowerEdge 6400). Both servers have 4GB RAM, but the Linux
> kernel uses about 500MB less memory in the newer machine.

This is a FAQ on the Linux-PowerEdge mailing list, and is captured here.
http://lists.us.dell.com/fom-serve/cache/68.html

I have 4GB (or more) RAM in my system. How come Linux sees less than
that?

BIOS must reserve some address space below 4GB for PCI devices such as
RAID controllers, SCSI controllers, NICs, etc. RAID controllers in
particular may request and be given 256MB each. This is address space
that would normally be occupied by RAM, but instead is used by PCI
devices.

RAM addresses start at 0 and grow up. PCI device addresses start at
~4GB and grow down. As long as there is no overlap, the OS will see
all available RAM and make use of it. If there is overlap, the PCI
devices win, and that RAM is not made available to the OS.

This is working as designed per PCI, BIOS, and system chipset
specifications. 


You may wish to subscribe to the Linux-PowerEdge@dell.com mailing list
at http://lists.us.dell.com/.

Thanks,
Matt

-- 
Matt Domsch
Sr. Software Engineer, Lead Engineer
Dell Linux Solutions linux.dell.com & www.dell.com/linux
Linux on Dell mailing lists @ http://lists.us.dell.com

HIGHMEM

[Nagy Tibor]

Hi,

We have two Dell Poweredge servers, an older one (PowerEdge 6300) and a 
newer one (PowerEdge 6400). Both servers have 4GB RAM, but the Linux 
kernel uses about 500MB less memory in the newer machine.

See /var/log/boot.msg on the old one:

<4>Linux version 2.4.20 (root@dell632) (gcc version 2.95.3 20010315
(SuSE)) #4 SMP Fri Jan 10 12:07:00 CET 2003
<6>BIOS-provided physical RAM map:
<4> BIOS-e820: 0000000000000000 - 00000000000a0000 (usable)
<4> BIOS-e820: 0000000000100000 - 00000000fbffe000 (usable)
<4> BIOS-e820: 00000000fbffe000 - 00000000fc000000 (reserved)
<4> BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved)
<4> BIOS-e820: 00000000fee00000 - 00000000fee10000 (reserved)
<4> BIOS-e820: 00000000fff80000 - 0000000100000000 (reserved)
<5>3135MB HIGHMEM available.
<5>896MB LOWMEM available.

and on the new one:

<4>Linux version 2.4.20 (root@alfa) (gcc version 2.95.3 20010315 (SuSE))
#10 SMP Fri Mar 28 15:40:45 CET 2003
<6>BIOS-provided physical RAM map:
<4> BIOS-e820: 0000000000000000 - 00000000000a0000 (usable)
<4> BIOS-e820: 0000000000100000 - 00000000dfff0000 (usable)
<4> BIOS-e820: 00000000dfff0000 - 00000000dfffec00 (ACPI data)
<4> BIOS-e820: 00000000dfffec00 - 00000000dffff000 (reserved)
<4> BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved)
<4> BIOS-e820: 00000000fee00000 - 00000000fee10000 (reserved)
<4> BIOS-e820: 00000000fff80000 - 0000000100000000 (reserved)
<5>2687MB HIGHMEM available.
<5>896MB LOWMEM available.


There is a big hole between 00000000dffff000 and 00000000fec00000, which
is not used on the new machine. What can I do?

Thanks for your help.

Tibor


------------------------------------------------------------------------
Tibor Nagy
E-mail: nagyt@otpbank.hu
------------------------------------------------------------------------

Re: HIGHMEM

[William Lee Irwin III]

On Tue, Apr 15, 2003 at 04:39:30PM +0200, Nagy Tibor wrote:
> <6>BIOS-provided physical RAM map:
> <4> BIOS-e820: 0000000000000000 - 00000000000a0000 (usable)
> <4> BIOS-e820: 0000000000100000 - 00000000fbffe000 (usable)
> <4> BIOS-e820: 00000000fbffe000 - 00000000fc000000 (reserved)
> <4> BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved)
> <4> BIOS-e820: 00000000fee00000 - 00000000fee10000 (reserved)
> <4> BIOS-e820: 00000000fff80000 - 0000000100000000 (reserved)
> <5>3135MB HIGHMEM available.
> <5>896MB LOWMEM available.
[...]
> <6>BIOS-provided physical RAM map:
> <4> BIOS-e820: 0000000000000000 - 00000000000a0000 (usable)
> <4> BIOS-e820: 0000000000100000 - 00000000dfff0000 (usable)
> <4> BIOS-e820: 00000000dfff0000 - 00000000dfffec00 (ACPI data)
> <4> BIOS-e820: 00000000dfffec00 - 00000000dffff000 (reserved)
> <4> BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved)
> <4> BIOS-e820: 00000000fee00000 - 00000000fee10000 (reserved)
> <4> BIOS-e820: 00000000fff80000 - 0000000100000000 (reserved)
> <5>2687MB HIGHMEM available.
> <5>896MB LOWMEM available.
> There is a big hole between 00000000dffff000 and 00000000fec00000, which
> is not used on the new machine. What can I do?

Presumably that was lost to ACPI. The hole between 0xdffff000 and
0xfec00000 appears to not be covered by the e820.

Try turning ACPI off in your .config since there's something that
looks relevant to it different between 2.4 and 2.5. You also might
want to follow up with .config's just in case. I'll look at 2.5's e820
stuff, but no promises.


-- wli

Re: HIGHMEM

[Samuel Flory]

Nagy Tibor wrote:

>
>
> There is a big hole between 00000000dffff000 and 00000000fec00000, which
> is not used on the new machine. What can I do?
>

  You might also try making sure you have the latest bios, and try 
turning off acpi in the bios.

-- 
There is no such thing as obsolete hardware.
Merely hardware that other people don't want.
(The Second Rule of Hardware Acquisition)
Sam Flory  <sflory@rackable.com>