X86_64 kernel support MAX memory.

X86_64 kernel support MAX memory.

[YhLu]

Andi,

How much is max RAM 2.6.11 x86_64 support on AMD64?
64G or 128G?

YH

Re: X86_64 kernel support MAX memory.

[Andi Kleen]

On Mon, Feb 14, 2005 at 07:32:42PM -0800, YhLu wrote:
> Andi,
> 
> How much is max RAM 2.6.11 x86_64 support on AMD64?
> 64G or 128G?

46bits in theory (64TB), however current CPUs only support
upto 40bits (AMD) or 36bits (Intel).  There is some other
code that is also limited to 40bits right now like AGP
or IOMMU or MTRR, that is all due to hardware limitations.

-Andi

RE: X86_64 kernel support MAX memory.

[YhLu]

I got a system with 8 way Opteron. Every CPU has 16G memory.

2.6 kernel x86_64, it will crash when it start the Fifth node.

YH

> -----Original Message-----
> From: Andi Kleen [mailto:ak@muc.de] 
> Sent: Tuesday, February 15, 2005 4:08 AM
> To: YhLu
> Cc: linux-kernel@vger.kernel.org
> Subject: Re: X86_64 kernel support MAX memory.
> 
> On Mon, Feb 14, 2005 at 07:32:42PM -0800, YhLu wrote:
> > Andi,
> > 
> > How much is max RAM 2.6.11 x86_64 support on AMD64?
> > 64G or 128G?
> 
> 46bits in theory (64TB), however current CPUs only support 
> upto 40bits (AMD) or 36bits (Intel).  There is some other 
> code that is also limited to 40bits right now like AGP or 
> IOMMU or MTRR, that is all due to hardware limitations.
> 
> -Andi
> 

Re: X86_64 kernel support MAX memory.

[Andi Kleen]

On Tue, Feb 15, 2005 at 10:49:05AM -0800, YhLu wrote:
> I got a system with 8 way Opteron. Every CPU has 16G memory.
> 
> 2.6 kernel x86_64, it will crash when it start the Fifth node.

The kernel has been successfully booted on 8 CPU Opteron systems before.
Most likely it is something specific to your system.

-Andi

RE: X86_64 kernel support MAX memory.

[YhLu]

If only use 64G RAM, it works well.

YH 

> -----Original Message-----
> From: Andi Kleen [mailto:ak@muc.de] 
> Sent: Tuesday, February 15, 2005 12:15 PM
> To: YhLu
> Cc: linux-kernel@vger.kernel.org
> Subject: Re: X86_64 kernel support MAX memory.
> 
> On Tue, Feb 15, 2005 at 10:49:05AM -0800, YhLu wrote:
> > I got a system with 8 way Opteron. Every CPU has 16G memory.
> > 
> > 2.6 kernel x86_64, it will crash when it start the Fifth node.
> 
> The kernel has been successfully booted on 8 CPU Opteron 
> systems before.
> Most likely it is something specific to your system.
> 
> -Andi
> 

Re: X86_64 kernel support MAX memory.

[Andi Kleen]

On Tue, Feb 15, 2005 at 02:34:17PM -0800, YhLu wrote:
> If only use 64G RAM, it works well.

Are you sure the RAM is not broken?  The more you have of it
the more likely one DIMM is bad.

Otherwise debug it. What's the oops dump?

-Andi

Re: X86_64 kernel support MAX memory.

[Andi Kleen]

On Tue, Feb 15, 2005 at 02:57:14PM -0800, YhLu wrote:
> It passed the memtest86+ 3.1a

Are you sure it even tests the full 128GB? Traditionally PAE only
supports 64GB. 

> 
> No oops dump, it just restart the system.

At what point exactly? You probably have a serial
console. What are the last lines.

That could well be an ECC error. You can see if mcelog
logs something after reboot (kernel should preserve 
machine check events) 

Or you could switch around the DIMMs of the CPUs for 
testing.

-Andi

RE: X86_64 kernel support MAX memory.

[YhLu]

It passed the memtest86+ 3.1a

No oops dump, it just restart the system.

YH 

> -----Original Message-----
> From: Andi Kleen [mailto:ak@muc.de] 
> Sent: Tuesday, February 15, 2005 2:42 PM
> To: YhLu
> Cc: linux-kernel@vger.kernel.org
> Subject: Re: X86_64 kernel support MAX memory.
> 
> On Tue, Feb 15, 2005 at 02:34:17PM -0800, YhLu wrote:
> > If only use 64G RAM, it works well.
> 
> Are you sure the RAM is not broken?  The more you have of it 
> the more likely one DIMM is bad.
> 
> Otherwise debug it. What's the oops dump?
> 
> -Andi
> 

RE: X86_64 kernel support MAX memory.

[YhLu]

Thanks.

I will test 64G on node 4-7 only or 64G on node 0-3.

YH

> -----Original Message-----
> From: Andi Kleen [mailto:ak@muc.de] 
> Sent: Tuesday, February 15, 2005 2:55 PM
> To: YhLu
> Cc: linux-kernel@vger.kernel.org
> Subject: Re: X86_64 kernel support MAX memory.
> 
> On Tue, Feb 15, 2005 at 02:57:14PM -0800, YhLu wrote:
> > It passed the memtest86+ 3.1a
> 
> Are you sure it even tests the full 128GB? Traditionally PAE 
> only supports 64GB. 
> 
> > 
> > No oops dump, it just restart the system.
> 
> At what point exactly? You probably have a serial console. 
> What are the last lines.
> 
> That could well be an ECC error. You can see if mcelog logs 
> something after reboot (kernel should preserve machine check events) 
> 
> Or you could switch around the DIMMs of the CPUs for testing.
> 
> -Andi
>