Kernel 2.6.17.8 on Quad AMD Opteron 852 with 16x 4GB Modules (64GB RAM)

Kernel 2.6.17.8 on Quad AMD Opteron 852 with 16x 4GB Modules (64GB RAM)

[Björn Engelhardt]

Hello,

we upgraded a Server from 32 GB RAM to 64 GB. Now we try to get a Linux 
(FC5) with kernel 2.6.17.8 on a Quad Opteron (852; 64bit)-system with 
16x 4GB modules to run.
With 32 GB (8x 4GB modules) the system starts without any problems, but 
above I get kernelpanics.
The output then gives me several memoryaddresses bevore the panic 
appears. The board (a Tyan K8QW,model S4881) should support up to 64GB 
Ram. A Memorytest under Linux recognizes the 64GB and continues without 
an error.
I tried several BIOS-Settings.
Does the kernel support the new 4GB-Modules by 64GB Ram?

Thanks for every help, I have no more ideas in the moment

Regards
    Björn Engelhardt

PS: Sorry for my bad english

Re: Kernel 2.6.17.8 on Quad AMD Opteron 852 with 16x 4GB Modules (64GB RAM)

[H. Peter Anvin]

Björn Engelhardt wrote:
> Hello,
> 
> we upgraded a Server from 32 GB RAM to 64 GB. Now we try to get a Linux 
> (FC5) with kernel 2.6.17.8 on a Quad Opteron (852; 64bit)-system with 
> 16x 4GB modules to run.
> With 32 GB (8x 4GB modules) the system starts without any problems, but 
> above I get kernelpanics.
> The output then gives me several memoryaddresses bevore the panic 
> appears. The board (a Tyan K8QW,model S4881) should support up to 64GB 
> Ram. A Memorytest under Linux recognizes the 64GB and continues without 
> an error.
> I tried several BIOS-Settings.
> Does the kernel support the new 4GB-Modules by 64GB Ram?
> 

Sounds like your memory is bad.

	-hpa

Re: Kernel 2.6.17.8 on Quad AMD Opteron 852 with 16x 4GB Modules (64GB RAM)

[Andi Kleen]

Björn Engelhardt <bjoern2@xqueue.de> writes:

> we upgraded a Server from 32 GB RAM to 64 GB. Now we try to get a
> Linux (FC5) with kernel 2.6.17.8 on a Quad Opteron (852; 64bit)-system
> with 16x 4GB modules to run.
> With 32 GB (8x 4GB modules) the system starts without any problems,
> but above I get kernelpanics.

It 99.9+% likely a hardware or BIOS or power supply/VRM/cooling
problem of some sort. So many DIMMs are quite stress full
to the system and can expose issues which were hidden before.
Or you might run into some other BIOS/hardware issue.

You can post the exact text of the panics from a serial console
if you want, but most likely you will get the same answer even then.

I would start talking to your hardware vendor.

> The output then gives me several memoryaddresses bevore the panic
> appears. The board (a Tyan K8QW,model S4881) should support up to 64GB
> Ram. A Memorytest under Linux recognizes the 64GB and continues
> without an error.
> I tried several BIOS-Settings.
> Does the kernel support the new 4GB-Modules by 64GB Ram?

The kernel doesn't know anything about DIMMs, it just gets a memory
map from the BIOS.

-Andi

Re: Kernel 2.6.17.8 on Quad AMD Opteron 852 with 16x 4GB Modules (64GB RAM)

[Maciek Zobniow]

>Björn Engelhardt wrote:
>> Hello,
>> 
>> we upgraded a Server from 32 GB RAM to 64 GB. Now we try to get a Linux 
>> (FC5) with kernel 2.6.17.8 on a Quad Opteron (852; 64bit)-system with 
>> 16x 4GB modules to run.
>> With 32 GB (8x 4GB modules) the system starts without any problems, but 
>> above I get kernelpanics.
>> The output then gives me several memoryaddresses bevore the panic 
>> appears. The board (a Tyan K8QW,model S4881) should support up to 64GB 
>> Ram. A Memorytest under Linux recognizes the 64GB and continues without 
>> an error.
>> I tried several BIOS-Settings.
>> Does the kernel support the new 4GB-Modules by 64GB Ram?
>> 
>
>Sounds like your memory is bad.
>
>	-hpa

My friend Bjoern has forgotten to write that we actually checked the memory with only 32GB and kernel booted normally. 
The problem appears when we want to put more, even just next 4GB, over the 32GB. It seems to be a kernel problem to me. 
Will try to retrive and send oops tonight.

Thanks for any suggestion!

Regards
Maciek