[U-Boot] at91 failures under temperature?

[U-Boot] at91 failures under temperature?

[Joey Oravec]

Hi -

I've been investigating a problem involving the at91sam9261 processor. 
Across 2 or 3 years we've manufactured two products with:

Crystal: 16.000 MHz
Processor Clock: 240 MHz (PLLA *15)
Master Clock: 120 MHz
VDDCORE: 1.2v

Based on testing history, about 10% of the assemblies fail starting 
above 30 degC. The simplest testcase to demonstrate is: 1) use uboot 
relocated to SDRAM as normal 2) enable caches 3) fill 16mb of SDRAM with 
a known pattern 4) use a test command to continually calculate and print 
CRC. As you apply heat to the processor the testcase will print an 
incorrect CRC. If the unit has not already crashed you can decrease the 
temperature and see the correct CRC again, which demonstrates that SDRAM 
hasn't flipped any bits.

Extending ram timings or lowering master clock (ie. sdram bus clock) has 
no impact. Lowering the processor clock to 224 MHz makes the problem go 
away -- at this speed I can heat the processor to the rated 85 C and the 
testcase above will not fail. Also replacing the processor usually makes 
the problem go away. You won't see the problem with any variable 
changed: caches disabled, any processor clock below 240 MHz, or VDDCORE 
at 1.3v.

I'm using PLL R/C calculated by the spreadsheet, and 240 MHz @ 1.2v 
should be within spec up to 85 C. I've requested failure analysis on 
some of the processors but I don't expect an answer any time soon.

Is anybody else successful with a 240 MHz processor clock, or are you 
using a lower processor clock for greater reliability? Can you recommend 
any other diagnostics that might identify the root cause? In the 
meantime for maximum reliability, I recommend using a processor clock 
less than 240 MHz and checking your hardware with a similar testcase.

-joey

[U-Boot] at91 failures under temperature?

[Remy Bohmer]

Hi,

2011/2/7 Joey Oravec <joravec@drewtech.com>:
> Hi -
>
> I've been investigating a problem involving the at91sam9261 processor.
> Across 2 or 3 years we've manufactured two products with:
>
> Crystal: 16.000 MHz
> Processor Clock: 240 MHz (PLLA *15)
> Master Clock: 120 MHz
> VDDCORE: 1.2v
>
> Based on testing history, about 10% of the assemblies fail starting
> above 30 degC. The simplest testcase to demonstrate is: 1) use uboot
> relocated to SDRAM as normal 2) enable caches 3) fill 16mb of SDRAM with
> a known pattern 4) use a test command to continually calculate and print
> CRC. As you apply heat to the processor the testcase will print an
> incorrect CRC. If the unit has not already crashed you can decrease the
> temperature and see the correct CRC again, which demonstrates that SDRAM
> hasn't flipped any bits.
>
> Extending ram timings or lowering master clock (ie. sdram bus clock) has
> no impact. Lowering the processor clock to 224 MHz makes the problem go
> away -- at this speed I can heat the processor to the rated 85 C and the
> testcase above will not fail. Also replacing the processor usually makes
> the problem go away. You won't see the problem with any variable
> changed: caches disabled, any processor clock below 240 MHz, or VDDCORE
> at 1.3v.
>
> I'm using PLL R/C calculated by the spreadsheet, and 240 MHz @ 1.2v
> should be within spec up to 85 C. I've requested failure analysis on
> some of the processors but I don't expect an answer any time soon.
>
> Is anybody else successful with a 240 MHz processor clock, or are you
> using a lower processor clock for greater reliability? Can you recommend
> any other diagnostics that might identify the root cause? In the
> meantime for maximum reliability, I recommend using a processor clock
> less than 240 MHz and checking your hardware with a similar testcase.

Funny... I always thought the Master-CLK could not go above 100MHz...
I do not know the exact reason any more...
Either way, we use it for several years now at 200/100Mhz with a 10MHz crystal.

Anyway: This problem is not related to U-boot in any way, and is
therefor OT. You should discuss this with Atmel.
And some good news: We had a similar problem last year. I will explain off-list.

Kind regards,

Remy