[U-Boot-Users] at91sam9260-ek intermittenly fails at u-boot init

[U-Boot-Users] at91sam9260-ek intermittenly fails at u-boot init

[Norman Cheung]

I am struggling with this problem for a week now and not making any progress. 
So any suggestions, ideas or pointer will be greatly appreciated.

The board works with u-boot 1.3.2 and at91bootstrap(1.5 I only have the binary)
booting from NAND.  We have changes the SDRAM from 32bit databus to 16bit and
NAND to 32M.   I am using U-boot 1.3.2 and AT91bootStrap 1.8. 

I changed the SDRAMC and PIOC init per Atmel's application notes in 
at91bootstrap.

U-boot fails intermittently at the start.  When it fails, it didn't make it to
my first printf statement.  When I use JTAG to debug it, it always works.  

If the board has been let without power for 1+ hour, it will also boot up when 
I apply power to it(something to do with the battery perhaps).  Once it booted
up,it is very stable.  I can run Mtest etc.  

We tried replacing NAND, SDRAM, etc.  Along the way we are battling bugs with 
SAM-BA as well.  We think that we have observed the same behavior on another
board.  We also experimented with different bootstraps and u-boot binaries.  

The only thing we can think of is Reset cycle or some h/ware init code in 
u-boot that might not be compatible with at91bootstrap's init code. 
   
We had at91bootstrap 1.5 working well in 32-bit mode with u-boot 1.3.2.  But I
don't have the source.  I wonder what is the current stable version of u-boot
and at91bootstrap? 

Thanks in advance for your help.

Norman

[U-Boot-Users] at91sam9260-ek intermittenly fails at u-boot init

[Andrejs Cainikovs]

Norman Cheung wrote:
> U-boot fails intermittently at the start.  When it fails, it didn't make it to
> my first printf statement.  When I use JTAG to debug it, it always works.  
>
> If the board has been let without power for 1+ hour, it will also boot up when 
> I apply power to it(something to do with the battery perhaps).  Once it booted
> up,it is very stable.  I can run Mtest etc.  
>
> ...
>
> The only thing we can think of is Reset cycle or some h/ware init code in 
> u-boot that might not be compatible with at91bootstrap's init code.
We had *exactly* the same problem with our AT91RM9200 based custom board 
recently.
After battling lots of days trying to find a bug within software we 
realized that problem actually hides in hardware. Boards reset line is 
controlled via small supervisor AVR. After tuning that watchdog properly 
according to datasheets and leaving small reserve, finally board is 
working as supposed.
Trying to fix this I believe we read half of the internet topics related 
to that :), and I have found similar problem description in details 
(don't remember where, though). There was one guy claiming that there 
should be correct reset procedure during cold reset, or AT91 CPU's 
memory controller will fail to initialize.
Based on that, we had reviewed our watchdog timings, have separated NRST 
and NTRST (!) giving each personal delay, and now it rocks.

I sincerely believe that french engineers of Atmel's EK/DK boards are 
too competent to make such a mistake, but probably it is worth trying to 
test this.

Kind regards,
Andrejs Cainikovs.

[U-Boot-Users] at91sam9260-ek intermittenly fails at u-boot init

[Stelian Pop]

Le mercredi 25 juin 2008 ? 01:36 +0000, Norman Cheung a ?crit :
> I am struggling with this problem for a week now and not making any progress. 
> So any suggestions, ideas or pointer will be greatly appreciated.
[...]

> The board works with u-boot 1.3.2 and at91bootstrap(1.5 I only have the binary)
> booting from NAND.

The source (for the latest version) is freely available on the Atmel
site: http://www.atmel.com/dyn/products/tools_card.asp?tool_id=4093

If you really need the source for an older version, I'm sure the Atmel
support will send it to you upon request.

> U-boot fails intermittently at the start.  When it fails, it didn't make it to
> my first printf statement.  When I use JTAG to debug it, it always works.  
> 
> If the board has been let without power for 1+ hour, it will also boot up when 
> I apply power to it(something to do with the battery perhaps).

Did you try disconnecting the battery ? Or replacing the battery with a
new one ?

[...]
   
> We had at91bootstrap 1.5 working well in 32-bit mode with u-boot 1.3.2.  But I
> don't have the source.

See above.

>   I wonder what is the current stable version of u-boot
> and at91bootstrap? 

AT91Bootstrap seems to be at v1.9.

As for U-Boot, try the latest git (which will soon be u-boot v1.3.4).

(BTW, if Wolfgang is listening, it would be nice to have proper tags in
git as well as releases on the ftp server for the -pre version. Since
the merge window is closed, I guess we're up to v1.3.4-pre1 now, but
this version tag cannot be found anywhere...)

Stelian.
-- 
Stelian Pop <stelian@popies.net>

[U-Boot-Users] at91sam9260-ek intermittenly fails at u-boot init

[Jean-Christophe PLAGNIOL-VILLARD]

> (BTW, if Wolfgang is listening, it would be nice to have proper tags in
> git as well as releases on the ftp server for the -pre version. Since
> the merge window is closed, I guess we're up to v1.3.4-pre1 now, but
> this version tag cannot be found anywhere...)
good idea

Best  Regards,
J.

[U-Boot-Users] at91sam9260-ek intermittenly fails at u-boot init

[Norman Cheung]

Many thanks Stelian for the replies.  My comments inline.

Norman

Stelian Pop wrote:
> Le mercredi 25 juin 2008 ? 01:36 +0000, Norman Cheung a ?crit :
>   
>> I am struggling with this problem for a week now and not making any progress. 
>> So any suggestions, ideas or pointer will be greatly appreciated.
>>     
> [...]
>
>   
Since the post earlier, I have narrowed the problem to the bootstrap 
during the copy from NAND to SDRAM, memories are corrupted.  I think I 
have a marginal NAND chip.  I played around the  SMC  cycle timing  in 
bootstrap,  I have a setting that I can boot u-boot with power cycle 6 
out of 10 times.  So I am convinced that it is not an u-boot issue, Thanks!
>> The board works with u-boot 1.3.2 and at91bootstrap(1.5 I only have the binary)
>> booting from NAND.
>>     
>
> The source (for the latest version) is freely available on the Atmel
> site: http://www.atmel.com/dyn/products/tools_card.asp?tool_id=4093
>
> If you really need the source for an older version, I'm sure the Atmel
> support will send it to you upon request.
>
>   
>> U-boot fails intermittently at the start.  When it fails, it didn't make it to
>> my first printf statement.  When I use JTAG to debug it, it always works.  
>>
>> If the board has been let without power for 1+ hour, it will also boot up when 
>> I apply power to it(something to do with the battery perhaps).
>>     
>
> Did you try disconnecting the battery ? Or replacing the battery with a
> new one ?
>
> [...]
>   
I tried disconnecting the battery and with the new SMC timing changes, I 
can boot up only one time after disconnecting the battery.  I will go 
and get a new battery.
>    
>   
>> We had at91bootstrap 1.5 working well in 32-bit mode with u-boot 1.3.2.  But I
>> don't have the source.
>>     
>
> See above.
>
>   
>>   I wonder what is the current stable version of u-boot
>> and at91bootstrap? 
>>     
>
> AT91Bootstrap seems to be at v1.9.
>
> As for U-Boot, try the latest git (which will soon be u-boot v1.3.4).
>
> (BTW, if Wolfgang is listening, it would be nice to have proper tags in
> git as well as releases on the ftp server for the -pre version. Since
> the merge window is closed, I guess we're up to v1.3.4-pre1 now, but
> this version tag cannot be found anywhere...)
>
> Stelian.
>   

[U-Boot-Users] at91sam9260-ek intermittenly fails at u-boot init

[Norman Cheung]

Dear all,

I am still struggling with this problem, but getting to reproduce this 
problem repeatably.  Hoping folks in the forum can give me some 
pointers.  Our Atmel FAE is clueless also.

This is repeatable, on two boards (both 9260 Rev A and Rev B):
- bootstrap, u-boot, kernel, RFS all downloaded to NAND with SAM-BA 2.6
- Without recycle power, just push reset button; u-boot will come up.  
And I can continue to boot Kernel (2.6.25)
- Push reset second time(and subsequent times), bootstrap will run and 
fail in the first few instruction of u-boot
- Sometimes power cycle, u-boot will come up again

- When it runs, if I has u-boot boot kernel, linux will run. 
- Push reset button in Linux board will reboot and u-boot will run again 
(all the times)
- If I use linux reboot, the board will stop after running bootstrap 
and  (I believe) branched to u-boot.

Our h/ware guy has checked all the timing NAND, SDRAM etc and we have 
experimented with all the settings. 

Thanks in advance for your help,
Norman

Norman Cheung wrote:
> Many thanks Stelian for the replies.  My comments inline.
>
> Norman
>
> Stelian Pop wrote:
>> Le mercredi 25 juin 2008 ? 01:36 +0000, Norman Cheung a ?crit :
>>  
>>> I am struggling with this problem for a week now and not making any 
>>> progress. So any suggestions, ideas or pointer will be greatly 
>>> appreciated.
>>>     
>> [...]
>>
>>   
> Since the post earlier, I have narrowed the problem to the bootstrap 
> during the copy from NAND to SDRAM, memories are corrupted.  I think I 
> have a marginal NAND chip.  I played around the  SMC  cycle timing  in 
> bootstrap,  I have a setting that I can boot u-boot with power cycle 6 
> out of 10 times.  So I am convinced that it is not an u-boot issue, 
> Thanks!
>>> The board works with u-boot 1.3.2 and at91bootstrap(1.5 I only have 
>>> the binary)
>>> booting from NAND.
>>>     
>>
>> The source (for the latest version) is freely available on the Atmel
>> site: http://www.atmel.com/dyn/products/tools_card.asp?tool_id=4093
>>
>> If you really need the source for an older version, I'm sure the Atmel
>> support will send it to you upon request.
>>
>>  
>>> U-boot fails intermittently at the start.  When it fails, it didn't 
>>> make it to
>>> my first printf statement.  When I use JTAG to debug it, it always 
>>> works. 
>>> If the board has been let without power for 1+ hour, it will also 
>>> boot up when I apply power to it(something to do with the battery 
>>> perhaps).
>>>     
>>
>> Did you try disconnecting the battery ? Or replacing the battery with a
>> new one ?
>>
>> [...]
>>   
> I tried disconnecting the battery and with the new SMC timing changes, 
> I can boot up only one time after disconnecting the battery.  I will 
> go and get a new battery.
>>     
>>> We had at91bootstrap 1.5 working well in 32-bit mode with u-boot 
>>> 1.3.2.  But I
>>> don't have the source.
>>>     
>>
>> See above.
>>
>>  
>>>   I wonder what is the current stable version of u-boot
>>> and at91bootstrap?     
>>
>> AT91Bootstrap seems to be at v1.9.
>>
>> As for U-Boot, try the latest git (which will soon be u-boot v1.3.4).
>>
>> (BTW, if Wolfgang is listening, it would be nice to have proper tags in
>> git as well as releases on the ftp server for the -pre version. Since
>> the merge window is closed, I guess we're up to v1.3.4-pre1 now, but
>> this version tag cannot be found anywhere...)
>>
>> Stelian.
>>   
>
>

[U-Boot-Users] at91sam9260-ek intermittenly fails at u-boot init

[Stelian Pop]

Le vendredi 27 juin 2008 ? 14:31 -0700, Norman Cheung a ?crit :
> Dear all,

Hi Norman,

> I am still struggling with this problem, but getting to reproduce this 
> problem repeatably.  Hoping folks in the forum can give me some 
> pointers.  Our Atmel FAE is clueless also.
> 
> This is repeatable, on two boards (both 9260 Rev A and Rev B):
> - bootstrap, u-boot, kernel, RFS all downloaded to NAND with SAM-BA 2.6
> - Without recycle power, just push reset button; u-boot will come up.  
> And I can continue to boot Kernel (2.6.25)
> - Push reset second time(and subsequent times), bootstrap will run and 
> fail in the first few instruction of u-boot
> - Sometimes power cycle, u-boot will come up again
> 
> - When it runs, if I has u-boot boot kernel, linux will run. 
> - Push reset button in Linux board will reboot and u-boot will run again 
> (all the times)
> - If I use linux reboot, the board will stop after running bootstrap 
> and  (I believe) branched to u-boot.
> 
> Our h/ware guy has checked all the timing NAND, SDRAM etc and we have 
> experimented with all the settings. 

It looks like you have a SDRAM initialisation problem. To confirm this
(since you said that you cannot reproduce this problem with a JTAG), you
need to modify the source code for AT91Bootstrap, and add a 'verify'
pass (using a checksum for example) after the copy from NAND to SDRAM.
This way you'll find out if the reading from NAND is going wrong, or the
writing to SDRAM (I suspect the latter).

Next, you may encounter an issue with the reset sequence. I know that on
some newer Atmel boards (AT91CAP9 and AT572D940HF) there is a problem
when an external reset is performed while a SDRAM burst access is
occuring. The workaround for this issue is to catch the external
interrupt generated by the reset switch, and do the reset by software
(see the AT572D940HF patches posted recently by Antonio Costa for the
full code). I'm not sure the 9260 exhibits the same bug, but you may
query your FAE for that.

Hope this gives some clues.

Stelian.
-- 
Stelian Pop <stelian@popies.net>