[U-Boot] mpc8313 SPL, relocate_code, large page NAND

[U-Boot] mpc8313 SPL, relocate_code, large page NAND

[Peter Vollmer]

Hi all,

I'm currently seeing a weird heisen-bug kind of problem on a mpc8313-based  
board with large page NAND flash. The problem happens in u-boot-spl in  
relocate_code. When I set a breakpoint after the first cycle of

...cpu/mpc83xx/start.S

relocate_code:
1: lwzu    r0,4(r8)
    stwu    r0,4(r7)
    bdnz    1b         <- breakpoint

    addi    r0,r5,3

   ...
and simply go on with the execution after it gets hit, everything is  
working perfectly well and I get to the u-boot CLI . But if I move the  
breakpoint to the next instruction (addi), it never gets hit. Looking@ 
the relocation destination (0x10000) afterwards in that case, I see that  
only the first 32 bytes were copied over to RAM.

I then inserted 8 pairs of lwzu/stwu inside the loop to see when exactly  
the problem occurs:

1: lwzu    r0,4(r8)
    stwu    r0,4(r7)
    lwzu    r0,4(r8)
    stwu    r0,4(r7)
    ...
    lwzu    r0,4(r8)
    stwu    r0,4(r7)
    bdnz    1b

and the last breakpoint location that would get hit is after 6 times of  
lwzu/stwu. I tested this with both a BDI2000 and a Ronetix Peedi emulator  
with exactly the same result. The jtag emulators do not do any register  
modification I know of, they just set the breakpoint.

I also tried to insert a delay of a couple of milliseconds before  
relocate_code to no avail.

I'm working with u-boot v2009.11 based on MPC8313ERDB. The RCW is read  
 from EEPROM, with ROMLOC set to large page NAND:
RCWHR = 0xa4540000
RCWLR = 0x65040000

I can see the 4kb bootblock from the first NAND page correctly mapped to  
0xfff00000.

Could anybody give me a clue whats happening here ? Could it be a cache  
related problem, or wrong RAM timing settings?

Thanks and best regards

-- 
Peter Vollmer
Innominate Security Technologies AG
Berlin / Germany

[U-Boot] mpc8313 SPL, relocate_code, large page NAND

[Peter Vollmer]

On Tue, 20 Apr 2010 10:36:08 +0200, Peter Vollmer  
<pvollmer-u-boot@innominate.com> wrote:


> I then inserted 8 pairs of lwzu/stwu inside the loop to see when exactly
> the problem occurs:
>
> 1: lwzu    r0,4(r8)
>     stwu    r0,4(r7)
>     lwzu    r0,4(r8)
>     stwu    r0,4(r7)
>     ...
>     lwzu    r0,4(r8)
>     stwu    r0,4(r7)
>     bdnz    1b
>
> and the last breakpoint location that would get hit is after 6 times of
> lwzu/stwu.

Further testing showed that after 7 consecutive stwu r0,4(r7) instructions  
alone the problem already occurs, i.e. it seems to have nothing to do with  
the local bus controller, but with DDR RAM access, right ?


-- 
Peter Vollmer
Innominate Security Technologies AG
Berlin / Germany

[U-Boot] mpc8313 SPL, relocate_code, large page NAND

[Joakim Tjernlund]

>
> On Tue, 20 Apr 2010 10:36:08 +0200, Peter Vollmer
> <pvollmer-u-boot@innominate.com> wrote:
>
>
> > I then inserted 8 pairs of lwzu/stwu inside the loop to see when exactly
> > the problem occurs:
> >
> > 1: lwzu    r0,4(r8)
> >     stwu    r0,4(r7)
> >     lwzu    r0,4(r8)
> >     stwu    r0,4(r7)
> >     ...
> >     lwzu    r0,4(r8)
> >     stwu    r0,4(r7)
> >     bdnz    1b
> >
> > and the last breakpoint location that would get hit is after 6 times of
> > lwzu/stwu.
>
> Further testing showed that after 7 consecutive stwu r0,4(r7) instructions
> alone the problem already occurs, i.e. it seems to have nothing to do with
> the local bus controller, but with DDR RAM access, right ?

Probably a BDI2000 issue. If memory serves right BDI2000 flushes the cache
when it hits a BP.

Try this addin this to your BDI .cfg file:
TSZ4    0xe6000000   0xe60001100 ; init stack space for cache
; needs a 'sap 0' in BDI

Then reboot BDI 2000, telnet to it and enter "sap 0" at the cmd prompt.
Start debugging.

Don't ask me what it does though. I got this tip long time ago from Abatron

  Jocke

[U-Boot] mpc8313 SPL, relocate_code, large page NAND

[Peter Vollmer]

On Tue, 20 Apr 2010 11:57:09 +0200, Joakim Tjernlund  
<joakim.tjernlund@transmode.se> wrote:
> Probably a BDI2000 issue. If memory serves right BDI2000 flushes the  
> cache when it hits a BP.
>
> Try this addin this to your BDI .cfg file:
> TSZ4    0xe6000000   0xe60001100 ; init stack space for cache
> ; needs a 'sap 0' in BDI
>
> Then reboot BDI 2000, telnet to it and enter "sap 0" at the cmd prompt.
> Start debugging.

Ok, but my problem is that it works with the BP enabled, so I can step  
through the whole relocate_code function, or simply continue, without a  
problem. It however does not work without the BP, or with the BP set too  
late, or if the board runs standalone without the emulator. So I guess I  
need to fix something in u-boot, not in the emulator configuration ...

But the bit about the BDI flushing the cache when a breakpoint gets hit,  
could that indicate that I have to flush the cache prior to entering the  
relocate_code loop? In case of a cache problem, wouldn't I just have to  
expect bogus data copied into RAM?
Instead, the core is frozen, with the PC at 0xfff00ab0 , and it refuses to  
step any further afterwards. Would it maybe help to try to include the  
exception traps in the spl bootloader ?

-- 
Peter Vollmer
Innominate Security Technologies AG
Berlin / Germany

[U-Boot] mpc8313 SPL, relocate_code, large page NAND

[Norbert van Bolhuis]

It sure looks like an mpc8313 problem I once looked into
for quite some time. The solution was:

http://lists.denx.de/pipermail/u-boot/2009-March/049175.html


It could also be your DDR RAM not properly functioning. You
can let u-boot test it with the POST MEM test (which happens
before relocation).

[U-Boot] mpc8313 SPL, relocate_code, large page NAND

[Joakim Tjernlund]

"Peter Vollmer" <pvollmer-u-boot@innominate.com> wrote on 2010/04/20 14:12:41:
>
> On Tue, 20 Apr 2010 11:57:09 +0200, Joakim Tjernlund
> <joakim.tjernlund@transmode.se> wrote:
> > Probably a BDI2000 issue. If memory serves right BDI2000 flushes the
> > cache when it hits a BP.
> >
> > Try this addin this to your BDI .cfg file:
> > TSZ4    0xe6000000   0xe60001100 ; init stack space for cache
> > ; needs a 'sap 0' in BDI
> >
> > Then reboot BDI 2000, telnet to it and enter "sap 0" at the cmd prompt.
> > Start debugging.
>
> Ok, but my problem is that it works with the BP enabled, so I can step

Oh, didn't read that carfully.

> through the whole relocate_code function, or simply continue, without a
> problem. It however does not work without the BP, or with the BP set too
> late, or if the board runs standalone without the emulator. So I guess I
> need to fix something in u-boot, not in the emulator configuration ...

Then I suspect that your DRAM isn't configured properly, perhaps burst access
is unreliable.

>
> But the bit about the BDI flushing the cache when a breakpoint gets hit,
> could that indicate that I have to flush the cache prior to entering the
> relocate_code loop? In case of a cache problem, wouldn't I just have to
> expect bogus data copied into RAM?
> Instead, the core is frozen, with the PC at 0xfff00ab0 , and it refuses to
> step any further afterwards. Would it maybe help to try to include the
> exception traps in the spl bootloader ?

Dunno, try running with any i/d cache for the DRAM. If that works
it is very likely you have misconfigured the DRAM

[U-Boot] mpc8313 SPL, relocate_code, large page NAND

[Peter Vollmer]

Thanks for all the valuable input. So this is what I have checked so far.  
Until now the behaviour is still exactly as described in the first post.

On Tue, 20 Apr 2010 15:30:56 +0200, Joakim Tjernlund  
<joakim.tjernlund@transmode.se> wrote:
> Dunno, try running with any i/d cache for the DRAM. If that works
> it is very likely you have misconfigured the DRAM

I checked that I have valid BAT entries and tried to remove anything  
unused compared to the MPC8313ERDB(like PCIe , NOR Flash etc.). These are  
my current BAT settings, which work when I start with enabled BP as  
described.

#define CONFIG_HIGH_BATS        1       /* High BATs supported */

/* 128 MB of DDR @ 0x00000000 */
#define CONFIG_SYS_IBAT0L       (CONFIG_SYS_SDRAM_BASE | BATL_PP_10 |  
BATL_CACHEINHIBIT | BATL_GUARDEDSTORAGE)
#define CONFIG_SYS_IBAT0U       (CONFIG_SYS_SDRAM_BASE | BATU_BL_128M |  
BATU_VS | BATU_VP)

/* IMMRBAR @ 0xE0000000 */
#define CONFIG_SYS_IBAT1L       (CONFIG_SYS_IMMR | BATL_PP_10 |  
BATL_CACHEINHIBIT | BATL_GUARDEDSTORAGE)
#define CONFIG_SYS_IBAT1U       (CONFIG_SYS_IMMR | BATU_BL_256M | BATU_VS  
| BATU_VP)

#define CONFIG_SYS_IBAT2L       (0xF0000000 | BATL_PP_10 |   
BATL_GUARDEDSTORAGE)
#define CONFIG_SYS_IBAT2U       (0xF0000000 | BATU_BL_256M | BATU_VS |  
BATU_VP)


> It sure looks like an mpc8313 problem I once looked into
> for quite some time. The solution was:
> http://lists.denx.de/pipermail/u-boot/2009-March/049175.html

I worked through that thread, but as it seems I have no unguarded memory  
range left. Each BAT entry has BATL_GUARDEDSTORAGE set.
By using the BATL_CACHEINHIBIT for the RAM bock I suppose I have the  
dcache disabled, right?


On Tue, 20 Apr 2010 14:44:10 +0200, Norbert van Bolhuis  
<nvbolhuis@aimvalley.nl> wrote:
> It could also be your DDR RAM not properly functioning. You
> can let u-boot test it with the POST MEM test (which happens
> before relocation).

I checked RAM using mtest in the CLI (I'll also let the memory test run  
through the night). The board has 128MB, so I did "mtest 0x3000  
0x03e97000" , and no problems so far. If I start u-boot through the right  
breakpoint, I can see no issues with DDR or NAND whatsoever.

Is it possible to enable the exception handlers (maybe at least  
MachineCheckException) in the SPL loader or how can I get the sytem into a  
defined state when the problem happens ? Currently I'm even unsure where  
the core is jumping to, so maybe that would be a good start. If the  
exception happens while still running in the high BMS, would the  
MachineCheckException hit address 0xfff00200 , or would it already try to  
jump to 0x00000200 (with a potentially uninitialised DDR RAM) ?

Best Regards

-- 
Peter Vollmer
Innominate Security Technologies AG
Berlin / Germany

[U-Boot] mpc8313 SPL, relocate_code, large page NAND

[Scott Wood]

On Tue, Apr 20, 2010 at 07:02:53PM +0200, Peter Vollmer wrote:
> I worked through that thread, but as it seems I have no unguarded memory  
> range left. Each BAT entry has BATL_GUARDEDSTORAGE set.
> By using the BATL_CACHEINHIBIT for the RAM bock I suppose I have the  
> dcache disabled, right?

Hmm, how is lock_ram_in_cache going to work then?

> On Tue, 20 Apr 2010 14:44:10 +0200, Norbert van Bolhuis  
> <nvbolhuis@aimvalley.nl> wrote:
> > It could also be your DDR RAM not properly functioning. You
> > can let u-boot test it with the POST MEM test (which happens
> > before relocation).
> 
> I checked RAM using mtest in the CLI (I'll also let the memory test run  
> through the night). The board has 128MB, so I did "mtest 0x3000  
> 0x03e97000" , and no problems so far. If I start u-boot through the right  
> breakpoint, I can see no issues with DDR or NAND whatsoever.
> 
> Is it possible to enable the exception handlers (maybe at least  
> MachineCheckException) in the SPL loader 

You could try to get some custom code in there maybe, but there's no room
for the standard exception code.

> or how can I get the sytem into a  
> defined state when the problem happens ? Currently I'm even unsure where  
> the core is jumping to, so maybe that would be a good start. If the  
> exception happens while still running in the high BMS, would the  
> MachineCheckException hit address 0xfff00200 , or would it already try to  
> jump to 0x00000200 (with a potentially uninitialised DDR RAM) ?

Depends on what the value of MSR[IP] is at that point.  It looks like
MSR[IP] will not be cleared in the NAND SPL.

-Scott