[U-Boot-Users] [PATCH] ppc4xx: Don't use last 256 bytes of SDRAM, workaround for 440EPx CHIP 11 errata

[U-Boot-Users] [PATCH] ppc4xx: Don't use last 256 bytes of SDRAM, workaround for 440EPx CHIP 11 errata

[Stefan Roese]

Since 440EPx/GRx has problems with accessing the last 256 bytes of SDRAM via
the Denali DDR/DDR2 controller, we set CONFIG_PRAM to 1 and reserve 1kByte
of protected RAM. This way this memory will not get "touched" by U-Boot. And
by passing "mem=${mem}" to the Linux kernel, Linux will not use this area
either.

This patch adds this workaround for the following 440EPx boards:
sequoia, lwmon5. Others should probably follow this example.

Here the description from the AMCC documentation:

CHIP_11: End of memory range area restricted access.
Category: 3

Overview:
The 440EPx DDR controller does not acknowledge any
transaction which is determined to be crossing over the
end-of-memory-range boundary, even if the starting address is
within valid memory space. Any such transaction from any PLB4
master will result in a PLB time-out on PLB4 bus.

Impact:
In case of such misaligned bursts, PLB4 masters will not
retrieve any data at all, just the available data up to the
end of memory, especially the 440 CPU. For example, if a CPU
instruction required an operand located in memory within the
last 7 words of memory, the DCU master would burst read 8
words to update the data cache and cross over the
end-of-memory-range boundary. Such a DCU read would not be
answered by the DDR controller, resulting in a PLB4 time-out
and ultimately in a Machine Check interrupt. The data would
be inaccessible to the CPU.

Workaround:
Forbid any application to access the last 256 bytes of DDR
memory. For example, make your operating system believe that
the last 256 bytes of DDR memory are absent. AMCC has a patch
that does this, available for Linux.

Signed-off-by: Stefan Roese <sr@denx.de>
---
 include/configs/lwmon5.h  |    7 +++++--
 include/configs/sequoia.h |    2 ++
 2 files changed, 7 insertions(+), 2 deletions(-)

diff --git a/include/configs/lwmon5.h b/include/configs/lwmon5.h
index c3f10c7..62c0e58 100644
--- a/include/configs/lwmon5.h
+++ b/include/configs/lwmon5.h
@@ -1,5 +1,5 @@
 /*
- * (C) Copyright 2007
+ * (C) Copyright 2007-2008
  * Stefan Roese, DENX Software Engineering, sr at denx.de.
  *
  * This program is free software; you can redistribute it and/or
@@ -145,6 +145,8 @@
 #else
 #define CFG_POST_ECC_ON		0
 #endif
+#define CONFIG_PRAM		1	/* don't use last 256 bytes	*/
+					/* 440EPx errata CHIP 11	*/
 
 /* POST support */
 #define CONFIG_POST		(CFG_POST_CACHE    | \
@@ -205,7 +207,8 @@
 		"ip=${ipaddr}:${serverip}:${gatewayip}:${netmask}"	\
 		":${hostname}:${netdev}:off panic=1\0"			\
 	"addtty=setenv bootargs ${bootargs} console=ttyS1,${baudrate}\0"\
-	"addmisc=setenv bootargs ${bootargs} rtc-pcf8563.probe=0,0x51\0"\
+	"addmisc=setenv bootargs ${bootargs} rtc-pcf8563.probe=0,0x51"	\
+		" mem=${mem}\0"						\
 	"flash_nfs=run nfsargs addip addtty addmisc;"			\
 		"bootm ${kernel_addr}\0"				\
 	"flash_self=run ramargs addip addtty addmisc;"			\
diff --git a/include/configs/sequoia.h b/include/configs/sequoia.h
index dfa8779..541f8bc 100644
--- a/include/configs/sequoia.h
+++ b/include/configs/sequoia.h
@@ -221,6 +221,8 @@
 #if !defined(CONFIG_NAND_U_BOOT) && !defined(CONFIG_NAND_SPL)
 #define CONFIG_DDR_DATA_EYE		/* use DDR2 optimization	*/
 #endif
+#define CONFIG_PRAM		1	/* don't use last 256 bytes	*/
+					/* 440EPx errata CHIP 11	*/
 
 /*
  * I2C
-- 
1.5.4.4

[U-Boot-Users] [PATCH] ppc4xx: Don't use last 256 bytes of SDRAM, workaround for 440EPx CHIP 11 errata

[Wolfgang Denk]

Hello Stefan,

in message <1206006397-16582-1-git-send-email-sr@denx.de> you wrote:
> Since 440EPx/GRx has problems with accessing the last 256 bytes of SDRAM via
> the Denali DDR/DDR2 controller, we set CONFIG_PRAM to 1 and reserve 1kByte
> of protected RAM. This way this memory will not get "touched" by U-Boot. And
> by passing "mem=${mem}" to the Linux kernel, Linux will not use this area
> either.

Note that this workaround is incomplete.

If you enable shared (between U-Boot and Linux) log buffer, then  the
log  buffer  will  go at the end of memory, and the pRAM area will be
allocated below. In this case the problems is not only still present,
but also very likely to hit pretty soon.

> This patch adds this workaround for the following 440EPx boards:
> sequoia, lwmon5. Others should probably follow this example.

OK, the default configs for Sequoia doesn't use shared log buffer, and
the lwmon5 uses CONFIG_ALT_LB_ADDR (i. e. it puts the log buffer in
OCM), so these boards will work.

But any changes to the configuration may easily break this again.


I don't like the idea that  each  affected  board  has  to  fix  this
(probably  differently)  in  it's board config file. I would prefer a
fix (common to all affected boards) in one central location.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
"Unix is simple, but it takes a genius to understand the simplicity."
					             - Dennis Ritchie

[U-Boot-Users] [PATCH] ppc4xx: Don't use last 256 bytes of SDRAM, workaround for 440EPx CHIP 11 errata

[Stefan Roese]

Hi Wolfgang,

On Thursday 20 March 2008, Wolfgang Denk wrote:
> > This patch adds this workaround for the following 440EPx boards:
> > sequoia, lwmon5. Others should probably follow this example.
>
> OK, the default configs for Sequoia doesn't use shared log buffer, and
> the lwmon5 uses CONFIG_ALT_LB_ADDR (i. e. it puts the log buffer in
> OCM), so these boards will work.
>
> But any changes to the configuration may easily break this again.

Understood.

> I don't like the idea that  each  affected  board  has  to  fix  this
> (probably  differently)  in  it's board config file. I would prefer a
> fix (common to all affected boards) in one central location.

I don't like this either. But I have not come up with a "generic" solution 
till now. It's not so easy since the 440EPx SDRAM setup code is sometimes 
common (cpu/ppc4xx/denali_spd_ddr2.c) and sometimes board specific. And it 
gets even more complicated when ECC initialization is needed too. So 
currently I have no "better" solution as to do it in a board specific way.

As always, patches implementing a "better" more generic solution are welcome.

Thanks.

Best regards,
Stefan

=====================================================================
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-0 Fax: +49-8142-66989-80  Email: office at denx.de
=====================================================================

[U-Boot-Users] [PATCH] ppc4xx: Don't use last 256 bytes of SDRAM, workaround for 440EPx CHIP 11 errata

[Wolfgang Denk]

In message <200803201129.06602.sr@denx.de> you wrote:
> 
> I don't like this either. But I have not come up with a "generic" solution 
> till now. It's not so easy since the 440EPx SDRAM setup code is sometimes 
> common (cpu/ppc4xx/denali_spd_ddr2.c) and sometimes board specific. And it 
> gets even more complicated when ECC initialization is needed too. So 
> currently I have no "better" solution as to do it in a board specific way.
> 
> As always, patches implementing a "better" more generic solution are welcome.

At the cost of an #ifdef, this could be added to the memory
reservation code in "lib_ppc/board.c", i. e. somewhere after

 ...
 427          * Reserve memory at end of RAM for (top down in that order):
 428          *  - kernel log buffer
 429          *  - protected RAM
 430          *  - LCD framebuffer
 431          *  - monitor code
 432          *  - board info struct
 433          */
 434         len = (ulong)&_end - CFG_MONITOR_BASE;
 435
 436         addr = CFG_SDRAM_BASE + get_effective_memsize();

Of course you need to make sure that  the  linux  kernel  understands
this changed behaviour / shifted addresses, too.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Minds are like parachutes - they only function when open.

[U-Boot-Users] [PATCH] ppc4xx: Don't use last 256 bytes of SDRAM, workaround for 440EPx CHIP 11 errata

[Kumar Gala]

On Mar 20, 2008, at 6:16 AM, Wolfgang Denk wrote:
> In message <200803201129.06602.sr@denx.de> you wrote:
>>
>> I don't like this either. But I have not come up with a "generic"  
>> solution
>> till now. It's not so easy since the 440EPx SDRAM setup code is  
>> sometimes
>> common (cpu/ppc4xx/denali_spd_ddr2.c) and sometimes board specific.  
>> And it
>> gets even more complicated when ECC initialization is needed too. So
>> currently I have no "better" solution as to do it in a board  
>> specific way.
>>
>> As always, patches implementing a "better" more generic solution  
>> are welcome.
>
> At the cost of an #ifdef, this could be added to the memory
> reservation code in "lib_ppc/board.c", i. e. somewhere after
>
> ...
> 427          * Reserve memory at end of RAM for (top down in that  
> order):
> 428          *  - kernel log buffer
> 429          *  - protected RAM
> 430          *  - LCD framebuffer
> 431          *  - monitor code
> 432          *  - board info struct
> 433          */
> 434         len = (ulong)&_end - CFG_MONITOR_BASE;
> 435
> 436         addr = CFG_SDRAM_BASE + get_effective_memsize();
>
> Of course you need to make sure that  the  linux  kernel  understands
> this changed behaviour / shifted addresses, too.

I haven't looked at the exact details but with the new-image branch of  
u-boot-testing the introduction of using the lmb to keep track of  
memory regions used might make this a little easier.  It clearly will  
not help in determining when to do a lmb_reserve().

- k

[U-Boot-Users] [PATCH] ppc4xx: Don't use last 256 bytes of SDRAM, workaround for 440EPx CHIP 11 errata

[Stefan Roese]

On Thursday 20 March 2008, Wolfgang Denk wrote:
> At the cost of an #ifdef, this could be added to the memory
> reservation code in "lib_ppc/board.c", i. e. somewhere after
>
>  ...
>  427          * Reserve memory at end of RAM for (top down in that order):
>  428          *  - kernel log buffer
>  429          *  - protected RAM
>  430          *  - LCD framebuffer
>  431          *  - monitor code
>  432          *  - board info struct
>  433          */
>  434         len = (ulong)&_end - CFG_MONITOR_BASE;
>  435
>  436         addr = CFG_SDRAM_BASE + get_effective_memsize();

OK. I'll add it here with an additional #ifdef here. How about something like 
this:

	len = (ulong)&_end - CFG_MONITOR_BASE;

+#ifdef CONFIG_MEM_TOP_HIDE
+	/*
+	 * Subtract specified amount of memory to hide so that it won't
+	 * get "touched" at all by U-Boot. By fixing up gd->ram_size
+	 * the Linux kernel should now get passed the now "corrected"
+	 * memory size and won't touch it either. This should work
+	 * for arch/ppc and arch/powerpc. Only Linux board ports in
+	 * arch/powerpc with bootwrapper support, that recalculate the
+	 * memory size from the SDRAM controller setup will have to
+	 * get fixed.
+	 */
+	gd->ram_size -= CONFIG_MEM_TOP_HIDE;
+#endif

	addr = CFG_SDRAM_BASE + get_effective_memsize();

Or even better this somewhere above:

#ifndef CONFIG_MEM_TOP_HIDE
#define CONFIG_MEM_TOP_HIDE	0
#endif

and we can drop the #ifdef completely.

> Of course you need to make sure that  the  linux  kernel  understands
> this changed behaviour / shifted addresses, too.

Right. By fixing up gd->ram_size in the new patch version above, this should 
get handled for arch/ppc and arch/powerpc kernel versions. So no need 
for 'mem=xxx' in the bootargs anymore. Or did I miss something here?

Thanks.

Best regards,
Stefan

=====================================================================
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-0 Fax: +49-8142-66989-80  Email: office at denx.de
=====================================================================

[U-Boot-Users] [PATCH] ppc4xx: Don't use last 256 bytes of SDRAM, workaround for 440EPx CHIP 11 errata

[Dave Littell]

Stefan Roese wrote:
> Hi Wolfgang,
> 
> On Thursday 20 March 2008, Wolfgang Denk wrote:
>>> This patch adds this workaround for the following 440EPx boards:
>>> sequoia, lwmon5. Others should probably follow this example.
>> OK, the default configs for Sequoia doesn't use shared log buffer, and
>> the lwmon5 uses CONFIG_ALT_LB_ADDR (i. e. it puts the log buffer in
>> OCM), so these boards will work.
>>
>> But any changes to the configuration may easily break this again.
> 
> Understood.
> 
>> I don't like the idea that  each  affected  board  has  to  fix  this
>> (probably  differently)  in  it's board config file. I would prefer a
>> fix (common to all affected boards) in one central location.
> 
> I don't like this either. But I have not come up with a "generic" solution 
> till now. It's not so easy since the 440EPx SDRAM setup code is sometimes 
> common (cpu/ppc4xx/denali_spd_ddr2.c) and sometimes board specific. And it 
> gets even more complicated when ECC initialization is needed too. So 
> currently I have no "better" solution as to do it in a board specific way.
> 
> As always, patches implementing a "better" more generic solution are welcome.
> 

Hi all,

This may not be the "better, more generic solution", but I just added a
CONFIG_CHIP_11_ERRATA option and code in lib_ppc/board.c to trim off
only 256 bytes before the optional log buffer is carved out.

There's a kernel patch for this errata that trims the memory length.  If
that patch is applied, does the kernel still need to be made to
understand the changed logbuffer location?


Thanks,
Dave

[U-Boot-Users] [PATCH] ppc4xx: Don't use last 256 bytes of SDRAM, workaround for 440EPx CHIP 11 errata

[Stefan Roese]

On Thursday 20 March 2008, Dave Littell wrote:
> This may not be the "better, more generic solution", but I just added a
> CONFIG_CHIP_11_ERRATA option and code in lib_ppc/board.c to trim off
> only 256 bytes before the optional log buffer is carved out.

OK. I'll submit a similar patch in a short while. Please give it a try and let 
me know if you see problems with it.

> There's a kernel patch for this errata that trims the memory length.  If
> that patch is applied, does the kernel still need to be made to
> understand the changed logbuffer location?

I don't think so. With my next patch it should not be necessary anymore. With 
one exception: Linux kernel ports that have fixed memory sizes or calculate 
the memory size from the SDRAM controller setup (as done in some arch/powerpc 
kernels with bootwrapper support).

Best regards,
Stefan

=====================================================================
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-0 Fax: +49-8142-66989-80  Email: office at denx.de
=====================================================================

[U-Boot-Users] [PATCH] ppc4xx: Don't use last 256 bytes of SDRAM, workaround for 440EPx CHIP 11 errata

[Larry Johnson]

Stefan Roese wrote:
> Hi Wolfgang,
> 
> On Thursday 20 March 2008, Wolfgang Denk wrote:
>>> This patch adds this workaround for the following 440EPx boards:
>>> sequoia, lwmon5. Others should probably follow this example.
>> OK, the default configs for Sequoia doesn't use shared log buffer, and
>> the lwmon5 uses CONFIG_ALT_LB_ADDR (i. e. it puts the log buffer in
>> OCM), so these boards will work.
>>
>> But any changes to the configuration may easily break this again.
> 
> Understood.
> 
>> I don't like the idea that  each  affected  board  has  to  fix  this
>> (probably  differently)  in  it's board config file. I would prefer a
>> fix (common to all affected boards) in one central location.
> 
> I don't like this either. But I have not come up with a "generic" solution 
> till now. It's not so easy since the 440EPx SDRAM setup code is sometimes 
> common (cpu/ppc4xx/denali_spd_ddr2.c) and sometimes board specific. And it 
> gets even more complicated when ECC initialization is needed too. So 
> currently I have no "better" solution as to do it in a board specific way.
> 
> As always, patches implementing a "better" more generic solution are welcome.
> 
> Thanks.
> 
> Best regards,
> Stefan

Hi Stefan,

Having multiple implementations of the 440EXp SDRAM setup code is, in
itself, less than ideal.  One alternative is to have a 440EPX board with
on-board SDRAM chips fake an array of SPD bytes describing the chips,
and pass it to the "denali_spd_ddr2.c" code.  The SPD code then becomes
the single location for the technology of how to configure the Denali
SDRAM controller.

The additional benefit (for me, anyway :-) ) is having more eyeballs on
the SPD code, with more opportunities for testing, bug fixing, and
enhancements.

Best regards,
Larry

[U-Boot-Users] [PATCH] ppc4xx: Don't use last 256 bytes of SDRAM, workaround for 440EPx CHIP 11 errata

[Stefan Roese]

Hi Larry,

On Thursday 20 March 2008, Larry Johnson wrote:
> Having multiple implementations of the 440EXp SDRAM setup code is, in
> itself, less than ideal.  One alternative is to have a 440EPX board with
> on-board SDRAM chips fake an array of SPD bytes describing the chips,
> and pass it to the "denali_spd_ddr2.c" code.  The SPD code then becomes
> the single location for the technology of how to configure the Denali
> SDRAM controller.

Generally a good idea. Unfortunately it can't be done. Think about 440EPx 
boards booting from NAND flash. Here the complete CPU setup code including 
SDRAM setup needs to fit into 4k. This can only be done with a fixed SDRAM 
setup. :-(

But your are right. Other boards without such a image size restriction should 
probably use the SPD code. One reason they aren't doing it right now is, that 
the SPD code is quite new and didn't exist when those board port were done. 
But they can be converted to use this code at some time.

> The additional benefit (for me, anyway :-) ) is having more eyeballs on
> the SPD code, with more opportunities for testing, bug fixing, and
> enhancements.

Ack.

BTW: Can you test your board with ECC modules? We need to change the ECC code 
in the Denali SPD routines to not touch the last 256 bytes here too. Best 
would be if you could provide a patch for this. :)

Thanks.

Best regards,
Stefan

=====================================================================
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-0 Fax: +49-8142-66989-80  Email: office at denx.de
=====================================================================

[U-Boot-Users] [PATCH] ppc4xx: Don't use last 256 bytes of SDRAM, workaround for 440EPx CHIP 11 errata

[Larry Johnson]

Stefan Roese wrote:
> [...]
> 
> BTW: Can you test your board with ECC modules? We need to change the ECC code 
> in the Denali SPD routines to not touch the last 256 bytes here too. Best 
> would be if you could provide a patch for this. :)
> 
> Thanks.
> 
> Best regards,
> Stefan

Yes, we normally use ECC modules in our testing.  I've been looking at a
patch for "initdram()" Denali SPD, but I've been waiting to see how your
"CFG_MEM_TOP_HIDE" patch would turn out.

As things stand now, can I assume that boards using the Denali SPD will
also define "CFG_MEM_TOP_HIDE", and therefore initdram() should continue
to return the full size of the memory?

The only place that the last 256 bytes of memory are touched is when
"dflush()" is called to zero the SDRAM.  This does not cause a Machine
Check interrupt.  I am guessing that all the writes from "dflush()" are
aligned, and therefore there are no burst that access beyond the end of
of the SDRAM memory space.  If so, then my inclination is not to change
this part of the code.  Does this make sense?

Best regards,
Larry

[U-Boot-Users] [PATCH] ppc4xx: Don't use last 256 bytes of SDRAM, workaround for 440EPx CHIP 11 errata

[Stefan Roese]

On Thursday 27 March 2008, Larry Johnson wrote:
> Yes, we normally use ECC modules in our testing.  I've been looking at a
> patch for "initdram()" Denali SPD, but I've been waiting to see how your
> "CFG_MEM_TOP_HIDE" patch would turn out.
>
> As things stand now, can I assume that boards using the Denali SPD will
> also define "CFG_MEM_TOP_HIDE", and therefore initdram() should continue
> to return the full size of the memory?

Correct.

> The only place that the last 256 bytes of memory are touched is when
> "dflush()" is called to zero the SDRAM.

To be more specific, in the dcbz_area() call.

> This does not cause a Machine 
> Check interrupt.  I am guessing that all the writes from "dflush()" are
> aligned, and therefore there are no burst that access beyond the end of
> of the SDRAM memory space.  If so, then my inclination is not to change
> this part of the code.  Does this make sense?

I recommend to change it this way:

        /* Zero the memory */
        debug("Zeroing SDRAM...");
-       dcbz_area(CFG_SDRAM_BASE, dram_size);
+       dcbz_area(CFG_SDRAM_BASE, dram_size - CFG_MEM_TOP_HIDE);
        dflush();

And perhaps put something like:

#ifndef CFG_MEM_TOP_HIDE
#error "Please define CFG_MEM_TOP_HIDE (see README) in your board config 
file!"
#endif

at the top of the file. What do you think?

Best regards,
Stefan

=====================================================================
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-0 Fax: +49-8142-66989-80  Email: office at denx.de
=====================================================================

[U-Boot-Users] [PATCH] ppc4xx: Don't use last 256 bytes of SDRAM, workaround for 440EPx CHIP 11 errata

[Larry Johnson]

Stefan Roese wrote:
> On Thursday 27 March 2008, Larry Johnson wrote:
>> Yes, we normally use ECC modules in our testing.  I've been looking at a
>> patch for "initdram()" Denali SPD, but I've been waiting to see how your
>> "CFG_MEM_TOP_HIDE" patch would turn out.
>>
>> As things stand now, can I assume that boards using the Denali SPD will
>> also define "CFG_MEM_TOP_HIDE", and therefore initdram() should continue
>> to return the full size of the memory?
> 
> Correct.
> 
>> The only place that the last 256 bytes of memory are touched is when
>> "dflush()" is called to zero the SDRAM.
> 
> To be more specific, in the dcbz_area() call.
> 
>> This does not cause a Machine 
>> Check interrupt.  I am guessing that all the writes from "dflush()" are
>> aligned, and therefore there are no burst that access beyond the end of
>> of the SDRAM memory space.  If so, then my inclination is not to change
>> this part of the code.  Does this make sense?
> 
> I recommend to change it this way:
> 
>         /* Zero the memory */
>         debug("Zeroing SDRAM...");
> -       dcbz_area(CFG_SDRAM_BASE, dram_size);
> +       dcbz_area(CFG_SDRAM_BASE, dram_size - CFG_MEM_TOP_HIDE);
>         dflush();
> 
> And perhaps put something like:
> 
> #ifndef CFG_MEM_TOP_HIDE
> #error "Please define CFG_MEM_TOP_HIDE (see README) in your board config 
> file!"
> #endif
> 
> at the top of the file. What do you think?
> 
> Best regards,
> Stefan

That sound fine, I'll try to get a patch out soon.

Best regards,
Larry