AMCC PPC440EPx/sequoia stability question...

AMCC PPC440EPx/sequoia stability question...

[Dave Littell]

Hi all,

I'm working on an AMCC PPC440EPx-based platform that's similar to the
Sequoia.  The board runs pretty well, but occasionally takes exceptions
both in U-Boot and while running Linux.  The exceptions vary from
Illegal Instructions to FP Unavailable to FP Unimplemented to ITLB and
DTLB Errors, to DSI, etc., which made us believe there's a problem with
SDRAM.  Sometimes there are simple hard lockups from which even the JTAG
can't regain control.  However, the SDRAM physical/electrical and DDR
Controller configurations have been investigated in detail (and some
corrections made) with no resulting improvement in the exception behavior.

We see problems primarily at 667 MHz but have noted some issues at 533
MHz as well.  Some boards seem particularly susceptible to this while
others rarely (if ever) exhibit the problem.

At this point all possibilities are on the table and I'm looking for any
input from anyone with experience (good, bad, or whatever) with this
processor and/or designs similar to the Sequoia.


Thanks very much,
Dave

[U-Boot-Users] AMCC PPC440EPx/sequoia stability question...

[Dave Littell]

Hi all,

I'm working on an AMCC PPC440EPx-based platform that's similar to the
Sequoia.  The board runs pretty well, but occasionally takes exceptions
both in U-Boot and while running Linux.  The exceptions vary from
Illegal Instructions to FP Unavailable to FP Unimplemented to ITLB and
DTLB Errors, to DSI, etc., which made us believe there's a problem with
SDRAM.  Sometimes there are simple hard lockups from which even the JTAG
can't regain control.  However, the SDRAM physical/electrical and DDR
Controller configurations have been investigated in detail (and some
corrections made) with no resulting improvement in the exception behavior.

We see problems primarily at 667 MHz but have noted some issues at 533
MHz as well.  Some boards seem particularly susceptible to this while
others rarely (if ever) exhibit the problem.

At this point all possibilities are on the table and I'm looking for any
input from anyone with experience (good, bad, or whatever) with this
processor and/or designs similar to the Sequoia.


Thanks very much,
Dave

Re: [U-Boot-Users] AMCC PPC440EPx/sequoia stability question...

[Stefan Roese]

On Wednesday 23 April 2008, Dave Littell wrote:
> I'm working on an AMCC PPC440EPx-based platform that's similar to the
> Sequoia.  The board runs pretty well, but occasionally takes exceptions
> both in U-Boot and while running Linux.  The exceptions vary from
> Illegal Instructions to FP Unavailable to FP Unimplemented to ITLB and
> DTLB Errors, to DSI, etc., which made us believe there's a problem with
> SDRAM.

Yes, this is my first idea too. Almost every time such "random" errors are 
seen, SDRAM setup/initialization is the cause for it.

> Sometimes there are simple hard lockups from which even the JTAG 
> can't regain control.  However, the SDRAM physical/electrical and DDR
> Controller configurations have been investigated in detail (and some
> corrections made) with no resulting improvement in the exception behavior.

Is the SDRAM termination similar to the one used on Sequoia too? Are you using 
soldered chips and not DIMM modules? Is ECC available?

> We see problems primarily at 667 MHz but have noted some issues at 533
> MHz as well.  Some boards seem particularly susceptible to this while
> others rarely (if ever) exhibit the problem.
>
> At this point all possibilities are on the table and I'm looking for any
> input from anyone with experience (good, bad, or whatever) with this
> processor and/or designs similar to the Sequoia.

If you scan the U-Boot mailing list for 440EPx/Denali DDR2 problems, you will 
most likely find some references.

Please note that I recently introduced a CFG_MEM_TOP_HIDE option for the 
440EPx CHIP 11 errata. I suggest you take a look at this too and see if this 
changes your behavior.

Best regards,
Stefan

=====================================================================
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-0 Fax: +49-8142-66989-80  Email: office@denx.de
=====================================================================

[U-Boot-Users] AMCC PPC440EPx/sequoia stability question...

[Stefan Roese]

On Wednesday 23 April 2008, Dave Littell wrote:
> I'm working on an AMCC PPC440EPx-based platform that's similar to the
> Sequoia.  The board runs pretty well, but occasionally takes exceptions
> both in U-Boot and while running Linux.  The exceptions vary from
> Illegal Instructions to FP Unavailable to FP Unimplemented to ITLB and
> DTLB Errors, to DSI, etc., which made us believe there's a problem with
> SDRAM.

Yes, this is my first idea too. Almost every time such "random" errors are 
seen, SDRAM setup/initialization is the cause for it.

> Sometimes there are simple hard lockups from which even the JTAG 
> can't regain control.  However, the SDRAM physical/electrical and DDR
> Controller configurations have been investigated in detail (and some
> corrections made) with no resulting improvement in the exception behavior.

Is the SDRAM termination similar to the one used on Sequoia too? Are you using 
soldered chips and not DIMM modules? Is ECC available?

> We see problems primarily at 667 MHz but have noted some issues at 533
> MHz as well.  Some boards seem particularly susceptible to this while
> others rarely (if ever) exhibit the problem.
>
> At this point all possibilities are on the table and I'm looking for any
> input from anyone with experience (good, bad, or whatever) with this
> processor and/or designs similar to the Sequoia.

If you scan the U-Boot mailing list for 440EPx/Denali DDR2 problems, you will 
most likely find some references.

Please note that I recently introduced a CFG_MEM_TOP_HIDE option for the 
440EPx CHIP 11 errata. I suggest you take a look at this too and see if this 
changes your behavior.

Best regards,
Stefan

=====================================================================
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-0 Fax: +49-8142-66989-80  Email: office at denx.de
=====================================================================

Re: [U-Boot-Users] AMCC PPC440EPx/sequoia stability question...

[Josh Boyer]

On Wed, 2008-04-23 at 14:49 +0200, Stefan Roese wrote:
> On Wednesday 23 April 2008, Dave Littell wrote:
> > At this point all possibilities are on the table and I'm looking for any
> > input from anyone with experience (good, bad, or whatever) with this
> > processor and/or designs similar to the Sequoia.
> 
> If you scan the U-Boot mailing list for 440EPx/Denali DDR2 problems, you will 
> most likely find some references.
> 
> Please note that I recently introduced a CFG_MEM_TOP_HIDE option for the 
> 440EPx CHIP 11 errata. I suggest you take a look at this too and see if this 
> changes your behavior.

Explain this a bit more please?  Is a kernel change needed here?

josh

[U-Boot-Users] AMCC PPC440EPx/sequoia stability question...

[Josh Boyer]

On Wed, 2008-04-23 at 14:49 +0200, Stefan Roese wrote:
> On Wednesday 23 April 2008, Dave Littell wrote:
> > At this point all possibilities are on the table and I'm looking for any
> > input from anyone with experience (good, bad, or whatever) with this
> > processor and/or designs similar to the Sequoia.
> 
> If you scan the U-Boot mailing list for 440EPx/Denali DDR2 problems, you will 
> most likely find some references.
> 
> Please note that I recently introduced a CFG_MEM_TOP_HIDE option for the 
> 440EPx CHIP 11 errata. I suggest you take a look at this too and see if this 
> changes your behavior.

Explain this a bit more please?  Is a kernel change needed here?

josh

Re: [U-Boot-Users] AMCC PPC440EPx/sequoia stability question...

[Stefan Roese]

On Thursday 24 April 2008, Josh Boyer wrote:
> > Please note that I recently introduced a CFG_MEM_TOP_HIDE option for the
> > 440EPx CHIP 11 errata. I suggest you take a look at this too and see if
> > this changes your behavior.
>
> Explain this a bit more please?  Is a kernel change needed here?

This depends. When the bootwrapper version is used then yes, the kernel should 
get changed. This is because the bootwrapper detects the SDRAM size from the 
DDR2 controller and passes it to Linux.

Without bootwrapper no changes are needed, since U-Boot already passes the 
corrected memory size to Linux (totalsize-4k currently).

Best regards,
Stefan

=====================================================================
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-0 Fax: +49-8142-66989-80  Email: office@denx.de
=====================================================================

[U-Boot-Users] AMCC PPC440EPx/sequoia stability question...

[Stefan Roese]

On Thursday 24 April 2008, Josh Boyer wrote:
> > Please note that I recently introduced a CFG_MEM_TOP_HIDE option for the
> > 440EPx CHIP 11 errata. I suggest you take a look at this too and see if
> > this changes your behavior.
>
> Explain this a bit more please?  Is a kernel change needed here?

This depends. When the bootwrapper version is used then yes, the kernel should 
get changed. This is because the bootwrapper detects the SDRAM size from the 
DDR2 controller and passes it to Linux.

Without bootwrapper no changes are needed, since U-Boot already passes the 
corrected memory size to Linux (totalsize-4k currently).

Best regards,
Stefan

=====================================================================
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-0 Fax: +49-8142-66989-80  Email: office at denx.de
=====================================================================

Re: [U-Boot-Users] AMCC PPC440EPx/sequoia stability question...

[Josh Boyer]

On Thu, 2008-04-24 at 06:36 +0200, Stefan Roese wrote:
> On Thursday 24 April 2008, Josh Boyer wrote:
> > > Please note that I recently introduced a CFG_MEM_TOP_HIDE option for the
> > > 440EPx CHIP 11 errata. I suggest you take a look at this too and see if
> > > this changes your behavior.
> >
> > Explain this a bit more please?  Is a kernel change needed here?
> 
> This depends. When the bootwrapper version is used then yes, the kernel should 
> get changed. This is because the bootwrapper detects the SDRAM size from the 
> DDR2 controller and passes it to Linux.
> 
> Without bootwrapper no changes are needed, since U-Boot already passes the 
> corrected memory size to Linux (totalsize-4k currently).

Hm.  Given that AMCC ships the boards with an older U-Boot that requires
cuImages, I think we'll need to patch the wrapper.  Does 440GRx share
this same errata?  I would think so.

josh

[U-Boot-Users] AMCC PPC440EPx/sequoia stability question...

[Josh Boyer]

On Thu, 2008-04-24 at 06:36 +0200, Stefan Roese wrote:
> On Thursday 24 April 2008, Josh Boyer wrote:
> > > Please note that I recently introduced a CFG_MEM_TOP_HIDE option for the
> > > 440EPx CHIP 11 errata. I suggest you take a look at this too and see if
> > > this changes your behavior.
> >
> > Explain this a bit more please?  Is a kernel change needed here?
> 
> This depends. When the bootwrapper version is used then yes, the kernel should 
> get changed. This is because the bootwrapper detects the SDRAM size from the 
> DDR2 controller and passes it to Linux.
> 
> Without bootwrapper no changes are needed, since U-Boot already passes the 
> corrected memory size to Linux (totalsize-4k currently).

Hm.  Given that AMCC ships the boards with an older U-Boot that requires
cuImages, I think we'll need to patch the wrapper.  Does 440GRx share
this same errata?  I would think so.

josh

Re: [U-Boot-Users] AMCC PPC440EPx/sequoia stability question...

[Stefan Roese]

On Thursday 24 April 2008, Josh Boyer wrote:
> > This depends. When the bootwrapper version is used then yes, the kernel
> > should get changed. This is because the bootwrapper detects the SDRAM
> > size from the DDR2 controller and passes it to Linux.
> >
> > Without bootwrapper no changes are needed, since U-Boot already passes
> > the corrected memory size to Linux (totalsize-4k currently).
>
> Hm.  Given that AMCC ships the boards with an older U-Boot that requires
> cuImages, I think we'll need to patch the wrapper.

Yes, this should be done too. I forgot about it since I usually don't use the 
wrapper.

> Does 440GRx share 
> this same errata?  I would think so.

Yep. Same problem on 440GRx.

Best regards,
Stefan

=====================================================================
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-0 Fax: +49-8142-66989-80  Email: office@denx.de
=====================================================================

[U-Boot-Users] AMCC PPC440EPx/sequoia stability question...

[Stefan Roese]

On Thursday 24 April 2008, Josh Boyer wrote:
> > This depends. When the bootwrapper version is used then yes, the kernel
> > should get changed. This is because the bootwrapper detects the SDRAM
> > size from the DDR2 controller and passes it to Linux.
> >
> > Without bootwrapper no changes are needed, since U-Boot already passes
> > the corrected memory size to Linux (totalsize-4k currently).
>
> Hm.  Given that AMCC ships the boards with an older U-Boot that requires
> cuImages, I think we'll need to patch the wrapper.

Yes, this should be done too. I forgot about it since I usually don't use the 
wrapper.

> Does 440GRx share 
> this same errata?  I would think so.

Yep. Same problem on 440GRx.

Best regards,
Stefan

=====================================================================
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-0 Fax: +49-8142-66989-80  Email: office at denx.de
=====================================================================

[U-Boot-Users] AMCC PPC440EPx/sequoia stability question...

[Dave Littell]

Stefan Roese wrote:
> On Wednesday 23 April 2008, Dave Littell wrote:
>> I'm working on an AMCC PPC440EPx-based platform that's similar to the
>> Sequoia.  The board runs pretty well, but occasionally takes exceptions
>> both in U-Boot and while running Linux.  The exceptions vary from
>> Illegal Instructions to FP Unavailable to FP Unimplemented to ITLB and
>> DTLB Errors, to DSI, etc., which made us believe there's a problem with
>> SDRAM.
> 
> Yes, this is my first idea too. Almost every time such "random" errors are 
> seen, SDRAM setup/initialization is the cause for it.
> 
>> Sometimes there are simple hard lockups from which even the JTAG 
>> can't regain control.  However, the SDRAM physical/electrical and DDR
>> Controller configurations have been investigated in detail (and some
>> corrections made) with no resulting improvement in the exception behavior.
> 
> Is the SDRAM termination similar to the one used on Sequoia too? Are you using 
> soldered chips and not DIMM modules? Is ECC available?
> 

Termination: very likely - they stayed very close to Sequoia, but I'll ask.

We're using soldered chips and there's no ECC.

>> We see problems primarily at 667 MHz but have noted some issues at 533
>> MHz as well.  Some boards seem particularly susceptible to this while
>> others rarely (if ever) exhibit the problem.
>>
>> At this point all possibilities are on the table and I'm looking for any
>> input from anyone with experience (good, bad, or whatever) with this
>> processor and/or designs similar to the Sequoia.
> 
> If you scan the U-Boot mailing list for 440EPx/Denali DDR2 problems, you will 
> most likely find some references.
> 

Will do, thanks for the pointer.

> Please note that I recently introduced a CFG_MEM_TOP_HIDE option for the 
> 440EPx CHIP 11 errata. I suggest you take a look at this too and see if this 
> changes your behavior.
> 

I remember this - PPC440EPx Revision A errata CHIP_11, right?


Thanks,
Dave