Still trying to get a stable 2.6.20.4 running on 8349ITX eval board

Still trying to get a stable 2.6.20.4 running on 8349ITX eval board

[Bruce_Leonard]

Hi,

Posted a cry for help a few days ago on this issue ;)> .  Got a couple of 
answers that didn't get me any farther so I'm on another fishing 
expedition.  Basic set up is as follows; MPC8349ITX eval board; 2.6.20.4 
kernel; a ubuntu/dapper FS via NFS.  The problem seems to be a random 
lock-up in either a DSI exception (0x300), a program exception (0x700) or 
a DTLB miss on store exception (0x1200).  I'm sure it's not truly random 
(I'm sure there's probably only a single underlying problem), but it's 
random in how it manifests.  When trying to mount the root FS via NFS, it 
always successfully reaches the point where it runs "/sbin/init" in 
main.c, but which script it fails in varies and it never gets to a login.

I've tried passing "init=/bin/sh" on the command line and I can reach a sh 
prompt, but it doesn't take long before it locks up.  I can do a "find . 
-name '*gcc*'" about four times before it stops.  I have a BDI2K hooked up 
so I know where it's stopping (most commonly in the DTLB miss mentioned 
above), but backtrace from GDB gets me nothing, and SRR0 and SRR1 don't 
contain any useful information (frequently they're both zero).  I expect 
it may be an interaction with the BDI, tho I'm not sure.

I've also tried a couple of other file systems (one right off the CD that 
ships with the eval board and the latest from ELDK) as well as pulling 
Kumar's powerpc kernel tree.  All possible combinations of these kernels 
and file systems fail to run for any period of time.

Long winded way to get around to my questions (sorry :(  ):
1) Can any one point me to good references in using the BDI2K to find my 
way back from exceptions?
2) I assume someone (Kumar?) has gotten some combination of kernel/FS to 
boot and run on this box.  What combination is that?

Thanks.

Bruce

------------------------------------------------

This e-mail may contain SEL confidential information.  The opinions
expressed are not necessarily those of SEL.  Any unauthorized disclosure,
distribution or other use is prohibited.  If you received this e-mail in
error, please notify the sender, permanently delete it, and destroy any
printout.

Thank you.

------------------------------------------------

RE: Still trying to get a stable 2.6.20.4 running on 8349ITX eval board

[Benedict, Michael]

>I've tried passing "init=3D/bin/sh" on the command line and I=20
>can reach a sh=20
>prompt, but it doesn't take long before it locks up.  I can do=20
>a "find .=20
>-name '*gcc*'" about four times before it stops.=20

I, too, am still experiencing this issue.   If it helps debugging, it is
only VFS accesses that seem to be locked.  I was able to run a
network-based IO app fine for almost a day (I rebooted the machine
then), even though the root filesystem "stops".

>2) I assume someone (Kumar?) has gotten some combination of=20
>kernel/FS to=20
>boot and run on this box.  What combination is that?

I am thinking that these issues are closely coupled to root NFS.  I know
this isn't good news, but I think the combo is running the root
filesystem on the hard drive or ramdisk.
	Good luck,
		Michael

RE: Still trying to get a stable 2.6.20.4 running on 8349ITX eval board

[Bruce_Leonard]

Michael,

linuxppc-embedded-bounces+brucle=selinc.com@ozlabs.org wrote on 04/11/2007 
01:59:11 PM:

> 
> >2) I assume someone (Kumar?) has gotten some combination of 
> >kernel/FS to 
> >boot and run on this box.  What combination is that?
> 
> I am thinking that these issues are closely coupled to root NFS.  I know
> this isn't good news, but I think the combo is running the root
> filesystem on the hard drive or ramdisk.
>    Good luck,
>       Michael
> 
For the heck of it, we tried formatting the hard drive, put ubuntu/dapper 
on a ext3 partition, and used that as the root FS.  Same results.  We 
booted with init=/bin/sh but when we run debootstrap --second-stage it got 
just a few seconds into it and locked up in a program exception.

Thanks for the input though.  I'll keep poking and let you know what I 
figure out.  I keep hoping some guru out there will go "ahhhh......" 
'cause I ain't no guru!  :)

Bruce


------------------------------------------------

This e-mail may contain SEL confidential information.  The opinions
expressed are not necessarily those of SEL.  Any unauthorized disclosure,
distribution or other use is prohibited.  If you received this e-mail in
error, please notify the sender, permanently delete it, and destroy any
printout.

Thank you.

------------------------------------------------

Re: Still trying to get a stable 2.6.20.4 running on 8349ITX eval board

[Ed Swierk]

On Wednesday 11 April 2007 13:47:55 Bruce_Leonard@selinc.com wrote:
> Posted a cry for help a few days ago on this issue ;)> .  Got a couple of
> answers that didn't get me any farther so I'm on another fishing
> expedition.  Basic set up is as follows; MPC8349ITX eval board; 2.6.20.4
> kernel; a ubuntu/dapper FS via NFS.  The problem seems to be a random
> lock-up in either a DSI exception (0x300), a program exception (0x700) or
> a DTLB miss on store exception (0x1200).  I'm sure it's not truly random
> (I'm sure there's probably only a single underlying problem), but it's
> random in how it manifests.  When trying to mount the root FS via NFS, it
> always successfully reaches the point where it runs "/sbin/init" in
> main.c, but which script it fails in varies and it never gets to a login.

I spent about 8 hours yesterday tracking down a problem on a similar board 
(Embedded Planet 8343M) booting Fedora 6 via NFS. Linux was getting a bit 
farther along than it is for you--I got a shell and could run python--but the 
CPU kept locking up somewhere near 0x600 or 0x1200, without a peep from the 
kernel. It usually occurred after a lot of memory allocation activity, so I 
tried tracing the out-of-memory code in the kernel, but then found I could 
reproduce the problem with the mtest command in U-Boot.

The problem turned out to be an incorrect DDR SDRAM configuration setting in 
U-Boot for my board (which has no SPD EEPROM, so it's U-Boot's job to 
configure the memory controller). I was using CSCONFIG_ROW_BIT_12 rather than 
CSCONFIG_ROW_BIT_13 in CFG_DDR_CONFIG, which apparently affected the way the 
CPU accesses and refreshes the DRAM. I was pretty shocked that Linux was able 
to boot at all with a configuration bug like this.

Anyway, it might be worth double-checking the DDR SDRAM code in U-Boot to make 
sure it's doing the right thing for the parts on your board.

--Ed

Re: Still trying to get a stable 2.6.20.4 running on 8349ITX eval board

[Vitaly Bordug]

On Wed, 11 Apr 2007 13:47:55 -0700
Bruce_Leonard@selinc.com wrote:

> Hi,
> 
> Posted a cry for help a few days ago on this issue ;)> .  Got a couple of 
> answers that didn't get me any farther so I'm on another fishing 
> expedition.  Basic set up is as follows; MPC8349ITX eval board; 2.6.20.4 
> kernel; a ubuntu/dapper FS via NFS.  The problem seems to be a random 
> lock-up in either a DSI exception (0x300), a program exception (0x700) or 
> a DTLB miss on store exception (0x1200).  I'm sure it's not truly random 
> (I'm sure there's probably only a single underlying problem), but it's 
> random in how it manifests.  When trying to mount the root FS via NFS, it 
> always successfully reaches the point where it runs "/sbin/init" in 
> main.c, but which script it fails in varies and it never gets to a login.
> 
> I've tried passing "init=/bin/sh" on the command line and I can reach a sh 
> prompt, but it doesn't take long before it locks up.  I can do a "find . 
> -name '*gcc*'" about four times before it stops.  I have a BDI2K hooked up 
> so I know where it's stopping (most commonly in the DTLB miss mentioned 
> above), but backtrace from GDB gets me nothing, and SRR0 and SRR1 don't 
> contain any useful information (frequently they're both zero).  I expect 
> it may be an interaction with the BDI, tho I'm not sure.
> 
> I've also tried a couple of other file systems (one right off the CD that 
> ships with the eval board and the latest from ELDK) as well as pulling 
> Kumar's powerpc kernel tree.  All possible combinations of these kernels 
> and file systems fail to run for any period of time.
> 
> Long winded way to get around to my questions (sorry :(  ):
> 1) Can any one point me to good references in using the BDI2K to find my 
> way back from exceptions?
> 2) I assume someone (Kumar?) has gotten some combination of kernel/FS to 
> boot and run on this box.  What combination is that?
> 
> 

It worked for me with both MontaVista and ELDK rfs for 6xx.

Your problem looks very much like odd DDR or memctl misconfiguration early in u-boot.
Please try upgrading your firmware with the latest u-boot from git tree at denx.de. Kernel from Paul Mackerras (powerpc.git) is also a good thing to give a try.

Also, there is Vitesse switch on this target, which should be initialized(Normally with vendor-shipped image you'll figure out how by print thirdp_init. Starting kernel without that may result in some instability too.

Hope it helps.

-- 
Sincerely, 
Vitaly

RE: Still trying to get a stable 2.6.20.4 running on 8349ITX evalboard

[Benedict, Michael]

>
>Your problem looks very much like odd DDR or memctl=20
>misconfiguration early in u-boot.
>Please try upgrading your firmware with the latest u-boot from=20
>git tree at denx.de. Kernel from Paul Mackerras (powerpc.git)=20
>is also a good thing to give a try.
>
I can verify this bug exists with the latest Denx git u-boot.  Thank
you, I will try Paul Mackerras kernel.

>Also, there is Vitesse switch on this target, which should be=20
>initialized(Normally with vendor-shipped image you'll figure=20
>out how by print thirdp_init. Starting kernel without that may=20
>result in some instability too.
>

I have to admit, this is over my head.  What do you mean by "print
thirdp_init"? =20

Thank you Vitaly!
	-Michael

Re: Still trying to get a stable 2.6.20.4 running on 8349ITX eval board

[Bruce_Leonard]

Vitaly,

Vitaly Bordug <vbordug@ru.mvista.com> wrote on 04/12/2007 08:36:58 AM:

> 
> It worked for me with both MontaVista and ELDK rfs for 6xx.
> 
> Your problem looks very much like odd DDR or memctl misconfiguration
> early in u-boot.
>

Ed made a similar observation.  I've been assuming ;) that the DDR RAM was 
configured correctly since it's been a 'supported' eval board since at 
least u-boot 1.1.6, but there may be a problem there.  At least I'm going 
to make sure that it's all okay before I continue.
 
> Also, there is Vitesse switch on this target, which should be 
> initialized(Normally with vendor-shipped image you'll figure out how
> by print thirdp_init. Starting kernel without that may result in 
> some instability too.

Thanks.  Again I was assuming all the HW was correctly configured with 
u-boot.  I'll dig into this as well.  Soon as I find anything for sure 
I'll post a follow up.

Bruce

------------------------------------------------

This e-mail may contain SEL confidential information.  The opinions
expressed are not necessarily those of SEL.  Any unauthorized disclosure,
distribution or other use is prohibited.  If you received this e-mail in
error, please notify the sender, permanently delete it, and destroy any
printout.

Thank you.

------------------------------------------------

RE: Still trying to get a stable 2.6.20.4 running on 8349ITX evalboard

[Benedict, Michael]

>Your problem looks very much like odd DDR or memctl=20
>misconfiguration early in u-boot.
>Please try upgrading your firmware with the latest u-boot from=20
>git tree at denx.de. Kernel from Paul Mackerras (powerpc.git)=20
>is also a good thing to give a try.

This is definetly something related to u-boot.  I was using a new Denx
git u-boot for flat device tree support.  If I boot a kernel with
hardcoded ethernet-phy registers on the shipping u-boot the system is
much more stable.  Now I am trying to track down the differences between
the u-boot the freescale shipped on the device verse the newest git
sources.  Hopefully some initialization is missing that will be very
obvious.
	Cheers,
		Michael

RE: Still trying to get a stable 2.6.20.4 running on 8349ITX evalboard

[Bruce_Leonard]

Hi all,

"Benedict, Michael" <MBenedict@twacs.com> wrote on 04/12/2007 12:27:35 PM:

> >Your problem looks very much like odd DDR or memctl 
> >misconfiguration early in u-boot.
> >Please try upgrading your firmware with the latest u-boot from 
> >git tree at denx.de. Kernel from Paul Mackerras (powerpc.git) 
> >is also a good thing to give a try.
> 
> This is definetly something related to u-boot.  I was using a new Denx
> git u-boot for flat device tree support.  If I boot a kernel with
> hardcoded ethernet-phy registers on the shipping u-boot the system is
> much more stable.  Now I am trying to track down the differences between
> the u-boot the freescale shipped on the device verse the newest git
> sources.  Hopefully some initialization is missing that will be very
> obvious.
>    Cheers,
>       Michael
> 

SUCCESS!!!  (Or at least a HUGE step forward.)  Once I got to thinking 
about memory timing I remembered that the 8349itx u-boot port allows the 
choice of using SPD for detecting the DDR RAM or using hard coded defaults 
in the memory controller.  I figured what the heck, I'll give the hard 
coded values a try even though they were probably wrong for the particular 
part I've got installed.  And the bloody thing ran!  I had it executing a 
forever loop for 35 minutes, which is about 34 and a half minutes longer 
than it's ever run before!

It still augured in on me, but the processor was so locked up/confused 
that I couldn't even get the BDI to reset it, I had to cycle power.  I'm 
inclined to think it was probably an ESD event, since I've got the cover 
off the box.

Thanks to everyone who offered suggestions, especially Ed and Vitaly for 
suggesting looking at the RAM.  I was in no way, shape, or form thinking 
along those lines.  Now I've got something I can actually dig on.  I'll 
post a followup here and on the u-boot mailing list if I find anything 
wrong in "public" code.

Thanks again.

Bruce


------------------------------------------------

This e-mail may contain SEL confidential information.  The opinions
expressed are not necessarily those of SEL.  Any unauthorized disclosure,
distribution or other use is prohibited.  If you received this e-mail in
error, please notify the sender, permanently delete it, and destroy any
printout.

Thank you.

------------------------------------------------

DMA Puzzle

[Charles Krinke]

My apologies for the complexity of this e-mail but DMA seems to be a
complex subject. Thanks to Ed.S a while back, I have DMA working fine
from U-Boot, but I am having a problem in Linux with the 8541 processor
using the 2.6.17.11 kernel. I need to DMA through an outbound address
translation window setup at 0x8800_0000 and the chip I am DMA'ing to is
responding to 64bit DAC (Dual Address Cycles) and its memory is setup
beginning at 0x1_0000_0000.

It seems, that the MR0/SR0 indicate the DMA has stalled and I have not
figured out why yet. Here is the progression in the DMA driver in Linux

First obtain a pointer to my RAM buffer using pci_map_single. This
returns the buffer in kernel space at 0xC768_0000 (or thereabouts) to
0x07680000 and prepare to transfer 0x10000 (64K bytes) from 0x0_c7680000
to 0x1_0000_0000.

This is the first of several printk's in the DMA driver where the POTAR2
is being used to hold the hi part of the 64bit address.

a src_low_addr:07680000 winBase:00000000 POTAR2:00100000
newLowAddr:07680000 dir:1 size:00010000  =20

Next we write all f's to SR0 to clear it and here is the next printk.

z MR0:00000000 SR0:00000000 begin

Now we write the low part of the source address to LSAR0.=20
A constant value of 0x50000 to DATR0.
The size in bytes to the BCR (Byte Count Register) is next

Then set the MR0 to 0x0f03c404 indicating a)disable bandwidth sharing,
DAHTS & SAHTS both 8 bytes, SRW set & CDSM/SWSM cleared to indicate
start the DMA by writing the destination.

Lastly, we write the destination to the DAR0 register.

At this point, the next printk has a problem. The MR0 register has its
CS bit set and the SR0 register has its CB bit set. This indicates to me
that the channel is busy, it stalled and cannot continue.

 c MR0:0F03C405 SR0:00000004 SATR:00050000 DATR:00050000 dst_low_addr:0

Just for reference, here are two final printk's with some of the other
pertinent registers.

 d POTAR2:00100000 LSAR0:07690000 DAR0:88010000 BCR:0 DGSR:00000000
 e POTEAR2:00000000 POWBAR2:00088000 POWAR 0:80044019 1:8004401A
2:8004401B

So, the question becomes, "What is going awry here and where should I be
looking in order to understand how to close the DMA loop back to
Linux?".

Charles Krinke

Re: DMA Puzzle

[Kelsey Dawes]

On Thu, Apr 12, 2007 at 04:23:00PM -0700, Charles Krinke wrote:
> My apologies for the complexity of this e-mail but DMA seems to be a
> complex subject. Thanks to Ed.S a while back, I have DMA working fine
> from U-Boot, but I am having a problem in Linux with the 8541 processor
> using the 2.6.17.11 kernel. I need to DMA through an outbound address
> translation window setup at 0x8800_0000 and the chip I am DMA'ing to is
> responding to 64bit DAC (Dual Address Cycles) and its memory is setup

<SNIP>

Hi Charles - have you checked the device errata for 8541?  I do not have the details in front of me, but do recall several items specific to DMA  signal to complete DMA transaction.  


Worth a look anyway - 

Kelsey

Anyone using the MPC8313E-RDB?

[David Hawkins]

Hi all,

Is anyone using the MPC8313E-RDB?

I see Freescale is having a series of PowerQUICC seminars
for $99, and you get one of the MPC8313E-RDB boards
(which according to the fact sheet goes for $299).

http://www.freescale.com/quiccseminars

I'm using the 8349E ... but I love collecting and playing
with evaluation boards ;)

Cheers
Dave