linuxppc embedded boot problems.

linuxppc embedded boot problems.

[Brendan Simon]

Just when I thought I had things sussed out, something changes and I'm
back to square one.  Sigh.......

I have HAD linuxppc (embedded-2.2.5) working on a custom MPC860 board.
By working I mean booting to the bash prompt and being able to view the
initrd filesystem with ls.  I have compiled some simple apps that print
things to the console but they only work if I compile with -static
option.

I now want to get the network going and really want to get root nfs
stuff going so development is easy.  I decided to put a whole lot of
printk statements in the enet.c code to see what was happening.  I
loaded this code and all the printk statements are output to the console
when I try "root=/dev/nfs ......" and the boot prompt and I even saw
that statement that the root file system was mounted.  Cool, but the
console hung and there is no bash shell.  I put some more printks in
init/main.c to see how far the code was getting.
NOW HERE IS WHERE THE WIERDNESS STARTS.  The kernel no longer boots.
WHAT !!!  The boot code seems to uncompress the kernel image into RAM
but then nothing else happens.  Here is the output:

loaded at:     FF801000 FF812BAC
relocated to:  00100000 00111BAC
board data at: 001001C4 001001E0
relocated to:  00200100 0020011C
zimage at:     FF807000 FF868100
avail ram:     00201000 01000000

Linux/PPC load:
Uncompressing Linux...done.
Now booting the kernel

That is all I get.  What the hell could be causing this.  OK, time to
take all the printk statements out so I can get back to where I left
off.  I can't believe it; the same thing happens.  I have appended the
output sections of my image.  It looks OK and I have had this thing
booting before so there must be something I'm missing.  I'm a bit
concerned about the possible overlap of the .bss and image sections.
Could some bss data be overwriting the image data at runtime ???

Sections:
Idx Name          Size      VMA       LMA       File off  Algn
  0 .text         00004870  ff801000  ff801000  00001000  2**2
                  CONTENTS, ALLOC, LOAD, READONLY, CODE
  1 .rodata       00000470  ff805870  ff805870  00005870  2**4
                  CONTENTS, ALLOC, LOAD, READONLY, DATA
  2 .data         00000300  ff806000  ff806000  00006000  2**2
                  CONTENTS, ALLOC, LOAD, DATA
  3 .bss          0000bbac  ff807000  ff807000  00007000  2**2
                  ALLOC
  4 image         00061104  ff807000  ff807000  00007000  2**0
                  CONTENTS, ALLOC, LOAD, DATA


Thanks,
Brendan Simon.


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: linuxppc embedded boot problems.

[Dan Malek]

Brendan Simon wrote:


> I have HAD linuxppc (embedded-2.2.5) working on a custom MPC860 board.
> By working I mean booting to the bash prompt and being able to view the
> initrd filesystem with ls.


Debug-101.....Back out the changes until you get to the
original working version.  That usually helps uncover information
necessary to solve the problem.


> ....  I have compiled some simple apps that print
> things to the console but they only work if I compile with -static
> option.


Perhaps because you are compiling/linking on a system with
different libraries that are running on the target.


> .......  I decided to put a whole lot of
> printk statements in the enet.c code to see what was happening.


Bad idea.  Maybe one or two that you keep moving around as you
discover information.


> ........  Cool, but the
> console hung and there is no bash shell.  I put some more printks in
> init/main.c to see how far the code was getting.

Again, just one or two around some key functions calls will tell
you lots of information.


> loaded at:     FF801000 FF812BAC
> relocated to:  00100000 00111BAC
> board data at: 001001C4 001001E0
> relocated to:  00200100 0020011C
> zimage at:     FF807000 FF868100
> avail ram:     00201000 01000000
> 

> Sections:
> Idx Name          Size      VMA       LMA       File off  Algn
>   0 .text         00004870  ff801000  ff801000  00001000  2**2
>                   CONTENTS, ALLOC, LOAD, READONLY, CODE
>   1 .rodata       00000470  ff805870  ff805870  00005870  2**4
>                   CONTENTS, ALLOC, LOAD, READONLY, DATA
>   2 .data         00000300  ff806000  ff806000  00006000  2**2
>                   CONTENTS, ALLOC, LOAD, DATA
>   3 .bss          0000bbac  ff807000  ff807000  00007000  2**2
>                   ALLOC
>   4 image         00061104  ff807000  ff807000  00007000  2**0
>                   CONTENTS, ALLOC, LOAD, DATA


Well, the output above doesn't match this set of headers.  The
VMA/LMA are not supposed to be the Flash ROM offsets, they are
supposed to be the 0x00100000 addresses where the code
really belongs in RAM.  The first stage of the Linux boot will
discover it is running at some address other than RAM, and
relocate itself to the linked address.

Do not ever change the load/offset addresses in Makefiles.
Use or write a program that will take the exact bits and place
them into a flash rom.  It couldn't be easier.  The zImage is
a perfect bag of bits that can be loaded (almost) anywhere
into memory or a flash rom without any modification.


Good Luck.


	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: linuxppc embedded boot problems.

[Brendan Simon]

Dan Malek wrote:

> Brendan Simon wrote:
>
> > I have HAD linuxppc (embedded-2.2.5) working on a custom MPC860 board.
> > By working I mean booting to the bash prompt and being able to view the
> > initrd filesystem with ls.
>
> Debug-101.....Back out the changes until you get to the
> original working version.  That usually helps uncover information
> necessary to solve the problem.

Yep.  Did that but still the same result.  I think I am back to the same
spot but nothing is in revision control at this stage so I can't garauntee
it.


> > .......  I decided to put a whole lot of
> > printk statements in the enet.c code to see what was happening.
>
> Bad idea.  Maybe one or two that you keep moving around as you
> discover information.

Why is it a bad idea.  I just put a few at the entry of each function to see
what was happening.  I prefix every thing with my initials so that I can
delete them later.  Is there a limit to printk statements ?


> Well, the output above doesn't match this set of headers.  The
> VMA/LMA are not supposed to be the Flash ROM offsets, they are
> supposed to be the 0x00100000 addresses where the code
> really belongs in RAM.  The first stage of the Linux boot will
> discover it is running at some address other than RAM, and
> relocate itself to the linked address.

Sorry for the misleading information.  The screen output was from last night
and I had made some more modifications and compiled the kernel since.


> Do not ever change the load/offset addresses in Makefiles.
> Use or write a program that will take the exact bits and place
> them into a flash rom.  It couldn't be easier.  The zImage is
> a perfect bag of bits that can be loaded (almost) anywhere
> into memory or a flash rom without any modification.

I haven't touched the makefile.  I use a script to massage the zImage and
zImage.initrd outputs.  It basically is a few powerpc-linux-objcopy commands
to move sections around to where the image expects them to be and also to
convert to a hex file so I can program the linear flash SIMMs.

I think the problems I am seeing are more fundamental to the 860 setup.  I
have a bootloader which essentially initialises the chip selects for ROM and
DRAM and sets up the UPMA for the DRAM.  There is also some port
initialisation for SMC1(console) and SCC4(ethernet).  Apart from that the
bootloader just jumps the start of the application code (the kernel image in
this case).

What I don't understand is why it starts to boot (ie. uncompresses into
memory) but then nothing happens.  Are there things that the kernel expects
to be setup besides the flash/dram memory ?  eg. Interrupt vectors, timers,
etc.  I'll have a look through the fadsrom code to see if this will shed any
light on my problems.

Brendan Simon.


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: linuxppc embedded boot problems.

[Brendan Simon]

Dan Malek wrote:

> Considering you are doing your own UPM, the Linux kernel will
> find lots of timing errors with it.  It enables the MMU and
> caches very early, so the bus timing will see something new
> for the first time.  I hope you are peforming the proper cache
> initialization sequence and setting all of the necessary
> processor registers in the boot rom.
>
> You can disable the caches in arch/ppc/kernel/head.S.  Try that.

I commented out (using #if0 and #endif) the bits of code that enable the
caches in arch/ppc/kernel/head.S and arch/ppc/mbxboot/head.S.  The kernel
now boots.  This indicates one or more of the following:
1) there is something I'm doing wrong which affects the cache setup.
2) Some of the code has moved in memory which makes it work.
3) I just got lucky this time.
I really hope it is number 1).

I now get the attached output.  Notice the "VFS: Mounted root (NFS
filesystem) readonly." message.  So the kernel things that the root
filesystem is mounted via NFS and it is ok with it.  But the shell never
runs.  The root filesystem is EXACTLY the same one I put into initrd and
that one seems to work.  So maybe the rootnfs stuff is not working properly
for me.  Is it the ethernet driver I ported to SCC4 ?  If it is, then how
can the root NFS filesystem be mounted.  Any clues ?

Thanks again,
Brendan Simon.


[lots of other stuff removed]
...
VFS: Mounted root (NFS filesystem) readonly.
BJS: INITRD: 1
BJS: freeing memory
Freeing unused kernel memory: 32k init
BJS: memory freed
BJS: kernel unlocked


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: linuxppc embedded boot problems.

[Dan Malek]

Brendan Simon wrote:

> What I don't understand is why it starts to boot (ie. uncompresses into
> memory) but then nothing happens.


Could be lots of things.  I hope you can reset your board
without losing memory.  If so, find the address of 'log_buf'
in the System.map and dump it out (convert the virtual address
to physical by masking the upper few bits).  This is the staging
area for kernel printk messages, and lots of information is
here before the console is enabled.

> ....  Are there things that the kernel expects
> to be setup besides the flash/dram memory ?


Considering you are doing your own UPM, the Linux kernel will
find lots of timing errors with it.  It enables the MMU and
caches very early, so the bus timing will see something new
for the first time.  I hope you are peforming the proper cache
initialization sequence and setting all of the necessary
processor registers in the boot rom.

You can disable the caches in arch/ppc/kernel/head.S.  Try that.


	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: linuxppc embedded boot problems.

[Dan Malek]

Brendan Simon wrote:


> I now get the attached output.  Notice the "VFS: Mounted root (NFS
> filesystem) readonly." message.



At some point, long ago, the generic kernel root mounting had
a subtle change for some reason.  The result is that all file
systems are now mounted read-only by default.  All you need
to do is add "rw" to the command line.  It should look like
this: root=/dev/nfs rw......Now, the more interesting part is
I thought I did that as part of the default command line in
arch/ppc/mbxboot/misc.c......where did it go?



	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

linuxppc-embedded: NFS boot options.

[Brendan Simon]

Dan Malek wrote:

> Brendan Simon wrote:
>
> > I now get the attached output.  Notice the "VFS: Mounted root (NFS
> > filesystem) readonly." message.
>
> At some point, long ago, the generic kernel root mounting had
> a subtle change for some reason.  The result is that all file
> systems are now mounted read-only by default.  All you need
> to do is add "rw" to the command line.  It should look like
> this: root=/dev/nfs rw......Now, the more interesting part is
> I thought I did that as part of the default command line in
> arch/ppc/mbxboot/misc.c......where did it go?

I am working with the embedded-2.2.5.tgz archive.  The default option
for NFS is "root=/dev/nfs".  There is no rw option.  Anyway, I use the
boot prompt to specify my inet addresses rather than setup a bootp
server.  I use the following options
"root=/dev/nfs nfsaddrs=203.21.127.150:203.21.127.160
nfsroot=203.21.127.160:/sys/myroot rw debug".  The rw option works as
the VFS message no longer says readonly.  I can't see any difference
with debug option specified or not.

I still get the same results.  ie. /bin/sh gets invoked but I get no
prompt and I can't type anything.  I will try the nfsd in the foreground
as suggested by someone else.  tcpdump suggests that the nfs server is
doing arp requests for the 860 system but is getting no repsonse.  Maybe
the network driver has some problems (most likely with my port to SCC4)
?

Brendan Simon.


** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/