init does not run on 405GP system

init does not run on 405GP system

[Robert Schwebel]

Hi,

I'm currently porting u-boot and Linux to an IBM 405GP based board. The
problem is now that init seems not to be running and does not give any
output. Up to that point where init should make some noise the kernel
boots smoothly (serial console), I see all output and NFS-Root is
mounted via an Intel 82559 network chip. The kernel threads are also
running, I see kupdated & friends being put into the run queue from time
to time.

I have replaced /sbin/init by a statically linked "hello world" (which
also does not give any output). My impression is that the binary code of
the init ELF binary is never run. When I switch on the SHOW_SYSCALLs
macro in arch/ppc/kernel/entry.S I see the system calls for open(),
dup(), dup() and execve() which come from init/main.c. Opening the
console works, execve() to /sbin/init as well. When I follow the path of
execution up to load_elf_binary() in fs/binfmt_elf.c I can even see the
correct code being load and pointed to by elf_entry in that file. But
there is never any output from init, nor does something happen when I
replace init by a piece of code which should immediately make a zero
pointer exception.

Nevertheless, the kernel runs smoothly. I can ping the machine, I can
even floodping it with 0% packet loss. Only that there is no userspace
running.

Has anybody seen something like this before?

- Kernel is 2.4.21-rc2 with bitkeeper from 20030515 plus board port
- userland was tested with Debian bootdisks, Denx 4xx boot image and
  others
- toolchain is the Debian powerpc-linux cross toolchain.

Robert
--
 Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
 Pengutronix - Linux Solutions for Science and Industry
   Braunschweiger Str. 79,  31134 Hildesheim, Germany
   Handelsregister:  Amtsgericht Hildesheim, HRA 2686
    Phone: +49-5121-28619-0 |  Fax: +49-5121-28619-4

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: init does not run on 405GP system

[Wolfgang Denk]

In message <20030610141047.GU9379@pengutronix.de> you wrote:
>
> I'm currently porting u-boot and Linux to an IBM 405GP based board. The
> problem is now that init seems not to be running and does not give any
> output. Up to that point where init should make some noise the kernel
...
> Nevertheless, the kernel runs smoothly. I can ping the machine, I can
> even floodping it with 0% packet loss. Only that there is no userspace
> running.

Do user space program like init really don;t run,  or  is  there  any
chance that you just might see no output?

You can do a couple of things:

* Check your adaptions of the kernel source and your kernel  configu-
  ration is you really have a working serial console port.

* If booting over NFS realy works, you can try to start a (statically
  linked small test program) that does something else but a printf to
  the console. For example, open  a  file  in  /tmp  for  output  and
  frpintf to that file.

* Or  start  your  test  program  with  a  known  system  call  (like
  gettimeofday() or maybe even getuid() etc.) and set a breakpoint at
  the system call handler code inthe kernel.


You wrote "opening the console works" - set a breakpoint, and  verify
that your console driver is really initialized and used.


Best regards,

Wolfgang Denk

--
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-4596-87  Fax: (+49)-8142-4596-88  Email: wd@denx.de
In general, if you think something isn't in Perl, try it out, because
it usually is :-) - Larry Wall in <1991Jul31.174523.9447@netlabs.com>

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: init does not run on 405GP system

[Robert Schwebel]

On Tue, Jun 10, 2003 at 04:39:06PM +0200, Wolfgang Denk wrote:
> Do user space program like init really don;t run,  or  is  there  any
> chance that you just might see no output?

I'm not sure...

> * Check your adaptions of the kernel source and your kernel  configu-
>   ration is you really have a working serial console port.

I suppose yes - printk() works just fine and I see the complete kernel
output on the serial line.

> * If booting over NFS realy works, you can try to start a (statically
>   linked small test program) that does something else but a printf to
>   the console. For example, open  a  file  in  /tmp  for  output  and
>   frpintf to that file.

Tried - does not work. Even when I end the init program with return 0; I
would expect the kernel panic which does not come. the fprintf() should
also trigger sys_write() being called which does not happen. This all
lets me assume that the code of the init process is not really run.

The strange thing is that there is also no error message. I would
normally expect that when some mapping is broken the kernel would just
blow up with all kinds of uggly noise. When I output which task is
currently being scheduled init runs more or less all the time, only
interrupted by the kernel daemons. I see no idle task.

> You wrote "opening the console works" - set a breakpoint, and  verify
> that your console driver is really initialized and used.

Well, as I can use printk() from kernel space, shouldn't the driver
work? But nevertheless, I've hooked a printk() into
drivers/char/serial.c:rs_open() and I get the output.

Robert
--
 Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
 Pengutronix - Linux Solutions for Science and Industry
   Braunschweiger Str. 79,  31134 Hildesheim, Germany
   Handelsregister:  Amtsgericht Hildesheim, HRA 2686
    Phone: +49-5121-28619-0 |  Fax: +49-5121-28619-4

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: init does not run on 405GP system

[Hollis Blanchard]

On Tuesday, Jun 10, 2003, at 09:10 US/Central, Robert Schwebel wrote:
>
> I'm currently porting u-boot and Linux to an IBM 405GP based board. The
> problem is now that init seems not to be running and does not give any
> output. Up to that point where init should make some noise the kernel
> boots smoothly (serial console), I see all output and NFS-Root is
> mounted via an Intel 82559 network chip. The kernel threads are also
> running, I see kupdated & friends being put into the run queue from
> time
> to time.

That's where I've seen output stop when I've forgotten to put a getty
on /dev/ttyS0 in /etc/inittab...

--
Hollis Blanchard
IBM Linux Technology Center


** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: init does not run on 405GP system

[Robert Schwebel]

On Tue, Jun 10, 2003 at 10:02:09AM -0500, Hollis Blanchard wrote:
> That's where I've seen output stop when I've forgotten to put a getty
> on /dev/ttyS0 in /etc/inittab...

But that cannot be the reason why a hello_world does not work and why I
do not see the sys_write() syscall being called at all. Btw, init on my
other rootfs is busybox init. I suppose I should always see someting
useful when init is starting.

Robert
--
 Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
 Pengutronix - Linux Solutions for Science and Industry
   Braunschweiger Str. 79,  31134 Hildesheim, Germany
   Handelsregister:  Amtsgericht Hildesheim, HRA 2686
    Phone: +49-5121-28619-0 |  Fax: +49-5121-28619-4

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: init does not run on 405GP system

[Wolfgang Denk]

In message <20030610151217.GV9379@pengutronix.de> you wrote:
>
> But that cannot be the reason why a hello_world does not work and why I

Show us your "hello_world" code. Does it actually open "/dev/console" ?

> do not see the sys_write() syscall being called at all. Btw, init on my
> other rootfs is busybox init. I suppose I should always see someting
> useful when init is starting.

Why do you assume that?


Best regards,

Wolfgang Denk

--
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-4596-87  Fax: (+49)-8142-4596-88  Email: wd@denx.de
"Though a program be but three lines long,
someday it will have to be maintained."
- The Tao of Programming

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: init does not run on 405GP system

[Robert Schwebel]

[-- Attachment #1: Type: text/plain, Size: 1026 bytes --]

On Tue, Jun 10, 2003 at 06:52:51PM +0200, Wolfgang Denk wrote:
> Show us your "hello_world" code. Does it actually open "/dev/console" ?

Attached - source + compiled binary (this one is dynamically linked).
Could you verify that this hello world works with your hardware when
copied to /sbin/init? I would assume it prints out the text and then the
kernel panics because of the return.

> > do not see the sys_write() syscall being called at all. Btw, init on my
> > other rootfs is busybox init. I suppose I should always see someting
> > useful when init is starting.
>
> Why do you assume that?

Normally init makes some output, something like

	message(MAYBE_CONSOLE | LOG, "init started:  %s", bb_msg_full_version);

from busybox-init.

Robert
--
 Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
 Pengutronix - Linux Solutions for Science and Industry
   Braunschweiger Str. 79,  31134 Hildesheim, Germany
   Handelsregister:  Amtsgericht Hildesheim, HRA 2686
    Phone: +49-5121-28619-0 |  Fax: +49-5121-28619-4

[-- Attachment #2: hello.c --]
[-- Type: text/plain, Size: 145 bytes --]

#include <stdio.h>

int main(void) {

	FILE *f;

	f = fopen("/dev/console","w");
	fprintf(f,"hello penguin world\n");
	fclose(f);

	return 0;

}

[-- Attachment #3: hello --]
[-- Type: application/octet-stream, Size: 9420 bytes --]

Re: init does not run on 405GP system

[Robert Schwebel]

On Tue, Jun 10, 2003 at 04:10:47PM +0200, Robert Schwebel wrote:
> Nevertheless, the kernel runs smoothly. I can ping the machine, I can
> even floodping it with 0% packet loss. Only that there is no userspace
> running.
>
> Has anybody seen something like this before?

It is solved. The On Chip Memory was mapped to an address below
0x8000'0000 - what we didn't know was that this is a _virtual_ address,
because the OCM is not simply located "behind" the MMU. After changing
the mapping it works smoothly now.

Robert
--
 Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
 Pengutronix - Linux Solutions for Science and Industry
   Braunschweiger Str. 79,  31134 Hildesheim, Germany
   Handelsregister:  Amtsgericht Hildesheim, HRA 2686
    Phone: +49-5121-28619-0 |  Fax: +49-5121-28619-4

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: init does not run on 405GP system

[Bill Fink]

On Tue, 10 Jun 2003, Robert Schwebel wrote:

> I'm currently porting u-boot and Linux to an IBM 405GP based board. The
> problem is now that init seems not to be running and does not give any
> output. Up to that point where init should make some noise the kernel
> boots smoothly (serial console), I see all output and NFS-Root is
> mounted via an Intel 82559 network chip. The kernel threads are also
> running, I see kupdated & friends being put into the run queue from time
> to time.
>
> I have replaced /sbin/init by a statically linked "hello world" (which
> also does not give any output). My impression is that the binary code of
> the init ELF binary is never run. When I switch on the SHOW_SYSCALLs
> macro in arch/ppc/kernel/entry.S I see the system calls for open(),
> dup(), dup() and execve() which come from init/main.c. Opening the
> console works, execve() to /sbin/init as well. When I follow the path of
> execution up to load_elf_binary() in fs/binfmt_elf.c I can even see the
> correct code being load and pointed to by elf_entry in that file. But
> there is never any output from init, nor does something happen when I
> replace init by a piece of code which should immediately make a zero
> pointer exception.

This is probably not the issue, but you do have CONFIG_BINFMT_ELF
enabled in the kernel, don't you (although I would assume you would
get some kind of kernel error message when you tried to exec /sbin/init
if CONFIG_BINFMT_ELF wasn't enabled).

						-Bill

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: init does not run on 405GP system

[Robert Schwebel]

On Tue, Jun 10, 2003 at 11:21:57PM -0400, Bill Fink wrote:
> This is probably not the issue, but you do have CONFIG_BINFMT_ELF
> enabled in the kernel, don't you (although I would assume you would
> get some kind of kernel error message when you tried to exec /sbin/init
> if CONFIG_BINFMT_ELF wasn't enabled).

The option is always defined y for PowerPC, you cannot deselect it. But
I've checked it anyway ;)

Robert
--
 Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
 Pengutronix - Linux Solutions for Science and Industry
   Braunschweiger Str. 79,  31134 Hildesheim, Germany
   Handelsregister:  Amtsgericht Hildesheim, HRA 2686
    Phone: +49-5121-28619-0 |  Fax: +49-5121-28619-4

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: init does not run on 405GP system

[Bill Fink]

On Wed, 11 Jun 2003, Robert Schwebel wrote:

> On Tue, Jun 10, 2003 at 11:21:57PM -0400, Bill Fink wrote:
> > This is probably not the issue, but you do have CONFIG_BINFMT_ELF
> > enabled in the kernel, don't you (although I would assume you would
> > get some kind of kernel error message when you tried to exec /sbin/init
> > if CONFIG_BINFMT_ELF wasn't enabled).
>
> The option is always defined y for PowerPC, you cannot deselect it. But
> I've checked it anyway ;)

One other simple thing to try if you haven't already is the kernel
boot parameter init=/bin/bash.  Also make sure any necessary shared
libraries are available at boot time.  On my system:

gwiz% ldd /sbin/init
        libc.so.6 => /lib/libc.so.6 (0x0fea0000)
        /lib/ld.so.1 => /lib/ld.so.1 (0x30000000)

Also on my system:

gwiz% strings - /sbin/init | grep '^/'
/lib/ld.so.1
/e=
/sbin/init
/dev/console
/dev/tty0
/dev/null
/etc/ioctl.save
/etc/initscript
/bin/sh
/etc/inittab
/etc/initrunlvl
/var/log/initrunlvl
/dev/initctl
/etc/powerstatus
/var/run/utmp
/var/log/wtmp
/var/run/utmp
/sbin/sulogin

Your system may be slightly different.  Perhaps the absence of one of
these may cause init to die (although not all of them exist even on my
system).

						-Bill

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

Re: init does not run on 405GP system

[Robert Schwebel]

On Wed, Jun 11, 2003 at 02:22:15AM -0400, Bill Fink wrote:
> One other simple thing to try if you haven't already is the kernel
> boot parameter init=/bin/bash.

Well, I've replaced init by a statically linked hello-world program. So
there should be no need for any shared library besides ld.so.1, the ELF
processor, but that one is loaded correctly and as far as I can see it
interprets the ELF file right; the right code is loaded (compared the
binary mem content at the entry point with objdump output) and the
correct entry point is called, but exactly after that I don't get an
answer any more. Unfortunately this is the point where the BDI doesn't
help any more because it cannot follow the user space memory mapping.

> gwiz% ldd /sbin/init
>         libc.so.6 => /lib/libc.so.6 (0x0fea0000)
>         /lib/ld.so.1 => /lib/ld.so.1 (0x30000000)

metis:/tftpboot/solidcard/sbin# file init
init: ELF 32-bit MSB executable, PowerPC or cisco 4500, version 1 (SYSV), statically linked, stripped

And I don't see any call to the open() system call.

Robert
--
 Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
 Pengutronix - Linux Solutions for Science and Industry
   Braunschweiger Str. 79,  31134 Hildesheim, Germany
   Handelsregister:  Amtsgericht Hildesheim, HRA 2686
    Phone: +49-5121-28619-0 |  Fax: +49-5121-28619-4

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/

init does not run on 405GP system

[Kabir Ahsan-r9aahw]

[-- Attachment #1: Type: text/plain, Size: 1445 bytes --]

Hi
I found this thread on the web. The reason I am writing is that I am
also seeing some problem with my /sbin/init. Can you elaborate on the
solution? You mentioned that "The On Chip Memory was mapped to an
address below 0x8000'0000 - what we didn't know was that this is a
_virtual_ address, because the OCM is not simply located "behind" the
MMU." 
What do you mean by OCM? What do you mean by address below 0x80000000?
So you had to redefine your MMU definition to get through this error?
Regard,
Ahsan
 
On Tue, Jun 10, 2003 at 04:10:47PM +0200, Robert Schwebel wrote:
> Nevertheless, the kernel runs smoothly. I can ping the machine, I can
> even floodping it with 0% packet loss. Only that there is no
userspace> running.
>
> Has anybody seen something like this before?

It is solved. The On Chip Memory was mapped to an address below
0x8000'0000 - what we didn't know was that this is a _virtual_ address,
because the OCM is not simply located "behind" the MMU. After changing
the mapping it works smoothly now.

Robert
--
Dipl.-Ing. Robert Schwebel | http://www.pengutronix.de
<http://www.pengutronix.de/> 
Pengutronix - Linux Solutions for Science and Industry
   Braunschweiger Str. 79,  31134 Hildesheim, Germany
   Handelsregister:  Amtsgericht Hildesheim, HRA 2686
    Phone: +49-5121-28619-0 |  Fax: +49-5121-28619-4

** Sent via the linuxppc-dev mail list. See http://lists.linuxppc.org/




[-- Attachment #2: Type: text/html, Size: 2321 bytes --]

Re: init does not run on 405GP system

[Wolfgang Denk]

In message <351763BE49AC8743AFCCF925821FF6BE04A503@az33exm22.fsl.freescale.net> you wrote:
> 
> I found this thread on the web. The reason I am writing is that I am
> also seeing some problem with my /sbin/init. Can you elaborate on the
> solution? You mentioned that "The On Chip Memory was mapped to an
> address below 0x8000'0000 - what we didn't know was that this is a
> _virtual_ address, because the OCM is not simply located "behind" the
> MMU."=20
> What do you mean by OCM? What do you mean by address below 0x80000000?

OCM = On Chip Memory (this should be obvious from the previous text).

And address "below 0x80000000" is  one  in  the  address  range  from
0x0000 up to and including 0x7FFFFFFF (this should be obvious, too).

> So you had to redefine your MMU definition to get through this error?

No, just fix your memory map.

Best regards,

Wolfgang Denk

-- 
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de
The use of anthropomorphic terminology when  dealing  with  computing
systems is a symptom of professional immaturity.   -- Edsger Dijkstra