Request For Help - Embedded x86, Kernel 2.6.33 boots, Mounts JFFS2 rootfs, Stops at init

Request For Help - Embedded x86, Kernel 2.6.33 boots, Mounts JFFS2 rootfs, Stops at init

[Graeme Russ]

Hi All,

I've been working for a while now getting the x86 U-Boot port up to speed
to boot a linux kernel

I now have a root fs (build from Linux-From-Scratch and trimmed to <15M
using qemu) but all attempts to get any serial console output past
launching of init have failed.

I have taken the root fs which successfully boots using qemu and created a
jffs2 image and flashed it onto one of the on-board flash chips. U-Boot can
successfully 'ls' the root fs

My bootags are:

  root=/dev/mtdblock1 rw rootfstype=jffs2 console=ttyS0,115200

as Linux can mount the root partition:
[    1.447819] Serial: 8250/16550 driver, 6 ports, IRQ sharing disabled
[    1.450027] serial8250: ttyS0 at I/O 0x3f8 (irq = 4) is a 16550A
[    1.452421] serial8250: ttyS1 at I/O 0x2f8 (irq = 3) is a 16550A
[    1.505824] brd: module loaded
[    1.507350] Uniform Multi-Platform E-IDE driver
[    1.509245] ide-gd driver 1.18
[    1.512852] Serck eNET flash device: 0x1000000 at 0x10000000
[    1.523853] eNET_Flash_0: Found 1 x16 devices at 0x0 in 16-bit bank
[    1.524828]  Intel/Sharp Extended Query Table at 0x0031
[    1.526492]  Intel/Sharp Extended Query Table at 0x0031
[    1.527492] Using buffer write method
[    1.528242] cfi_cmdset_0001: Erase suspend on write enabled
[    1.529242] Serck eNET flash device: 0x1000000 at 0x11000000
[    1.539872] eNET_Flash_1: Found 1 x16 devices at 0x0 in 16-bit bank
[    1.540951]  Intel/Sharp Extended Query Table at 0x0031
[    1.543632]  Intel/Sharp Extended Query Table at 0x0031
[    1.544426] Using buffer write method
[    1.545495] cfi_cmdset_0001: Erase suspend on write enabled
[    1.547452] Serck eNET flash device: 0x80000 at 0x38000000
[    1.549664] Found: AMD AM29LV040B
[    1.550136] eNET_Boot_Flash: Found 1 x8 devices at 0x0 in 8-bit bank
[    1.551136] number of JEDEC chips: 1
[    1.551975] Adding Device 0
[    1.556819] Partitioning Device 1
[    1.558150] Creating 1 MTD partitions on "eNET_Flash_1":
[    1.559150] 0x000000000000-0x000001000000 : "rootfs"
[    1.563816] Adding Device 2
[    1.568821] 8139too Fast Ethernet driver 0.9.28
[    1.573803] eth0: RealTek RTL8139 at 0xc8896000, 00:40:f4:6e:67:ac, IRQ 10
[    1.578796] eth1: RealTek RTL8139 at 0xc889a100, 00:40:f4:6e:67:ac, IRQ 9
[    1.582822] TCP cubic registered
[    1.584516] Using IPI Shortcut mode
[    1.901864] VFS: Mounted root (jffs2 filesystem) on device 31:1.
[    1.902926] Freeing unused kernel memory: 204k freed
[    1.909817] Write protecting the kernel text: 1924k
[    1.911896] Write protecting the kernel read-only data: 596k
[    1.915842] kernel_execve(/sbin/init)

(I've added a printk in kernel_execve() which gives me the last line)

I  initially had hot-plug support compiled into the kernel and was getting
a lot of kernel_execve(/sbin/hotplug) messages as well (plus a udev:
starting version 151 message which appear 100+ seconds and the above
messages), but I have since removed that and used mknod to create
/dev/ttyS0 directly

I have also tried the following test script:

#!/bin/bash
stty -F /dev/ttyS0 115200
cat hello > /dev/ttyS0

but still no luck

Does anyone have any advise on how I can debug this problem?

Regards,

Graeme

Re: Request For Help - Embedded x86, Kernel 2.6.33 boots, Mounts JFFS2 rootfs, Stops at init

[Mike Rapoport]

Graeme Russ wrote:
> Hi All,
> 
> I've been working for a while now getting the x86 U-Boot port up to speed
> to boot a linux kernel
> 
> I now have a root fs (build from Linux-From-Scratch and trimmed to <15M
> using qemu) but all attempts to get any serial console output past
> launching of init have failed.
> 
> I have taken the root fs which successfully boots using qemu and created a
> jffs2 image and flashed it onto one of the on-board flash chips. U-Boot can
> successfully 'ls' the root fs
> 
> My bootags are:
> 
>   root=/dev/mtdblock1 rw rootfstype=jffs2 console=ttyS0,115200
> 
> as Linux can mount the root partition:
> [    1.447819] Serial: 8250/16550 driver, 6 ports, IRQ sharing disabled

[ snip ]

> [    1.911896] Write protecting the kernel read-only data: 596k
> [    1.915842] kernel_execve(/sbin/init)
> 
> (I've added a printk in kernel_execve() which gives me the last line)
> 
> I  initially had hot-plug support compiled into the kernel and was getting
> a lot of kernel_execve(/sbin/hotplug) messages as well (plus a udev:
> starting version 151 message which appear 100+ seconds and the above
> messages), but I have since removed that and used mknod to create
> /dev/ttyS0 directly
> 
> I have also tried the following test script:
> 
> #!/bin/bash
> stty -F /dev/ttyS0 115200
> cat hello > /dev/ttyS0
> 
> but still no luck
> 
> Does anyone have any advise on how I can debug this problem?

Have you tried with init=/bin/sh?
Also, make sure you have /dev/null and /dev/console nodes.

> 
> Regards,
> 
> Graeme
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/


-- 
Sincerely yours,
Mike.

Re: Request For Help - Embedded x86, Kernel 2.6.33 boots, Mounts JFFS2 rootfs, Stops at init

[Alexander Clouter]

Graeme Russ <graeme.russ@gmail.com> wrote:
> 
> I've been working for a while now getting the x86 U-Boot port up to speed
> to boot a linux kernel
>
...ewww x86.
 
> [snipped]
>
> [    1.911896] Write protecting the kernel read-only data: 596k
> [    1.915842] kernel_execve(/sbin/init)
> 
> (I've added a printk in kernel_execve() which gives me the last line)
> 
> I  initially had hot-plug support compiled into the kernel and was getting
> a lot of kernel_execve(/sbin/hotplug) messages as well (plus a udev:
> starting version 151 message which appear 100+ seconds and the above
> messages), but I have since removed that and used mknod to create
> /dev/ttyS0 directly
> 
> I have also tried the following test script:
> 
> #!/bin/bash
> stty -F /dev/ttyS0 115200
> cat hello > /dev/ttyS0
> 
> but still no luck
> 
> Does anyone have any advise on how I can debug this problem?
> 
What does you /dev directory look like?  You need to have at least the 
following:
----
mknod console c 5 1
mknod null c 1 3
mknod ttyS0 c 4 64
----

Cheers

-- 
Alexander Clouter
.sigmonster says: Your love life will be... interesting.

Re: Request For Help - Embedded x86, Kernel 2.6.33 boots, Mounts JFFS2 rootfs, Stops at init

[Graeme Russ]

On Thu, Apr 29, 2010 at 1:37 AM, Alexander Clouter <alex@digriz.org.uk> wrote:
> Graeme Russ <graeme.russ@gmail.com> wrote:
>>
>> I've been working for a while now getting the x86 U-Boot port up to speed
>> to boot a linux kernel
>>
> ...ewww x86.

Yeah, nothing like a challange ;)

>
>> [snipped]
>>
>> [    1.911896] Write protecting the kernel read-only data: 596k
>> [    1.915842] kernel_execve(/sbin/init)
>>
>> (I've added a printk in kernel_execve() which gives me the last line)
>>
>> I  initially had hot-plug support compiled into the kernel and was getting
>> a lot of kernel_execve(/sbin/hotplug) messages as well (plus a udev:
>> starting version 151 message which appear 100+ seconds and the above
>> messages), but I have since removed that and used mknod to create
>> /dev/ttyS0 directly
>>
>> I have also tried the following test script:
>>
>> #!/bin/bash
>> stty -F /dev/ttyS0 115200
>> cat hello > /dev/ttyS0
>>
>> but still no luck
>>
>> Does anyone have any advise on how I can debug this problem?
>>
> What does you /dev directory look like?  You need to have at least the
> following:
> ----
> mknod console c 5 1
> mknod null c 1 3
> mknod ttyS0 c 4 64
> ----
>

tick, tick, tick

A little further background (should have included this last night but it
was late and I thought the problem must be trivial)...

I had originally mounted this root fs using NFS with (very) limited
success. After mounting and performing a few NFS requests (observed using
wireshark) the NFS activity would simply stop with 'server not responding'
messages. I turned on debugging in the 8139too and NFS drivers. With
debugging output enabled I was still getting 'server not responding'
messages, but (eventually) the boot sequence would continue. This even
resulted in logs in /var/log of the NFS share for the device (filled with
raw network packet data).

But no console...

What if I init with the following:

#!/bin/bash
echo hello > /hello.txt

and then rebooted and checked the root fs in U-Boot? How long will it
take for hello.txt to be flushed? Can I force the flush?

Regards,

Graeme

Re: Request For Help - Embedded x86, Kernel 2.6.33 boots, Mounts ?JFFS2 rootfs, Stops at init

[Alexander Clouter]

Graeme Russ <graeme.russ@gmail.com> wrote:
>
> On Thu, Apr 29, 2010 at 1:37 AM, Alexander Clouter <alex@digriz.org.uk> wrote:
>
>> What does you /dev directory look like?  You need to have at least the
>> following:
>> ----
>> mknod console c 5 1
>> mknod null c 1 3
>> mknod ttyS0 c 4 64
>> ----
> 
> tick, tick, tick
> 
> A little further background (should have included this last night but it
> was late and I thought the problem must be trivial)...
> 
> I had originally mounted this root fs using NFS with (very) limited
> success. After mounting and performing a few NFS requests (observed using
> wireshark) the NFS activity would simply stop with 'server not responding'
> messages. I turned on debugging in the 8139too and NFS drivers. With
> debugging output enabled I was still getting 'server not responding'
> messages, but (eventually) the boot sequence would continue. This even
> resulted in logs in /var/log of the NFS share for the device (filled with
> raw network packet data).
>
I got this behaviour when setting up a PXE booting workstation network 
registration system.  My issue was that the IP changed and was running
running TCP based NFS; I think the kernel continues to try to use the 
old IP, same socket, but as it is no longer there...Game Over.   
Depending on how your boot up worked, at the PXE booting phase (or 
just where the kernel uses DHCP to get a lease) things work, however 
then you get to the network configuring part of the boot up, dhclient 
uses a different identifier so you get a different IP.
 
Well...that's what happened to us and the simpliest fix was to move to 
UDP for NFS.

> But no console...
>
...instead of LFS, have you looked at buildroot[1]?

> What if I init with the following:
> 
> #!/bin/bash
> echo hello > /hello.txt
> 
....put a 'while (true); do sleep 1; done' on the end of that.  Although 
you should see a 'attempted to kill init' without it there.

> and then rebooted and checked the root fs in U-Boot? How long will it
> take for hello.txt to be flushed? Can I force the flush?
> 
Have a look at:

http://www.spinics.net/lists/mips/msg37361.html

Especially my 'readelf' bit, I'm guessing you might have forgotten a 
library or two in your keenness to trim your root mountpoint.

Cheers

[1] http://buildroot.uclibc.org/

-- 
Alexander Clouter
.sigmonster says: Time to be aggressive.  Go after a tattooed Virgo.

Re: Request For Help - Embedded x86, Kernel 2.6.33 boots, Mounts JFFS2 rootfs, Stops at init

[Graeme Russ]

Graeme Russ wrote:
> On Thu, Apr 29, 2010 at 1:37 AM, Alexander Clouter <alex@digriz.org.uk> wrote:
>> Graeme Russ <graeme.russ@gmail.com> wrote:
>>> I've been working for a while now getting the x86 U-Boot port up to speed
>>> to boot a linux kernel
>>>
>> ...ewww x86.
> 
>>>
>> What does you /dev directory look like?  You need to have at least the
>> following:
>> ----
>> mknod console c 5 1
>> mknod null c 1 3
>> mknod ttyS0 c 4 64
>> ----
>>
> 
> tick, tick, tick
> 

Success!!!

I feel _really_ stupid about the cause of the problem.... I hadn't set up
the interrupt mapping of the serial ports in my board initialisation
routines in U-Boot. Once I mapped IRQ4 to UART0 it all sprang to life ;)

Big thanks to everyone for helping and a big thanks to all the linux and
GNU developers for creating such a wonderful 'toy' ;)

Graeme