[Adeos-main] adeos-ipipe-2.6.19-arm-1.7-00 on iMX31: init doesn't complete!

[Adeos-main] adeos-ipipe-2.6.19-arm-1.7-00 on iMX31: init doesn't complete!

[Hank]

[-- Attachment #1.1: Type: text/plain, Size: 1425 bytes --]

Hello Guys,
I'm porting xenomai to Freescale iMX31 processor (on iMX31 LiteKit) as per
porting guidelines. In the first phase, applied
adeos-ipipe-2.6.19-arm-1.7-00 patch found in xenomai-2.4.4 to kernel
2.6.19.2 and have implemented required machine specific code. Everything
went smooth and after fixing a few bugs, kernel booted but with a problem -
init doesn't complete. Following are last few kernel boot log messages:


Looking up port of RPC 100003/2 on 192.168.1.25
Looking up port of RPC 100005/1 on 192.168.1.25
VFS: Mounted root (nfs filesystem).
Freeing init memory: 120K
====== No More Messages =======


As one might suspect, It's not a lockup or freeze - the console is
responsive and echoes back. The kernel is apparently up as it can be ping'ed
from outside. If IPIPE is disabled, kernel boots OK into the shell.

Have tried booting with init=/bin/sh and even with a statically/dynamically
linked simple app but same response. printk debugging shows that
run_init_process() goes upto kernel_execve() call, do_execve() completes
without error but while switching to
user space, it's lost!

PS: I've even tried applying adeos-ipipe-2.6.20-arm-1.8-03.patch (I know the
kernel version difference, but I somehow happened to fix all the failed
HUNKS!). Unfortunately, the response is exactly the same as with 1.7.
Any help would be highly appreciated.

Attached is complete kernel boot log for reference.

Thanks

[-- Attachment #1.2: Type: text/html, Size: 1962 bytes --]

[-- Attachment #2: kernlog.txt --]
[-- Type: text/plain, Size: 7579 bytes --]

Uncompressing Linux............................................................................................................... done, booting the kernel.
Linux version 2.6.19.2 (hank@linux) (gcc version 4.1.1) #19 Wed Sep 17 11:15:55 BST 2008

CPU: Some Random V6 Processor [4107b364] revision 4 (ARMv6TEJ), cr=00c5387f

Machine: Freescale i.MX31 litekit

Memory policy: ECC disabled, Data cache writeback

CPU0: D VIPT write-back cache

CPU0: I cache: 16384 bytes, associativity 4, 32 byte lines, 128 sets

CPU0: D cache: 16384 bytes, associativity 4, 32 byte lines, 128 sets

Built 1 zonelists.  Total pages: 28448

Kernel command line: noinitrd console=ttymxc0 root=/dev/nfs nfsroot=192.168.1.25:/tftpboot/rootfs rw init=/linuxrc ip=dhcp

MXC IRQ initialized

PID hash table entries: 512 (order: 9, 2048 bytes)

Clock input source is 26000000

Actual CLOCK_TICK_RATE is 16625000 Hz

I-pipe 1.7-00: pipeline enabled.

Console: colour dummy device 80x30

Dentry cache hash table entries: 16384 (order: 4, 65536 bytes)

Inode-cache hash table entries: 8192 (order: 3, 32768 bytes)

Memory: 112MB = 112MB total

Memory: 109952KB available (2808K code, 751K data, 120K init)

Mount-cache hash table entries: 512

CPU: Testing write buffer coherency: ok

NET: Registered protocol family 16

MXC GPIO hardware

system_rev is: 0x20

  XXXXXXXXXXXXXXXXXXXX pll = 190320000, usb_pdf = 1, usb_prepdf = 1, retval = 47580000

Irq init for eth0

L2 cache: WB

kobject_add failed for MX31ADS/MX31LITE PC (-13)

[<c002af68>] (dump_stack+0x0/0x14) from [<c013f320>] (kobject_add+0x15c/0x19c)

[<c013f1c4>] (kobject_add+0x0/0x19c) from [<c0175058>] (device_add+0x88/0x4a8)

[<c0174fd0>] (device_add+0x0/0x4a8) from [<c0179080>] (platform_device_add+0x100/0x158)

[<c0178f80>] (platform_device_add+0x0/0x158) from [<c01790fc>] (platform_device_register+0x24/0x28)

 r7 = 00000000  r6 = C04B8000  r5 = C00215F8  r4 = C02EBEE8

[<c01790d8>] (platform_device_register+0x0/0x28) from [<c00109b8>] (mxc_init_devices+0xa0/0xd8)

 r4 = C0021D84 

[<c0010918>] (mxc_init_devices+0x0/0xd8) from [<c00260c8>] (init+0x9c/0x290)

[<c002602c>] (init+0x0/0x290) from [<c0048564>] (do_exit+0x0/0x820)

 r7 = 00000000  r6 = 00000000  r5 = 00000000  r4 = 00000000

Using SDMA I.API

MXC DMA API initialized

usb: Host 1 registered

usb: Host 2 registered

usb: OTG HS Host registered

usb: OTG HS Gadget registered

Generic PHY: Registered new driver

SCSI subsystem initialized



spi_active 1

CSPI: mxc_spi-2 probed

usbcore: registered new interface driver usbfs

usbcore: registered new interface driver hub

usbcore: registered new device driver usb

MXC I2C driver

mc13783 Rev 3.3 FinVer 0 detected

Device spi2.0 probed

PMIC Character device: successfully loaded

PMIC RTC successfully probed

PMIC Light successfully loaded

PMIC Power successfully probed

NET: Registered protocol family 2

IP route cache hash table entries: 1024 (order: 0, 4096 bytes)

TCP established hash table entries: 4096 (order: 2, 16384 bytes)

TCP bind hash table entries: 2048 (order: 1, 8192 bytes)

TCP: Hash tables configured (established 4096 bind 2048)

TCP reno registered

Low-Level PM Driver module loaded

Freescale i.MX31 Dynamic Power Management.

I-pipe: Domain Xenomai registered.

Xenomai: hal/arm started.

Xenomai: real-time nucleus v2.4.4 (Made Of Tears) loaded.

Xenomai: starting native API services.

Xenomai: starting POSIX services.

Xenomai: starting RTDM services.

JFFS2 version 2.2. (NAND) (C) 2001-2006 Red Hat, Inc.

io scheduler noop registered

io scheduler anticipatory registered

io scheduler deadline registered

io scheduler cfq registered (default)

FS453/4 driver, (c) 2005 Freescale Semiconductor, Inc.

mxc_ipu mxc_ipu: Warning: channel already initialized

Console: switching to colour frame buffer device 100x30

mxcfb: fb registered, using mode Promate-WVGA

Real TIme clock Driver v1.0 

MXC WatchDog Driver 2.0

MXC Watchdog # 0 Timer: initial timeout 60 sec

Serial: MXC Internal UART driver

mxcintuart.0: ttymxc0 at MMIO 0x43f90000 (irq = 45) is a Freescale MXC

mxcintuart.1: ttymxc1 at MMIO 0x43f94000 (irq = 32) is a Freescale MXC

mxcintuart.2: ttymxc2 at MMIO 0x5000c000 (irq = 18) is a Freescale MXC

mxcintuart.4: ttymxc4 at MMIO 0x43fb4000 (irq = 47) is a Freescale MXC

RAMDISK driver initialized: 16 RAM disks of 16384K size 1024 blocksize

loop: loaded (max 8 devices)

SMSC LAN83C185: Registered new driver

cs89x0:cs89x0_probe(0x0)

cs89x0: no cs8900 or cs8920 detected.  Be sure to disable PnP with SETUP

Linux video capture interface: v2.00

MXC Video Output MXC Video Output.0: Registered device video16

MXC MTD nor Driver 2.0

mxc_nor_flash.0: Found 1 x16 devices at 0x0 in 16-bit bank

Support for command set 0003 not present

gen_probe: No supported Vendor Command Set found

mxc_nor_flash: probe of mxc_nor_flash.0 failed with error -5

MXC MTD nand Driver 2.0

NAND device: Manufacturer ID: 0x20, Chip ID: 0x76 (ST Micro NAND 64MiB 3,3V 8-bit)

Scanning device for bad blocks

Bad eraseblock 793 at 0x00c64000

Creating 4 MTD partitions on "NAND 64MiB 3,3V 8-bit":

0x00000000-0x00200000 : "nand.kernel"

0x00200000-0x04000000 : "nand.rootfs"

0x00000000-0x04000000 : "<NULL>"

0x00000000-0x04000000 : "<NULL>"

usbmon: debugfs is not available

Initializing USB Mass Storage driver...

usbcore: registered new interface driver usb-storage

USB Mass Storage support registered.

MXC keypad loaded

input: mxckpd as /class/input/input0

i2c /dev entries driver

Driver for 1-wire Dallas network protocol.

Serial: MXC OWire driver

IPU Post-filter loading

SSI module loaded successfully

PMIC Audio driver loading...

PMIC Battery successfully probed

PMIC Connectivity driver loading..

DPTC controller initialized

DVFS controller initialized

DVFS frequency #0: 532000000

DVFS frequency #1: 266000000

DVFS frequency #2: 133000000

DVFS frequency #3: 133000000

RNG Driver: Loading

SEC: mxc_sec_mod_init() called 

MXC MMC/SD driver

mxcmci-0 found

Advanced Linux Sound Architecture Driver Version 1.0.13 (Tue Nov 28 14:07:24 2006 UTC).

Control ALSA component registered

ALSA device list:

  #0: MXC Freescale with PMIC

oprofile: using arm/arm11

TCP cubic registered

NET: Registered protocol family 1

NET: Registered protocol family 17

ieee80211: 802.11 data/management/control stack, git-1.1.13

ieee80211: Copyright (C) 2004-2005 Intel Corporation <jketreno@linux.intel.com>

Power Management for Freescale MX31

VFP support v0.3: implementor 41 architecture 1 part 20 variant b rev 2

input: mxc_ts as /class/input/input1

mxc input touchscreen loaded

eth0: SMSC911x/921x identified at 0xc7a00000, IRQ: 90

eth0: SMSC911x MAC Address: 00:08:ee:01:e2:57

eth0: link down

Sending DHCP requests .<6>eth0: link up, 100Mbps, full-duplex, lpa 0x45E1

., OK

IP-Config: Got DHCP answer from 0.0.0.0, my address is 192.168.1.20

IP-Config: Complete:

      device=eth0, addr=192.168.1.20, mask=255.255.255.0, gw=192.168.1.105,

     host=192.168.1.20, domain=dev.local, nis-domain=(none),

     bootserver=0.0.0.0, rootserver=192.168.1.25, rootpath=

Looking up port of RPC 100003/2 on 192.168.1.25

Looking up port of RPC 100005/1 on 192.168.1.25

VFS: Mounted root (nfs filesystem).

Freeing init memory: 120K

Re: [Adeos-main] adeos-ipipe-2.6.19-arm-1.7-00 on iMX31: init doesn't complete!

[Gilles Chanteperdrix]

Hank wrote:
> Hello Guys,
> I'm porting xenomai to Freescale iMX31 processor (on iMX31 LiteKit) as per
> porting guidelines. In the first phase, applied
> adeos-ipipe-2.6.19-arm-1.7-00 patch found in xenomai-2.4.4 to kernel
> 2.6.19.2 and have implemented required machine specific code. Everything
> went smooth and after fixing a few bugs, kernel booted but with a problem -
> init doesn't complete. Following are last few kernel boot log messages:

Did you try to follow the second advice in the "Tips and tricks" section ?
-- 
                                                 Gilles.

Re: [Adeos-main] adeos-ipipe-2.6.19-arm-1.7-00 on iMX31: init doesn't complete!

[Gilles Chanteperdrix]

Hank wrote:
> Hello Guys,
> I'm porting xenomai to Freescale iMX31 processor (on iMX31 LiteKit) as per
> porting guidelines. In the first phase, applied
> adeos-ipipe-2.6.19-arm-1.7-00 patch found in xenomai-2.4.4 to kernel
> 2.6.19.2 and have implemented required machine specific code. Everything
> went smooth and after fixing a few bugs, kernel booted but with a problem -
> init doesn't complete. Following are last few kernel boot log messages:
> 
> 
> Looking up port of RPC 100003/2 on 192.168.1.25
> Looking up port of RPC 100005/1 on 192.168.1.25
> VFS: Mounted root (nfs filesystem).
> Freeing init memory: 120K

Since you are booting via NFS, wireshark may help ?

Do you have the same problem if you disable support for VFP ?

-- 
                                                 Gilles.

Re: [Adeos-main] adeos-ipipe-2.6.19-arm-1.7-00 on iMX31: init doesn't complete!

[Hank]

[-- Attachment #1: Type: text/plain, Size: 1089 bytes --]

On 9/18/08, Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org> wrote:
>
> Hank wrote:
> > Hello Guys,
> > I'm porting xenomai to Freescale iMX31 processor (on iMX31 LiteKit) as
> per
> > porting guidelines. In the first phase, applied
> > adeos-ipipe-2.6.19-arm-1.7-00 patch found in xenomai-2.4.4 to kernel
> > 2.6.19.2 and have implemented required machine specific code. Everything
> > went smooth and after fixing a few bugs, kernel booted but with a problem
> -
> > init doesn't complete. Following are last few kernel boot log messages:
> >
> >
> > Looking up port of RPC 100003/2 on 192.168.1.25
> > Looking up port of RPC 100005/1 on 192.168.1.25
> > VFS: Mounted root (nfs filesystem).
> > Freeing init memory: 120K
>
> Since you are booting via NFS, wireshark may help ?
>
> Do you have the same problem if you disable support for VFP ?
>
> --
>                                                 Gilles.

Thanks Gilles.

I've tried everything. Wireshark shows NFS read requests being issued and
busybox etc. being sent back. Already played with VFP without any luck!

Regards

[-- Attachment #2: Type: text/html, Size: 1841 bytes --]

Re: [Adeos-main] adeos-ipipe-2.6.19-arm-1.7-00 on iMX31: init doesn't complete!

[Gilles Chanteperdrix]

Hank wrote:
> On 9/18/08, Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org> wrote:
>> Hank wrote:
>>> Hello Guys,
>>> I'm porting xenomai to Freescale iMX31 processor (on iMX31 LiteKit) as
>> per
>>> porting guidelines. In the first phase, applied
>>> adeos-ipipe-2.6.19-arm-1.7-00 patch found in xenomai-2.4.4 to kernel
>>> 2.6.19.2 and have implemented required machine specific code. Everything
>>> went smooth and after fixing a few bugs, kernel booted but with a problem
>> -
>>> init doesn't complete. Following are last few kernel boot log messages:
>>>
>>>
>>> Looking up port of RPC 100003/2 on 192.168.1.25
>>> Looking up port of RPC 100005/1 on 192.168.1.25
>>> VFS: Mounted root (nfs filesystem).
>>> Freeing init memory: 120K
>> Since you are booting via NFS, wireshark may help ?
>>
>> Do you have the same problem if you disable support for VFP ?
>>
>> --
>>                                                 Gilles.
> 
> Thanks Gilles.
> 
> I've tried everything. Wireshark shows NFS read requests being issued and
> busybox etc. being sent back. Already played with VFP without any luck!

And the advice in the "tips and tricks" section of the howto ?

-- 
                                                 Gilles.

Re: [Adeos-main] adeos-ipipe-2.6.19-arm-1.7-00 on iMX31: init doesn't complete!

[Hank]

[-- Attachment #1: Type: text/plain, Size: 1497 bytes --]

On 9/18/08, Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org> wrote:
>
> Hank wrote:
> > On 9/18/08, Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org>
> wrote:
> >> Hank wrote:
> >>> Hello Guys,
> >>> I'm porting xenomai to Freescale iMX31 processor (on iMX31 LiteKit) as
> >> per
> >>> porting guidelines. In the first phase, applied
> >>> adeos-ipipe-2.6.19-arm-1.7-00 patch found in xenomai-2.4.4 to kernel
> >>> 2.6.19.2 and have implemented required machine specific code.
> Everything
> >>> went smooth and after fixing a few bugs, kernel booted but with a
> problem
> >> -
> >>> init doesn't complete. Following are last few kernel boot log messages:
> >>>
> >>>
> >>> Looking up port of RPC 100003/2 on 192.168.1.25
> >>> Looking up port of RPC 100005/1 on 192.168.1.25
> >>> VFS: Mounted root (nfs filesystem).
> >>> Freeing init memory: 120K
> >> Since you are booting via NFS, wireshark may help ?
> >>
> >> Do you have the same problem if you disable support for VFP ?
> >>
> >> --
> >>                                                 Gilles.
> >
> > Thanks Gilles.
> >
> > I've tried everything. Wireshark shows NFS read requests being issued and
> > busybox etc. being sent back. Already played with VFP without any luck!
>
> And the advice in the "tips and tricks" section of the howto ?
>
> --
>                                                 Gilles.


Well, some of the tricks helped during initial stages but other don't seem
to apply now as there is no crash here!

[-- Attachment #2: Type: text/html, Size: 2711 bytes --]

Re: [Adeos-main] adeos-ipipe-2.6.19-arm-1.7-00 on iMX31: init doesn't complete!

[Gilles Chanteperdrix]

Hank wrote:
> Well, some of the tricks helped during initial stages but other don't seem
> to apply now as there is no crash here!

Oh yes ? Are you sure the timer is ticking correctly ?

-- 
                                                 Gilles.

Re: [Adeos-main] adeos-ipipe-2.6.19-arm-1.7-00 on iMX31: init doesn't complete!

[Hank]

[-- Attachment #1: Type: text/plain, Size: 519 bytes --]

On 9/18/08, Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org> wrote:
>
> Hank wrote:
> > Well, some of the tricks helped during initial stages but other don't
> seem
> > to apply now as there is no crash here!
>
> Oh yes ? Are you sure the timer is ticking correctly ?
>
> --
>                                                 Gilles.


Yes. Also, forced it to a soft lockup crash in ret_to_user() and trace
showed timer interrupt ticking and updating process timers all the way up!





-- 
Freedom is everything

[-- Attachment #2: Type: text/html, Size: 1110 bytes --]

Re: [Adeos-main] adeos-ipipe-2.6.19-arm-1.7-00 on iMX31: init doesn't complete!

[Gilles Chanteperdrix]

Hank wrote:
> On 9/18/08, Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org> wrote:
>> Hank wrote:
>>> Well, some of the tricks helped during initial stages but other don't
>> seem
>>> to apply now as there is no crash here!
>> Oh yes ? Are you sure the timer is ticking correctly ?
>>
>> --
>>                                                 Gilles.
> 
> 
> Yes. Also, forced it to a soft lockup crash in ret_to_user() and trace
> showed timer interrupt ticking and updating process timers all the way up!

I am afraid I can not help much. The patch you use is too old, I fixed
some bugs recently in the traps interception code which should lead to a
clear crash instead of some lockup. I also added support for VFP.

So, do you observe the same phenomenon with a vanilla 2.6.26 and the
latest patch (AFAIK Litekit is supported by mainline) ?

I also suspect a bug when using Xenomai with NO_HZ on ARM. So, you may
try to disable NO_HZ.

-- 
                                                 Gilles.

Re: [Adeos-main] adeos-ipipe-2.6.19-arm-1.7-00 on iMX31: init doesn't complete!

[Hank]

[-- Attachment #1: Type: text/plain, Size: 1323 bytes --]

On 9/18/08, Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org> wrote:
>
> Hank wrote:
> > On 9/18/08, Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org>
> wrote:
> >> Hank wrote:
> >>> Well, some of the tricks helped during initial stages but other don't
> >> seem
> >>> to apply now as there is no crash here!
> >> Oh yes ? Are you sure the timer is ticking correctly ?
> >>
> >> --
> >>                                                 Gilles.
> >
> >
> > Yes. Also, forced it to a soft lockup crash in ret_to_user() and trace
> > showed timer interrupt ticking and updating process timers all the way
> up!
>
> I am afraid I can not help much. The patch you use is too old, I fixed
> some bugs recently in the traps interception code which should lead to a
> clear crash instead of some lockup. I also added support for VFP.
>
> So, do you observe the same phenomenon with a vanilla 2.6.26 and the
> latest patch (AFAIK Litekit is supported by mainline) ?
>
> I also suspect a bug when using Xenomai with NO_HZ on ARM. So, you may
> try to disable NO_HZ.
>
> --
>                                                 Gilles.


Thanks Gilles. Well, looks like your suspicions about VFP were right! I
certainly didn't test it recently without VFP. It just booted to shell after
disabling VFP.

Thanks for helping out.

[-- Attachment #2: Type: text/html, Size: 2438 bytes --]

Re: [Adeos-main] adeos-ipipe-2.6.19-arm-1.7-00 on iMX31: init doesn't complete!

[Gilles Chanteperdrix]

Hank wrote:
> On 9/18/08, Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org> wrote:
>> Hank wrote:
>>> On 9/18/08, Gilles Chanteperdrix <gilles.chanteperdrix@xenomai.org>
>> wrote:
>>>> Hank wrote:
>>>>> Well, some of the tricks helped during initial stages but other don't
>>>> seem
>>>>> to apply now as there is no crash here!
>>>> Oh yes ? Are you sure the timer is ticking correctly ?
>>>>
>>>> --
>>>>                                                 Gilles.
>>>
>>> Yes. Also, forced it to a soft lockup crash in ret_to_user() and trace
>>> showed timer interrupt ticking and updating process timers all the way
>> up!
>>
>> I am afraid I can not help much. The patch you use is too old, I fixed
>> some bugs recently in the traps interception code which should lead to a
>> clear crash instead of some lockup. I also added support for VFP.
>>
>> So, do you observe the same phenomenon with a vanilla 2.6.26 and the
>> latest patch (AFAIK Litekit is supported by mainline) ?
>>
>> I also suspect a bug when using Xenomai with NO_HZ on ARM. So, you may
>> try to disable NO_HZ.
>>
>> --
>>                                                 Gilles.
> 
> 
> Thanks Gilles. Well, looks like your suspicions about VFP were right! I
> certainly didn't test it recently without VFP. It just booted to shell after
> disabling VFP.

Ok. If you intend to use VFP, you have to use the latest version of
Xenomai (not released yet), and the latest I-pipe patch. Otherwise, you
will run into problems.

-- 
                                                 Gilles.