ppc32 kernel boot problem (pmu related?)

ppc32 kernel boot problem (pmu related?)

[Nathan Pilatzke]

Hello, I am not a member of this list and I hope this is the correct
place to report such problems and ask for help.

I am running an eMac first generation powerpc (700MHz) and the Gentoo
distribution.  Kernel 2.6.15 (Gentoo revision 1) is working quite well
for me.  A few days ago I compiled 2.6.17-rc3 from the vanilla sources
ebuild and am having troubles with the pmu code.

I can boot successfully when my config has
# CONFIG_ADB_PMU is not set
but cannot boot if my config has
CONFIG_ADB_PMU=3Dy

I can post anything you need to help narrow down the problem (ie.
kernel configs, boot msgs).  Would turning on the kernel debugging
give more information on the problem?  Any help would be greatly
appreciated... even if I can just narrow down the problem further so
that I can make a very specific bug report.

Here is the cpuinfo (as reported under 2.6.17-rc3 without PMU):
cat /proc/cpuinfo
processor       : 0
cpu             : 7450, altivec supported
clock           : 700.000000MHz
revision        : 0.1 (pvr 8000 0201)
bogomips        : 49.79
timebase        : 24913389
platform        : PowerMac
machine         : PowerMac4,4
motherboard     : PowerMac4,4 MacRISC2 MacRISC Power Macintosh
detected as     : 80 (eMac)
pmac flags      : 00000010
L2 cache        : 256K unified
pmac-generation : NewWorld

Thanks.

Re: ppc32 kernel boot problem (pmu related?)

[Benjamin Herrenschmidt]

On Thu, 2006-05-11 at 15:07 -0400, Nathan Pilatzke wrote:
> Hello, I am not a member of this list and I hope this is the correct
> place to report such problems and ask for help.
> 
> I am running an eMac first generation powerpc (700MHz) and the Gentoo
> distribution.  Kernel 2.6.15 (Gentoo revision 1) is working quite well
> for me.  A few days ago I compiled 2.6.17-rc3 from the vanilla sources
> ebuild and am having troubles with the pmu code.
> 
> I can boot successfully when my config has
> # CONFIG_ADB_PMU is not set
> but cannot boot if my config has
> CONFIG_ADB_PMU=y

What happens when you enable that option ?

> I can post anything you need to help narrow down the problem (ie.
> kernel configs, boot msgs).  Would turning on the kernel debugging
> give more information on the problem?  Any help would be greatly
> appreciated... even if I can just narrow down the problem further so
> that I can make a very specific bug report.
> 
> Here is the cpuinfo (as reported under 2.6.17-rc3 without PMU):
> cat /proc/cpuinfo
> processor       : 0
> cpu             : 7450, altivec supported
> clock           : 700.000000MHz
> revision        : 0.1 (pvr 8000 0201)
> bogomips        : 49.79
> timebase        : 24913389
> platform        : PowerMac
> machine         : PowerMac4,4
> motherboard     : PowerMac4,4 MacRISC2 MacRISC Power Macintosh
> detected as     : 80 (eMac)
> pmac flags      : 00000010
> L2 cache        : 256K unified
> pmac-generation : NewWorld
> 
> Thanks.
> _______________________________________________
> Linuxppc-dev mailing list
> Linuxppc-dev@ozlabs.org
> https://ozlabs.org/mailman/listinfo/linuxppc-dev

Re: ppc32 kernel boot problem (pmu related?)

[Nathan Pilatzke]

On 5/11/06, Benjamin Herrenschmidt <benh@kernel.crashing.org> wrote:
> On Thu, 2006-05-11 at 15:07 -0400, Nathan Pilatzke wrote:
> > I am running an eMac first generation powerpc (700MHz) and the Gentoo
> > distribution.  Kernel 2.6.15 (Gentoo revision 1) is working quite well
> > for me.  A few days ago I compiled 2.6.17-rc3 from the vanilla sources
> > ebuild and am having troubles with the pmu code.
> >
> > I can boot successfully when my config has
> > # CONFIG_ADB_PMU is not set
> > but cannot boot if my config has
> > CONFIG_ADB_PMU=3Dy
>
> What happens when you enable that option ?
>
Sorry, I was a little light on details... I thought this might be a
known problem.  With the pmu option enabled (and early open firmware
messages enabled) I get:

Using PowerMac machine description
Total memory =3D 640MB; using 2048kB for hash table (at cfe00000)
Linux version 2.6.17-rc3 (root@the_emac) (gcc version 3.4.5 (Gentoo 3.4.5-r=
1, ss
p-3.4.5-1.0, pie-8.7.9)) #11 Thu May 11 09:53:39 EDT 2006
Found UniNorth memory controller & host bridge @ 0xf8000000 revision: 0x11
Mapped at 0xfdfc0000
Found a Keylargo mac-io controller, rev: 3, mapped at 0xfdf40000
Processor NAP mode on idle enabled.
PowerMac motherboard: eMac
setup_arch: bootmem
Found UniNorth PCI host bridge at 0xf0000000. Firmware bus number: 0->0
Found UniNorth PCI host bridge at 0xf2000000. Firmware bus number: 0->0
Found UniNorth PCI host bridge at 0xf4000000. Firmware bus number: 0->0
via-pmu: Server Mode is disabled
PMU driver v2 initialized for Core99, firmware: 0c
arch: exit
Top of RAM: 0x28000000, Total RAM: 0x28000000
Memory hole size: 0MB
Built 1 zonelists
Kernel command line: root=3D/dev/sda8 ro video=3Dofonly root=3D/dev/sda8 ro=
otdelay=3D10
resume=3D/dev/sda9
mpic: Setting up MPIC " MPIC 1   " version 1.2 at 80040000, max 4 CPUs
mpic: ISU size: 64, shift: 6, mask: 3f
mpic: Initializing for 64 sources
PID hash table entries: 4096 (order: 12, 16384 bytes)
time_init: decrementer frequency =3D 24.913389 MHz
time_init: processor frequency   =3D 700.000000 MHz

and then the machine freezes and requires a power cycle to restart.
Without that option enabled I get this:

 PowerMac motherboard: eMac
-setup_arch: bootmem
 Found UniNorth PCI host bridge at 0xf0000000. Firmware bus number: 0->0
 Found UniNorth PCI host bridge at 0xf2000000. Firmware bus number: 0->0
 Found UniNorth PCI host bridge at 0xf4000000. Firmware bus number: 0->0
-via-pmu: Server Mode is disabled
-PMU driver v2 initialized for Core99, firmware: 0c
-arch: exit
+WARNING ! Your machine is PMU-based but your kernel
+          wasn't compiled with CONFIG_ADB_PMU option !
 Top of RAM: 0x28000000, Total RAM: 0x28000000
 Memory hole size: 0MB
+On node 0 totalpages: 163840
+  DMA zone: 163840 pages, LIFO batch:31
 Built 1 zonelists
 Kernel command line: root=3D/dev/sda8 ro video=3Dofonly root=3D/dev/sda8
rootdelay=3D10
resume=3D/dev/sda9
 mpic: Setting up MPIC " MPIC 1   " version 1.2 at 80040000, max 4 CPUs
 mpic: ISU size: 64, shift: 6, mask: 3f
 mpic: Initializing for 64 sources
 PID hash table entries: 4096 (order: 12, 16384 bytes)
 time_init: decrementer frequency =3D 24.913389 MHz
 time_init: processor frequency   =3D 700.000000 MHz
+Console: colour dummy device 80x25
+Dentry cache hash table entries: 131072 (order: 7, 524288 bytes)
...

and it continues booting as normal.  As a side note it should be known
that I could not coerce the PowerMac ALSA sound driver to compile
without the PMU enabled either, so there probably should be a config
dependency on it if that is the case.

Anything else I can do?

Re: ppc32 kernel boot problem (pmu related?)

[Benjamin Herrenschmidt]

> Using PowerMac machine description
> Total memory = 640MB; using 2048kB for hash table (at cfe00000)
> Linux version 2.6.17-rc3 (root@the_emac) (gcc version 3.4.5 (Gentoo 3.4.5-r1, ss
> p-3.4.5-1.0, pie-8.7.9)) #11 Thu May 11 09:53:39 EDT 2006
> Found UniNorth memory controller & host bridge @ 0xf8000000 revision: 0x11
> Mapped at 0xfdfc0000
> Found a Keylargo mac-io controller, rev: 3, mapped at 0xfdf40000
> Processor NAP mode on idle enabled.
> PowerMac motherboard: eMac
> setup_arch: bootmem
> Found UniNorth PCI host bridge at 0xf0000000. Firmware bus number: 0->0
> Found UniNorth PCI host bridge at 0xf2000000. Firmware bus number: 0->0
> Found UniNorth PCI host bridge at 0xf4000000. Firmware bus number: 0->0
> via-pmu: Server Mode is disabled
> PMU driver v2 initialized for Core99, firmware: 0c
> arch: exit
> Top of RAM: 0x28000000, Total RAM: 0x28000000
> Memory hole size: 0MB
> Built 1 zonelists
> Kernel command line: root=/dev/sda8 ro video=ofonly root=/dev/sda8 rootdelay=10
> resume=/dev/sda9
> mpic: Setting up MPIC " MPIC 1   " version 1.2 at 80040000, max 4 CPUs
> mpic: ISU size: 64, shift: 6, mask: 3f
> mpic: Initializing for 64 sources
> PID hash table entries: 4096 (order: 12, 16384 bytes)
> time_init: decrementer frequency = 24.913389 MHz
> time_init: processor frequency   = 700.000000 MHz
> 
> and then the machine freezes and requires a power cycle to restart.
> Without that option enabled I get this:

Hrm... that's pretty much when console_init() gets called, which means
no message until the fbdev kicks in, thus we just "miss" the messages
from there to the crash.... Can you edit arch/powerpc/kernel/udbg.c,
there is a function called disable_early_printk() which contains a #if
1, turn that into #if 0 and tell me if you get a more useful log after
the time_init thing. Thanks !

Re: ppc32 kernel boot problem (pmu related?)

[Nathan Pilatzke]

Ok, here is the new log:

time_init: processor frequency =3D 700.000000 MHz
Console: colour dummy device 80x25
Dentry cache hash table entries: 131072 (order: 7,524288 bytes)
Inode-cache hash table entries: 65536 (order: 6,262144 bytes)
Memory: 643456k/655360k available (2908k kernel code, 11564k reserved,
272k data, 131k bss, 156k init)
Mount-cache hash table entries: 512
device-tree: Duplicate name in /cpus/PowerPC,G4@0, renamed to "l2-cache#1"
NET: Registered protocol family 16

KeyWest i2c @0xf8001003 irq 42 /uni-n@f8000000/i2c@f8001000
 channel 0 bus <multibus>
 channel 1 bus <multibus>
KeyWest i2c @0x80018000 irq 26 /pci@f2000000/mac-io@17/i2c@18000
 channel 0 bus <multibus>
PMU i2c /pci@f2000000/mac-io@17/via-pmu@16000/pmu-i2c
 channel 1 bus <multibus>
 channel 2 bus <multibus>
PCI: Probing PCI hardware

And that's where is locks hard.  Just as an alternative I tried
compiling without i2c support, but the kernel locks hard at the exact
same point.  Compiling without the pmu code is the only way I can get
the kernel to boot.  There are some kernel messages missing from a
kernel compiled with pmu versus a kernel without pmu, namely:

On node 0 total pages: 163840
 DMA zone: 163840 pages, LIFO batch: 31

and

Calibrating delay loop... 49.79 BogoMIPS (lpj=3D99584)

but they do not seem to be anything serious.  Where to go from here?
Thanks for the debugging suggestions so far, keep them coming please.


On 5/14/06, Benjamin Herrenschmidt <benh@kernel.crashing.org> wrote:
>
> Hrm... that's pretty much when console_init() gets called, which means
> no message until the fbdev kicks in, thus we just "miss" the messages
> from there to the crash.... Can you edit arch/powerpc/kernel/udbg.c,
> there is a function called disable_early_printk() which contains a #if
> 1, turn that into #if 0 and tell me if you get a more useful log after
> the time_init thing. Thanks !
>
>
>

Re: ppc32 kernel boot problem (pmu related?)

[Benjamin Herrenschmidt]

> And that's where is locks hard.  Just as an alternative I tried
> compiling without i2c support, but the kernel locks hard at the exact
> same point.  Compiling without the pmu code is the only way I can get
> the kernel to boot.  There are some kernel messages missing from a
> kernel compiled with pmu versus a kernel without pmu, namely:
> 
> On node 0 total pages: 163840
>  DMA zone: 163840 pages, LIFO batch: 31
> 
> and
> 
> Calibrating delay loop... 49.79 BogoMIPS (lpj=99584)
> 
> but they do not seem to be anything serious.  Where to go from here?
> Thanks for the debugging suggestions so far, keep them coming please.

Can you try editing arch/powerpc/platforms/powermac/low_i2c.c, in
function pmac_i2c_init(), comment out the call to pmac_i2c_devscan() and
tell me if that makes a difference.

Thanks,
Ben.

Re: ppc32 kernel boot problem (pmu related?)

[Benjamin Herrenschmidt]

Oh and also send me a tarball of /proc/device-tree

Ben.

Re: ppc32 kernel boot problem (pmu related?)

[Nathan Pilatzke]

[-- Attachment #1: Type: text/plain, Size: 986 bytes --]

On 5/16/06, Benjamin Herrenschmidt <benh@kernel.crashing.org> wrote:
>
> Can you try editing arch/powerpc/platforms/powermac/low_i2c.c, in
> function pmac_i2c_init(), comment out the call to pmac_i2c_devscan() and
> tell me if that makes a difference.
>
> Thanks,
> Ben.
>
>
Commenting out the call to pmac_i2c_devscan() in function
pmac_i2c_init() allowed the kernel to boot.  I am typing this email
booted under it.  The kernel messages were printed a bit strangely
though.  There was no scrolling or clearing of the screen.  The kernel
messages would start at the top and overwrite old messages, with the
cursor jumping back to the top of the screen every so often.  The
/proc/device-tree is attached for the booting 2.6.17-rc3 kernel with
pmu enabled, as is the new dmesg output.  If you need the
/proc/device-tree from a 2.6.15 working kernel with pmu enabled
instead, let me know.

Is this kernel fairly safe/stable to use?  What can be done next?

Nathan

[-- Attachment #2: device-tree_2.6.17-rc3-pmuworking.tar.bz2 --]
[-- Type: application/x-bzip2, Size: 15848 bytes --]

[-- Attachment #3: dmesg_2.6.17-rc3-pmuworking.bz2 --]
[-- Type: application/x-bzip2, Size: 3942 bytes --]

Re: ppc32 kernel boot problem (pmu related?)

[Benjamin Herrenschmidt]

On Tue, 2006-05-16 at 11:04 -0400, Nathan Pilatzke wrote:
> On 5/16/06, Benjamin Herrenschmidt <benh@kernel.crashing.org> wrote:
> >
> > Can you try editing arch/powerpc/platforms/powermac/low_i2c.c, in
> > function pmac_i2c_init(), comment out the call to pmac_i2c_devscan() and
> > tell me if that makes a difference.
> >
> > Thanks,
> > Ben.
> >
> >
> Commenting out the call to pmac_i2c_devscan() in function
> pmac_i2c_init() allowed the kernel to boot.  I am typing this email
> booted under it.  The kernel messages were printed a bit strangely
> though.  There was no scrolling or clearing of the screen.  The kernel
> messages would start at the top and overwrite old messages, with the
> cursor jumping back to the top of the screen every so often.

That's because of the change you did to udbg.c for debugging :)

> The /proc/device-tree is attached for the booting 2.6.17-rc3 kernel with
> pmu enabled, as is the new dmesg output.  If you need the
> /proc/device-tree from a 2.6.15 working kernel with pmu enabled
> instead, let me know.

No, it should be the same regardless of the kernel

> Is this kernel fairly safe/stable to use?  What can be done next?

Well, I need to figure out what's going on with the i2c stuff... there
is a chip that hangs off the PMU and that says something like "do that
at boot"... but when I do it, it causes the crash you are seeing. So
either I'm not doing what it tells me to do properly or I should just
blacklist this machine (Apple doesn't execute the platform functions
from the device-tree on older machines I think).

I'm investigating, will come back to you.

Ben.