Troubles booting sparc64 SMP 2.4.18 kernel

Troubles booting sparc64 SMP 2.4.18 kernel

[Erik de Castro Lopo]

Hi all,

I've got linux running on an dual CPU E220R. The base system is Debian Woody and 
I've installed the sparc64 compiler and binutils

    root@razor > sparc64-linux-gcc -v
    Reading specs from /usr/lib/gcc-lib/sparc-linux/3.0.3/specs
    Configured with: ../src/configure -v --enable-languages=c --prefix=/usr 
      --infodir=/share/info --mandir=/share/man --enable-shared --with-gnu-as 
      --with-gnu-ld --with-system-zlib --enable-long-long --enable-nls --without-x 
      --without-included-gettext --disable-checking --with-cpu=v7 sparc-linux
    Thread model: single
    gcc version 3.0.3
    root@razor > ld -V
    GNU ld version 2.11.92.0.12.3 20011121 Debian/GNU Linux
      Supported emulations:
       elf32_sparc
       sparclinux
       elf64_sparc
       sun4

I am able to build a working uni-processor sparc64 kernel which boots and works 
as expected :

    root@razor > cat sizeof.c 
    #include <stdio.h>
    
    int main (void)
    {	printf ("sizeof (void*) = %d\n", sizeof (void*)) ;
        return 0 ;
    } /* main */

    root@razor > sparc64-linux-gcc -m64 sizeof.c -o sizeof
    root@razor > ./sizeof 
    sizeof (void*) = 8

This seems to suggest that this is a 64 bit aware kernel.

If I then take the working uni-processor kernel tree, switch
on SMP support and then do 

    make dep ; make clean ; make ; make modules ; 
    make modules_install

and copy the kernel to /boot, run SILO and reboot, it starts off
OK but the seems to fail just after init is run. This is the last 
part of the boot messages:

    IP: routing cache hash table of 4096 buckets, 64Kbytes
    TCP: Hash tables configured (established 32768 bind 32768)
    NET4: Unix domain sockets 1.0/SMP for Linux NET4.0.
    VFS: Mounted root (ext2 filesystem) readonly.
    version 2.84 booting
    cannot ®¨Høcannot fork, retry..
    cannot fork, retry..
    cannot fork, retry..
    ¹½Ñ™½É­É•ÑÉåÿ

I have also tried to boot a 32 bit only SMP kernel but that fails in the
same way. Anybody have any clues?

Cheers,
Erik
-- 
+-----------------------------------------------------------+
  Erik de Castro Lopo  nospam@mega-nerd.com (Yes it's valid)
+-----------------------------------------------------------+
Seen on usenet (possibly a quote from an IBM exec):
"Each large company needs its Vietnam, and Microsoft will
 experience it with NT..."

Re: Troubles booting sparc64 SMP 2.4.18 kernel

[Keith Owens]

On Sun, 17 Mar 2002 11:11:32 +1100, 
Erik de Castro Lopo <nospam@mega-nerd.com> wrote:
>If I then take the working uni-processor kernel tree, switch
>on SMP support and then do 
>
>    make dep ; make clean ; make ; make modules ; 
>    make modules_install

You must make mrproper after switching SMP on/off.  Kernel build 2.4
does not correctly rebuild after changing CONFIG_SMP.

Re: Troubles booting sparc64 SMP 2.4.18 kernel

[Erik de Castro Lopo]

On Sun, 17 Mar 2002 12:17:59 +1100
Keith Owens <kaos@ocs.com.au> wrote:

> On Sun, 17 Mar 2002 11:11:32 +1100, 
> Erik de Castro Lopo <nospam@mega-nerd.com> wrote:
> >If I then take the working uni-processor kernel tree, switch
> >on SMP support and then do 
> >
> >    make dep ; make clean ; make ; make modules ; 
> >    make modules_install
> 
> You must make mrproper after switching SMP on/off.  Kernel build 2.4
> does not correctly rebuild after changing CONFIG_SMP.

Thanks Keith. I think that fixed the problem a little but I now have a new
problem.

I grab a clean 2.4.18 kernel source tree (from a pristine 2.4.18 tarball) and 
copy in the .config file I used to create a working uni-processor kernel from 
the same kernel version. I then do:

  - make menuconfig and enable SMP
  - edit the top level makefile and set EXTRAVERSION to 'smp' so that
    all modules will end up in a different directory to the uni-processor
    kernel
  - make dep
  - make
  - make modules modules_install
  - reboot

The new kernel does this at boot:

    boot: test
    Uncompressing image...
    -
    Remapping the kernel... done.
    Booting Linux...
    Starting CPU 2... OK

and then hangs. I can't even "send brk" to drop it back to the OBP prompt,
I have to power cycle it.

The only thing I can think of that might be causing trouble is that this is
a relatively recent box and the CPUs may be a previously unseen version. Here
are the details (after booting the single uni-processor kernel):

    root@razor > cat /proc/cpuinfo 
    cpu             : TI UltraSparc II  (BlackBird)
    fpu             : UltraSparc II integrated FPU
    promlib         : Version 3 Revision 23
    prom            : 3.23.1
    type            : sun4u
    ncpus probed    : 2
    ncpus active    : 1
    Cpu0Bogo        : 897.84
    Cpu0ClkTck      : 000000001ad2819f

This machine runs happily on the single processor 2.4.18 kernel I built. It also
runs Solaris on 2 processors without a hitch. I added a debug print statements
and found that the last place I get any useful behaviour out of the machine is
just before calling cpu_idel().

Anybody got any clues on how to debug this further?

Cheers,
Erik
-- 
+-----------------------------------------------------------+
  Erik de Castro Lopo  nospam@mega-nerd.com (Yes it's valid)
+-----------------------------------------------------------+
The National Multiple Sclerosis Society of America recently started an
advertising campaign with the slogan "MS: It's not a software company".

Seasoned IT professionals will have no trouble telling the two MS's 
apart. One is a debilitating and surprisingly widespread affliction 
that renders the sufferer barely able to perform the simplest task. 
The other is a disease.

Re: Troubles booting sparc64 SMP 2.4.18 kernel

[David S. Miller]

Please try 2.4.19-pre3 or later.  There were some problems in 2.4.18
vanilla on sparc64.

Re: Troubles booting sparc64 SMP 2.4.18 kernel

[Erik de Castro Lopo]

On Tue, 19 Mar 2002 00:19:46 -0800 (PST)
"David S. Miller" <davem@redhat.com> wrote:

> 
> Please try 2.4.19-pre3 or later.  There were some problems in 2.4.18
> vanilla on sparc64.

Thanks Dave,

I had to fix a few things to get this compiling:

   1) Edited include/asm-sparc64/pgtable.h adding :

      #define flush_icache_user_range(vma,pg,adr,len)   do { } while (0)

   2) I got compile errors in the ppp, ide and usb areas so I disabled all of
      these (I don't need them at the moment).

I then got a working compile with an SMP kernel which still didn't boot. The
last boot message I see is this:

    ttyS01 at 0x1fff1400040 (irq = 12,7eb) is a SAB82532 V3.2
    Console: ttyS0 (SAB82532)
    power: Control reg at 000001fff1724000 ... not using powerd.
    Linux NET4.0 for Linux 2.4
    Based upon Swansea University Computer Society NET3.039
    Initializing RT netlink socket
    Starting kswapd
    pty: 256 Unix98 ptys configured
    rtc_init: no PC rtc found
    block: 128 slots per queue, batch2
    sunhme.c:v1.99 12/Sep/99 David S. Miller (davem@redhat.com)
    eth0: HAPPY MEAL (PCI/CheerIO) 10/100BaseT Ethernet 08:00:20:a1:70:11 
    Linux video capture interface: v1.00
    SCSI subsystem driver Revision: 1.00
    qlogicfc0 : new isp2x00 revision ID (4)
    qlogicfc0 : link is not up

I am now going to try and disable fiber channel support and see if I can get 
it any further.

Cheers,
Erik
-- 
+-----------------------------------------------------------+
  Erik de Castro Lopo  nospam@mega-nerd.com (Yes it's valid)
+-----------------------------------------------------------+
The mouse has been moved. Windows must be restarted for the change
to take effect. Reboot now?

Re: Troubles booting sparc64 SMP 2.4.18 kernel

[David S. Miller]

   From: Erik de Castro Lopo <nospam@mega-nerd.com>
   Date: Wed, 20 Mar 2002 06:59:21 +1100

       qlogicfc0 : new isp2x00 revision ID (4)
       qlogicfc0 : link is not up
   
   I am now going to try and disable fiber channel support and see if
   I can get it any further.

It could definitely be the problem.  Where is this fibre channel
interface, on the motherboard or on a PCI card?  If so, what is
this PCI card?

Re: Troubles booting sparc64 SMP 2.4.18 kernel

[Erik de Castro Lopo]

On Tue, 19 Mar 2002 14:30:54 -0800 (PST)
"David S. Miller" <davem@redhat.com> wrote:

> It could definitely be the problem.  Where is this fibre channel
> interface, on the motherboard or on a PCI card?  

I don't currently have physical access to this machine so I can't tell you 
this till tomorrow.

> If so, what is this PCI card?

Its the second one from the bottom.

root@razor > lspci 
00:00.0 Host bridge: Sun Microsystems Computer Corp. Psycho PCI Bus Module
00:01.0 Bridge: Sun Microsystems Computer Corp. EBUS (rev 01)
00:01.1 Ethernet controller: Sun Microsystems Computer Corp. Happy Meal (rev 01)
00:03.0 SCSI storage controller: LSI Logic / Symbios Logic (formerly NCR) 53c875 (rev 14)
00:03.1 SCSI storage controller: LSI Logic / Symbios Logic (formerly NCR) 53c875 (rev 14)
00:04.0 SCSI storage controller: QLogic Corp. QLA2100 64-bit Fibre Channel Adapter (rev 04)
01:00.0 Host bridge: Sun Microsystems Computer Corp. Psycho PCI Bus Module

Erik
-- 
+-----------------------------------------------------------+
  Erik de Castro Lopo  nospam@mega-nerd.com (Yes it's valid)
+-----------------------------------------------------------+
"Software is largely a service industry operating under the persistent 
but unfounded delusion that it is a manufacturing industry."
-- Eric S. Raymond

Re: Troubles booting sparc64 SMP 2.4.18 kernel

[David S. Miller]

   From: Erik de Castro Lopo <nospam@mega-nerd.com>
   Date: Wed, 20 Mar 2002 10:55:22 +1100

   On Tue, 19 Mar 2002 14:30:54 -0800 (PST)
   "David S. Miller" <davem@redhat.com> wrote:
   
   > It could definitely be the problem.  Where is this fibre channel
   > interface, on the motherboard or on a PCI card?  
   
   I don't currently have physical access to this machine so I can't tell you 
   this till tomorrow.
   
If you send me '/usr/sbin/prtconf -pv' I can figure it out without
you having physical access to the machine :-)

Re: Troubles booting sparc64 SMP 2.4.18 kernel

[Erik de Castro Lopo]

On Tue, 19 Mar 2002 14:30:54 -0800 (PST)
"David S. Miller" <davem@redhat.com> wrote:

>    From: Erik de Castro Lopo <nospam@mega-nerd.com>
>    Date: Wed, 20 Mar 2002 06:59:21 +1100
> 
>        qlogicfc0 : new isp2x00 revision ID (4)
>        qlogicfc0 : link is not up
>    
>    I am now going to try and disable fiber channel support and see if
>    I can get it any further.
> 
> It could definitely be the problem.  

The other weird thing is that even though I have disabled "Fibre Channel 
support" and "Fusion MPT device support" in make menuconfig and did a make
clean before doing make dep, I still get this qlogicfc0 device in the
boot messages. I have even made sure that I deleted al qlogic* drivers
in the /lib/modules/2.4.19-pre3 tree.

To prevent this driver from being compiled into the kernel I had to copy 
the .config file out, do "make mrproper", copy the .config file back in,
make oldconfig and then "make dep" etc.

The upshot of all this is that I do finnaly have this machine booting a
SMP Sparc64 kernel :-) although IDE, PPOE, USB and fiber channel are 
disabled. I can start doing the work on this machine that i was hoping 
to do :->.

BTW, this device:

   00:04.0 SCSI storage controller: QLogic Corp. QLA2100 64-bit Fibre Channel Adapter (rev 04)

is indeed a plugin PCI card. I just asked someone to have a look at the
back end of the machine :-).

Here's the "/usr/sbin/prtconf -pv" (hey I thought that was a Solaris 
command ???) output:

    https://mega-nerd.net/prtconf-pv.txt


Erik
-- 
+-----------------------------------------------------------+
  Erik de Castro Lopo  nospam@mega-nerd.com (Yes it's valid)
+-----------------------------------------------------------+
Reporter: "What do you think of Western Civilisation?" 
M.K. Gandhi: "I think it would be a good idea." 

Re: Troubles booting sparc64 SMP 2.4.18 kernel

[David S. Miller]

   From: Erik de Castro Lopo <erikd@mega-nerd.com>
   Date: Wed, 20 Mar 2002 15:11:20 +1100

   The other weird thing is that even though I have disabled "Fibre Channel 
   support" and "Fusion MPT device support" in make menuconfig and did a make
   clean before doing make dep, I still get this qlogicfc0 device in the
   boot messages. I have even made sure that I deleted al qlogic* drivers
   in the /lib/modules/2.4.19-pre3 tree.
   
It is listed under SCSI as CONFIG_SCSI_QLOGICFC, not fibre channel.

   Here's the "/usr/sbin/prtconf -pv" (hey I thought that was a Solaris 
   command ???) output:
   
       https://mega-nerd.net/prtconf-pv.txt
   
It is a Solaris command, we just reimplemented it for Linux.
Thanks.

Re: Troubles booting sparc64 SMP 2.4.18 kernel

[David S. Miller]

   From: Erik de Castro Lopo <erikd@mega-nerd.com>
   Date: Wed, 20 Mar 2002 15:11:20 +1100
   
[ Recap, his system won't boot with this QLA2100 card installed
  if drivers/net/qlogicfc.o tries to make use of it. ]

   BTW, this device:
   
      00:04.0 SCSI storage controller: QLogic Corp. QLA2100 64-bit Fibre Channel Adapter (rev 04)
   
   is indeed a plugin PCI card. I just asked someone to have a look at the
   back end of the machine :-).

Erik, would you be willing to give some patches a try to get
this card working?  After talking with Matt Jacob, we decided that
testing his driver out will help us isolate why this is failing.

Thanks.