SMP "unexpected IO-APIC"

[<u.milde@gdsys.de>]

Hello,

while booting a linux SMP kernel it advises me with

"WARNING: unexpected IO-APIC, please mail
          to linux-smp@verger.kernel.org"

And so I do. :-)

[warning: lengthy description, jump to the end when it gets tedious ;-)]

The system is a Asus CUV4X-D motherboard with VIA Apollo Pro 133A
chipset. The CPUs are two identical Pentium III Coppermine stepping 06.

Additonal hardware is a 3com 3c905C network adapter, IDE harddisk,
IDE DVD-ROM drive and a Matrox G450 graphics board and part of the
time a AVM B1 ISDN adapter.
SDRAM is a PC133 compliant 256MB module.

The platform is a fresh RedHat 7.1 installation, which was then updated
with the appropriate RPMs step by step after the problems first showed.

What works: Using one of Redhat's kernel RPMs of kernel 2.4.2, 2.4.3-
12, 2.4.9-31 for single processor work fine.

What doesn't work:
Using Redhat's *SMP versions of the same kernels will halt with the
warning message as above. To be more concrete that message appears
after  CPU1 has been found, the total of 2 processors has been
activated and then for Enabling the IO-APIC IRQS it is
"testing the IO APIC........................
  WARNING: unexpected IO-APIC, please mail
           to linux-smp@vger.kernel.org
 ....................... done."

After the warning message a few more lines appear 
"Using local APIC timer interrupts.;
 calibrating APIC time ...;
 ... CPU clock speed is 1004.5519 MHz.;
 ... host bus clock speed is 133.9398 MHz.;
 cpu: 0, clocks: 1339398, slice: 446466;
 CPU0<T0_1339392,T1892912,D:14,S:446466,C:1339398>"

and additionally in one case (rebooting often enough, with
 RH 2.4.9-31smp):
[...]
 cpu: 0, clocks: 1339227, slice: 446409
 CPU0<T0:1339216,T1:892800,D:7,S:446409,C:1339227
 cpu: 1, clocks: 1339227, slice: 446409
 CPU1<T0:1339216,T1:446384,D:14,S:446409,C:1339227
 checking TSC synchronization across XPUs:
 BIOS BUG: CPU#0 improperly initialized, has -5 usecs TSC skew! FIXED.
 BIOS BUG: CPU#1 improperly initialized, has 5 usecs TSC skew! FIXED.
 PCI: PCI BIOS revision 2.10 entry at 0xf0d20, last bus=1
 PCI: Using configuration type 1
 PCI: Probing PCI hardware
 Unknown bridge resource 0: assuming transparent
 PCI: Using IRQ router VIA [1106/0686] at 00:04.0
 PCI->APIC IRQ transform: (B0,I4,P3) -> 18
 PCI->APIC IRQ transform: (B0,I4,P3) -> 18
 PCI->APIC IRQ transform: (B0,I10,P0) -> 18
 PCI->APIC IRQ transform: (B0,I11,P0) -> 17
 PCI->APIC IRQ transform: (B1,I0,P0) -> 16
 PCI: Enabling Via external APIC routing
 PCI: Via IRQ fixup for 00:04.2, from 5 to 2
 PCI: Via IRQ fixup for 00:04.3, from 5 to 2
 isapnp: Scanning for PnP cards...
*hangs*


What I tried:
I tried removing additional hardware, namely the 3com network adapter,
the ISDN B1 wasn't present most of the time.
I tried removing the second CPU, it'll still hang after the IO-APIC
initialisation.
Clocked down CPU and RAM to 500MHz and 100MHz respectively. It's slower
but nothing else.
Tried out several CUV4X-D BIOS releases up to the current 1014
(formerly known as 1014beta5).

I tried obtaining newer Kernels (so far) up to 2.4.10-pre12 and playing
with compilation options. With the result that I can pack anything I
want into it or remove support for just about anything as long as SMP
stays switched off and it'll work (within the set limits). Compile SMP
support and it'll hang during boot -- oddly enough, though, not as
'reliably' in the same spot as the Redhat kernels do but just
*somewhere* after the IO-APIC init. Add or remove something from the
kernel and it seems to get a bit further or not.

Switching to the final 2.4.18 pretty much confirmed the above.
I compiled the kernel taking out most fancy stuff (ISDN-support and
 whatnot) and started adding options in the CPU and General section of
/menuconfig/. It wasn't necessary to enable SMP-support, but boot
already fails with "Local APIC" *and* "IO-APIC" selected. The boot
output looks slightly different from that described above:
[...]
Intel MultiProcessor Specification v1.4
   Virtual Wire compatibility mode.
OEM ID: OEM00000 Product ID: PROD0000000 APIC at: 0xFEE00000
Processor #3 Pentium(tm) Pro APIC version 17
Processor #0 Pentium(tm) Pro PAIC version 17
I/O APIC #2 Version 17 at 0xFEC00000.
Processors: 2
Initializing CPU#0
[... all still relating to CPU#0 ...]
 ENABLING IO-APIC IRQs
 Setting 2 in the phys_id_present_map
 ..changing IO-APIC physical APID_ID to 2 ... ok.
 ..TIMER: vector=0x31 pin1=2 pin2=0
 testing the IO APIC........................
 
 ....................... done.
 Using local APIC timer interrupts.
 calibrating APIC timer ...
 ..... CPU clock speed is 1004.5734 MHz.
 ..... host bus clock speed is 133.9428 MHz.
 cpu: 0, clocks: 1339428, slice: 669714

and then the machine hangs.


Getting a bit desperate I installed Windows 2000 Pro on the same
machine, and 'lo it detects and works with both CPUs.
So I suspect a hardware defect is quite unlikely.



So, my question is, would you suspect the behaviour is likely to result
from some bug or not-yet-implemented support/feature in the kernel? Or
am I totally on the wrong track and should look into something else? If
so, any idea into what?
What further info, if any, do you need?

Best regards
 Uwe Milde


PS: A Google search revealed for that motherboard following comment: "I
 had to configure the BIOS to use MP 1.1 (not the default 1.4) to make
 it work. Otherwise the systems hangs somewhere in the boot procedure."
 Now I am not quite sure how this is meant, otherwise I'd try it out.
 Of course I have noticed, as of the output quoted above, the version
 1.4 does show up...

PPS: Requests for comments from Asus on the matter have yet failed to
 produce a response.