APIC version and 8-bit APIC IDs

APIC version and 8-bit APIC IDs

[Martin Wilck]

Hi William, hello everyone,

The MP_valid_apicid() function [arch/i386/kernel/mpparse.c] checks 
whether the APIC version field is >=20 in order to determine whether the 
CPU supports 8-bit physical APIC ids.

We currently have two modern processors oin our labs (Intel Xeon MP, AMD 
Dual-Core Opteron 875) for which this test is wrong because their APIC 
ids are both 16, but they _do_ support 8-bit APIC ids. This leads to 
erratic error messages and to valid CPUs not being detected.

Unfortunately I cannot tell why this is so, and what test should be used 
instead to make sure a CPU supports 8-bit APIC IDs.

The AMD BIOS and kernel developer's guide for Athlon64 and Opteron 
processors says

"When both ApicExtId and ApicExtBrdCst in the HyperTransport" 
Transaction Control Register are set, all 8 bits of APIC ID are used."

This refers to the TCR register. Reading that would require PCI 
configuration space access before the APICs are set up, I don't know if 
that's possible.
(http://www.amd.com/us-en/assets/content_type/white_papers_and_tech_docs/26094.PDF)

The Intel Manual simply says
"For the Pentium 4 and Intel Xeon Processors, the xAPIC specification 
extends the local APIC field to 8 bits". The CPUs we have are Xeon MP 
(family 15, model 4); their local APIC version is 16, and they do 
support 8-bit APIC-IDs.

I guess it's up to the Intel an AMD people to have a final word on this,
but the current implementation is clearly wrong for these latest CPU types.

Regards
Martin

-- 
Martin Wilck                Phone: +49 5251 8 15113
Fujitsu Siemens Computers   Fax:   +49 5251 8 20409
Heinz-Nixdorf-Ring 1        mailto:Martin.Wilck@Fujitsu-Siemens.com
D-33106 Paderborn           http://www.fujitsu-siemens.com/primergy

Re: APIC version and 8-bit APIC IDs

[Andi Kleen]

Martin Wilck <martin.wilck@fujitsu-siemens.com> writes:

> Hi William, hello everyone,
> 
> The MP_valid_apicid() function [arch/i386/kernel/mpparse.c] checks
> whether the APIC version field is >=20 in order to determine whether
> the CPU supports 8-bit physical APIC ids.

Yes, it's broken. In fact I removed it in my physflat32 patch
which is needed for 16 core AMD systems. I don't think there
is a generic way to fix it because the XAPIC check breaks
on AMD systems and there is no good way to decide early 
on subarchitectures before doing this check. Also it's only
a sanity check for broken BIOS, and in this case it causes more problems
than it solves.

ftp://ftp.firstfloor.org/pub/ak/x86_64/x86_64-2.6.13rc3-1/patches/physflat32

Will hopefully be fixed in 2.6.14.

-Andi

Re: APIC version and 8-bit APIC IDs

[Martin Wilck]

Andi Kleen wrote:

> Yes, it's broken. In fact I removed it in my physflat32 patch
> which is needed for 16 core AMD systems. I don't think there
> is a generic way to fix it because the XAPIC check breaks
> on AMD systems 

on the Intel Xeon MP systems, too,

> and there is no good way to decide early 
> on subarchitectures before doing this check. Also it's only
> a sanity check for broken BIOS, and in this case it causes more problems
> than it solves.

agreed.

> ftp://ftp.firstfloor.org/pub/ak/x86_64/x86_64-2.6.13rc3-1/patches/physflat32

That is a beautiful patch, thank you.

Only one small point: I wonder whether it is correct to use the number 
of CPUs as criterion for this architecture. AFAICS, the Specs allow
having only 4 CPUS, but giving them APIC IDs e.g. 16,17,18,19. In this 
case, physflat32 should be used as well (in particular, the APIC ID 
broadcast and mask must be set to 0xff).

> Will hopefully be fixed in 2.6.14.

Great,

Martin

-- 
Martin Wilck                Phone: +49 5251 8 15113
Fujitsu Siemens Computers   Fax:   +49 5251 8 20409
Heinz-Nixdorf-Ring 1        mailto:Martin.Wilck@Fujitsu-Siemens.com
D-33106 Paderborn           http://www.fujitsu-siemens.com/primergy

Re: APIC version and 8-bit APIC IDs

[Andi Kleen]

On Fri, Aug 12, 2005 at 03:21:00PM +0200, Martin Wilck wrote:
> >Yes, it's broken. In fact I removed it in my physflat32 patch
> >which is needed for 16 core AMD systems. I don't think there
> >is a generic way to fix it because the XAPIC check breaks
> >on AMD systems 
> 
> on the Intel Xeon MP systems, too,

How so? The XAPIC version check should work there.

The problem on AMD happens because it reports an old APIC version.

> 
> >and there is no good way to decide early 
> >on subarchitectures before doing this check. Also it's only
> >a sanity check for broken BIOS, and in this case it causes more problems
> >than it solves.
> 
> agreed.
> 
> >ftp://ftp.firstfloor.org/pub/ak/x86_64/x86_64-2.6.13rc3-1/patches/physflat32
> 
> That is a beautiful patch, thank you.

It'll probably be revamped somewhat before inclusion. In particular
it has been suggested to merge it with bigsmp because the setup
should work on Intel too.

> 
> Only one small point: I wonder whether it is correct to use the number 
> of CPUs as criterion for this architecture. AFAICS, the Specs allow
> having only 4 CPUS, but giving them APIC IDs e.g. 16,17,18,19. In this 
> case, physflat32 should be used as well (in particular, the APIC ID 
> broadcast and mask must be set to 0xff).

I fixed that in the 64bit version already, but not in 32bit yet.
Yes, the value of the APIC IDs must be used as indicator.

-Andi

Re: APIC version and 8-bit APIC IDs

[Martin Wilck]

Andi Kleen wrote:

>>on the Intel Xeon MP systems, too,
> 
> How so? The XAPIC version check should work there.

ACPI: LAPIC (acpi_id[0x11] lapic_id[0x21] enabled)
Processor #33 15:4 APIC version 16
ACPI: LAPIC (acpi_id[0x12] lapic_id[0x26] enabled)
Processor #38 15:4 APIC version 16

This is a boot message excerpt from one of the "Potomac" systems we 
have. x86_64 kernel without xapic check, therefore the CPUs are 
activated, but they wouldn't be on recent i386.

> I fixed that in the 64bit version already, but not in 32bit yet.
> Yes, the value of the APIC IDs must be used as indicator.

Great.
Regards, Martin


-- 
Martin Wilck                Phone: +49 5251 8 15113
Fujitsu Siemens Computers   Fax:   +49 5251 8 20409
Heinz-Nixdorf-Ring 1        mailto:Martin.Wilck@Fujitsu-Siemens.com
D-33106 Paderborn           http://www.fujitsu-siemens.com/primergy

Re: APIC version and 8-bit APIC IDs

[Martin Wilck]

I wrote:

>> How so? The XAPIC version check should work there.
> 
> ACPI: LAPIC (acpi_id[0x11] lapic_id[0x21] enabled)
> Processor #33 15:4 APIC version 16
> ACPI: LAPIC (acpi_id[0x12] lapic_id[0x26] enabled)
> Processor #38 15:4 APIC version 16

Forget it. I have fallen prey to  this line:

	processor.mpc_apicver = 0x10; /* TBD: lapic version */

in arch/x86_64/kernel/mpparse.c.
I am used to get correct answers from Linux :-)

Cheers
Martin

-- 
Martin Wilck                Phone: +49 5251 8 15113
Fujitsu Siemens Computers   Fax:   +49 5251 8 20409
Heinz-Nixdorf-Ring 1        mailto:Martin.Wilck@Fujitsu-Siemens.com
D-33106 Paderborn           http://www.fujitsu-siemens.com/primergy

Re: APIC version and 8-bit APIC IDs

[Andi Kleen]

On Fri, Aug 12, 2005 at 04:55:40PM +0200, Martin Wilck wrote:
> I wrote:
> 
> >>How so? The XAPIC version check should work there.
> >
> >ACPI: LAPIC (acpi_id[0x11] lapic_id[0x21] enabled)
> >Processor #33 15:4 APIC version 16
> >ACPI: LAPIC (acpi_id[0x12] lapic_id[0x26] enabled)
> >Processor #38 15:4 APIC version 16
> 
> Forget it. I have fallen prey to  this line:
> 
> 	processor.mpc_apicver = 0x10; /* TBD: lapic version */
> 
> in arch/x86_64/kernel/mpparse.c.
> I am used to get correct answers from Linux :-)

Heh. Should probably fix that. Can you file a bug with the ACPI
people on http://bugzilla.kernel.org ? (or do a patch?)

-Andi

Re: APIC version and 8-bit APIC IDs

[yhlu]

So MPTABLE do not have problem with it, only acpi related...?

YH

On 8/12/05, Andi Kleen <ak@suse.de> wrote:
> On Fri, Aug 12, 2005 at 04:55:40PM +0200, Martin Wilck wrote:
> > I wrote:
> >
> > >>How so? The XAPIC version check should work there.
> > >
> > >ACPI: LAPIC (acpi_id[0x11] lapic_id[0x21] enabled)
> > >Processor #33 15:4 APIC version 16
> > >ACPI: LAPIC (acpi_id[0x12] lapic_id[0x26] enabled)
> > >Processor #38 15:4 APIC version 16
> >
> > Forget it. I have fallen prey to  this line:
> >
> >       processor.mpc_apicver = 0x10; /* TBD: lapic version */
> >
> > in arch/x86_64/kernel/mpparse.c.
> > I am used to get correct answers from Linux :-)
> 
> Heh. Should probably fix that. Can you file a bug with the ACPI
> people on http://bugzilla.kernel.org ? (or do a patch?)
> 
> -Andi
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: APIC version and 8-bit APIC IDs

[Andi Kleen]

On Fri, Aug 12, 2005 at 09:37:11AM -0700, yhlu wrote:
> So MPTABLE do not have problem with it, only acpi related...?

It's only a cosmetic problem I think with the printk being
wrong. The actual decision in the code should all use the true
value.

Another way would be to just remove the printk output.

-Andi

Re: APIC version and 8-bit APIC IDs

[yhlu]

why matrin need to make apic id to be greater than 0x10 when system is
only 2way?

too much io-apic in system?

YH

On 8/12/05, Andi Kleen <ak@suse.de> wrote:
> On Fri, Aug 12, 2005 at 09:37:11AM -0700, yhlu wrote:
> > So MPTABLE do not have problem with it, only acpi related...?
> 
> It's only a cosmetic problem I think with the printk being
> wrong. The actual decision in the code should all use the true
> value.
> 
> Another way would be to just remove the printk output.
> 
> -Andi
>

Re: APIC version and 8-bit APIC IDs

[Siddha, Suresh B]

On Fri, Aug 12, 2005 at 04:57:25PM +0200, Andi Kleen wrote:
> On Fri, Aug 12, 2005 at 04:55:40PM +0200, Martin Wilck wrote:
> > I wrote:
> > 
> > >>How so? The XAPIC version check should work there.
> > >
> > >ACPI: LAPIC (acpi_id[0x11] lapic_id[0x21] enabled)
> > >Processor #33 15:4 APIC version 16
> > >ACPI: LAPIC (acpi_id[0x12] lapic_id[0x26] enabled)
> > >Processor #38 15:4 APIC version 16
> > 
> > Forget it. I have fallen prey to  this line:
> > 
> > 	processor.mpc_apicver = 0x10; /* TBD: lapic version */
> > 
> > in arch/x86_64/kernel/mpparse.c.
> > I am used to get correct answers from Linux :-)
> 
> Heh. Should probably fix that. Can you file a bug with the ACPI
> people on http://bugzilla.kernel.org ? (or do a patch?)

Here is the patch.

--
Fix the apic version that gets printed during boot. 

Signed-off-by: Suresh Siddha <suresh.b.siddha@intel.com>

--- linux-2.6.13-rc6/arch/x86_64/kernel/mpparse.c~	2005-08-12 10:19:07.037696416 -0700
+++ linux-2.6.13-rc6/arch/x86_64/kernel/mpparse.c	2005-08-12 10:19:38.098974384 -0700
@@ -707,7 +707,7 @@
 
 	processor.mpc_type = MP_PROCESSOR;
 	processor.mpc_apicid = id;
-	processor.mpc_apicver = 0x10; /* TBD: lapic version */
+	processor.mpc_apicver = GET_APIC_VERSION(apic_read(APIC_LVR));
 	processor.mpc_cpuflag = (enabled ? CPU_ENABLED : 0);
 	processor.mpc_cpuflag |= (boot_cpu ? CPU_BOOTPROCESSOR : 0);
 	processor.mpc_cpufeature = (boot_cpu_data.x86 << 8) | 

Re: APIC version and 8-bit APIC IDs

[Andi Kleen]

> --- linux-2.6.13-rc6/arch/x86_64/kernel/mpparse.c~	2005-08-12 10:19:07.037696416 -0700
> +++ linux-2.6.13-rc6/arch/x86_64/kernel/mpparse.c	2005-08-12 10:19:38.098974384 -0700
> @@ -707,7 +707,7 @@
>  
>  	processor.mpc_type = MP_PROCESSOR;
>  	processor.mpc_apicid = id;
> -	processor.mpc_apicver = 0x10; /* TBD: lapic version */
> +	processor.mpc_apicver = GET_APIC_VERSION(apic_read(APIC_LVR));

That code executes on the BP. What happens when different CPUs have
different APIC versions?  Not sure if that can happen, but it 
still looks a bit like "cheating.

-Andi

Re: APIC version and 8-bit APIC IDs

[Siddha, Suresh B]

On Fri, Aug 12, 2005 at 08:22:28PM +0200, Andi Kleen wrote:
> > --- linux-2.6.13-rc6/arch/x86_64/kernel/mpparse.c~	2005-08-12 10:19:07.037696416 -0700
> > +++ linux-2.6.13-rc6/arch/x86_64/kernel/mpparse.c	2005-08-12 10:19:38.098974384 -0700
> > @@ -707,7 +707,7 @@
> >  
> >  	processor.mpc_type = MP_PROCESSOR;
> >  	processor.mpc_apicid = id;
> > -	processor.mpc_apicver = 0x10; /* TBD: lapic version */
> > +	processor.mpc_apicver = GET_APIC_VERSION(apic_read(APIC_LVR));
> 
> That code executes on the BP. What happens when different CPUs have
> different APIC versions?  Not sure if that can happen, but it 
> still looks a bit like "cheating.

Not sure if we ever support CPUs with different APIC versions. That will
probably require some ACPI SPEC changes too..

I would say, for now just follow the i386 code.

thanks,
suresh

Re: APIC version and 8-bit APIC IDs

[Andi Kleen]

> Not sure if we ever support CPUs with different APIC versions. That will
> probably require some ACPI SPEC changes too..
> 
> I would say, for now just follow the i386 code.

Ok I applied the patch. Thanks.

-Andi

Re: APIC version and 8-bit APIC IDs

[Martin Wilck]

yhlu wrote:
> why matrin need to make apic id to be greater than 0x10 when system is
> only 2way?

It's a scalable system where multiple boards may be combined. Anyway, I 
see nothing in the specs that says you must start counting CPUs from zero.

Regards
Martin



-- 
Martin Wilck                Phone: +49 5251 8 15113
Fujitsu Siemens Computers   Fax:   +49 5251 8 20409
Heinz-Nixdorf-Ring 1        mailto:Martin.Wilck@Fujitsu-Siemens.com
D-33106 Paderborn           http://www.fujitsu-siemens.com/primergy

Re: APIC version and 8-bit APIC IDs

[Maciej W. Rozycki]

On Mon, 22 Aug 2005, Martin Wilck wrote:

> It's a scalable system where multiple boards may be combined. Anyway, I see
> nothing in the specs that says you must start counting CPUs from zero.

 Well, Intel's "Multiprocessor Specification" mandates that (see section 
3.6.1 and also the compliance list in Appendix C).  I does not mandate 
local APIC IDs to be consecutive though.

  Maciej

Re: APIC version and 8-bit APIC IDs

[Martin Wilck]

Maciej W. Rozycki wrote:

>  Well, Intel's "Multiprocessor Specification" mandates that (see section 
> 3.6.1 and also the compliance list in Appendix C).  I does not mandate 
> local APIC IDs to be consecutive though.

Unless I am mistaken, the MP spec does not say that _CPUs_ must start 
from 0. We had an IO-APIC at 0. The MP spec says that the IDs must be 
unique (I am told this isn't true any more because an IO APIC and a CPU 
may have the same ID) and _need not_ be consecutive.

We tried different setups; one had IO APICs at 0,1,2 and CPUs starting 
at 16. I can't see that this is forbidden (the reason is that the 
IO-APICs have only 4-bit APIC ID registers). Anyway we changed it now to 
have both IO-APICs and CPUs start at 0.

Regards
Martin

-- 
Martin Wilck                Phone: +49 5251 8 15113
Fujitsu Siemens Computers   Fax:   +49 5251 8 20409
Heinz-Nixdorf-Ring 1        mailto:Martin.Wilck@Fujitsu-Siemens.com
D-33106 Paderborn           http://www.fujitsu-siemens.com/primergy

Re: APIC version and 8-bit APIC IDs

[Maciej W. Rozycki]

On Fri, 26 Aug 2005, Martin Wilck wrote:

> Unless I am mistaken, the MP spec does not say that _CPUs_ must start from 0.
> We had an IO-APIC at 0. The MP spec says that the IDs must be unique (I am
> told this isn't true any more because an IO APIC and a CPU may have the same
> ID) and _need not_ be consecutive.

 You are unfortunately mistaken -- the spec is explicit about *local* APIC 
IDs having to start at 0.  There are at least two places in the spec that 
refer to that.

> We tried different setups; one had IO APICs at 0,1,2 and CPUs starting at 16.
> I can't see that this is forbidden (the reason is that the IO-APICs have only
> 4-bit APIC ID registers). Anyway we changed it now to have both IO-APICs and
> CPUs start at 0.

 You can always assign 0, 16, 17, etc. to local APICs and then 1, 2, 3, 
etc. for I/O APICs.  Frankly I don't know what the actual justification 
behind the requirement is.  Note that the ID of 0 need not necessarily 
belong to the BSP.

  Maciej

Re: APIC version and 8-bit APIC IDs

[Martin Wilck]

Maciej W. Rozycki wrote:

>  You are unfortunately mistaken -- the spec is explicit about *local* APIC 
> IDs having to start at 0.  There are at least two places in the spec that 
> refer to that.

I see. Thanks for the clarification.

>  You can always assign 0, 16, 17, etc. to local APICs and then 1, 2, 3, 
> etc. for I/O APICs.  

Actually, that's what we did (I messed thinks up in my posting). I think 
that's an ugly configuration, though, and totally misleading in terms of 
system topology.

> Frankly I don't know what the actual justification 
> behind the requirement is.  Note that the ID of 0 need not necessarily 
> belong to the BSP.

Starting both local APICs and IO-APICs at 0 isa working and 
aesthetically pleasing solution. We're pursuing that now.

Thanks again, Martin

-- 
Martin Wilck                Phone: +49 5251 8 15113
Fujitsu Siemens Computers   Fax:   +49 5251 8 20409
Heinz-Nixdorf-Ring 1        mailto:Martin.Wilck@Fujitsu-Siemens.com
D-33106 Paderborn           http://www.fujitsu-siemens.com/primergy

Re: APIC version and 8-bit APIC IDs

[Martin Wilck]

Hi Andi, hi everyone,

>>The MP_valid_apicid() function [arch/i386/kernel/mpparse.c] checks
>>whether the APIC version field is >=20 in order to determine whether
>>the CPU supports 8-bit physical APIC ids.
> 
> Yes, it's broken. ... . Also it's only
> a sanity check for broken BIOS, and in this case it causes more problems
> than it solves.

We have another issue with the APIC IDs: the GET_APIC_ID and 
APIC_ID_MASK macros from the subarch code. In all subarchs except summit 
and es7000, these macros use a hard-coded mask 0x0F for the APIC ID. 
Thus, any APIC ID >15 will be truncated, leading to obscure errors.

We are wondering why these masks are there in the subarch code at all. 
After all, whether or not 8-bit APIC IDs are supported depends mainly on 
the CPU type used. Why wouldn't it possible to have a "default" 
architecture with APIC IDs > 15, if the CPUs allow it?

In other words: What would be broken if we just used an APIC ID mask of 
0xFF everywhere?

The current situation with MP_valid_apicid() on the one hand (masking 
the APIC ID as a function of local APIC version) and APIC_ID_MASK 
(masking the APIC as a function of subarch) on the other hand is 
inconsistent. A correct approach must take both CPU and architecture 
constraints into account, and use a CPU-type-dependent variable mask in 
the subarch code.

Regards
Martin

-- 
Martin Wilck                Phone: +49 5251 8 15113
Fujitsu Siemens Computers   Fax:   +49 5251 8 20409
Heinz-Nixdorf-Ring 1        mailto:Martin.Wilck@Fujitsu-Siemens.com
D-33106 Paderborn           http://www.fujitsu-siemens.com/primergy

Re: APIC version and 8-bit APIC IDs

[Maciej W. Rozycki]

On Wed, 31 Aug 2005, Martin Wilck wrote:

> We are wondering why these masks are there in the subarch code at all. After
> all, whether or not 8-bit APIC IDs are supported depends mainly on the CPU
> type used. Why wouldn't it possible to have a "default" architecture with APIC
> IDs > 15, if the CPUs allow it?

 It actually depends on the APIC type, rather than the CPU.  E.g. with 
Pentium systems the width of the ID is either 4 bits or 8 bits, depending 
on whether the integrated or an external 82489DX APIC is used.  This 
should be able to be determined by the APIC version; for v <= 0xf the ID 
is 8-bit and for v >= 0x10 it used to be 4-bit.  Now you only need to 
determine what is the value of v above 0x10 that makes the ID 8-bit again.

  Maciej

Re: APIC version and 8-bit APIC IDs

[Martin Wilck]

Hi Maciej,

> It actually depends on the APIC type, rather than the CPU.  E.g. with
> Pentium systems the width of the ID is either 4 bits or 8 bits,
> depending on whether the integrated or an external 82489DX APIC is
> used.  This should be able to be determined by the APIC version; for
> v <= 0xf the ID is 8-bit and for v >= 0x10 it used to be 4-bit.  Now
> you only need to determine what is the value of v above 0x10 that
> makes the ID 8-bit again.

That would be v>=0x14 for Intel. But that is wrong for AMD CPUs. The 
actual Dual-Core Athlon CPUs we have report an APIC version of 0x10. 
Please refer to the start of this thread.

Anyway, I understand that you agree this does not belong into the 
subarch code?

Regards
Martin

-- 
Martin Wilck                Phone: +49 5251 8 15113
Fujitsu Siemens Computers   Fax:   +49 5251 8 20409
Heinz-Nixdorf-Ring 1        mailto:Martin.Wilck@Fujitsu-Siemens.com
D-33106 Paderborn           http://www.fujitsu-siemens.com/primergy

Re: APIC version and 8-bit APIC IDs

[Andi Kleen]

On Wednesday 31 August 2005 15:13, Martin Wilck wrote:

> In other words: What would be broken if we just used an APIC ID mask of
> 0xFF everywhere?

Nothing I think. It's more historical reasons. The physflat subarchitecture 
patch essentially removed it, but it needs some rework and merging
with bigsmp now.

> The current situation with MP_valid_apicid() on the one hand (masking
> the APIC ID as a function of local APIC version) and APIC_ID_MASK
> (masking the APIC as a function of subarch) on the other hand is
> inconsistent. A correct approach must take both CPU and architecture
> constraints into account, and use a CPU-type-dependent variable mask in
> the subarch code.

Yes, it's broken right now.

-Andi

Re: APIC version and 8-bit APIC IDs

[Martin Wilck]

[ Putting everyone cc again and leaving maciej's reply intact for 
reference ]

Maciej W. Rozycki wrote:

>>The ID is 0x14 for Intel. But that is wrong for AMD CPUs. The actual Dual-Core
> 
> 
>  Why can't hw designers ever get such things right?  Sigh...
> 
> 
>>Athlon CPUs we have report an APIC version of 0x10. Please refer to the start
>>of this thread.
> 
> 
>  Frankly, such a change implies a change to the inter-APIC communication 
> protocol, so the major revision should have been bumped, like it happened 
> for the I/O APIC (0x20); I hope they do not worry of changing the design 
> so many times they run out of numbers!  Thus none of the implementers has 
> done their job properly, but for Intel there is at least some change, 
> while for AMD there seems no generic way of determining that -- 0x10 
> implies a "traditional" integrated APIC as implemented in Pentium in 1995, 
> using a three-wire serial bus for communication...
> 
>  Perhaps a variable should be added holding the "architectural revision" 
> of the APIC subsystem, set up by the early CPU/APIC initialization code 
> and used from there on instead of direct references to the version 
> register as implemented in the hardware.  It does not have to be based on 
> what hardware uses, e.g.:
> 
> - 0: no APIC,
> 
> - 1: 82489DX -- communication using a five-wire inter-APIC bus, 8-bit 
>      physical ID, 32-bit logical ID, etc.,
> 
> - 2: Pentium-style -- communication using a three-wire inter-APIC bus, 
>      4-bit physical ID, 8-bit logical ID, etc.,
> 
> - 3: P4-style -- communication using the system bus, 8-bit physical ID, 
>      8-bit logical ID, etc.

AMD has their HyperTransport bus, which is yet a different thing.

According to what AMD told us, all K8 CPUs support 8-bit APIC IDs 
("extended APIC IDs" in their speak), but they are only enabled
"when both ApicExtId and ApicExtBrdCst in the HyperTransport Transaction 
Control Register are set".

As we cannot read PCI config space at APIC configuration time, my 
suggestion would be something like

unsigned char phys_apic_id_mask(int apic_version)
         if (apic_version < 0x10 || apic_version >= 0x14 ||
		(boot_cpu_data.vendor == X86_VENDOR_AMD &&
          	 boot_cpu_data.family >= 0x0f))
		return 0xff;
	else
		return 0x0f;
}

That would assume that no BIOS is so stupid to set APIC IDs >0xf and at 
the same time forget to enable the respective bits in the HyperTransport 
Transaction Control Register.

I do not oversee whether boot_cpu_data is initialized when the APIC IDs 
need to be tested/masked, but somebody else certainly does.

> 
> 
>>Anyway, I understand that you agree this does not belong into the subarch
>>code?
> 
> 
>  That's CPU/chipset specific indeed.  It shouldn't really depend on the 
> platform these are used in.  Exceptions, if any, can be handled on a case 
> by case basis.
> 
>   Maciej

Regards
Martin

-- 
Martin Wilck                Phone: +49 5251 8 15113
Fujitsu Siemens Computers   Fax:   +49 5251 8 20409
Heinz-Nixdorf-Ring 1        mailto:Martin.Wilck@Fujitsu-Siemens.com
D-33106 Paderborn           http://www.fujitsu-siemens.com/primergy