SMP processor rework help needed

SMP processor rework help needed

[Tommy Faasen]

Hi,

I have this unique situation where cpu 1 has less features (like fxsr) then
cpu 0.

However the linux kernel only looks at the cpu 0 and assumes the all the
other cpu's have the same capabilities. Which can cause it to oops.
Several of you have been so kind to help me find this problem. Now I can
manually fit it in processor.h to change #define cpu_has_fxsr 0.
I would like to rewrite this bit so it tests all the cpu's if it has a
certain capability and only then activates this feature, I think this would
be a more clean way of handling things.
Anyway I have been looking through processor.h and bitops.h and figured it
does a some asm to extract this information.

So again I am stuck as my limited knowledge stops here, so I was hoping that
someone could give me a helping hand or point me in a direction.

Ps 2.4.12-ac2 works fine with my little modification on my aic78xxx lx smp
system in case you wanted to know.

Re: SMP processor rework help needed

[Andi Kleen]

In article <000b01c154ee$1d6a2610$6400a8c0@it0>,
"Tommy Faasen" <tommy@vuurwerk.nl> writes:
> Hi,
> I have this unique situation where cpu 1 has less features (like fxsr) then
> cpu 0.

I used to have such an AMP machine too: a dual PII-300 with one Katmai and one
Deschutes. It's technically a violation of the specs; the Intel SMP spec
requires that the non boot cpus need to have a superset of the features
of the boot CPU. One CPU died, so it is symmetric now.

For most capabilities it should already work in 2.4 after hpa's cpu 
set rewrite, but FXSAVE is unfortunately a bit of a special case because
it is used in the scheduler context switch and that is required early
in the initialization for SMP bootup and changing it would be very
intrusive.

In the 2.2 SuSE kernel it was fixed instead by adding a new kernel
command line option nofxsave that overrides the FXSAVE bit on the first
CPU. That is ok because such setup is very rare and is only generated by
people who build their own boxes; and these should also know how to pass
kernel command line arguments.

Such an option should also be easy to add to 2.4.
-Andi

Re: SMP processor rework help needed

[Gerhard Mack]

On 14 Oct 2001, Andi Kleen wrote:

> In article <000b01c154ee$1d6a2610$6400a8c0@it0>,
> "Tommy Faasen" <tommy@vuurwerk.nl> writes:
> > Hi,
> > I have this unique situation where cpu 1 has less features (like fxsr) then
> > cpu 0.
> 
> I used to have such an AMP machine too: a dual PII-300 with one Katmai and one
> Deschutes. It's technically a violation of the specs; the Intel SMP spec
> requires that the non boot cpus need to have a superset of the features
> of the boot CPU. One CPU died, so it is symmetric now.
> 
> For most capabilities it should already work in 2.4 after hpa's cpu 
> set rewrite, but FXSAVE is unfortunately a bit of a special case because
> it is used in the scheduler context switch and that is required early
> in the initialization for SMP bootup and changing it would be very
> intrusive.
> 
> In the 2.2 SuSE kernel it was fixed instead by adding a new kernel
> command line option nofxsave that overrides the FXSAVE bit on the first
> CPU. That is ok because such setup is very rare and is only generated by
> people who build their own boxes; and these should also know how to pass
> kernel command line arguments.

This may sound like a dumb question but wouldn't simply swapping the CPUs
have the same affect?

	Gerhard



--
Gerhard Mack

gmack@innerfire.net

<>< As a computer I find your faith in technology amusing.

Re: SMP processor rework help needed

[Andi Kleen]

On Sun, Oct 14, 2001 at 10:50:50PM +0200, Gerhard Mack wrote:
> This may sound like a dumb question but wouldn't simply swapping the CPUs
> have the same affect?

In theory yes, assuming the determination of the boot cpu is fully
deterministic. the spec says it is the one with the lowest apic number; but
who knows if that is true in every weird board.

-Andi

Re: SMP processor rework help needed

[Tommy Faasen]

Yes it would however in my case, the cooler,which I can't get off, is too
big for the first slot!

It's just 1 more way linux can be more flexible with hardware.
----- Original Message -----
From: "Gerhard Mack" <gmack@innerfire.net>
To: "Andi Kleen" <ak@muc.de>
Cc: "Tommy Faasen" <tommy@vuurwerk.nl>; <linux-kernel@vger.kernel.org>
Sent: Sunday, October 14, 2001 10:50 PM
Subject: Re: SMP processor rework help needed


> On 14 Oct 2001, Andi Kleen wrote:
>
> > In article <000b01c154ee$1d6a2610$6400a8c0@it0>,
> > "Tommy Faasen" <tommy@vuurwerk.nl> writes:
> > > Hi,
> > > I have this unique situation where cpu 1 has less features (like fxsr)
then
> > > cpu 0.
> >
> > I used to have such an AMP machine too: a dual PII-300 with one Katmai
and one
> > Deschutes. It's technically a violation of the specs; the Intel SMP spec
> > requires that the non boot cpus need to have a superset of the features
> > of the boot CPU. One CPU died, so it is symmetric now.
> >
> > For most capabilities it should already work in 2.4 after hpa's cpu
> > set rewrite, but FXSAVE is unfortunately a bit of a special case because
> > it is used in the scheduler context switch and that is required early
> > in the initialization for SMP bootup and changing it would be very
> > intrusive.
> >
> > In the 2.2 SuSE kernel it was fixed instead by adding a new kernel
> > command line option nofxsave that overrides the FXSAVE bit on the first
> > CPU. That is ok because such setup is very rare and is only generated by
> > people who build their own boxes; and these should also know how to pass
> > kernel command line arguments.
>
> This may sound like a dumb question but wouldn't simply swapping the CPUs
> have the same affect?
>
> Gerhard
>
>
>
> --
> Gerhard Mack
>
> gmack@innerfire.net
>
> <>< As a computer I find your faith in technology amusing.
>
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: SMP processor rework help needed

[Pascal Schmidt]

On 14 Oct 2001, Andi Kleen wrote:

> I used to have such an AMP machine too: a dual PII-300 with one Katmai and
> one Deschutes. It's technically a violation of the specs; the Intel SMP
> spec requires that the non boot cpus need to have a superset of the
> features of the boot CPU. One CPU died, so it is symmetric now.
Errrrm, excuse me, but why not simply swap the two CPUs so that the one
with less features becomes the boot CPU?

-- 
Ciao, Pascal

-<[ pharao90@tzi.de, netmail 2:241/215.72, home http://cobol.cjb.net/) ]>-

Re: SMP processor rework help needed

[Eric W. Biederman]

Andi Kleen <ak@muc.de> writes:

> On Sun, Oct 14, 2001 at 10:50:50PM +0200, Gerhard Mack wrote:
> > This may sound like a dumb question but wouldn't simply swapping the CPUs
> > have the same affect?
> 
> In theory yes, assuming the determination of the boot cpu is fully
> deterministic. the spec says it is the one with the lowest apic number; but
> who knows if that is true in every weird board.

I do recall that the apics have programmable numbers.  We even test
that as part of our cpu initialization.  So that means little.  

For intel the initial determination is made having the cpus race on
the apic bus.  The cpu that sends a message first gets the lowest
apicid.  Though I need to see how the P4 Xeon does it, as the apic
bus is actually unused.

Also many boards have logic so that allows the second cpu to become the
boot cpu if the first cpu fails to boot.  This logic might be as
simple as round-robin, so even a deterministic may make this
difficult.

So the only reliable way to force the boot cpu is with software that
runs before the operating system, usually the firmware.

I'll keep this in mind for linuxBIOS, as that would be an ideal place
to implement something like that.  

Eric

Re: SMP processor rework help needed

[Manfred Spraul]

 
Content-Type: text/plain; charset=us-ascii
Content-Transfer-Encoding: 7bit

> For intel the initial determination is made having the cpus race on
> the apic bus.  The cpu that sends a message first gets the lowest
> apicid.  Though I need to see how the P4 Xeon does it, as the apic
> bus is actually unused.

Huh?

24547202.pdf: (i.e. volume 3 of the ia32 SDM)
<<<<<<<<
The APIC ID register is loaded at power up by sampling configuration
data that is driven onto pins of the processor. For the Pentium 4 and P6
family processors, pins A11# and A12# and pins BR0# through BR3# are
sampled; for the Pentium processor, pins BE0# through BE3# are sampled.
<<<<<<

--
	Manfred

Re: SMP processor rework help needed

[Bill Davidsen]

On 14 Oct 2001, Andi Kleen wrote:

> I used to have such an AMP machine too: a dual PII-300 with one Katmai and one
> Deschutes. It's technically a violation of the specs; the Intel SMP spec
> requires that the non boot cpus need to have a superset of the features
> of the boot CPU. One CPU died, so it is symmetric now.
> 
> For most capabilities it should already work in 2.4 after hpa's cpu 
> set rewrite, but FXSAVE is unfortunately a bit of a special case because
> it is used in the scheduler context switch and that is required early
> in the initialization for SMP bootup and changing it would be very
> intrusive.
> 
> In the 2.2 SuSE kernel it was fixed instead by adding a new kernel
> command line option nofxsave that overrides the FXSAVE bit on the first
> CPU. That is ok because such setup is very rare and is only generated by
> people who build their own boxes; and these should also know how to pass
> kernel command line arguments.
> 
> Such an option should also be easy to add to 2.4.

Given that this is an unlikely case, and init code is released after boot,
if this were going to be done at all it might be desirable to make it a
build option and check all capabilities on all CPUs. While I can't
identify a current set of CPUs with at least one feature missing from
each, given Linux running on Intel and AMD processors, and many other type
as well (still multiple SPARC vendors, I think), just checking all the
things which might be broken or missing would be the safest way.

No, I have no idea how hard it really is, one quick look at the code makes
it look less than hard and more than non-trivial for iA86.

-- 
bill davidsen <davidsen@tmr.com>
  CTO, TMR Associates, Inc
Doing interesting things with little computers since 1979.

Re: SMP processor rework help needed

[Alan Cox]

> Huh?
> 
> 24547202.pdf: (i.e. volume 3 of the ia32 SDM)
> <<<<<<<<
> The APIC ID register is loaded at power up by sampling configuration
> data that is driven onto pins of the processor. For the Pentium 4 and P6
> family processors, pins A11# and A12# and pins BR0# through BR3# are
> sampled; for the Pentium processor, pins BE0# through BE3# are sampled.
> <<<<<<

Dual Pentium at least has a mode where the boot processor is decided by a
boot time resolution - its valuable for HA uses.  I'm not sure if it affects
the APIC ident