It hurts when I shoot myself in the foot

It hurts when I shoot myself in the foot

[Chris]

A Cautionary Tale for Silly People
----------------------------------

For the past year now, I've had "ping" problems.

It pauses when it runs, and always returns warnings like so:

"Warning: time of day goes back, taking countermeasures"

I looked _everywhere_ on the net trying to find the problem.

I upgraded the kernel many times.
I upgraded glibc a few times.
I upgraded iputils a few times as well.

Nothing helped.

The clock was sync'd with an atomic clock every night.

Still, I kept getting the problem.

Then while moving log files around today, I noticed that the clock 'second'
didn't move.  Weird.

So I ran this:
  
  while :
  do
    date "+%H:%M:%S"
  done

I got interesting results:

17:05:24
17:05:24
17:05:24
17:05:33
17:05:33
17:05:25
17:05:25
17:05:33
17:05:25
17:05:25

Nice huh!

 Why?  

I looked inside the box and found a Pentium II 400, and a Pentium II 450.

Oddly enough they run together as a 266.

[root@hercules root]#cat /proc/cpuinfo
processor	: 0
vendor_id	: GenuineIntel
cpu family	: 6
model		: 5
model name	: Pentium II (Deschutes)
stepping	: 2
cpu MHz		: 265.915
cache size	: 512 KB
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 2
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr
bogomips	: 530.84

processor	: 1
vendor_id	: GenuineIntel
cpu family	: 6
model		: 5
model name	: Pentium II (Deschutes)
stepping	: 1
cpu MHz		: 265.915
cache size	: 512 KB
fdiv_bug	: no
hlt_bug		: no
f00f_bug	: no
coma_bug	: no
fpu		: yes
fpu_exception	: yes
cpuid level	: 2
wp		: yes
flags		: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr
bogomips	: 663.55

Re: It hurts when I shoot myself in the foot

[H. Peter Anvin]

Followup to:  <200205221615.g4MGFCH30271@directcommunications.net>
By author:    Chris <chris@directcommunications.net>
In newsgroup: linux.dev.kernel
> 
> I looked inside the box and found a Pentium II 400, and a Pentium II 450.
> 
> Oddly enough they run together as a 266.
> 

The fact that they have different BogoMIPS figures indicate that that
is not really the case.  It looks more like the second processor is
running at 333 MHz or something.  You definitely have a bizarre box
here, and you probably should be running with the "notsc" option.
Heck, maybe you can reconfigure your mobo and actually run both
processors at 400 MHz.  You'd get quite a performance boost, too...

	-hpa
-- 
<hpa@transmeta.com> at work, <hpa@zytor.com> in private!
"Unix gives you enough rope to shoot yourself in the foot."
http://www.zytor.com/~hpa/puzzle.txt	<amsp@zytor.com>

Re: It hurts when I shoot myself in the foot

[Andreas Dilger]

On May 22, 2002  16:39 -0700, H. Peter Anvin wrote:
> Followup to:  <200205221615.g4MGFCH30271@directcommunications.net>
> By author:    Chris <chris@directcommunications.net>
> In newsgroup: linux.dev.kernel
> > 
> > I looked inside the box and found a Pentium II 400, and a Pentium II 450.
> > 
> > Oddly enough they run together as a 266.
> 
> The fact that they have different BogoMIPS figures indicate that that
> is not really the case.  It looks more like the second processor is
> running at 333 MHz or something.  You definitely have a bizarre box
> here, and you probably should be running with the "notsc" option.
> Heck, maybe you can reconfigure your mobo and actually run both
> processors at 400 MHz.  You'd get quite a performance boost, too...

There was a kernel patch posted about5 or so months ago which would
"handle" this setup (CPUs with the same clock speed, but different
multipliers).  Alan Cox said it probably was a bad idea, so it wouldn't
go into the kernel, but the patch may still be usable.

This is sometimes called "asymmetric multiprocessing", and the thread
is at http://marc.theaimsgroup.com/?l=linux-kernel&m=98519070331478&w=4

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/

Re: It hurts when I shoot myself in the foot

[Mike Fedyk]

On Wed, May 22, 2002 at 09:48:21PM -0600, Andreas Dilger wrote:
> On May 22, 2002  16:39 -0700, H. Peter Anvin wrote:
> > Followup to:  <200205221615.g4MGFCH30271@directcommunications.net>
> > By author:    Chris <chris@directcommunications.net>
> > In newsgroup: linux.dev.kernel
> > > 
> > > I looked inside the box and found a Pentium II 400, and a Pentium II 450.
> > > 
> > > Oddly enough they run together as a 266.
> > 
> > The fact that they have different BogoMIPS figures indicate that that
> > is not really the case.  It looks more like the second processor is
> > running at 333 MHz or something.  You definitely have a bizarre box
> > here, and you probably should be running with the "notsc" option.
> > Heck, maybe you can reconfigure your mobo and actually run both
> > processors at 400 MHz.  You'd get quite a performance boost, too...
> 
> There was a kernel patch posted about5 or so months ago which would
> "handle" this setup (CPUs with the same clock speed, but different
> multipliers).  Alan Cox said it probably was a bad idea, so it wouldn't
> go into the kernel, but the patch may still be usable.
> 
> This is sometimes called "asymmetric multiprocessing", and the thread
> is at http://marc.theaimsgroup.com/?l=linux-kernel&m=98519070331478&w=4

I thought asymmetric multiprocessing would support CPUs with different
speeds.  ie, 400 & 450mhz.  How would you get different multipliers and same
Mhz when the CPUs are on the same FSB(ignoring AMD SMP where each processor
has an exclusive FSB, and this might be possible)?

IIRC, Linux supports SMP on CPUs with different stepping as long as the
features are the same, and even then it'll use the features of the boot CPU.
There was a patch to compare the different features on the CPUs available
and use the subset available on all processors.

Mike

Re: It hurts when I shoot myself in the foot

[Andreas Dilger]

On May 22, 2002  21:49 -0700, Mike Fedyk wrote:
> On Wed, May 22, 2002 at 09:48:21PM -0600, Andreas Dilger wrote:
> > There was a kernel patch posted about 5 or so months ago which would
> > "handle" this setup (CPUs with the same clock speed, but different
> > multipliers).  Alan Cox said it probably was a bad idea, so it wouldn't
> > go into the kernel, but the patch may still be usable.
> > 
> > This is sometimes called "asymmetric multiprocessing", and the thread
> > is at http://marc.theaimsgroup.com/?l=linux-kernel&m=98519070331478&w=4
> 
> I thought asymmetric multiprocessing would support CPUs with different
> speeds.  ie, 400 & 450mhz.  How would you get different multipliers and same
> Mhz when the CPUs are on the same FSB(ignoring AMD SMP where each processor
> has an exclusive FSB, and this might be possible)?

That was what I was trying to say: same FSB speed * different multipliers
= different CPU MHZ, like what the original poster is asking about.
I don't think it is possible to configure a motherboard to have different
FSB speeds for two processors.

> There was a patch to compare the different features on the CPUs available
> and use the subset available on all processors.

Hmm, maybe if you had actually read my email and followed the URL I
posted, you would have found the patch to which you refer ;-).

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/

Re: It hurts when I shoot myself in the foot

[Mike Fedyk]

On Wed, May 22, 2002 at 11:42:19PM -0600, Andreas Dilger wrote:
> On May 22, 2002  21:49 -0700, Mike Fedyk wrote:
> > On Wed, May 22, 2002 at 09:48:21PM -0600, Andreas Dilger wrote:
> > > There was a kernel patch posted about 5 or so months ago which would
> > > "handle" this setup (CPUs with the same clock speed, but different
> > > multipliers).  Alan Cox said it probably was a bad idea, so it wouldn't
> > > go into the kernel, but the patch may still be usable.
> > > 
> > > This is sometimes called "asymmetric multiprocessing", and the thread
> > > is at http://marc.theaimsgroup.com/?l=linux-kernel&m=98519070331478&w=4
> > 
> > I thought asymmetric multiprocessing would support CPUs with different
> > speeds.  ie, 400 & 450mhz.  How would you get different multipliers and same
> > Mhz when the CPUs are on the same FSB(ignoring AMD SMP where each processor
> > has an exclusive FSB, and this might be possible)?
> 
> That was what I was trying to say: same FSB speed * different multipliers
> = different CPU MHZ, like what the original poster is asking about.
> I don't think it is possible to configure a motherboard to have different
> FSB speeds for two processors.
>

Me neither, but it seems theoretically possible.

> > There was a patch to compare the different features on the CPUs available
> > and use the subset available on all processors.
> 
> Hmm, maybe if you had actually read my email 

I did. ;)

> and followed the URL I posted, 

That I didn't do. :(

>you would have found the patch to which you refer ;-).

And you are correct.  That is the exact patch to which I was referring.

Mike

Re: It hurts when I shoot myself in the foot

[Vojtech Pavlik]

On Thu, May 23, 2002 at 10:33:05AM -0700, Mike Fedyk wrote:
> On Wed, May 22, 2002 at 11:42:19PM -0600, Andreas Dilger wrote:
> > On May 22, 2002  21:49 -0700, Mike Fedyk wrote:
> > > On Wed, May 22, 2002 at 09:48:21PM -0600, Andreas Dilger wrote:
> > > > There was a kernel patch posted about 5 or so months ago which would
> > > > "handle" this setup (CPUs with the same clock speed, but different
> > > > multipliers).  Alan Cox said it probably was a bad idea, so it wouldn't
> > > > go into the kernel, but the patch may still be usable.
> > > > 
> > > > This is sometimes called "asymmetric multiprocessing", and the thread
> > > > is at http://marc.theaimsgroup.com/?l=linux-kernel&m=98519070331478&w=4
> > > 
> > > I thought asymmetric multiprocessing would support CPUs with different
> > > speeds.  ie, 400 & 450mhz.  How would you get different multipliers and same
> > > Mhz when the CPUs are on the same FSB(ignoring AMD SMP where each processor
> > > has an exclusive FSB, and this might be possible)?
> > 
> > That was what I was trying to say: same FSB speed * different multipliers
> > = different CPU MHZ, like what the original poster is asking about.
> > I don't think it is possible to configure a motherboard to have different
> > FSB speeds for two processors.
> >
> 
> Me neither, but it seems theoretically possible.

It is not, they are both on the same FSB, at least in Pentium II/III case.

-- 
Vojtech Pavlik
SuSE Labs

Re: It hurts when I shoot myself in the foot

[Mike Fedyk]

On Fri, May 24, 2002 at 12:28:29AM +0200, Vojtech Pavlik wrote:
> On Thu, May 23, 2002 at 10:33:05AM -0700, Mike Fedyk wrote:
> > On Wed, May 22, 2002 at 11:42:19PM -0600, Andreas Dilger wrote:
> > > On May 22, 2002  21:49 -0700, Mike Fedyk wrote:
> > > > On Wed, May 22, 2002 at 09:48:21PM -0600, Andreas Dilger wrote:
> > > > > There was a kernel patch posted about 5 or so months ago which would
> > > > > "handle" this setup (CPUs with the same clock speed, but different
> > > > > multipliers).  Alan Cox said it probably was a bad idea, so it wouldn't
> > > > > go into the kernel, but the patch may still be usable.
> > > > > 
> > > > > This is sometimes called "asymmetric multiprocessing", and the thread
> > > > > is at http://marc.theaimsgroup.com/?l=linux-kernel&m=98519070331478&w=4
> > > > 
> > > > I thought asymmetric multiprocessing would support CPUs with different
> > > > speeds.  ie, 400 & 450mhz.  How would you get different multipliers and same
> > > > Mhz when the CPUs are on the same FSB(ignoring AMD SMP where each processor
> > > > has an exclusive FSB, and this might be possible)?
> > > 
> > > That was what I was trying to say: same FSB speed * different multipliers
> > > = different CPU MHZ, like what the original poster is asking about.
> > > I don't think it is possible to configure a motherboard to have different
> > > FSB speeds for two processors.
> > >
> > 
> > Me neither, but it seems theoretically possible.
> 
> It is not, they are both on the same FSB, at least in Pentium II/III case.

And if you read even what is quoted above in this message you would notice
that we are not talking about Intel SMP, but AMD SMP, or more correctly, AMD
SMP licensed from Alpha.  What you say is true for Intel SMP though.

Re: It hurts when I shoot myself in the foot

[Vojtech Pavlik]

On Fri, May 24, 2002 at 12:32:16AM -0700, Mike Fedyk wrote:
> On Fri, May 24, 2002 at 12:28:29AM +0200, Vojtech Pavlik wrote:
> > On Thu, May 23, 2002 at 10:33:05AM -0700, Mike Fedyk wrote:
> > > On Wed, May 22, 2002 at 11:42:19PM -0600, Andreas Dilger wrote:
> > > > On May 22, 2002  21:49 -0700, Mike Fedyk wrote:
> > > > > On Wed, May 22, 2002 at 09:48:21PM -0600, Andreas Dilger wrote:
> > > > > > There was a kernel patch posted about 5 or so months ago which would
> > > > > > "handle" this setup (CPUs with the same clock speed, but different
> > > > > > multipliers).  Alan Cox said it probably was a bad idea, so it wouldn't
> > > > > > go into the kernel, but the patch may still be usable.
> > > > > > 
> > > > > > This is sometimes called "asymmetric multiprocessing", and the thread
> > > > > > is at http://marc.theaimsgroup.com/?l=linux-kernel&m=98519070331478&w=4
> > > > > 
> > > > > I thought asymmetric multiprocessing would support CPUs with different
> > > > > speeds.  ie, 400 & 450mhz.  How would you get different multipliers and same
> > > > > Mhz when the CPUs are on the same FSB(ignoring AMD SMP where each processor
> > > > > has an exclusive FSB, and this might be possible)?
> > > > 
> > > > That was what I was trying to say: same FSB speed * different multipliers
> > > > = different CPU MHZ, like what the original poster is asking about.
> > > > I don't think it is possible to configure a motherboard to have different
> > > > FSB speeds for two processors.
> > > >
> > > 
> > > Me neither, but it seems theoretically possible.
> > 
> > It is not, they are both on the same FSB, at least in Pentium II/III case.
> 
> And if you read even what is quoted above in this message you would notice
> that we are not talking about Intel SMP, but AMD SMP, or more correctly, AMD
> SMP licensed from Alpha.  What you say is true for Intel SMP though.

"ignoring AMD SMP" doesn't seem you're talking about it ... anyway even
with AMD chipsets while there are two different FSBs, they're still
driven by a single northbridge with a single clock. Now with
HyperTransport (like for example nVidia uses now in their nForce), I
think it could work ...

-- 
Vojtech Pavlik
SuSE Labs

Re: It hurts when I shoot myself in the foot

[Alan Cox]

> That was what I was trying to say: same FSB speed * different multipliers
> = different CPU MHZ, like what the original poster is asking about.
> I don't think it is possible to configure a motherboard to have different
> FSB speeds for two processors.

The Intel pentium II/III FSB at least can't handle this with normal chipsets. 
You can run same FSB different multipliers but this not officially supported.
I have code to detect the multiplier of each CPU but never got around to
making the kernel detect this and turn off TSC usage so you need to boot
with "notsc" in such a case

Re: It hurts when I shoot myself in the foot

[Kasper Dupont]

Alan Cox wrote:
> 
> I have code to detect the multiplier of each CPU but never got around to
> making the kernel detect this and turn off TSC usage so you need to boot
> with "notsc" in such a case

If the kernel knew multipliers couldn't it actually use the TSCs
anyway? Of course it would take some work, but is there any
reason why it would not be posible?

-- 
Kasper Dupont -- der bruger for meget tid på usenet.
For sending spam use mailto:razor-report@daimi.au.dk

Re: It hurts when I shoot myself in the foot

[Alan Cox]

> If the kernel knew multipliers couldn't it actually use the TSCs
> anyway? Of course it would take some work, but is there any
> reason why it would not be posible?

In 2.4 yes. In 2.5 it would be close to impossible due to the pre-empt code

Re: It hurts when I shoot myself in the foot

[Kasper Dupont]

Alan Cox wrote:
> 
> > If the kernel knew multipliers couldn't it actually use the TSCs
> > anyway? Of course it would take some work, but is there any
> > reason why it would not be posible?
> 
> In 2.4 yes. In 2.5 it would be close to impossible due to the pre-empt code

Couldn't that be solved in one of the following ways?

1) Disable pre-emption while reading TSC and CPU nr.
2) Use affinity for processes pre-empted in kernel mode.
3) Disable pre-emption for SMP systems.

-- 
Kasper Dupont -- der bruger for meget tid på usenet.
For sending spam use mailto:razor-report@daimi.au.dk

Re: It hurts when I shoot myself in the foot

[Alan Cox]

> Couldn't that be solved in one of the following ways?
> 
> 1) Disable pre-emption while reading TSC and CPU nr.
> 2) Use affinity for processes pre-empted in kernel mode.
> 3) Disable pre-emption for SMP systems.

You can solve it by disabling pre-emption (and given its questionable
value doing so permanently might not be a bad idea). However if you simply
disable pre-emption during udelay() calls then you've just screwed yourself
by removing 99% of the use pre-emption had.

Given all the pain its probably better to not use the TSC

Re: It hurts when I shoot myself in the foot

[Kasper Dupont]

Alan Cox wrote:
> 
> > Couldn't that be solved in one of the following ways?
> >
> > 1) Disable pre-emption while reading TSC and CPU nr.
> > 2) Use affinity for processes pre-empted in kernel mode.
> > 3) Disable pre-emption for SMP systems.
> 
> You can solve it by disabling pre-emption (and given its questionable
> value doing so permanently might not be a bad idea).

Questionable value of what? TSC or preemption?

> However if you simply
> disable pre-emption during udelay() calls then you've just screwed yourself
> by removing 99% of the use pre-emption had.

I wouldn't want to disable preemption during udelays.
Either I would disable and enable preemption on every
pass through the loop. Or I would just manually check
for every pass if I should give up the CPU. This
obviously requires more computation for every pass,
but being a busy waiting loop I don't see a problem.

Otherwise I would lock the process to a fixed CPU for
the duration of udelay.

> 
> Given all the pain its probably better to not use the TSC

Do we have better alternatives for high resolution
time meassurements?

-- 
Kasper Dupont -- der bruger for meget tid på usenet.
For sending spam use mailto:razor-report@daimi.au.dk

Re: It hurts when I shoot myself in the foot

[Alan Cox]

> > You can solve it by disabling pre-emption (and given its questionable
> > value doing so permanently might not be a bad idea).
> 
> Questionable value of what? TSC or preemption?

Pre-emption

> I wouldn't want to disable preemption during udelays.
> Either I would disable and enable preemption on every
> pass through the loop. Or I would just manually check

You've already made it too expensive too bother with

Re: It hurts when I shoot myself in the foot

[Austin Gonyou]

On Fri, 2002-05-24 at 10:36, Kasper Dupont wrote:
> Alan Cox wrote:
> > 
> > > If the kernel knew multipliers couldn't it actually use the TSCs
> > > anyway? Of course it would take some work, but is there any
> > > reason why it would not be posible?
> > 
> > In 2.4 yes. In 2.5 it would be close to impossible due to the pre-empt code
> 
> Couldn't that be solved in one of the following ways?
> 
> 1) Disable pre-emption while reading TSC and CPU nr.
> 2) Use affinity for processes pre-empted in kernel mode.
> 3) Disable pre-emption for SMP systems.
> 
Sorry to chime in on this if it isn't appreciated, but wouldn't #3 open
some old wounds again? It seems that it would FMPOV. 

Austin