Low bogomips on IBM x445 (kernel 2.6.5)

Low bogomips on IBM x445 (kernel 2.6.5)

[Christoph Pohl]

Hello,

I'm currently configuring an IBM x445 box with 4 Xeon CPUs (3GHz) and 
hyperthreading enabled. Everything seems to work fine since kernel 2.6.5 
but I keep wondering about the *very low* Bogomips numbers. Here is what 
I see in /proc/cpuinfo:

processor       : 0
vendor_id       : GenuineIntel
cpu family      : 15
model           : 2
model name      : Intel(R) Xeon(TM) CPU 3.00GHz
stepping        : 7
cpu MHz         : 2996.175
cache size      : 512 KB
physical id     : 0
siblings        : 2
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 2
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge 
mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe cid
bogomips        : 193.53

(...)

processor       : 7
vendor_id       : GenuineIntel
cpu family      : 15
model           : 2
model name      : Intel(R) Xeon(TM) CPU 3.00GHz
stepping        : 7
cpu MHz         : 2996.175
cache size      : 512 KB
physical id     : 25
siblings        : 2
fdiv_bug        : no
hlt_bug         : no
f00f_bug        : no
coma_bug        : no
fpu             : yes
fpu_exception   : yes
cpuid level     : 2
wp              : yes
flags           : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge 
mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe cid
bogomips        : 199.16

199 bogomips seems to be pretty low for my taste, although the box seems 
to work fine (and fast).

I came accross two bugs that looked very similar. However, they were 
filed agains 2.4.x and should have been fixed for 2.6.x:

http://www.ussg.iu.edu/hypermail/linux/kernel/0311.0/0329.html

http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=115061

Any ideas?

Best regards,
Christoph

PS: Please CC me, I didn't subscribe to the list!

Re: Low bogomips on IBM x445 (kernel 2.6.5)

[Jurriaan]

From: Christoph Pohl <christoph.pohl@inf.tu-dresden.de>
Date: Tue, Apr 27, 2004 at 01:01:08PM +0200
> Hello,
> 
> I'm currently configuring an IBM x445 box with 4 Xeon CPUs (3GHz) and 
> hyperthreading enabled. Everything seems to work fine since kernel 2.6.5 
> but I keep wondering about the *very low* Bogomips numbers. Here is what 
> I see in /proc/cpuinfo:
> 
What does 

cat /proc/mtrr

say and how much memory is in that thing?

Jurriaan
-- 
Is it a son? Is it a daughter? It's an Addams!
        Addams Family II
Debian (Unstable) GNU/Linux 2.6.6-rc2-mm2 2x6062 bogomips 2.58 1.39

Re: Low bogomips on IBM x445 (kernel 2.6.5)

[Christoph Pohl]

Jurriaan wrote:
>>I'm currently configuring an IBM x445 box with 4 Xeon CPUs (3GHz) and 
>>hyperthreading enabled. Everything seems to work fine since kernel 2.6.5 
>>but I keep wondering about the *very low* Bogomips numbers. Here is what 
>>I see in /proc/cpuinfo:
>>
> 
> What does 
> 
> cat /proc/mtrr

reg00: base=0x00000000 (   0MB), size=2048MB: write-back, count=1
reg01: base=0x80000000 (2048MB), size=1024MB: write-back, count=1
reg02: base=0xc0000000 (3072MB), size= 512MB: write-back, count=1
reg03: base=0xe0000000 (3584MB), size= 256MB: write-back, count=1
reg04: base=0x100000000 (4096MB), size=4096MB: write-back, count=1
reg05: base=0x200000000 (8192MB), size=8192MB: write-back, count=1

> say and how much memory is in that thing?

8192MB (8GB), plus 2GB swap space.
Does memory size somehow affect bogomips calculation?

There are however some noteworthy lines in syslog during reboot:
(...)
Apr 26 14:25:40 x445 kernel: Using local APIC timer interrupts.
Apr 26 14:25:40 x445 kernel: calibrating APIC timer ...
Apr 26 14:25:40 x445 kernel: ..... CPU clock speed is 2993.0654 MHz.
Apr 26 14:25:40 x445 kernel: ..... host bus clock speed is 99.0788 MHz.
Apr 26 14:25:40 x445 kernel: checking TSC synchronization across 8 CPUs:
Apr 26 14:25:40 x445 kernel: BIOS BUG: CPU#0 improperly initialized, has 
7358270 usecs TSC skew! FIXED.
Apr 26 14:25:40 x445 kernel: BIOS BUG: CPU#1 improperly initialized, has 
7358270 usecs TSC skew! FIXED.
Apr 26 14:25:40 x445 kernel: BIOS BUG: CPU#2 improperly initialized, has 
-7358271 usecs TSC skew! FIXED.
Apr 26 14:25:40 x445 kernel: BIOS BUG: CPU#3 improperly initialized, has 
-7358271 usecs TSC skew! FIXED.
Apr 26 14:25:40 x445 kernel: BIOS BUG: CPU#4 improperly initialized, has 
7358271 usecs TSC skew! FIXED.
Apr 26 14:25:40 x445 kernel: BIOS BUG: CPU#5 improperly initialized, has 
7358270 usecs TSC skew! FIXED.
Apr 26 14:25:40 x445 kernel: BIOS BUG: CPU#6 improperly initialized, has 
-7358269 usecs TSC skew! FIXED.
Apr 26 14:25:40 x445 kernel: BIOS BUG: CPU#7 improperly initialized, has 
-7358271 usecs TSC skew! FIXED.
Apr 26 14:25:40 x445 kernel: Brought up 8 CPUs
(...)

I don't know if that's any help.

Tell me if you need more details like .config, syslog etc.

Please CC me when replying!

Thanks,
Christoph

Re: Low bogomips on IBM x445 (kernel 2.6.5)

[Jurriaan]

From: Christoph Pohl <christoph.pohl@inf.tu-dresden.de>
Date: Wed, Apr 28, 2004 at 07:50:42AM +0200
> >cat /proc/mtrr
> 
> reg00: base=0x00000000 (   0MB), size=2048MB: write-back, count=1
> reg01: base=0x80000000 (2048MB), size=1024MB: write-back, count=1
> reg02: base=0xc0000000 (3072MB), size= 512MB: write-back, count=1
> reg03: base=0xe0000000 (3584MB), size= 256MB: write-back, count=1
> reg04: base=0x100000000 (4096MB), size=4096MB: write-back, count=1
> reg05: base=0x200000000 (8192MB), size=8192MB: write-back, count=1
> memory:
> 8192MB (8GB), plus 2GB swap space.
> Does memory size somehow affect bogomips calculation?
> 
> There are however some noteworthy lines in syslog during reboot:
> (...)
> Apr 26 14:25:40 x445 kernel: Using local APIC timer interrupts.
> Apr 26 14:25:40 x445 kernel: calibrating APIC timer ...
> Apr 26 14:25:40 x445 kernel: ..... CPU clock speed is 2993.0654 MHz.
> Apr 26 14:25:40 x445 kernel: ..... host bus clock speed is 99.0788 MHz.
> Apr 26 14:25:40 x445 kernel: checking TSC synchronization across 8 CPUs:
> Apr 26 14:25:40 x445 kernel: BIOS BUG: CPU#0 improperly initialized, has 
> 7358270 usecs TSC skew! FIXED.
> Apr 26 14:25:40 x445 kernel: BIOS BUG: CPU#1 improperly initialized, has 
> 7358270 usecs TSC skew! FIXED.
> Apr 26 14:25:40 x445 kernel: BIOS BUG: CPU#2 improperly initialized, has 
> -7358271 usecs TSC skew! FIXED.
> Apr 26 14:25:40 x445 kernel: BIOS BUG: CPU#3 improperly initialized, has 
> -7358271 usecs TSC skew! FIXED.
> Apr 26 14:25:40 x445 kernel: BIOS BUG: CPU#4 improperly initialized, has 
> 7358271 usecs TSC skew! FIXED.
> Apr 26 14:25:40 x445 kernel: BIOS BUG: CPU#5 improperly initialized, has 
> 7358270 usecs TSC skew! FIXED.
> Apr 26 14:25:40 x445 kernel: BIOS BUG: CPU#6 improperly initialized, has 
> -7358269 usecs TSC skew! FIXED.
> Apr 26 14:25:40 x445 kernel: BIOS BUG: CPU#7 improperly initialized, has 
> -7358271 usecs TSC skew! FIXED.
> Apr 26 14:25:40 x445 kernel: Brought up 8 CPUs
> (...)
> 
> I don't know if that's any help.

I recall some earlier messages where some crucial part of memory was
marked 'non-cacheable' in /proc/mtrr, and linux ran very slow. But on
re-reading, your problem is more that linux isn't slow, but bogomips
(and only bogomips) don't seem right?


In that case, you may need the help of a real kernel hacker, not a
piddling amateur like me :-)

Good luck,
Jurriaan
-- 
Nothing is a waste of time if you use the experience wisely.
        Rodin
Debian (Unstable) GNU/Linux 2.6.6-rc2-mm2 2x6062 bogomips 0.07 0.02

Re: Low bogomips on IBM x445 (kernel 2.6.5)

[john stultz]

On Tue, 2004-04-27 at 04:01, Christoph Pohl wrote:
> I'm currently configuring an IBM x445 box with 4 Xeon CPUs (3GHz) and 
> hyperthreading enabled. Everything seems to work fine since kernel 2.6.5 
> but I keep wondering about the *very low* Bogomips numbers. Here is what 
> I see in /proc/cpuinfo:
> 
> cpu MHz         : 2996.175
[snip]
> bogomips        : 193.53

This is expected. Since the IBM x440/x445 are NUMA systems, we cannot
use the TSC (cpu cycle counter) as a time source. Instead we use an off
chip performance counter which runs at 100Mhz. This then translates to a
bogoMIPS value of ~200. 


> I came accross two bugs that looked very similar. However, they were 
> filed agains 2.4.x and should have been fixed for 2.6.x:
> 
> http://www.ussg.iu.edu/hypermail/linux/kernel/0311.0/0329.html
> 
> http://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=115061

These posts are about an old bug in 2.4 that was related to the
performance counter time source on the x44x systems. However in those
cases, bogoMIPS would be mis-calculated to ~3, which caused problems.
This issue does not affect 2.6 or recent 2.4 kernels.

thanks
-john

Re: Low bogomips on IBM x445 (kernel 2.6.5)

[Chris Friesen]

john stultz wrote:

> This is expected. Since the IBM x440/x445 are NUMA systems, we cannot
> use the TSC (cpu cycle counter) as a time source. Instead we use an off
> chip performance counter which runs at 100Mhz. This then translates to a
> bogoMIPS value of ~200. 

That sounds very strange.  Bogomips is supposed to be how many busy-wait loops the cpu can do in a 
second, or at least that's what I've seen in all the books.  It shouldn't matter what the time 
source is.

I would expect a NUMA machine to have different bogomips values for the different CPUs, but the 
values should still be the same as if that cpu was on a uniprocessor system, no?

Chris

Re: Low bogomips on IBM x445 (kernel 2.6.5)

[john stultz]

On Wed, 2004-04-28 at 14:11, Chris Friesen wrote:
> john stultz wrote:
> 
> > This is expected. Since the IBM x440/x445 are NUMA systems, we cannot
> > use the TSC (cpu cycle counter) as a time source. Instead we use an off
> > chip performance counter which runs at 100Mhz. This then translates to a
> > bogoMIPS value of ~200. 
> 
> That sounds very strange.  Bogomips is supposed to be how many busy-wait loops the cpu can do in a 
> second, or at least that's what I've seen in all the books.  It shouldn't matter what the time 
> source is.

Well, sort of. bogoMIPS is derived from loops_per_jiffy, which when
using the PIT as a time source is how many busy-wait loops occur in a
timer tick.  However when using the TSC as a timesource, the __delay()
function busy waits for a number of cycles. This increases the accuracy,
as interrupts taken do not affect the delay time. So in this case
bogoMIPS becomes the number of cpu cycles per tick. When using the
cyclone time source (the off chip performance counter) on the x440, it
becomes the number of cyclone cycles per tick.

thanks
-john