Re: sched_clock - cont'd

Re: sched_clock - cont'd

[Zoltan Menyhart]

David,

Since I applied your patch, I re-booted my Tiger-4 about 10 times
and I got twice the following Oops:

CPU 3: base freq\x199.453MHz, ITC ratio\x13/2, ITC freq\x1296.444MHz+/--1ppm
Calibrating delay loop... 1943.56 BogoMIPS
Oops: timer tick before it's due (itc—c85f8eed,itm—c87b85d7)
Oops: timer tick before it's due (itc—c8d4a7c7,itm—c8dc1d93)
Oops: timer tick before it's due (itc—c949bc39,itm—c95006db)

... some 100 of similar lines ...

Oops: timer tick before it's due (itc˜59824596,itm˜5985c2ef)
Oops: timer tick before it's due (itc˜59f75936,itm˜59f9ac37)
Oops: timer tick before it's due (itc˜5a6c6c40,itm˜5a6d957f)
CPU 3: synchronized ITC with CPU 0 (last diff -3 cycles, maxerr 494 cycles)

Once it is over, everything seems to be correct. E.g. I can stress
the system by compiling the kernel with "make -j100".
It's a 2.6.5 kernel.
I'm sure the new "sched_clock" code is O.K. I guess this modification
has brought to light a hidden bug in the initial CPU synchronization
or timing set up code.
Have you ever seen a similar problem ?

The 4 CPUs in the box are the same, like CPU #3:
processor  : 3
vendor     : GenuineIntel
arch       : IA-64
family     : Itanium 2
model      : 1
revision   : 5
archrev    : 0
features   : branchlong
cpu number : 0
cpu regs   : 4
cpu MHz    : 1296.444994
itc MHz    : 1296.444994
BogoMIPS   : 1941.96

Thanks,

Zoltán

Re: sched_clock - cont'd

[David Mosberger]

>>>>> On Fri, 04 Jun 2004 16:41:45 +0200, Zoltan Menyhart <Zoltan.Menyhart_AT_bull.net@nospam.org> said:

  Zoltan> Oops: timer tick before it's due
  Zoltan> (itc˜59824596,itm˜5985c2ef) Oops: timer tick before it's
  Zoltan> due (itc˜59f75936,itm˜59f9ac37) Oops: timer tick before
  Zoltan> it's due (itc˜5a6c6c40,itm˜5a6d957f) CPU 3: synchronized
  Zoltan> ITC with CPU 0 (last diff -3 cycles, maxerr 494 cycles)

  Zoltan> Once it is over, everything seems to be correct. E.g. I can
  Zoltan> stress the system by compiling the kernel with "make -j100".
  Zoltan> It's a 2.6.5 kernel.  I'm sure the new "sched_clock" code is
  Zoltan> O.K. I guess this modification has brought to light a hidden
  Zoltan> bug in the initial CPU synchronization or timing set up
  Zoltan> code.  Have you ever seen a similar problem ?

No.  sched_clock() has nothing to do with the timer-tick or
time-of-day, so the problem must be something else.

	--david

Re: sched_clock - cont'd

[David Mosberger]

>>>>> On Fri, 04 Jun 2004 16:41:45 +0200, Zoltan Menyhart <Zoltan.Menyhart_AT_bull.net@nospam.org> said:

  Zoltan> David, Since I applied your patch, I re-booted my Tiger-4
  Zoltan> about 10 times and I got twice the following Oops:

  Zoltan> CPU 3: base freq\x199.453MHz, ITC ratio\x13/2, ITC
  Zoltan> freq\x1296.444MHz+/--1ppm Calibrating delay loop... 1943.56
  Zoltan> BogoMIPS Oops: timer tick before it's due
  Zoltan> (itc—c85f8eed,itm—c87b85d7) Oops: timer tick before it's
  Zoltan> due (itc—c8d4a7c7,itm—c8dc1d93) Oops: timer tick before
  Zoltan> it's due (itc—c949bc39,itm—c95006db)

  Zoltan> Once it is over, everything seems to be correct. E.g. I can
  Zoltan> stress the system by compiling the kernel with "make -j100".
  Zoltan> It's a 2.6.5 kernel.

I forgot to mention: make sure you have Bjorn's patch in your
tree:

Stable cset id: davidm@tiger.hpl.hp.com|ChangeSet|20040513223146|25847

ChangeSet@1.1608.9.11, 2004-05-13 15:31:46-07:00, davidm@tiger.hpl.hp.com
  ia64: fix spurious "timer tick before it's due" problem

  Patch Bjorn Helgaas: Fix the "timer tick before it's due" complaint
  from timer_interrupt().  The problem was that smp_callin() turned
  on the periodic timer tick before syncing the ITC with the BP.

This went in _after_ 2.6.6 so it's likely that this will fix your
problem (I never saw the issue on a 4-way, though).

	--david