Clock has stopped (time/date looping over 5 seconds), things are broken - what to check to debug?

[Roger Heflin <rogerheflin@gmail.com>]

So far what I have is that the clock is moving between
10:01:03 to 10:01:07 (when it gets to 07 it goes back to 03), doing rdate -s
results in things changing:

16:12:38 to 16:12:43 (resets back to :38).

Doing this:
while true ; do date; usleep 1000000; done
Fri Apr  4 16:12:39 CDT 2008
Fri Apr  4 16:12:40 CDT 2008
Fri Apr  4 16:12:41 CDT 2008
Fri Apr  4 16:12:42 CDT 2008
Fri Apr  4 16:12:43 CDT 2008

It stops at :43, ^C is required, and you can then restart it with repeatable 
results.

This F7 - 2.6.23.15-80.fc7

dmesg/messages contain nothing abnormal.

This machine has done it several times, a freqency of maybe 1x per every couple 
of weeks or so.   I believe it had also done this with: 2.6.22.9-91.fc7 so it
has been doing this for a while.   It used to work with some older kernel (I 
don't know which).

Given what the clock is doing, things that sleep at the wrong time hang forever, 
and a number of other things fail to work.

vmstat 1 results in a single line being printed out, and then a floating point 
exception.

"shutdown -r now" fails to complete, power cycle is required to get the machine
back up.

I don't believe any hardware failure that I can think of would cause the clock 
to do what mine is doing.

Ideas?

                                Roger