Re: [PATCH] TSC scaling for live migration between platforms with different TSC frequecies

[John Levon <levon@movementarian.org>]

On Thu, Jun 18, 2009 at 03:27:54PM -0700, Dan Magenheimer wrote:

> Even when restricted to physical hardware, using the TSC
> for such purposes seems ill-advised.

In practice it's not so bad, if you only do power management on P-state
invariant TSC CPUs, and disable C1 clock ramping. I'm sure there are all
sorts of fun caveats, but I don't think we've had many practical
problems.

> In a virtual data center, the data will be often useless.

It won't be happy across different machines indeed.

We haven't retested past 3.1, but the PV timer isn't even monotonic in
SMP guests. We have to global-sync to get one.

You mentioned the PV timer can't handle migration - why doesn't
tsc_to_system_mul account for it? If ever a subsystem badly needed a
detailed write-up...

> Is mstate accounting used for anything other than providing
> interesting performance data if one cares to look at it?

You make it sounds like that isn't critically important :)

> Does mstate accounting ignore negative values for delta TSC?

No, the system time is assumed monotonic (it's not in TSC units). The
TSC code
(http://src.opensolaris.org/source/xref/onnv/onnv-gate/usr/src/uts/i86pc/os/timestamp.c)
is expected to provide monotonicity across all CPUs. And /that/ code
assumes there's no inter-CPU drift. Deltas are allowed, but of course
HVM assumes that Xen has dealt with that (since it can't possible
compute deltas between VCPUs).

All this stuff is painful. What I wouldn't give for a single cheap
monotonic timer source that worked under all circumstances.

regards
john