TSC trouble in hvm linux, after host S3

[Tomasz Wroblewski <tomasz.wroblewski@citrix.com>]

Hello. I'm trying to debug a problem of linux hvms (3.11.7 dom0 kernel, xen 4.3.1, guest kernels tried were 3.2, 3.5, 3.8), using tsc as 
clocksource, hanging some time after host S3 has been performed. I'm using the default tsc_mode (also tried never emulate to make sure), 
vtsc is off.

It seems linux tsc clocksource code expects the counter to be reset to 0 after S3 resume, except on some Atom chips who guarantee constant 
tsc thru S3 (I'm not testing on these though). That doesn't seem to be case. I dumped results of rtdsc() both in dom0 and the linux guest. 
It seems that in dom0 the tsc after S3 restarts to 0 as expected, in the linux hvm however it goes into negative values. As soon as it wraps 
back to 0, timekeeping in linux gets broken.

The negativeness seem to be because there is some offsetting being done to the tsc in hvm guest -> I sort of expected that since vtsc on 
that hvm is off, the values returned by rdtsc() would match, that is however not the case, when the hvm guest boots the tsc in there seems 
to start from 0, not from the current rdtsc value in dom0. So, after host S3, my linux hvm has about equivalent time to live as the delta 
between xen boot and guest boot (i.e. if i booted the hvm 2mins after host boot, it will hang ~2mins after host s3 since thats when the tsc 
will wrap)

Any ideas on cause/fix? Why the tsc in hvm guest is offset from dom0 one even in TSC_MODE_NEVER_EMULATE?