From mboxrd@z Thu Jan 1 00:00:00 1970 From: Tomasz Wroblewski Subject: Re: TSC trouble in hvm linux, after host S3 Date: Wed, 20 Nov 2013 12:41:36 +0100 Message-ID: <528C9FF0.2050706@citrix.com> References: <528C95AE.6040308@citrix.com> Mime-Version: 1.0 Content-Type: text/plain; charset="us-ascii"; Format="flowed" Content-Transfer-Encoding: 7bit Return-path: In-Reply-To: <528C95AE.6040308@citrix.com> List-Unsubscribe: , List-Post: List-Help: List-Subscribe: , Sender: xen-devel-bounces@lists.xen.org Errors-To: xen-devel-bounces@lists.xen.org To: xen-devel@lists.xen.org List-Id: xen-devel@lists.xenproject.org On 11/20/2013 11:57 AM, Tomasz Wroblewski wrote: > Hello. I'm trying to debug a problem of linux hvms (3.11.7 dom0 kernel, xen 4.3.1, guest kernels tried were 3.2, 3.5, 3.8), using tsc as > clocksource, hanging some time after host S3 has been performed. I'm using the default tsc_mode (also tried never emulate to make sure), > vtsc is off. > > It seems linux tsc clocksource code expects the counter to be reset to 0 after S3 resume, except on some Atom chips who guarantee constant > tsc thru S3 (I'm not testing on these though). That doesn't seem to be case. I dumped results of rtdsc() both in dom0 and the linux guest. > It seems that in dom0 the tsc after S3 restarts to 0 as expected, in the linux hvm however it goes into negative values. As soon as it wraps > back to 0, timekeeping in linux gets broken. > > The negativeness seem to be because there is some offsetting being done to the tsc in hvm guest -> I sort of expected that since vtsc on > that hvm is off, the values returned by rdtsc() would match, that is however not the case, when the hvm guest boots the tsc in there seems > to start from 0, not from the current rdtsc value in dom0. So, after host S3, my linux hvm has about equivalent time to live as the delta > between xen boot and guest boot (i.e. if i booted the hvm 2mins after host boot, it will hang ~2mins after host s3 since thats when the tsc > will wrap) > > Any ideas on cause/fix? Why the tsc in hvm guest is offset from dom0 one even in TSC_MODE_NEVER_EMULATE? > I've since found out that VMCS seems to provide tsc offsetting capabilities; adding something like for_each_vcpu ( d, v ) { if (v->vcpu_id == 0) hvm_set_guest_tsc(v, 0); } inside hvm_s3_resume (xen/arch/x86/hvm/hvm.c) fixed this for me; any comment on such a solution?