From mboxrd@z Thu Jan 1 00:00:00 1970 From: Philipp Hahn Subject: Re: [BUG, PATCH-2.6.32] Fix a possible backwards warp of kvmclock Date: Mon, 5 Sep 2011 16:06:57 +0200 Message-ID: <201109051607.02349.hahn@univention.de> References: <1282291669-25709-1-git-send-email-zamsden@redhat.com> <1282291669-25709-17-git-send-email-zamsden@redhat.com> <201109022034.16517.hahn@univention.de> Mime-Version: 1.0 Content-Type: multipart/signed; boundary="nextPart1590380.EgY9hNe2Ju"; protocol="application/pgp-signature"; micalg=pgp-sha1 Content-Transfer-Encoding: 7bit Cc: kvm@vger.kernel.org, Avi Kivity , Marcelo Tosatti , Glauber Costa , Thomas Gleixner , John Stultz To: Zachary Amsden , =?utf-8?q?=E7=BC=96=E7=A0=81=E4=BA=BA?= , Xiao Guangrong , Nikola Ciprich Received: from mail.univention.de ([82.198.197.8]:2339 "EHLO mail.univention.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753494Ab1IEOHI (ORCPT ); Mon, 5 Sep 2011 10:07:08 -0400 In-Reply-To: <201109022034.16517.hahn@univention.de> Sender: kvm-owner@vger.kernel.org List-ID: --nextPart1590380.EgY9hNe2Ju Content-Type: multipart/mixed; boundary="Boundary-01=_CeNZOwn4qwwb7M3" Content-Transfer-Encoding: 7bit Content-Disposition: inline --Boundary-01=_CeNZOwn4qwwb7M3 Content-Type: text/plain; charset="utf-8" Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Hello, (cc:-ing lost of people who reported similar bugs on kvm-devel) > Changing clock in KVM host may cause VM to hang > 2.6.32 guest with paravirt clock enabled hangs on 2.6.37.6 host (w=20 qemu-kvm-0.13.0) I found a bug regarding PV-clock in the KVM-kernel module. The attached pat= ch=20 solves the problem of the guest being very slow after a reboot. Can you=20 please have a look and give it a try to see if it solves your problem as=20 well. Since the fix is only relevant for the stable 2.6.32 tree, where the code i= s=20 quiet different, please have a look and forward to stable@ as appropriate. Sincerely Philipp PS: This bug is tracked in our German Bugzilla at=20 =2D-=20 Philipp Hahn Open Source Software Engineer hahn@univention.de Univention GmbH Linux for Your Business fon: +49 421 22 232- 0 Mary-Somerville-Str.1 D-28359 Bremen fax: +49 421 22 232-99 http://www.univention.de/ =2D------------------------------------------------------------------------= =2D-- Treffen Sie Univention auf der IT&Business vom 20. bis 22. September 2011 auf dem Gemeinschaftsstand der Open Source Business Alliance in Stuttgart in Halle 3 Stand 3D27-7. --Boundary-01=_CeNZOwn4qwwb7M3 Content-Type: text/x-diff; charset="utf-8"; name="23258_kvm-clock-reset.diff" Content-Transfer-Encoding: quoted-printable Content-Disposition: attachment; filename="23258_kvm-clock-reset.diff" Bug #23257: Reset tsc_timestamp on TSC writes vcpu->last_guest_tsc is updated in vcpu_enter_guest() and kvm_arch_vcpu_put= () by getting the last value of the TSC from the guest. On reset, the SeaBIOS resets the TSC to 0, which triggers a bug on the next call to kvm_write_guest_time(): Since vcpu->hw_clock.tsc_timestamp still contains the old value before the reset, "max_kernel_ns =3D vcpu->last_gues= t_tsc =2D vcpu->hw_clock.tsc_timestamp" gets negative. Since the variable is u64,= it gets translated to a large positive value. [9333.197080] vcpu->last_guest_tsc =3D209_328_760_015 =E2=86=90 vcpu->hv_clock.tsc_timestamp=3D209_328_708_109 vcpu->last_kernel_ns =3D9_333_179_830_643 kernel_ns =3D9_333_197_073_429 max_kernel_ns =3D9_333_179_847_943 =E2=86=90 [9336.910995] vcpu->last_guest_tsc =3D9_438_510_584 =E2=86=90 vcpu->hv_clock.tsc_timestamp=3D211_080_593_143 vcpu->last_kernel_ns =3D9_333_763_732_907 kernel_ns =3D9_336_910_990_771 max_kernel_ns =3D6_148_296_831_006_663_830 =E2=86=90 =46or completeness, here are the values for my 3 GHz CPU: vcpu->hv_clock.tsc_shift =3D-1 vcpu->hv_clock.tsc_to_system_mul =3D2_863_019_502 This makes the guest kernel crawl very slowly when clocksource=3Dkvmclock is used: sleeps take way longer than expected and don't match wall clock any m= ore. The times printed with printk() don't match real time and the reboot often stalls for long times. In linux-git this isn't a problem, since on every MSR_IA32_TSC write vcpu->arch.hv_clock.tsc_timestamp is reset to 0, which disables above logic. The code there is only in arch/x86/kvm/x86.c, since much of the kvm-clock related code has been refactured for 2.6.37: 99e3e30a arch/x86/kvm/x86.c (Zachary Amsden 2010-08-19 22:07:17= -1000 1084) vcpu->arch.hv_clock.tsc_timestamp =3D 0; = =20 Since 1d5f066e0b63271b67eac6d3752f8aa96adcbddb from 2.6.37 was back-ported = to 2.6.32.40 as ad2088cabe0fd7f633f38ba106025d33ed9a2105, the following patch = is needed to add the needed reset logic to 2.6.32 as well. Signed-off-by: Philipp Hahn =2D-- a/arch/x86/kvm/vmx.c 2011-09-05 14:17:54.000000000 +0200 +++ b/arch/x86/kvm/vmx.c 2011-09-05 14:18:03.000000000 +0200 @@ -1067,6 +1067,7 @@ static int vmx_set_msr(struct kvm_vcpu * case MSR_IA32_TSC: rdtscll(host_tsc); guest_write_tsc(data, host_tsc); + vcpu->arch.hv_clock.tsc_timestamp =3D 0; break; case MSR_IA32_CR_PAT: if (vmcs_config.vmentry_ctrl & VM_ENTRY_LOAD_IA32_PAT) { =2D-- a/arch/x86/kvm/svm.c 2011-09-05 14:17:57.000000000 +0200 +++ b/arch/x86/kvm/svm.c 2011-09-05 14:18:00.000000000 +0200 @@ -2256,6 +2256,7 @@ static int svm_set_msr(struct kvm_vcpu * } =20 svm->vmcb->control.tsc_offset =3D tsc_offset + g_tsc_offset; + vcpu->arch.hv_clock.tsc_timestamp =3D 0; =20 break; } --Boundary-01=_CeNZOwn4qwwb7M3-- --nextPart1590380.EgY9hNe2Ju Content-Type: application/pgp-signature; name=signature.asc Content-Description: This is a digitally signed message part. -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.9 (GNU/Linux) iEYEABECAAYFAk5k14IACgkQYPlgoZpUDjkRFgCfXJsl0/KfMNEyr0t7wX7OPc+w 9cQAnRgnIX7z4aarKO1MvnDbq2MkdSxq =1Fnp -----END PGP SIGNATURE----- --nextPart1590380.EgY9hNe2Ju--