From mboxrd@z Thu Jan 1 00:00:00 1970 From: Marcelo Tosatti Subject: Re: KVM-Clock Date: Wed, 24 Feb 2016 08:33:06 -0300 Message-ID: <20160224113306.GA12114@amt.cnet> References: <20160224040919.GA10513@amt.cnet> Mime-Version: 1.0 Content-Type: text/plain; charset=iso-8859-1 Content-Transfer-Encoding: QUOTED-PRINTABLE Cc: "kvm@vger.kernel.org" To: Avi Cohen Return-path: Received: from mx1.redhat.com ([209.132.183.28]:36963 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1753149AbcBXLdY (ORCPT ); Wed, 24 Feb 2016 06:33:24 -0500 Content-Disposition: inline In-Reply-To: Sender: kvm-owner@vger.kernel.org List-ID: On Wed, Feb 24, 2016 at 08:15:46AM +0000, Avi Cohen wrote: > >>=20 > >> Hello, > >> Last week I've sent a mail regarding the kvm-clock accuracy. > >> Now I try to draw-up my question again, any answer/partial/hint i= s=20 > >> greatly appreciated > >>=20 > >> Our application is running in a Tenant's Virtual Machine in a dat= a-centre. > >> We have some OAM functions running in the VMs. > >> One OAM function is to measure one-way delay between VMa and VMb.=20 > >> One way delay measurement requires that all machines should be syn= chronized to a common central clock. > >> Accuracy requirement is in order of 10s nano-seconds, hence only t= he 1588v2/PTP is suitable here. > >> Since we cannot use HW timestamping in a virtual machine (we canno= t force using SR-IOV), I thought to run PTP on the physical machines an= d to sync the VMs to the host by the kvm-clock. > >> But now I see that the clock in the VM is far away from the host (= ~ Hundreds of micro-second) , and this before I even run the PTP in th= e host... > >> My test is very simple - I send a packet from host to the VM, I se= t the host time (tx_time) in the packet. When the guest receives the p= acket it reads its time (rx_time) and calculate the delay as : > >> Delay =3D rx_time - tx_time > >> I use the clock_gettime(REALTIME) in the host to set tx-time and i= n the guest to read rx_time.=20 > >> My questions : > >> 1. Assuming my HW support the paravirtualization clock requirement= s - (see below output of cpuinfo) , In Theory - Is it possible to ach= ieve 10s ns accuracy between VM clock and the host clock ?=20 > >> or I'm too na=EFve and have to abandon the idea to run this timin= g sensitive application on a VM, and instead run it in Linux containe= r for example? =20 >=20 > >You need realtime KVM to run a time sensitive application on a VM. = What are your requirements? >=20 > >RHEL-RT-KVM achieves 20us maximum latency (and that is effectively t= he best clock resolution you can give to users, irrespective of synchro= nization between guest clock and host clock). >=20 > The requirement is that the guest VM will be synced to the host clock= - I'm not talking on real-time in the meaning of speed, but I need tha= t the guest clock will measure the exact delay between packet transmiss= ion from the host to the packet received in the guest.=20 This makes no sense - if you want a "reliable clock synchronized up to = X microseconds", then you have to take into account any possible delay incurred by scheduling (as scheduling determines how reliable a clock read can be) into the clock reliability calculation. > My problem now is that sometimes when I receive the packet in the gue= st I calculate a negative delay because my clock in the guest is in del= ay relative to the host clock. Use the guest TSC and host TSC values (rdtsc in both). Then subtract from host TSC timestamps the tsc_offset value.=20 You'll get a "perfect" synchronized RDTSC clock.=20 To find out the tsc_offset value, enable kvm_write_tsc_offset tracepoin= t in the host (before the particular guest boots). > I expect to get a positive (jittered) delay in the order of 10s of us= =2E =20 > My test is very simple - I send a packet from host to the VM, I set t= he host time (tx_time) in the packet. When the guest receives the pack= et it reads its time (rx_time) and calculate the delay as : > Delay =3D rx_time - tx_time > I use the clock_gettime(REALTIME) in the host to set tx-time and in t= he guest to read rx_time.