From: Marcelo Tosatti <mtosatti@redhat.com>
To: Radim Krcmar <rkrcmar@redhat.com>
Cc: Miroslav Lichvar <mlichvar@redhat.com>,
kvm@vger.kernel.org, linux-kernel@vger.kernel.org,
Paolo Bonzini <pbonzini@redhat.com>,
Richard Cochran <richardcochran@gmail.com>
Subject: Re: [patch 3/3] PTP: add kvm PTP driver
Date: Wed, 18 Jan 2017 10:24:58 -0200 [thread overview]
Message-ID: <20170118122456.GC13762@amt.cnet> (raw)
In-Reply-To: <20170118121738.GA14832@amt.cnet>
On Wed, Jan 18, 2017 at 10:17:38AM -0200, Marcelo Tosatti wrote:
> On Tue, Jan 17, 2017 at 04:36:21PM +0100, Radim Krcmar wrote:
> > 2017-01-17 09:30-0200, Marcelo Tosatti:
> > > On Tue, Jan 17, 2017 at 09:03:27AM +0100, Miroslav Lichvar wrote:
> > >> On Mon, Jan 16, 2017 at 06:01:14PM -0200, Marcelo Tosatti wrote:
> > >> > On Mon, Jan 16, 2017 at 05:47:15PM -0200, Marcelo Tosatti wrote:
> > >> > > On Mon, Jan 16, 2017 at 05:36:55PM -0200, Marcelo Tosatti wrote:
> > >> > > > Sorry, unless i am misunderstanding how this works, it'll get the guest clock
> > >> > > > 2us behind, which is something not wanted.
> > >> > > >
> > >> > > > Miroslav, if ->gettime64 returns the host realtime at 2us in the past,
> > >> > > > this means Chrony will sync the guest clock to
> > >> > > >
> > >> > > > host realtime - 2us
> > >> > > >
> > >> > > > Is that correct?
> > >>
> > >> Probably. It depends on the error of both host and guest timestamps.
> > >> If the error is the same on both sides, it will cancel out. An
> > >> occasional spike in the delay shouldn't be a problem as the reading
> > >> will be filtered out, but for best accuracy it's necessary that the
> > >> host's timestamp is taken in the middle between the guest's
> > >> timestamps.
> > >
> > > The problem is that spikes can be far from occasional: it depends on activity of
> > > the host CPU and interrupts. Whose delay can be "intermittent": as long
> > > as interrupts are being sent to the host CPU, for example, the delay
> > > will be high (which can last minutes).
> > >
> > > The TSC reading in the guest KVM PTP driver corrects for that delay.
> > >
> > >> Users of the PTP_SYS_OFFSET ioctl assume that (ts[0]+ts[2])/2
> > >> corresponds to ts[1], (ts[2]+ts[4])/2 corresponds to ts[3], and so on.
> > >>
> > >> ts[1] ts[3]
> > >> Host time ---------+---------+........
> > >> | |
> > >> | |
> > >> Guest time ----+---------+---------+......
> > >> ts[0] ts[2] ts[4]
> >
> > KVM PTP delay moves host ts[i] to be close to guest ts[i+1] and makes
> > the offset very consistent, so the graph would look like:
> >
> > ts[1] ts[3]
> > Host time -------------+---------+........
> > | |
> > | |
> > Guest time ----+---------+---------+......
> > ts[0] ts[2] ts[4]
> >
> > which doesn't sound good if users assume that the host reading is in the
> > middle -- the guest time would be ahead of the host time.
> >
> > I'm wondering why is the PTP precision around 10ns, when the hypercall
> > takes around 2-3k cycles. Have you measured the guest<->host offset by
> > getting the output of the hypercall, i.e.
> > {host_sec @ tsc, host_nsec @ tsc, tsc}
> > and comparing it with guest time computed from the same tsc, i.e.
> > {guest_sec @ tsc, guest_nsec @ tsc}
> > ?
> >
> > Thanks.
>
> Testcase: run a guest and a loop sending SIGUSR1 to vcpu0 (emulating
> intense interrupts). Follows results:
>
> Without TSC delta calculation:
> =============================
>
> #* PHC0 0 3 377 2 -99ns[ +206ns] +/- 116ns
> #* PHC0 0 3 377 8 +202ns[ +249ns] +/- 111ns
> #* PHC0 0 3 377 8 -213ns[ +683ns] +/- 88ns
> #* PHC0 0 3 377 6 +77ns[ +319ns] +/- 56ns
> #* PHC0 0 3 377 4 -771ns[-1029ns] +/- 93ns
> #* PHC0 0 3 377 10 -49ns[ -58ns] +/- 121ns
> #* PHC0 0 3 377 9 +562ns[ +703ns] +/- 107ns
> #* PHC0 0 3 377 6 -2ns[ -3ns] +/- 94ns
> #* PHC0 0 3 377 4 +451ns[ +494ns] +/- 138ns
> #* PHC0 0 3 377 11 -67ns[ -74ns] +/- 113ns
> #* PHC0 0 3 377 8 +244ns[ +264ns] +/- 119ns
> #* PHC0 0 3 377 7 -696ns[ -890ns] +/- 89ns
> #* PHC0 0 3 377 4 +468ns[ +560ns] +/- 110ns
> #* PHC0 0 3 377 11 -310ns[ -430ns] +/- 72ns
> #* PHC0 0 3 377 9 +189ns[ +298ns] +/- 54ns
> #* PHC0 0 3 377 7 +594ns[ +473ns] +/- 96ns
> #* PHC0 0 3 377 5 +151ns[ +280ns] +/- 71ns
> #* PHC0 0 3 377 10 -590ns[ -696ns] +/- 94ns
> #* PHC0 0 3 377 8 +415ns[ +526ns] +/- 74ns
> #* PHC0 0 3 377 6 +1381ns[+1469ns] +/- 101ns
> #* PHC0 0 3 377 4 +571ns[+1304ns] +/- 54ns
> #* PHC0 0 3 377 8 -5ns[ +71ns] +/- 139ns
> #* PHC0 0 3 377 7 -247ns[ -502ns] +/- 69ns
> #* PHC0 0 3 377 5 -283ns[ +879ns] +/- 73ns
> #* PHC0 0 3 377 3 +148ns[ -109ns] +/- 61ns
>
> With TSC delta calculation:
> ============================
>
> #* PHC0 0 3 377 7 +379ns[ +432ns] +/- 53ns
> #* PHC0 0 3 377 9 +106ns[ +420ns] +/- 42ns
> #* PHC0 0 3 377 7 -58ns[ -136ns] +/- 62ns
> #* PHC0 0 3 377 12 +93ns[ -38ns] +/- 64ns
> #* PHC0 0 3 377 8 +84ns[ +107ns] +/- 69ns
> #* PHC0 0 3 377 3 -76ns[ -103ns] +/- 52ns
> #* PHC0 0 3 377 7 +52ns[ +63ns] +/- 50ns
> #* PHC0 0 3 377 11 +29ns[ +31ns] +/- 70ns
> #* PHC0 0 3 377 7 -47ns[ -56ns] +/- 42ns
> #* PHC0 0 3 377 10 -35ns[ -42ns] +/- 33ns
> #* PHC0 0 3 377 7 -32ns[ -34ns] +/- 42ns
> #* PHC0 0 3 377 11 -172ns[ -173ns] +/- 118ns
> #* PHC0 0 3 377 6 +65ns[ +76ns] +/- 23ns
> #* PHC0 0 3 377 9 +18ns[ +23ns] +/- 37ns
> #* PHC0 0 3 377 6 +41ns[ -60ns] +/- 30ns
> #* PHC0 0 3 377 10 +39ns[ +183ns] +/- 42ns
> #* PHC0 0 3 377 6 +50ns[ +102ns] +/- 86ns
> #* PHC0 0 3 377 11 +50ns[ +75ns] +/- 52ns
> #* PHC0 0 3 377 6 +50ns[ +116ns] +/- 100ns
> #* PHC0 0 3 377 10 +46ns[ +65ns] +/- 79ns
> #* PHC0 0 3 377 7 -38ns[ -51ns] +/- 29ns
> #* PHC0 0 3 377 10 -11ns[ -12ns] +/- 32ns
> #* PHC0 0 3 377 7 -31ns[ -32ns] +/- 99ns
> #* PHC0 0 3 377 10 +222ns[ +238ns] +/- 58ns
> #* PHC0 0 3 377 6 +185ns[ +207ns] +/- 39ns
> #* PHC0 0 3 377 10 -392ns[ -394ns] +/- 118ns
> #* PHC0 0 3 377 6 -9ns[ -50ns] +/- 35ns
> #* PHC0 0 3 377 10 -346ns[ -355ns] +/- 111ns
>
>
> Do you still want to drop it in favour of simplicity?
>
This is the output of "chronyc sources". See section "Time sources"
of https://chrony.tuxfamily.org/doc/2.4/chronyc.html.
next prev parent reply other threads:[~2017-01-18 12:24 UTC|newest]
Thread overview: 56+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-01-13 12:01 [patch 0/3] KVM virtual PTP driver Marcelo Tosatti
2017-01-13 12:01 ` [patch 1/3] KVM: x86: provide realtime host clock via vsyscall notifiers Marcelo Tosatti
2017-01-13 15:18 ` Radim Krcmar
2017-01-13 15:34 ` Marcelo Tosatti
2017-01-13 16:28 ` Radim Krcmar
2017-01-13 17:51 ` Marcelo Tosatti
2017-01-16 15:40 ` Radim Krcmar
2017-01-13 15:41 ` Konrad Rzeszutek Wilk
2017-01-13 15:46 ` Marcelo Tosatti
2017-01-13 15:46 ` Marcelo Tosatti
2017-01-13 15:41 ` Konrad Rzeszutek Wilk
2017-01-13 12:01 ` [patch 2/3] KVM: x86: add KVM_HC_CLOCK_OFFSET hypercall Marcelo Tosatti
2017-01-13 15:31 ` Radim Krcmar
2017-01-13 15:43 ` Marcelo Tosatti
2017-01-13 17:07 ` Radim Krcmar
2017-01-13 17:57 ` Marcelo Tosatti
2017-01-13 12:01 ` [patch 3/3] PTP: add kvm PTP driver Marcelo Tosatti
2017-01-13 15:56 ` Radim Krcmar
2017-01-13 17:40 ` Marcelo Tosatti
2017-01-16 16:26 ` Radim Krcmar
2017-01-16 16:54 ` Radim Krcmar
2017-01-16 17:08 ` Marcelo Tosatti
2017-01-16 17:27 ` Radim Krcmar
2017-01-16 17:39 ` Marcelo Tosatti
2017-01-16 18:01 ` Radim Krcmar
2017-01-16 19:36 ` Marcelo Tosatti
2017-01-16 19:47 ` Marcelo Tosatti
2017-01-16 20:01 ` Marcelo Tosatti
2017-01-17 8:03 ` Miroslav Lichvar
2017-01-17 11:30 ` Marcelo Tosatti
2017-01-17 15:36 ` Radim Krcmar
2017-01-18 12:17 ` Marcelo Tosatti
2017-01-18 12:24 ` Marcelo Tosatti [this message]
2017-01-18 12:46 ` Paolo Bonzini
2017-01-18 13:36 ` Miroslav Lichvar
2017-01-18 14:02 ` Paolo Bonzini
2017-01-18 14:50 ` Marcelo Tosatti
2017-01-18 15:35 ` Radim Krcmar
2017-01-18 15:45 ` Paolo Bonzini
2017-01-18 15:57 ` Marcelo Tosatti
2017-01-18 14:24 ` Marcelo Tosatti
2017-01-18 15:54 ` Miroslav Lichvar
2017-01-18 16:07 ` Paolo Bonzini
2017-01-18 16:14 ` Radim Krcmar
2017-01-18 14:37 ` Marcelo Tosatti
2017-01-18 14:53 ` Marcelo Tosatti
2017-01-18 15:20 ` Radim Krcmar
2017-01-18 15:28 ` Marcelo Tosatti
2017-01-20 14:18 ` Radim Krcmar
2017-01-18 15:59 ` Radim Krcmar
2017-01-16 17:04 ` Marcelo Tosatti
2017-01-16 17:46 ` Radim Krcmar
2017-01-16 19:33 ` Marcelo Tosatti
2017-01-14 15:26 ` Richard Cochran
2017-01-16 15:48 ` Radim Krcmar
-- strict thread matches above, loose matches on Subject: below --
2017-01-13 18:45 [patch 0/3] KVM virtual PTP driver (v2) Marcelo Tosatti
2017-01-13 18:46 ` [patch 3/3] PTP: add kvm PTP driver Marcelo Tosatti
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170118122456.GC13762@amt.cnet \
--to=mtosatti@redhat.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mlichvar@redhat.com \
--cc=pbonzini@redhat.com \
--cc=richardcochran@gmail.com \
--cc=rkrcmar@redhat.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.