From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1751226AbcICELW (ORCPT ); Sat, 3 Sep 2016 00:11:22 -0400 Received: from mail.kernel.org ([198.145.29.136]:55584 "EHLO mail.kernel.org" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1750719AbcICELU (ORCPT ); Sat, 3 Sep 2016 00:11:20 -0400 Date: Sat, 3 Sep 2016 13:04:10 +0900 From: Masami Hiramatsu To: Marcelo Tosatti Cc: Steven Rostedt , Stefan Hajnoczi , Luiz Capitulino , kvm@vger.kernel.org, linux-kernel@vger.kernel.org, pbonzini@redhat.com, rkrcmar@redhat.com, mhiramat@kernel.org Subject: Re: [PATCH 4/4] kvm: x86: export TSC offset to user-space Message-Id: <20160903130410.08e392d002d13c5fb9e0fc98@kernel.org> In-Reply-To: <20160903002310.GB12659@amt.cnet> References: <1472663145-1835-1-git-send-email-lcapitulino@redhat.com> <1472663145-1835-5-git-send-email-lcapitulino@redhat.com> <20160902134301.GC21771@stefanha-x1.localdomain> <20160902101541.2b670b09@gandalf.local.home> <20160903002310.GB12659@amt.cnet> X-Mailer: Sylpheed 3.5.0 (GTK+ 2.24.30; x86_64-pc-linux-gnu) Mime-Version: 1.0 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org On Fri, 2 Sep 2016 21:23:11 -0300 Marcelo Tosatti wrote: > On Fri, Sep 02, 2016 at 10:15:41AM -0400, Steven Rostedt wrote: > > On Fri, 2 Sep 2016 09:43:01 -0400 > > Stefan Hajnoczi wrote: > > > > > Can TSC offset changes occur at runtime? > > Yes, but Linux guests don't write to the TSC offset > after booting and unless user does manual TSC writes. > > > > One example is vcpu hotplug where the tracing tool would need to fetch > > > the new vcpu's TSC offset after tracing has already started. > > > > > > Another example is if QEMU or the guest change the TSC offset while > > > running. If the tracing tool doesn't notice this then trace events will have > > > incorrect timestamps. > > So what happens is this: > > HostTSC (a variable). > GuestTSC (variable) = GuestTSCOffset (fixed) + HostTSC (variable) The same idea has been done by Yoshihiro http://events.linuxfoundation.org/sites/events/files/cojp13_yunomae.pdf > Then the algorithm processes the trace as follows: > line = for each line(guest_trace) > line = line - GuestTSCOffset (only the timestamp of course) > > So from the moment the guest writes a new TSC offset, the host > should use the new TSC offset to subtract from the trace entries. > The trace entries are in fact: > > HostTSC + GuestTSCOffset > > So the guest trace should contain entries for "USE NEW TSC OFFSET, > VALUE: xxx", which can be done (hum not sure if guest entries > or host entries). > > However, correct me if i am wrong, the usecase seems to be: > > 1) Boot guest. > 2) run trace-cmd > 3) run workload > 4) read traces on host. IIRC, previous (current?) method is to run trace-cmd at first (before boot the guest) so that it can get tsc-offset event and can wait on a special unix domain socket. For above usecase, we have to have an interface to get the current tsc offset like Luis suggested. > > Another option is to have notifications as follows: record on a buffer > the following: > > [ EVENT: TSC offset write, VAL: [ host tsc value, guest tsc offset ] ] > [ EVENT: TSC offset write, VAL: [ host tsc value, guest tsc offset ] ] > > Then when merging the trace entries, you do: > > line = for each line(host trace) > write_to_merged_trace(line) > if (contains_tsc_offset_event(line)) { > GuestTSCOffset = line.GuestTSCOffset > if (!guest_tsc_offset_initialized) { > process_all_guest_lines( > line = line - GuestTSCOffset (only the timestamp of course) > } > } > > Aha, fail: the traces on the host are not sufficient to know when > to use which offset to subtract on the guest trace. > > So the only possibility is to have the guest inform the occurrence > of the events: however the guest does not have access to the TSC offset. > > So the host needs to inform the new tsc offset value and the guest needs > to inform when the event occurred on its side. So the algorithm can use > information on both traces to know which value to subtract on the > algorithm above. > > Is this necessary? Or people do: > 1) Boot guest. > 2) run trace-cmd > 3) run workload > 4) read traces on host. > > > I believe there are tracepoints for these events. They would obviously > > need to be enabled for the tracer to catch them. Yes, Yoshihiro introduced a tracepoint for that. http://lkml.iu.edu/hypermail/linux/kernel/1306.1/01741.html So, we have to trace this event in host side. > > > > I would also recommend that they go into their own instance to make > > sure other events do not overwrite them. > > > > -- Steve Thanks, -- Masami Hiramatsu