From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mail.kernel.org (mail.kernel.org [198.145.29.99]) by smtp.lore.kernel.org (Postfix) with ESMTP id A6EE3C433F5 for ; Fri, 1 Oct 2021 15:14:41 +0000 (UTC) Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by mail.kernel.org (Postfix) with ESMTPS id 7470161ABA for ; Fri, 1 Oct 2021 15:14:41 +0000 (UTC) DMARC-Filter: OpenDMARC Filter v1.4.1 mail.kernel.org 7470161ABA Authentication-Results: mail.kernel.org; dmarc=fail (p=none dis=none) header.from=redhat.com Authentication-Results: mail.kernel.org; spf=none smtp.mailfrom=lists.infradead.org DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:Content-Type: Content-Transfer-Encoding:List-Subscribe:List-Help:List-Post:List-Archive: List-Unsubscribe:List-Id:In-Reply-To:From:References:Cc:To:Subject: MIME-Version:Date:Message-ID:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=ZqWunw002cEROUhCT3+IuC7NDZKDV0qteOOr9ndwxvY=; b=NLo1xtBIxp1yZG D/Tr12MLdIIWDKOvWTqcz6+cY1+AHw0cr++xrVoI7P19+3EdkdocmzVzKcVGHaUNPxLbLdUGuMKpC POfp6hmWLQoIWc6sQvH63pYRXLePJcOq/e14GuLfLsdDefPFhCkBkBri8S3ucz84rIiHKutBXPeyC DeaLmSZGgKsVTm83nni8H3DsF9W3bqzAG+L6Rn/MPO1was/UIViBR1mbiT1uVsH6+cTdn5MnjqHU1 1jCT8AVFz+nuKZstbjnM1GG+z5qxRMtPNIiQCHBOud/hhbnfWeD06aBkd6vyZdRb4A713BdLGFZqX o6WUer5LawjGxVPOCPEA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1mWKDx-000gV0-2O; Fri, 01 Oct 2021 15:13:21 +0000 Received: from us-smtp-delivery-124.mimecast.com ([170.10.133.124]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1mWKDt-000gUE-8E for linux-arm-kernel@lists.infradead.org; Fri, 01 Oct 2021 15:13:19 +0000 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1633101194; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=LW6fp9rG/lvp5l8prGyc0xe8qMlUDZGZChEO4If5ox0=; b=Al/Db8XhhPODEAiBaLPPbsUI5HxzcbJi6pso+o/R6EujgOzsedjyRGy9dw22ZvwxPKO1L3 UvZfs1FCQpwRSGBklty8T/tQfH2lvPO5xlZB8QAWr22/qKlDl2rXA0wuNDzuYgDoHQCl9v 4wMhEl9c3d8guw4ks95ssDLaY41eops= Received: from mail-ed1-f72.google.com (mail-ed1-f72.google.com [209.85.208.72]) (Using TLS) by relay.mimecast.com with ESMTP id us-mta-202-bD-I8sRNOpaoSNSlon-Lhg-1; Fri, 01 Oct 2021 11:12:50 -0400 X-MC-Unique: bD-I8sRNOpaoSNSlon-Lhg-1 Received: by mail-ed1-f72.google.com with SMTP id x7-20020a50ba87000000b003dabd8354f1so6030161ede.7 for ; Fri, 01 Oct 2021 08:12:28 -0700 (PDT) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20210112; h=x-gm-message-state:message-id:date:mime-version:user-agent:subject :content-language:to:cc:references:from:in-reply-to :content-transfer-encoding; bh=LW6fp9rG/lvp5l8prGyc0xe8qMlUDZGZChEO4If5ox0=; b=gcFGcrnARSOBSr+UMBv5384gc6iWBIe779ofb9u6znMZGQg/fTSdYsok/jddjV+m13 KRRTJ9EnRWTVmCVb2h2LO1uUwy0CEw0Zazo6MMagHqc2gpIK5mF9wswpyMz9snomxz0w Mfk1ouv0JPafy3xx0iD6bOV8VprzY2DCB7aB5NiPtd8uwbBJqJJWsDTXjXUfbH36Vrap KQO3yXxc4I1VXBcb7/UnfROtzpFmRcqXqxlRozf2U3GJ9xSEmgieI2U0CzZ5sVYpNDNy OYb2tu3pF9tvv3a8XYDHSv6boLFqqkzraxBagparOGjK5co1Th7WYL4otmzyQjeNCe2S gFeA== X-Gm-Message-State: AOAM533tHm5pkXx/LPlZ45zOBtoM8tZXmt6DIktEi7KhHUIY3g9iVEc0 h/ALyGoRa1lUkPP31BEL1OPRk1lY4xiAsLhdDio6IN3C2z3FJgxRKoLnr4rGZ893kdRd9h/Va0y 0szCOn/4Tgr8ejwB7wp5PTIXUigb1lXXq8mk= X-Received: by 2002:a17:907:3e03:: with SMTP id hp3mr6732922ejc.183.1633101142626; Fri, 01 Oct 2021 08:12:22 -0700 (PDT) X-Google-Smtp-Source: ABdhPJxSleQpwCGZXW71w2qzpx6TzvbZz4Gx6iqhZH1Md89m/5OKOBFh+S0Y4ZfLrRfA1AXKxqbKQQ== X-Received: by 2002:a17:907:3e03:: with SMTP id hp3mr6732898ejc.183.1633101142335; Fri, 01 Oct 2021 08:12:22 -0700 (PDT) Received: from [192.168.10.118] ([93.56.162.200]) by smtp.gmail.com with ESMTPSA id ee13sm1227751edb.14.2021.10.01.08.12.20 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Fri, 01 Oct 2021 08:12:21 -0700 (PDT) Message-ID: <7901cb84-052d-92b6-1e6a-028396c2c691@redhat.com> Date: Fri, 1 Oct 2021 17:12:20 +0200 MIME-Version: 1.0 User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:91.0) Gecko/20100101 Thunderbird/91.1.0 Subject: Re: [PATCH v8 7/7] KVM: x86: Expose TSC offset controls to userspace To: Marcelo Tosatti Cc: Oliver Upton , kvm@vger.kernel.org, kvmarm@lists.cs.columbia.edu, Sean Christopherson , Marc Zyngier , Peter Shier , Jim Mattson , David Matlack , Ricardo Koller , Jing Zhang , Raghavendra Rao Anata , James Morse , Alexandru Elisei , Suzuki K Poulose , linux-arm-kernel@lists.infradead.org, Andrew Jones , Will Deacon , Catalin Marinas References: <20210916181538.968978-1-oupton@google.com> <20210916181538.968978-8-oupton@google.com> <20210930191416.GA19068@fuller.cnet> <48151d08-ee29-2b98-b6e1-f3c8a1ff26bc@redhat.com> <20211001103200.GA39746@fuller.cnet> From: Paolo Bonzini In-Reply-To: <20211001103200.GA39746@fuller.cnet> Authentication-Results: relay.mimecast.com; auth=pass smtp.auth=CUSA124A263 smtp.mailfrom=pbonzini@redhat.com X-Mimecast-Spam-Score: 0 X-Mimecast-Originator: redhat.com Content-Language: en-US X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20211001_081317_911207_AAB89CD0 X-CRM114-Status: GOOD ( 20.40 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Transfer-Encoding: 7bit Content-Type: text/plain; charset="us-ascii"; Format="flowed" Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On 01/10/21 12:32, Marcelo Tosatti wrote: >> +1. Invoke the KVM_GET_CLOCK ioctl to record the host TSC (t_0), + >> kvmclock nanoseconds (k_0), and realtime nanoseconds (r_0). + [...] >> +4. Invoke the KVM_SET_CLOCK ioctl, providing the kvmclock >> nanoseconds + (k_0) and realtime nanoseconds (r_0) in their >> respective fields. + Ensure that the KVM_CLOCK_REALTIME flag is >> set in the provided + structure. KVM will advance the VM's >> kvmclock to account for elapsed + time since recording the clock >> values. > > You can't advance both kvmclock (kvmclock_offset variable) and the > TSCs, which would be double counting. > > So you have to either add the elapsed realtime (1) between > KVM_GET_CLOCK to kvmclock (which this patch is doing), or to the > TSCs. If you do both, there is double counting. Am i missing > something? Probably one of these two (but it's worth pointing out both of them): 1) the attribute that's introduced here *replaces* KVM_SET_MSR(MSR_IA32_TSC), so the TSC is not added. 2) the adjustment formula later in the algorithm does not care about how much time passed between step 1 and step 4. It just takes two well known (TSC, kvmclock) pairs, and uses them to ensure the guest TSC is the same on the destination as if the guest was still running on the source. It is irrelevant that one of them is before migration and one is after, all it matters is that one is on the source and one is on the destination. Perhaps we can add to step 6 something like: > +6. Adjust the guest TSC offsets for every vCPU to account for (1) > time + elapsed since recording state and (2) difference in TSCs > between the + source and destination machine: + + new_off_n = t_0 > + off_n + (k_1 - k_0) * freq - t_1 + "off + t - k * freq" is the guest TSC value corresponding to a time of 0 in kvmclock. The above formula ensures that it is the same on the destination as it was on the source. Also, the names are a bit hard to follow. Perhaps t_0 tsc_src t_1 tsc_dest k_0 guest_src k_1 guest_dest r_0 host_src off_n ofs_src[i] new_off_n ofs_dest[i] Paolo _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel