From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 78465C636D6 for ; Thu, 23 Feb 2023 18:26:59 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=mUK2czMhZIEM+Zei4YL4vf8rurSDo2tdsuwqvN6X7ds=; b=x/TAhSN30A2N/r C8SqiUM3GK32rOXUUFawi0cfdbptM28Eb5Owa1XTyZpGdQlls8t2ZU/caVqWTiWti091pbSNFYzPQ evm0c8yK4W0Fqnzv5SQ9R/byQ8G+kTFf8eIczFlcszAlKriyIf1GEaS8kl/WuT3KTheCVLbZaY+iI M2wHNkaQTnOWjKXTE7AfSdpOvJ+DVJyd+6rUpCE0JBCSK8fHWORBWHF1HEjfRJCbDI+n5FfUdHwHE dN+HAjkuHXZbPUA2CUiJsmSZOfaJYg+ZtWLuJdGXhAWxn7l+y34uxiB3HZiMkHL9IN+tvbugbvVp+ W9UEeE4+LQ93j6gXcV6g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1pVGIA-00HVxl-5L; Thu, 23 Feb 2023 18:26:06 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1pVGI6-00HVwa-Nj for linux-arm-kernel@lists.infradead.org; Thu, 23 Feb 2023 18:26:04 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id 65781B80D92; Thu, 23 Feb 2023 18:26:01 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 10612C433D2; Thu, 23 Feb 2023 18:26:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1677176760; bh=faUGEHXZIAbc4vaLIXbC1j60CezNNU8Sq6BxoorVfHQ=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=bE8v63YP37SIEjHPA1H9PNoZanbayAVlPhToj40ymVgfkifJbsnsF/kUCIKAGfIqz yenpvkNeP89uUBfnorpr6A7rEcNbJNli9GWw4tnPlldq/LuIGL5A+6WIliODLSufJq O7eo1arRgPnWmF2JNoUC5quNHGVpon+bEL8H1MgvRUqnPcGpsOiQ3Vjd7vAWj1NWrY WexQtRqTiHvgMuqxqR9Q3iteRE3YkOTjmxZrEeeF2qSXVsj+mvcVJb5Qy38mdIu0Jx cWzgxcpfJhUb0mr9kbGOVYWb4QFNEoi+pqh2SCXw0mk4MVjcWQF40U8LGBKvofYC/H rBg8HfFNvAlmw== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1pVGI1-00CiDK-My; Thu, 23 Feb 2023 18:25:57 +0000 Date: Thu, 23 Feb 2023 18:25:57 +0000 Message-ID: <867cw8xmq2.wl-maz@kernel.org> From: Marc Zyngier To: Oliver Upton Cc: kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, James Morse , Suzuki K Poulose , Zenghui Yu , Ricardo Koller , Simon Veith , dwmw2@infradead.org Subject: Re: [PATCH 08/16] KVM: arm64: timers: Allow userspace to set the counter offsets In-Reply-To: References: <20230216142123.2638675-1-maz@kernel.org> <20230216142123.2638675-9-maz@kernel.org> <86k00gy4so.wl-maz@kernel.org> <86bkllyku2.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/28.2 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: oliver.upton@linux.dev, kvmarm@lists.linux.dev, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, ricarkol@google.com, sveith@amazon.de, dwmw2@infradead.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20230223_102603_085015_BF1E07D0 X-CRM114-Status: GOOD ( 50.36 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Wed, 22 Feb 2023 16:34:32 +0000, Oliver Upton wrote: > > On Wed, Feb 22, 2023 at 11:56:53AM +0000, Marc Zyngier wrote: > > > > Chewing on this a bit more, I don't think userspace has any business > > > messing with virtual and physical time independently, especially when > > > nested virtualization comes into play. > > > > Well, NV already ignores the virtual offset completely (see how the > > virtual timer gets its offset reassigned at reset time). > > I'll need to have a look at that, but if we need to ignore user input on > the shiny new interface for NV then I really do wonder if it is the > right fit. The thing is, I'm not keen making the interface modal. Modes suck and lead to invasive userspace changes. I'd rather ignore irrelevant parameters than change the way userspace works. And you don't *have* to provide the virtual offset with NV. > > I previously toyed with this idea, and I really like it. However, the > > problem with this is that it breaks the current behaviour of having > > two different values for CNTVCT and CNTPCT in the guest, and CNTPCT > > representing the counter value on the host. > > > > Such a VM cannot be migrated *today*, but not everybody cares about > > migration. My "dual offset" approach allows the current behaviour to > > persist, and such a VM to be migrated. The luser even gets the choice > > of preserving counter continuity in the guest or to stay without a > > physical offset and reflect the host's counter. > > > > Is it a good behaviour? Of course not. Does anyone depend on it? I > > have no idea, but odds are that someone does. Can we break their toys? > > The jury is still out. > > Well, I think the new interface presents an opportunity to change the > rules around counter migration, and the illusion of two distinct offsets > for physical / virtual counters will need to be broken soon enough for > NV. Broken in the kernel, sure. Do we need to involve userspace in what was an initial mis-design of the API? Changing these rules mean changing the way userspace works, and I'm not keen on going there. > Do we need to bend over backwards for a theoretical use case with > the new UAPI? If anyone depends on the existing behavior then they can > continue to use the old UAPI to partially migrate the guest counters. I don't buy the old/new thing. My take is that these things should be cumulative if there isn't a hard reason to break the existing API. > My previous suggestion of tying the physical and virtual counters > together at VM creation would definitely break such a use case, though, > so we'd be at the point of requiring explicit opt-in from userspace. I'm trying to find a middle ground, so bear with me. Here's the situation as I see it: (1) a VM that is migrating today can only set the virtual offset and doesn't affect the physical counter. This behaviour must be preserved in we cannot prove that nobody relies on it. (2) setting the physical offset could be done by two means: (a) writing the counter register (like we do for CNTVCT) (b) providing an offset via a side channel I think (1) must stay forever, just like we still support the old GICv2 implicit initialisation. (2a) is also desirable as it requires no extra work on the VMM side. Just restore the damn thing, and nothing changes (we're finally able to migrate the physical timer). (2b) really is icing on the cake. The question is whether we can come up with an API offering (2b) that still allows (1) and (2a). I'd be happy with a new API that, when used, resets both offsets to the same value, matching your pretty picture. But the dual offsetting still has to exist internally. When it comes to NV, it uses either the physical offset that has been provided by writing CNTPCT, or the one that has been written via the new API. Under the hood, this is the same piece of data, of course. The only meaningful difference with my initial proposal is that there is no new virtual offset API. It is either register writes that obey the same rules as before, or a single offset setting. > > And I think that's the point where we differ. I can completely imagine > > some in-VM code using the physical counter to export some timestamping > > to the host (for tracing purposes, amongst other things). > > So in this case the guest and userspace would already be in cahoots, so > userspace could choose to not use UAPI. Hell, if userspace did > absolutely nothing then it all continues to work. Userspace, yes. Not necessarily the VMM. Let's say my guest spits a bunch of timestamped traces over a network connection, which I then correlate with host traces (all similarities with a shipping product are completely fortuitous...). The VMM isn't involved at all here. > Oh, we're definitely on the hook for any existing misuse of observable > KVM behavior. I just think if we're at the point of adding new UAPI we > may as well lay down some new rules with userspace to avoid surprises. > > OTOH, ignoring the virtual offset for NV is another way out of the mess, > but it just bothers me we're about to ignore input on a brand new > UAPI... I think my single-offset suggestion should answer your questioning. Thoughts? M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel