From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id B3C75C4332F for ; Fri, 2 Dec 2022 09:41:17 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=Td66k9Xv1uQboaeBzSmgmOI1y9kvkWe5ELWRywwkkg4=; b=j7gsxgxnI5S5Oi F1PkxZxmt2apEmhCNtPDAYsoPMzQvjNuBc0Xwtgn7WSlbJD7KgeNLhXVO2cI54Kh+sAnlm3uudiXn 9ooeeIQ+t+MCZIfL/vp66F5JxNOSxC2bHjZi9/Iacz2mLcFYYx98eUuFEWPvcX5q5t8JjiWvIXHj8 VihQNt4Sdc5eZ/Mxv5zyVt/WYhXiJjCLe7dbUzTwKbJHywr8glBDARlleqIobjqcxnltLoE+VJf7M +kTYpYiNFOOZTvsxTG+Ql1obMd3cG1/sthku17b5axB5CfUt2u5/7wiIiwywG4ZD15gRFKKnWLMAm H0f2lxEy8xMRL9r6gNsA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1p12Wl-00F3Dv-Iw; Fri, 02 Dec 2022 09:40:15 +0000 Received: from ams.source.kernel.org ([2604:1380:4601:e00::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1p12Wi-00F3Cn-8w for linux-arm-kernel@lists.infradead.org; Fri, 02 Dec 2022 09:40:14 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by ams.source.kernel.org (Postfix) with ESMTPS id E6346B82076; Fri, 2 Dec 2022 09:40:10 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id ACD43C433D6; Fri, 2 Dec 2022 09:40:09 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1669974009; bh=C8ySU2FWQgsD68imRfeyGlSW0TJii+QC9V3BvojoXMw=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=AiAs0wGNZDPebO8ykbwc0tKssOFX09O6hEb0JF4/PrvRSSeDURg17GYfJ8XSlCsWf y8et5MdMSY9YTyVqI7He9Sydra7AF27/4cAMZLqPNl1ZWoK+31fJ3FeWvNv3VGUgp5 rA9+vP7VZKYYwSaMdPEPFci2Wd2x/F/KhmMPF4CHHOn6AxFJqm6KN/Igg+fmB/ps+z MOHhEVrvuM/kPONYYyPj3G27gySumUuXoGhg85sv6f9BBy5FY411YLVJ47Mcqs3HGQ C9ZJMxZlukJJ9KOcVsuEyuW9zml/+xYHdZR89SIC7R668Il17l19sd1rhd8HtkiAek pxdBZjOMXsthQ== Received: from 51-171-6-54-dynamic.agg9.chf.chf-qkr.eircom.net ([51.171.6.54] helo=wait-a-minute.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1p12Wd-00A2LU-9Q; Fri, 02 Dec 2022 09:40:07 +0000 Date: Fri, 02 Dec 2022 09:40:04 +0000 Message-ID: <87h6yeta8b.wl-maz@kernel.org> From: Marc Zyngier To: Akihiko Odaki Cc: linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, Mathieu Poirier , Oliver Upton , Suzuki K Poulose , Alexandru Elisei , James Morse , Will Deacon , Catalin Marinas , asahi@lists.linux.dev, Alyssa Rosenzweig , Sven Peter , Hector Martin Subject: Re: [PATCH 0/3] KVM: arm64: Handle CCSIDR associativity mismatches In-Reply-To: <525ff263-90b3-5b12-da31-171b09f9ad1b@daynix.com> References: <20221201104914.28944-1-akihiko.odaki@daynix.com> <867czbmlh1.wl-maz@kernel.org> <50499ee9-33fe-4f5d-9d0a-76ceef038333@daynix.com> <87lenqu37t.wl-maz@kernel.org> <525ff263-90b3-5b12-da31-171b09f9ad1b@daynix.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 51.171.6.54 X-SA-Exim-Rcpt-To: akihiko.odaki@daynix.com, linux-kernel@vger.kernel.org, kvmarm@lists.linux.dev, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org, mathieu.poirier@linaro.org, oliver.upton@linux.dev, suzuki.poulose@arm.com, alexandru.elisei@arm.com, james.morse@arm.com, will@kernel.org, catalin.marinas@arm.com, asahi@lists.linux.dev, alyssa@rosenzweig.io, sven@svenpeter.dev, marcan@marcan.st X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20221202_014012_627547_A33FD03E X-CRM114-Status: GOOD ( 44.76 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Fri, 02 Dec 2022 05:17:12 +0000, Akihiko Odaki wrote: > > >> On M2 MacBook Air, I have seen no other difference in standard ID > >> registers and CCSIDRs are exceptions. Perhaps Apple designed this way > >> so that macOS's Hypervisor can freely migrate vCPU, but I can't assure > >> that without more analysis. This is still enough to migrate vCPU > >> running Linux at least. > > > > I guess that MacOS hides more of the underlying HW than KVM does. And > > KVM definitely doesn't hide the MIDR_EL1 registers, which *are* > > different between the two clusters. > > It seems KVM stores a MIDR value of a CPU and reuse it as "invariant" > value for ioctls while it exposes the MIDR value each physical CPU > owns to vCPU. This only affects the VMM though, and not the guest which sees the MIDR of the CPU it runs on. The problem is that at short of pinning the vcpus, you don't know where they will run. So any value is fair game. > This may be a problem worth fixing. My understanding is that while > there is no serious application which requires vCPU migration among > physical clusters, Hey, I do that all the time with kvmtool! It's just that my guest do not care about being run on a CPU or another. > crosvm uses KVM on big.LITTLE processors by pinning > vCPU to physical CPU, and it is a real-world application which needs > to be supported. > > For an application like crosvm, you would expect the vCPU thread gets > the MIDR value of the physical CPU which the thread is pinned to when > it calls ioctl, but it can get one of another arbitrary CPU in > reality. No. It will get the MIDR of the CPU it runs on. Check again. What you describing above is solely for userspace. > > Fixing this problem will pose two design questions: > > 1. Should it expose a value consistent among clusters? > > For example, we can change the KVM initialization code so that it > initializes VPIDR with the value stored as "invariant". This would > help migrating vCPU among clusters, but if you pin each vCPU thread to > a distinct phyiscal CPU, you may rather want the vCPU to see the MIDR > value specific to each physical CPU and to apply quirks or tuning > parameters according to the value. Which is what happens. Not at the cluster level, but at the CPU level. The architecture doesn't describe what a *cluster* is. > 2. Should it be invariant or variable? > > Fortunately making it variable is easy. Arm provides VPIDR_EL1 > register to specify the value exposed as MPIDR_EL0 so there is no > trapping cost. And if you do that you make it impossible for the guest to mitigate errata, as most of the errata handling is based on the MIDR values. > ...or we may just say the value of MPIDR_EL0 (and possibly other I assume you meant MIDR_EL1 here, as MPIDR_EL1 is something else (and it has no _EL0 equivalent). > "invariant" registers) exposed via ioctl are useless and deprecated. Useless? Not really. The all are meaningful to the guest, and a change there will cause issues. CTR_EL0 must, for example, be an invariant. Otherwise, you need to trap all the CMOs when the {I,D}minLine values that are restored from userspace are bigger than the ones the HW has. Even worse, when the DIC/IDC bits are set from userspace while the HW has them cleared: you cannot mitigate that one, and you'll end up with memory corruption. I've been toying with the idea of exposing to guests the list of MIDR/REVIDR the guest is allowed to run on, as a PV service. This would allow that guest to enable all the mitigations it wants in one go. Not sure I have time for this at the moment, but that'd be something to explore. [...] > > So let's first build on top of HCR_EL2.TID2, and only then once we > > have an idea of the overhead add support for HCR_EL2.TID4 for the > > systems that have FEAT_EVT. > > That sounds good, I'll write a new series according to this idea. Thanks! M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel