From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 418B5CFD354 for ; Fri, 11 Oct 2024 11:46:43 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender:List-Subscribe:List-Help :List-Post:List-Archive:List-Unsubscribe:List-Id:Content-Type:MIME-Version: References:In-Reply-To:Subject:Cc:To:From:Message-ID:Date:Reply-To: Content-Transfer-Encoding:Content-ID:Content-Description:Resent-Date: Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID:List-Owner; bh=yaIRAzKPcRfmDf8AyfvUPPZEjI1yIbE1dnXNs2X2NG0=; b=bHMLZWqmSO/Vcw2ox2fALiCucK 0UTYDesjdl6VRh1C3zTgpjtennbXiCmtDOQA5g+F5kj/GVe8L2XmrP3JcqxX9AemJdyD6+X/WSOyX Ay5t7vuO5028AuhG+xEVDks9fTvgcHZR4mpUMRA/4m/LcJoB4h3KB+1UM66e3gDYTkrdzv0pJcoqf ttRKU1Bqd2p1qzPHdffu2smK9/D2NrEMf/VHfN1wc9UP5S5UrMOe6VwmLhtSm7OGgZJa7wa7WtbwG 7S7yBpXhialqA2eQvILv7jcXZmA8+5J5f2GPn2LjU5M2saSDap32G1huKau9TI1xqktGjnhktN0BV +X5PE0ww==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.98 #2 (Red Hat Linux)) id 1szE6M-0000000GBXm-1fNt; Fri, 11 Oct 2024 11:46:34 +0000 Received: from nyc.source.kernel.org ([147.75.193.91]) by bombadil.infradead.org with esmtps (Exim 4.98 #2 (Red Hat Linux)) id 1szD1V-0000000Fzbc-41fR for linux-arm-kernel@lists.infradead.org; Fri, 11 Oct 2024 10:37:32 +0000 Received: from smtp.kernel.org (transwarp.subspace.kernel.org [100.75.92.58]) by nyc.source.kernel.org (Postfix) with ESMTP id A9C47A43600; Fri, 11 Oct 2024 10:37:19 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 0394DC4AF0B; Fri, 11 Oct 2024 10:37:28 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1728643048; bh=sLOhLNx8TSmEQHIOZ1Me0lpvTVNMQpFT9QQNL/SL41k=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=cAeFBo//m21VS8fAeczPodqGmm+o/Fd8hONjTfiTVS3VDhADHkKWWno96FrScNT+N aJIbwApK1Jrj6PAB4p9VEeQ7+H9qMkLVSsIinl3+knEoOtQD//HA3pQhr+UcaKXWnO YAtBk6k3MEdeDlRvz98R3eMg2EPGG0k2CfNfrLW3xOeeB/4s8VEE5yTxEzsOPup2RX OKmxkMtmcSImdHdNRzSTtdk4b1IKE+XptlRHFdXEc6GUxDH93SeGu+C60+/nioUF3/ bw+6WCt2t6J425YMU0wrgQjEz+45CgykR9H0uvHfjRH6mamejcRJ6jazP8VXIM2GR1 kl1DF5jRuZZkg== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1szD1R-002Vx7-Lf; Fri, 11 Oct 2024 11:37:25 +0100 Date: Fri, 11 Oct 2024 11:37:24 +0100 Message-ID: <86jzef53iz.wl-maz@kernel.org> From: Marc Zyngier To: Shameer Kolothum Cc: , , , , , , , , , , , , , Subject: Re: [RFC PATCH 0/6] KVM: arm64: Errata management for VM Live migration In-Reply-To: <20241011075053.80540-1-shameerali.kolothum.thodi@huawei.com> References: <20241011075053.80540-1-shameerali.kolothum.thodi@huawei.com> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.4 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: shameerali.kolothum.thodi@huawei.com, kvmarm@lists.linux.dev, oliver.upton@linux.dev, catalin.marinas@arm.com, will@kernel.org, mark.rutland@arm.com, cohuck@redhat.com, eric.auger@redhat.com, yuzenghui@huawei.com, wangzhou1@hisilicon.com, jiangkunkun@huawei.com, jonathan.cameron@huawei.com, anthony.jebson@huawei.com, linux-arm-kernel@lists.infradead.org, linuxarm@huawei.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20241011_033730_152200_0433CD24 X-CRM114-Status: GOOD ( 37.36 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org Hi Shameer, Thanks for getting the ball rolling on this one, much appreciated. On Fri, 11 Oct 2024 08:50:47 +0100, Shameer Kolothum wrote: > > Hi, > > On ARM64 platforms most of the errata workarounds are based on CPU > MIDR/REVIDR values and a number of these workarounds need to be > implemented by the Guest kernel as well. This creates a problem when > Guest needs to be migrated to a platform that differs in these > MIDR/REVIDR values even if the VMM can come up with a common minimum > feature list for the Guest using the recently introduced "Writable > ID registers" support. > > (This is roughly based on a discussion I had with Marc and Oliver > at KVM forum. Marc outlined his idea for a solution and this is an > attempt to implement it. Thanks to both and I take all the blame > if this is nowhere near what is intended/required) > > This RFC proposes a solution to handle the above issue by introducing > the following, > > 1. A new VM IOCTL, > KVM_ARM_SET_MIGRN_TARGET_CPUS _IOW(KVMIO, 0xb7, struct kvm_arm_migrn_cpus) > This can be used by the userspace(VMM) to set the target CPUs the > Guest will run in its lifetime. See patch #2 > 2. Add hypercall support for Guest kernel to retrieve any migration > errata bitmap(ARM_SMCCC_VENDOR_HYP_KVM_MIGRN_ERRATA) > The above will return the bitmaps in R0-R3 registers. See patch #4 > 3. The "capability" field in struct arm64_cpu_capabilities is a generated > one at present and may get renumbered or reordered. Hence, we can't use > this directly for migration errata bitmaps. Instead, introduced > "migartion_safe_cap", which has to be set statically for any > erratum that needs to be enabled and is safe for migration > purposes. See patches 3 & 6. > 4. Rest of the patches includes the plumbing required to populate the > errata bitmap based on the target CPUs set by the VMM and update the > system_cap based on it. > > ToDos:- > -We still need a way to handle the error in setting the invariant > registers(MIDR/REVIDR/AIDR) during Guest migration. Perhaps we can > handle it in userspace? > - Possibly we could do better to avoid the additional "migartion_safe_cap" use. > Suggestions welcome. > -There are errata that require more than MIDR/REVIDR, eg: CTR_EL0. > How to handle those? > -Check for locking requirements if any. > > This is lightly tested on a HiSilicon ARM64 platform. > > Please take a look and let me know your thoughts. Having eyeballed this very superficially, I think we can do something simpler, and maybe more future-proof: - I don't think KVM should be concerned about the description of the target CPUs. The hypercall you defined is the right thing to do, but the VMM should completely handle it. That's an implementation detail, but it would make things much simpler. - I don't think the "errata bitmap" works. That's a construct that is specific to Linux, and that cannot be supported for other OSs. It also limits the described issues to those the host knows, instead of the guest. The host doesn't have a clue what the guest really wants. Really, the guest should have enough information to decide what to do based on its own view of the ID registers and the list of CPUs it runs on. - To answer your question about CTR_EL0: KVM should (and does) sanitise that register by trapping it. This should be the default behaviour for things that need to be mitigated outside of MIDR/REVIDR. Thanks, M. -- Without deviation from the norm, progress is not possible.