From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id D5DDFC4345F for ; Thu, 2 May 2024 16:46:23 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:In-Reply-To:MIME-Version:References: Message-ID:Subject:Cc:To:From:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=RUvmwVSrItVN6yy3726YDvl5jJf9Yx8czLwAfkcJxlc=; b=ZbgiURCUsKd01i quXQ2Gl4m8gygQCgcTHVr++fihb3GrNMObPgmXot6hXE2GM3nIYb7SSTjFmbSNEYJztpKzupHBN/N zLAoYJS9tZCE8OGdJ3CFdXCrziYXlUSZBFi3bQuok6ZEV5IJ6JpBmkQYctMFL3xGkvgDvQ8aHrjEb /94yNhvBA59yueeIpbKwX5Um7Wh4qXr92Dk6SJBwbM2ae44AjWvmA1wCrlOWz1cskbV3Uaur93mZW 285UO5NHZtBI1hzINEr1DZE3csbbMb444T6zJ+RwP0U5H/s9/s3ZlCp/VSGQG64nl3SGdQR+ptEkt fJidt/caj89rZcMXnfoA==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.97.1 #2 (Red Hat Linux)) id 1s2ZZT-0000000DKnO-2lXy; Thu, 02 May 2024 16:46:11 +0000 Received: from out-180.mta1.migadu.com ([2001:41d0:203:375::b4]) by bombadil.infradead.org with esmtps (Exim 4.97.1 #2 (Red Hat Linux)) id 1s2ZZP-0000000DKlz-32lc for linux-arm-kernel@lists.infradead.org; Thu, 02 May 2024 16:46:10 +0000 Date: Thu, 2 May 2024 09:45:53 -0700 DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=linux.dev; s=key1; t=1714668360; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: in-reply-to:in-reply-to:references:references; bh=PdV8LdHHGzyQZPGhLVN9kiXMb7hiNdhFo8cDZA2g52w=; b=mvBZtoMaK0qwqPeJOurDbR6tNZ4CqO/IZWdDCnh8WvLr6P8X37vHy0zfPhLa1MOSM0O7nM SVYiQHoLG/QU/7vsGKqYhcfcq844rTXTefHMbq4FNz4JwNK+qHGMHVepDzXyhJK6/wJCyK 8uZZJ6UTTeCdFRsVG/3HTo3EsDwXC/s= X-Report-Abuse: Please report any abuse attempt to abuse@migadu.com and include these headers. From: Oliver Upton To: "Russell King (Oracle)" Cc: Marc Zyngier , Catalin Marinas , Will Deacon , James Morse , Suzuki K Poulose , Zenghui Yu , linux-arm-kernel@lists.infradead.org, kvmarm@lists.linux.dev Subject: Re: [PATCH RFC] KVM: arm64: allow ID_MMFR4_EL1 to be writable Message-ID: References: MIME-Version: 1.0 Content-Disposition: inline In-Reply-To: X-Migadu-Flow: FLOW_OUT X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20240502_094607_931094_220A2780 X-CRM114-Status: GOOD ( 39.39 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Thu, May 02, 2024 at 03:40:38PM +0100, Russell King (Oracle) wrote: > On Wed, May 01, 2024 at 06:59:17PM +0000, Oliver Upton wrote: > > On Wed, May 01, 2024 at 07:08:05PM +0100, Russell King (Oracle) wrote: > > > Yes, it did strike me as odd, since the description seems to imply that > > > XNX affects EL2, which the VM wouldn't have access to. So I'm not sure > > > why we don't just force it to zero. > > > > Probably because we failed to catch it in the first place and setting to > > 0 now would be even more UAPI breakage. Meh :-/ I don't see any immediate > > issues with the patch, especially since it is fixing a genuine UAPI > > breakage in KVM. > > I think the only two ways around this would be to: > > 1) teach QEMU about the contents of these registers, with which fields > in these registers can be ignored when reloading a VMs context. > > 2) allow userspace to write to the XNX field such that it can be set > to values seen with previous kernels (thus allowing at least one- > way migration.) > > (1) has the advantage that reloading a VM state on older vs newer > kernels can work in either direction, whereas (2) would only work > for state saved on an older kernel loaded onto a newer kernel. Yeah, so this is something that has affected my employer as well. I think (1) should only be expected of VMMs that want rollback safety, i.e. the ability to migrate state back to an older kernel. Our userspace initializes vCPUs from a fixed set of feature ID register values that prevents VMs on new kernels from picking up new CPU features. It is quite tedious, but necessary as rollback safety is very much a non-goal of the KVM UAPI. OTOH, in cases where KVM screws up and breaks UAPI, the kernel needs to do something special to accept the previously-advertised state even if it were nonsensical. For example, there was a bug where KVM advertised an IMP DEF PMU to VMs even though the only thing KVM virtualizes is PMUv3. We fixed it in commit f90f9360c3d7 ("KVM: arm64: Rewrite IMPDEF PMU version as NI") by accepting the old value in the ioctl and changing the field to NI internally. I dislike these sort of hacks, but when we're caught between upholding UAPI and the architecture it seems to be the best option. I wonder if an approach similar to this would be sufficient to address the SPE change that you noticed. > I've been bitten before with KVM differences between kernel versions > in the past - where the number of registers that userspace sees has > changed despite being on the same hardware. This is intended behavior, as VMs are initialized to the maximum feature set KVM is able to support. Forward-compatibility for the set of exposed registers is tested, see the get-reg-list selftest. > I'm now wondering what > testing goes on to validate this part of the UAPI across different > kernel versions on the same hardware. We may've been a bit more relaxed in the past with this, but in recent history we've been careful about preserving UAPI. On top of that, we now have some generalized infrastructure for dealing with these things by way of the 'writable' / mutable ID register work. Although it isn't precisely what you're looking for, the set_id_regs selftest ensures we at least accept features that are valid for the underlying HW platform and explicitly tests downgrades. > I did knock up a test program that dumped the list of registers so > that one could trivially diff the output between various kernels. > Maybe I need to extend that to dump the register values themselves, > and then maybe we need to find a way to get some kind of automated > testing setup to highlight differences. (something maybe kernelci > could add?) Given the absolutely massive test matrix of implementations and kernel versions I question our ability to support this. Additionally the only thing we'd care about upstream is the unsafe removal of a feature. Nevertheless, a starting point could be to drop some pr_info() into set_id_regs.c to print the starting value of the ID registers which could be diffed. -- Thanks, Oliver _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel