From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from mm01.cs.columbia.edu (mm01.cs.columbia.edu [128.59.11.253]) by smtp.lore.kernel.org (Postfix) with ESMTP id E59D0C433FE for ; Tue, 8 Feb 2022 17:49:05 +0000 (UTC) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 58C8B4B0F1; Tue, 8 Feb 2022 12:49:05 -0500 (EST) X-Virus-Scanned: at lists.cs.columbia.edu Authentication-Results: mm01.cs.columbia.edu (amavisd-new); dkim=softfail (fail, message has been altered) header.i=@kernel.org Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id DdzMAnuCFUYG; Tue, 8 Feb 2022 12:49:04 -0500 (EST) Received: from mm01.cs.columbia.edu (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 389E64B0FC; Tue, 8 Feb 2022 12:49:04 -0500 (EST) Received: from localhost (localhost [127.0.0.1]) by mm01.cs.columbia.edu (Postfix) with ESMTP id 8CE744B0F7 for ; Tue, 8 Feb 2022 12:49:03 -0500 (EST) X-Virus-Scanned: at lists.cs.columbia.edu Received: from mm01.cs.columbia.edu ([127.0.0.1]) by localhost (mm01.cs.columbia.edu [127.0.0.1]) (amavisd-new, port 10024) with ESMTP id sJiPxzkGCqUR for ; Tue, 8 Feb 2022 12:49:02 -0500 (EST) Received: from dfw.source.kernel.org (dfw.source.kernel.org [139.178.84.217]) by mm01.cs.columbia.edu (Postfix) with ESMTPS id 3CC194B0F1 for ; Tue, 8 Feb 2022 12:49:02 -0500 (EST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C5A3B617E3; Tue, 8 Feb 2022 17:49:00 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 37060C004E1; Tue, 8 Feb 2022 17:49:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1644342540; bh=fiDA/ww0hH9Ihny0p1Mw46FtIdZcId1+1g9QAhPQmMQ=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=ThxP1jgwknTX415K1izO5hWFXB3QTxYCPffGDOViUTzgffNYlps4PC96hjrk8rNYX kPXzHQ7w/VGVhwJ8wny31enHFT4WCwLtBDdGPA6MvFOQvrN+t6uC5LWnq1aZ7bcXCb RZqX5hBcrqaJgPezu2NYCzG59/kw0liK9v2vslhT2m3jsIZNoPcmqwpCRi0oTytQSG 0mH53xrRqSNfyp2QbovIBFfev4rUCgKxexdyAB5joNbtkYe+cPLJyhzhsFPQCPMp7b z5UPlJow8XUsDNQmGt6RfkZj3r3+Xk5c+p7JZbDih2zS232alW7rH/3tqdjLcZziSH 7JlpPSXbwpWIw== Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1nHUbq-006L8W-8g; Tue, 08 Feb 2022 17:48:58 +0000 Date: Tue, 08 Feb 2022 17:48:57 +0000 Message-ID: <878rul5jw6.wl-maz@kernel.org> From: Marc Zyngier To: Sean Christopherson Subject: Re: KVM/arm64: Guest ABI changes do not appear rollback-safe In-Reply-To: References: <87ilyitt6e.wl-maz@kernel.org> <87lf3drmvp.wl-maz@kernel.org> <875yq88app.wl-maz@kernel.org> <878ruld72v.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: seanjc@google.com, oupton@google.com, rananta@google.com, drjones@redhat.com, kvmarm@lists.cs.columbia.edu, pshier@google.com, ricarkol@google.com, reijiw@google.com, jingzhangos@google.com, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, Alexandru.Elisei@arm.com, suzuki.poulose@arm.com, peter.maydell@linaro.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Cc: kvm@vger.kernel.org, pshier@google.com, kvmarm@lists.cs.columbia.edu, linux-arm-kernel@lists.infradead.org X-BeenThere: kvmarm@lists.cs.columbia.edu X-Mailman-Version: 2.1.14 Precedence: list List-Id: Where KVM/ARM decisions are made List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Errors-To: kvmarm-bounces@lists.cs.columbia.edu Sender: kvmarm-bounces@lists.cs.columbia.edu On Tue, 08 Feb 2022 16:58:26 +0000, Sean Christopherson wrote: > > On Tue, Feb 08, 2022, Oliver Upton wrote: > > Hi Marc, > > > > On Tue, Feb 8, 2022 at 1:46 AM Marc Zyngier wrote: > > > > > KVM currently restricts the vcpu features to be unified across vcpus, > > > > > but that's only an implementation choice. > > > > > > > > But that implementation choice has become ABI, no? How could support > > > > for asymmetry be added without requiring userspace opt-in or breaking > > > > existing VMMs that depend on feature unification? > > > > > > Of course, you'd need some sort of advertising of this new behaviour. > > > > > > One thing I would like to add to the current state of thing is an > > > indication of whether the effects of a sysreg being written from > > > userspace are global or local to a vcpu. You'd need a new capability, > > > and an extra flag added to the encoding of each register. > > > > Ah. I think that is a much more reasonable fit then. VMMs unaware of > > this can continue to migrate new bits (albeit at the cost of > > potentially higher lock contention for the per-VM stuff), and those > > that do can reap the benefits of writing such attributes exactly once. > > But the "proper" usage is no different than adding support for > VM-scoped variants of KVM_{G,S}ET_ONE_REG and friends, and a > VM-scoped variant is conceptually a lot cleaner IMO. And making > them truly VM-scoped means KVM can do things like support sysregs > that are immutable after vCPUs are created. It is different, because your approach requires you to update all the existing VMMs to find out which register is of which kind. Not to mention that global sysregs are an absolute oddity in the face of the architecture (there is none in the base architecture). > So long as KVM defaults to '0' for all such registers, lack of > migration support in userspace that isn't aware of the new API, > i.e. doesn't do KVM_GET_REG_LIST at a VM-scope, is a nop because > said userspace also won't modify the registers in the first place. We want any VMM that correctly uses the API today to seamlessly be able to save/restore any new feature. QEMU does that, and it should continue to work no matter what new feature we add to the list. M. -- Without deviation from the norm, progress is not possible. _______________________________________________ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from bombadil.infradead.org (bombadil.infradead.org [198.137.202.133]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.lore.kernel.org (Postfix) with ESMTPS id 68BB7C433EF for ; Tue, 8 Feb 2022 17:50:29 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; q=dns/txt; c=relaxed/relaxed; d=lists.infradead.org; s=bombadil.20210309; h=Sender: Content-Transfer-Encoding:Content-Type:List-Subscribe:List-Help:List-Post: List-Archive:List-Unsubscribe:List-Id:MIME-Version:References:In-Reply-To: Subject:Cc:To:From:Message-ID:Date:Reply-To:Content-ID:Content-Description: Resent-Date:Resent-From:Resent-Sender:Resent-To:Resent-Cc:Resent-Message-ID: List-Owner; bh=pA+mU+Drthuro/WdyK4odjCAZFPDMbFpOZ1/DGIKlFA=; b=Mg+SXHmly2ZHps ujZ95rOdUPNiWF2lTA3VpmKba1R1REYG554kbXCjtACKixOicqN9faO2Xp/SYhcZID7etKsmP1zaG AJJpjarRZsjhtkuPr8mRRFzVsOL/478B+RYhWvmUmudg0DxeOukU4rtxZgljcg1luoZJVQfEOzL9g tErn+PQZEctm05cS3Zm++iUxdgsHzhIKpeVJa6q9gH+MlAM0ZXEFQREYl5P+g5vryh21qt4LN48v2 XOBVo4CSoQvBYC/jirQcAC4VeXTNfZ7F3rwPJ/givV9sonrRSfWHy1E3bfTrtiX9ZbRLs/qwMpiw7 fOaIMv5cA35+403sEG0g==; Received: from localhost ([::1] helo=bombadil.infradead.org) by bombadil.infradead.org with esmtp (Exim 4.94.2 #2 (Red Hat Linux)) id 1nHUbx-00FAYP-5w; Tue, 08 Feb 2022 17:49:05 +0000 Received: from dfw.source.kernel.org ([2604:1380:4641:c500::1]) by bombadil.infradead.org with esmtps (Exim 4.94.2 #2 (Red Hat Linux)) id 1nHUbt-00FAY6-8i for linux-arm-kernel@lists.infradead.org; Tue, 08 Feb 2022 17:49:02 +0000 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C5A3B617E3; Tue, 8 Feb 2022 17:49:00 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 37060C004E1; Tue, 8 Feb 2022 17:49:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1644342540; bh=fiDA/ww0hH9Ihny0p1Mw46FtIdZcId1+1g9QAhPQmMQ=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=ThxP1jgwknTX415K1izO5hWFXB3QTxYCPffGDOViUTzgffNYlps4PC96hjrk8rNYX kPXzHQ7w/VGVhwJ8wny31enHFT4WCwLtBDdGPA6MvFOQvrN+t6uC5LWnq1aZ7bcXCb RZqX5hBcrqaJgPezu2NYCzG59/kw0liK9v2vslhT2m3jsIZNoPcmqwpCRi0oTytQSG 0mH53xrRqSNfyp2QbovIBFfev4rUCgKxexdyAB5joNbtkYe+cPLJyhzhsFPQCPMp7b z5UPlJow8XUsDNQmGt6RfkZj3r3+Xk5c+p7JZbDih2zS232alW7rH/3tqdjLcZziSH 7JlpPSXbwpWIw== Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1nHUbq-006L8W-8g; Tue, 08 Feb 2022 17:48:58 +0000 Date: Tue, 08 Feb 2022 17:48:57 +0000 Message-ID: <878rul5jw6.wl-maz@kernel.org> From: Marc Zyngier To: Sean Christopherson Cc: Oliver Upton , Raghavendra Rao Ananta , Andrew Jones , kvmarm@lists.cs.columbia.edu, pshier@google.com, ricarkol@google.com, reijiw@google.com, jingzhangos@google.com, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, Alexandru.Elisei@arm.com, suzuki.poulose@arm.com, Peter Maydell Subject: Re: KVM/arm64: Guest ABI changes do not appear rollback-safe In-Reply-To: References: <87ilyitt6e.wl-maz@kernel.org> <87lf3drmvp.wl-maz@kernel.org> <875yq88app.wl-maz@kernel.org> <878ruld72v.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: seanjc@google.com, oupton@google.com, rananta@google.com, drjones@redhat.com, kvmarm@lists.cs.columbia.edu, pshier@google.com, ricarkol@google.com, reijiw@google.com, jingzhangos@google.com, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, Alexandru.Elisei@arm.com, suzuki.poulose@arm.com, peter.maydell@linaro.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false X-CRM114-Version: 20100106-BlameMichelson ( TRE 0.8.0 (BSD) ) MR-646709E3 X-CRM114-CacheID: sfid-20220208_094901_428146_6D8029AD X-CRM114-Status: GOOD ( 29.82 ) X-BeenThere: linux-arm-kernel@lists.infradead.org X-Mailman-Version: 2.1.34 Precedence: list List-Id: List-Unsubscribe: , List-Archive: List-Post: List-Help: List-Subscribe: , Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit Sender: "linux-arm-kernel" Errors-To: linux-arm-kernel-bounces+linux-arm-kernel=archiver.kernel.org@lists.infradead.org On Tue, 08 Feb 2022 16:58:26 +0000, Sean Christopherson wrote: > > On Tue, Feb 08, 2022, Oliver Upton wrote: > > Hi Marc, > > > > On Tue, Feb 8, 2022 at 1:46 AM Marc Zyngier wrote: > > > > > KVM currently restricts the vcpu features to be unified across vcpus, > > > > > but that's only an implementation choice. > > > > > > > > But that implementation choice has become ABI, no? How could support > > > > for asymmetry be added without requiring userspace opt-in or breaking > > > > existing VMMs that depend on feature unification? > > > > > > Of course, you'd need some sort of advertising of this new behaviour. > > > > > > One thing I would like to add to the current state of thing is an > > > indication of whether the effects of a sysreg being written from > > > userspace are global or local to a vcpu. You'd need a new capability, > > > and an extra flag added to the encoding of each register. > > > > Ah. I think that is a much more reasonable fit then. VMMs unaware of > > this can continue to migrate new bits (albeit at the cost of > > potentially higher lock contention for the per-VM stuff), and those > > that do can reap the benefits of writing such attributes exactly once. > > But the "proper" usage is no different than adding support for > VM-scoped variants of KVM_{G,S}ET_ONE_REG and friends, and a > VM-scoped variant is conceptually a lot cleaner IMO. And making > them truly VM-scoped means KVM can do things like support sysregs > that are immutable after vCPUs are created. It is different, because your approach requires you to update all the existing VMMs to find out which register is of which kind. Not to mention that global sysregs are an absolute oddity in the face of the architecture (there is none in the base architecture). > So long as KVM defaults to '0' for all such registers, lack of > migration support in userspace that isn't aware of the new API, > i.e. doesn't do KVM_GET_REG_LIST at a VM-scope, is a nop because > said userspace also won't modify the registers in the first place. We want any VMM that correctly uses the API today to seamlessly be able to save/restore any new feature. QEMU does that, and it should continue to work no matter what new feature we add to the list. M. -- Without deviation from the norm, progress is not possible. _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id A18DBC433F5 for ; Tue, 8 Feb 2022 17:49:04 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1384389AbiBHRtE (ORCPT ); Tue, 8 Feb 2022 12:49:04 -0500 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:37704 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S234672AbiBHRtD (ORCPT ); Tue, 8 Feb 2022 12:49:03 -0500 Received: from dfw.source.kernel.org (dfw.source.kernel.org [IPv6:2604:1380:4641:c500::1]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 30903C061578 for ; Tue, 8 Feb 2022 09:49:01 -0800 (PST) Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by dfw.source.kernel.org (Postfix) with ESMTPS id C350E617CF for ; Tue, 8 Feb 2022 17:49:00 +0000 (UTC) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 37060C004E1; Tue, 8 Feb 2022 17:49:00 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1644342540; bh=fiDA/ww0hH9Ihny0p1Mw46FtIdZcId1+1g9QAhPQmMQ=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=ThxP1jgwknTX415K1izO5hWFXB3QTxYCPffGDOViUTzgffNYlps4PC96hjrk8rNYX kPXzHQ7w/VGVhwJ8wny31enHFT4WCwLtBDdGPA6MvFOQvrN+t6uC5LWnq1aZ7bcXCb RZqX5hBcrqaJgPezu2NYCzG59/kw0liK9v2vslhT2m3jsIZNoPcmqwpCRi0oTytQSG 0mH53xrRqSNfyp2QbovIBFfev4rUCgKxexdyAB5joNbtkYe+cPLJyhzhsFPQCPMp7b z5UPlJow8XUsDNQmGt6RfkZj3r3+Xk5c+p7JZbDih2zS232alW7rH/3tqdjLcZziSH 7JlpPSXbwpWIw== Received: from sofa.misterjones.org ([185.219.108.64] helo=why.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.94.2) (envelope-from ) id 1nHUbq-006L8W-8g; Tue, 08 Feb 2022 17:48:58 +0000 Date: Tue, 08 Feb 2022 17:48:57 +0000 Message-ID: <878rul5jw6.wl-maz@kernel.org> From: Marc Zyngier To: Sean Christopherson Cc: Oliver Upton , Raghavendra Rao Ananta , Andrew Jones , kvmarm@lists.cs.columbia.edu, pshier@google.com, ricarkol@google.com, reijiw@google.com, jingzhangos@google.com, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, Alexandru.Elisei@arm.com, suzuki.poulose@arm.com, Peter Maydell Subject: Re: KVM/arm64: Guest ABI changes do not appear rollback-safe In-Reply-To: References: <87ilyitt6e.wl-maz@kernel.org> <87lf3drmvp.wl-maz@kernel.org> <875yq88app.wl-maz@kernel.org> <878ruld72v.wl-maz@kernel.org> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/27.1 (x86_64-pc-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: seanjc@google.com, oupton@google.com, rananta@google.com, drjones@redhat.com, kvmarm@lists.cs.columbia.edu, pshier@google.com, ricarkol@google.com, reijiw@google.com, jingzhangos@google.com, kvm@vger.kernel.org, linux-arm-kernel@lists.infradead.org, james.morse@arm.com, Alexandru.Elisei@arm.com, suzuki.poulose@arm.com, peter.maydell@linaro.org X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org On Tue, 08 Feb 2022 16:58:26 +0000, Sean Christopherson wrote: > > On Tue, Feb 08, 2022, Oliver Upton wrote: > > Hi Marc, > > > > On Tue, Feb 8, 2022 at 1:46 AM Marc Zyngier wrote: > > > > > KVM currently restricts the vcpu features to be unified across vcpus, > > > > > but that's only an implementation choice. > > > > > > > > But that implementation choice has become ABI, no? How could support > > > > for asymmetry be added without requiring userspace opt-in or breaking > > > > existing VMMs that depend on feature unification? > > > > > > Of course, you'd need some sort of advertising of this new behaviour. > > > > > > One thing I would like to add to the current state of thing is an > > > indication of whether the effects of a sysreg being written from > > > userspace are global or local to a vcpu. You'd need a new capability, > > > and an extra flag added to the encoding of each register. > > > > Ah. I think that is a much more reasonable fit then. VMMs unaware of > > this can continue to migrate new bits (albeit at the cost of > > potentially higher lock contention for the per-VM stuff), and those > > that do can reap the benefits of writing such attributes exactly once. > > But the "proper" usage is no different than adding support for > VM-scoped variants of KVM_{G,S}ET_ONE_REG and friends, and a > VM-scoped variant is conceptually a lot cleaner IMO. And making > them truly VM-scoped means KVM can do things like support sysregs > that are immutable after vCPUs are created. It is different, because your approach requires you to update all the existing VMMs to find out which register is of which kind. Not to mention that global sysregs are an absolute oddity in the face of the architecture (there is none in the base architecture). > So long as KVM defaults to '0' for all such registers, lack of > migration support in userspace that isn't aware of the new API, > i.e. doesn't do KVM_GET_REG_LIST at a VM-scope, is a nop because > said userspace also won't modify the registers in the first place. We want any VMM that correctly uses the API today to seamlessly be able to save/restore any new feature. QEMU does that, and it should continue to work no matter what new feature we add to the list. M. -- Without deviation from the norm, progress is not possible.