From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: X-Spam-Checker-Version: SpamAssassin 3.4.0 (2014-02-07) on aws-us-west-2-korg-lkml-1.web.codeaurora.org Received: from vger.kernel.org (vger.kernel.org [23.128.96.18]) by smtp.lore.kernel.org (Postfix) with ESMTP id F25F5E7B5E1 for ; Wed, 4 Oct 2023 09:36:56 +0000 (UTC) Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S242006AbjJDJg5 (ORCPT ); Wed, 4 Oct 2023 05:36:57 -0400 Received: from lindbergh.monkeyblade.net ([23.128.96.19]:55320 "EHLO lindbergh.monkeyblade.net" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S232952AbjJDJg5 (ORCPT ); Wed, 4 Oct 2023 05:36:57 -0400 Received: from smtp.kernel.org (relay.kernel.org [52.25.139.140]) by lindbergh.monkeyblade.net (Postfix) with ESMTPS id 90B05A7 for ; Wed, 4 Oct 2023 02:36:53 -0700 (PDT) Received: by smtp.kernel.org (Postfix) with ESMTPSA id 3557DC433CA; Wed, 4 Oct 2023 09:36:53 +0000 (UTC) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/simple; d=kernel.org; s=k20201202; t=1696412213; bh=nhBCsDpwNVLWHwfW6eG2neTZ8vbY5dFCKwdiEzjew6g=; h=Date:From:To:Cc:Subject:In-Reply-To:References:From; b=H2yxlXhzPlYIHLltoKDNRWONtpESL2yyvO3vQ0xXK7cLaJT+NwID7qhDnLh3T2Jix 2MHuGV3NxRjIpRdLAHuMxrqL/NB56Y0aRWwYxw8dumjGBbEzLtMjSOEKucgyWpAEsp RtOvZheSDjgunwaZoHU1tmrGy8Ash02uOUoALjeS75bon9pkck3uiZ3FpuJ4BXB9PX lgsv5kYTF+0vvdNrWUl4uijsCyNaiMkyw4sp7dF0PGQUNfBQVJEwFMtrqoUdjKcpNk 1OET2bo0werD9FitDGaGc6jxNJvh5VCslDn/lrX3IDTlTrdhyEFU9joyMx1C/4pKX8 0q1GnLroq+DgA== Received: from sofa.misterjones.org ([185.219.108.64] helo=goblin-girl.misterjones.org) by disco-boy.misterjones.org with esmtpsa (TLS1.3) tls TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (Exim 4.95) (envelope-from ) id 1qnyJG-0010OJ-Q7; Wed, 04 Oct 2023 10:36:50 +0100 Date: Wed, 04 Oct 2023 10:36:50 +0100 Message-ID: <86o7heohjh.wl-maz@kernel.org> From: Marc Zyngier To: Oliver Upton Cc: kvmarm@lists.linux.dev, kvm@vger.kernel.org, James Morse , Suzuki K Poulose , Zenghui Yu , Jing Zhang , Cornelia Huck Subject: Re: [PATCH v11 10/12] KVM: arm64: Document vCPU feature selection UAPIs In-Reply-To: <20231003230408.3405722-11-oliver.upton@linux.dev> References: <20231003230408.3405722-1-oliver.upton@linux.dev> <20231003230408.3405722-11-oliver.upton@linux.dev> User-Agent: Wanderlust/2.15.9 (Almost Unreal) SEMI-EPG/1.14.7 (Harue) FLIM-LB/1.14.9 (=?UTF-8?B?R29qxY0=?=) APEL-LB/10.8 EasyPG/1.0.0 Emacs/29.1 (aarch64-unknown-linux-gnu) MULE/6.0 (HANACHIRUSATO) MIME-Version: 1.0 (generated by SEMI-EPG 1.14.7 - "Harue") Content-Type: text/plain; charset=US-ASCII X-SA-Exim-Connect-IP: 185.219.108.64 X-SA-Exim-Rcpt-To: oliver.upton@linux.dev, kvmarm@lists.linux.dev, kvm@vger.kernel.org, james.morse@arm.com, suzuki.poulose@arm.com, yuzenghui@huawei.com, jingzhangos@google.com, cohuck@redhat.com X-SA-Exim-Mail-From: maz@kernel.org X-SA-Exim-Scanned: No (on disco-boy.misterjones.org); SAEximRunCond expanded to false Precedence: bulk List-ID: X-Mailing-List: kvm@vger.kernel.org On Wed, 04 Oct 2023 00:04:06 +0100, Oliver Upton wrote: > > KVM/arm64 has a couple schemes for handling vCPU feature selection now, > which is a lot to put on userspace. Add some documentation about how > these interact and provide some recommendations for how to use the > writable ID register scheme. > > Signed-off-by: Oliver Upton > --- > Documentation/virt/kvm/api.rst | 4 ++ > Documentation/virt/kvm/arm/index.rst | 1 + > Documentation/virt/kvm/arm/vcpu-features.rst | 48 ++++++++++++++++++++ > 3 files changed, 53 insertions(+) > create mode 100644 Documentation/virt/kvm/arm/vcpu-features.rst > > diff --git a/Documentation/virt/kvm/api.rst b/Documentation/virt/kvm/api.rst > index d55c2b68c0a9..8d4050eedb26 100644 > --- a/Documentation/virt/kvm/api.rst > +++ b/Documentation/virt/kvm/api.rst > @@ -3370,6 +3370,8 @@ return indicates the attribute is implemented. It does not necessarily > indicate that the attribute can be read or written in the device's > current state. "addr" is ignored. > > +.. _KVM_ARM_VCPU_INIT: > + > 4.82 KVM_ARM_VCPU_INIT > ---------------------- > > @@ -6070,6 +6072,8 @@ writes to the CNTVCT_EL0 and CNTPCT_EL0 registers using the SET_ONE_REG > interface. No error will be returned, but the resulting offset will not be > applied. > > +.. _KVM_ARM_GET_REG_WRITABLE_MASKS: > + > 4.139 KVM_ARM_GET_REG_WRITABLE_MASKS > ------------------------------------------- > > diff --git a/Documentation/virt/kvm/arm/index.rst b/Documentation/virt/kvm/arm/index.rst > index e84848432158..7f231c724e16 100644 > --- a/Documentation/virt/kvm/arm/index.rst > +++ b/Documentation/virt/kvm/arm/index.rst > @@ -11,3 +11,4 @@ ARM > hypercalls > pvtime > ptp_kvm > + vcpu-features > diff --git a/Documentation/virt/kvm/arm/vcpu-features.rst b/Documentation/virt/kvm/arm/vcpu-features.rst > new file mode 100644 > index 000000000000..2d2f89c5781f > --- /dev/null > +++ b/Documentation/virt/kvm/arm/vcpu-features.rst > @@ -0,0 +1,48 @@ > +.. SPDX-License-Identifier: GPL-2.0 > + > +=============================== > +vCPU feature selection on arm64 > +=============================== > + > +KVM/arm64 provides two mechanisms that allow userspace to configure > +the CPU features presented to the guest. > + > +KVM_ARM_VCPU_INIT > +================= > + > +The ``KVM_ARM_VCPU_INIT`` ioctl accepts a bitmap of feature flags > +(``struct kvm_vcpu_init::features``). Features enabled by this interface are > +*opt-in* and may change/extend UAPI. See :ref:`KVM_ARM_VCPU_INIT` for complete > +documentation of the features controlled by the ioctl. > + > +Otherwise, all CPU features supported by KVM are described by the architected > +ID registers. > + > +The ID Registers > +================ > + > +The Arm architecture specifies a range of *ID Registers* that describe the set > +of architectural features supported by the CPU implementation. KVM initializes > +the guest's ID registers to the maximum set of CPU features supported by the > +system. The ID register values are VM-scoped in KVM, meaning that the values > +are identical for all vCPUs in a VM. I'm a bit reluctant to give this guarantee. Case in point: MPIDR_EL1 is part of the Feature ID space, and is definitely *not* a register that we can make global, even on a fully homogeneous system. I'd also like to give us more flexibility to change the implementation in the future without having to change the API again. IMO, the fact that we make our life simpler by only tracking a single copy is an implementation detail, not something that userspace should rely on. I would simply turn the "The ID register values are VM-scoped" into "The ID register values may be VM-scoped", which gives us that flexibility. > + > +KVM allows userspace to *opt-out* of certain CPU features described by the ID > +registers by writing values to them via the ``KVM_SET_ONE_REG`` ioctl. The ID > +registers are mutable until the VM has started, i.e. userspace has called > +``KVM_RUN`` on at least one vCPU in the VM. Userspace can discover what fields > +are mutable in the ID registers using the ``KVM_ARM_GET_REG_WRITABLE_MASKS``. > +See the :ref:`ioctl documentation ` for more > +details. > + > +Userspace is allowed to *limit* or *mask* CPU features according to the rules > +outlined by the architecture in DDI0487J 'D19.1.3 Principles of the ID scheme nit: consider spelling out the *full* version of the ARM ARM (DDI 0487J.a), just in case we get a J.b this side of Xmas and that this reference is renumbered... > +for fields in ID register'. KVM does not allow ID register values that exceed > +the capabilities of the system. > + > +.. warning:: > + It is **strongly recommended** that userspace modify the ID register values > + before accessing the rest of the vCPU's CPU register state. KVM may use the > + ID register values to control feature emulation. Interleaving ID register > + modification with other system register accesses may lead to unpredictable > + behavior. Thanks, M. -- Without deviation from the norm, progress is not possible.