From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from mail-pl1-f201.google.com (mail-pl1-f201.google.com [209.85.214.201]) (using TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 (128/128 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id 7A10D5028C for ; Tue, 21 Jan 2025 17:20:33 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=209.85.214.201 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737480035; cv=none; b=XPsCb0z+rvm2Dp46v7lrayd+9rvLV1lBxkemHPbsfarOzvo74z/YKbxI5ePb8QCFVlyMuUundoMkbJbYyUbvT7gq2ZAYUHCIA/EA9XaJ3CEq5cAdibNtFLWR0mRDvJAb0/zhv5KrSQosP6yeTnGg0yRDYHjx4w+UzMWqui6QLdg= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1737480035; c=relaxed/simple; bh=x5yMvkK517sWj3Rln5yfo0RN3XBjU/f6LgwdHrnqDiU=; h=Date:In-Reply-To:Mime-Version:References:Message-ID:Subject:From: To:Cc:Content-Type; b=DOy8InX7LH0heKm50h58Ggly2i16/KDPghyioPp2E+3LRneTuQidP6lb1vKzgEef1/+XcXRx6qAIn5VOehJaGdlxJZeltbdAYYBC4MVT4kZ82S0fzMi++g3x9bByy/1+MvV8Zf89Nd1oZzZkkIGYwfQdbO6HCzhEwGOwH31J5KA= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b=z84S7hzi; arc=none smtp.client-ip=209.85.214.201 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=reject dis=none) header.from=google.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=flex--seanjc.bounces.google.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (2048-bit key) header.d=google.com header.i=@google.com header.b="z84S7hzi" Received: by mail-pl1-f201.google.com with SMTP id d9443c01a7336-2166e907b5eso112920995ad.3 for ; Tue, 21 Jan 2025 09:20:33 -0800 (PST) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=20230601; t=1737480033; x=1738084833; darn=vger.kernel.org; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:from:to:cc:subject:date:message-id:reply-to; bh=ZKBBRZAROa15rhH3QqEBLkWUC6taneZwpUedt33fLOg=; b=z84S7hzikDotpTEeEs34Guj40yVjV7Erv/zN8Ezfgcqc6ERqSg1FZys3a3ySsCccTf cakzWNlHDPfcpZXFq6UEFbQxfsRVaHR7Z50Y/vCaBaUrxJFnU95anFlfU3QLMIMZ/VLQ z6Ze8TYIqL202I3CMRfHhODN5Bt3U3B3IY/c3lAMLJSxuXQFiUB8D8H3vu+EZ2g3i+DS YdiJVLuXA2KiyygGQFsy3yUjsHB1K4vA2uOugYMHSPndNldrujNSstwLIDXNp6B4HrIV CPk2D466e7pC4aCx0xpakViC0LDUdplVLAq8ut0a5EfyU/KlJpToVo7Tqk8QE6xNhFCs OztA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1737480033; x=1738084833; h=cc:to:from:subject:message-id:references:mime-version:in-reply-to :date:x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=ZKBBRZAROa15rhH3QqEBLkWUC6taneZwpUedt33fLOg=; b=UESUVkVovcVpf+Htvd/tMpwcRkZmOqayKbfpAh35SeRwwYERQ4haBsclWYQ2lVBs0M t32zZ6tL0Lc1SAP6hbdi2gTafOu8NfcHV7oD0xTtUi2i1F9bmUo+/z3UcpDJZHQrQfxw USU6A0otX0513v4l5rAD/JVZ0dEf1TkBu42DODAq8Aa6vE3vzgF4E3tiFN0z2uT8Ynys 1w2MZSuDU5kWuNN1kLgetlPs9t6m1PLz4+D33fM1F7QzgxD+OB+RN9YIxjQWTVxB0vDA 4WIvN+/kqdN+ZpMc58RfRhNvvqgSlp4UeRQ01Lo0h/AAEAzmTEom0ihxfOGmyrPS0yjk UqWQ== X-Forwarded-Encrypted: i=1; AJvYcCWUvbM/TaFYMutZBexwnn7viqqrMD644PCYztU1uMqug2rFRMyI9az/uNnTxyWw6zW9UQaHZupJcvTCbHw=@vger.kernel.org X-Gm-Message-State: AOJu0Yx63NAddsgpIrCZP8GfhlBTcoIRNbxAlJYNAac2kKs9S6MI00FL CDfY94zFTR0/WSkt52pMgZ8NIsMxyaGI4ggIsDckMBs/xvDFClXGAEDkyCQspvBqH9ZVnk/zBeH BPA== X-Google-Smtp-Source: AGHT+IE+M+zQgh7U6I+YCMtLJ1fFjwIPWZjsg2dteoxP8P0yc55/E0Tsxz4Z3WK9Fr+onXhzhXKYNxYhNKc= X-Received: from plgk6.prod.google.com ([2002:a17:902:ce06:b0:216:3eee:c9b7]) (user=seanjc job=prod-delivery.src-stubby-dispatcher) by 2002:a17:902:ea03:b0:215:b33b:e26d with SMTP id d9443c01a7336-21c3550c5d3mr298899905ad.21.1737480032730; Tue, 21 Jan 2025 09:20:32 -0800 (PST) Date: Tue, 21 Jan 2025 09:20:31 -0800 In-Reply-To: Precedence: bulk X-Mailing-List: linux-kernel@vger.kernel.org List-Id: List-Subscribe: List-Unsubscribe: Mime-Version: 1.0 References: <20241023124507.280382-1-pbonzini@redhat.com> <20241023124507.280382-6-pbonzini@redhat.com> Message-ID: Subject: Re: [PATCH 5/5] Documentation: kvm: introduce "VM plane" concept From: Sean Christopherson To: Nicolas Saenz Julienne Cc: Paolo Bonzini , linux-kernel@vger.kernel.org, kvm@vger.kernel.org, roy.hopkins@suse.com, michael.roth@amd.com, ashish.kalra@amd.com, jroedel@suse.de, thomas.lendacky@amd.com, anelkz@amazon.de, oliver.upton@linux.dev, isaku.yamahata@intel.com, maz@kernel.org, steven.price@arm.com, kai.huang@intel.com, rick.p.edgecombe@intel.com, James.Bottomley@hansenpartnership.com Content-Type: text/plain; charset="us-ascii" On Tue, Jan 21, 2025, Nicolas Saenz Julienne wrote: > Hi Sean, > > On Fri Jan 17, 2025 at 9:48 PM UTC, Sean Christopherson wrote: > > On Wed, Oct 23, 2024, Paolo Bonzini wrote: > >> @@ -6398,6 +6415,46 @@ the capability to be present. > >> `flags` must currently be zero. > >> > >> > >> +.. _KVM_CREATE_PLANE: > >> + > >> +4.144 KVM_CREATE_PLANE > >> +---------------------- > >> + > >> +:Capability: KVM_CAP_PLANE > >> +:Architectures: none > >> +:Type: vm ioctl > >> +:Parameters: plane id > >> +:Returns: a VM fd that can be used to control the new plane. > >> + > >> +Creates a new *plane*, i.e. a separate privilege level for the > >> +virtual machine. Each plane has its own memory attributes, > >> +which can be used to enable more restricted permissions than > >> +what is allowed with ``KVM_SET_USER_MEMORY_REGION``. > >> + > >> +Each plane has a numeric id that is used when communicating > >> +with KVM through the :ref:`kvm_run ` struct. While > >> +KVM is currently agnostic to whether low ids are more or less > >> +privileged, it is expected that this will not always be the > >> +case in the future. For example KVM in the future may use > >> +the plane id when planes are supported by hardware (as is the > >> +case for VMPLs in AMD), or if KVM supports accelerated plane > >> +switch operations (as might be the case for Hyper-V VTLs). > >> + > >> +4.145 KVM_CREATE_VCPU_PLANE > >> +--------------------------- > >> + > >> +:Capability: KVM_CAP_PLANE > >> +:Architectures: none > >> +:Type: vm ioctl (non default plane) > >> +:Parameters: vcpu file descriptor for the default plane > >> +:Returns: a vCPU fd that can be used to control the new plane > >> + for the vCPU. > >> + > >> +Adds a vCPU to a plane; the new vCPU's id comes from the vCPU > >> +file descriptor that is passed in the argument. Note that > >> + because of how the API is defined, planes other than plane 0 > >> +can only have a subset of the ids that are available in plane 0. > > > > Hmm, was there a reason why we decided to add KVM_CREATE_VCPU_PLANE, as opposed > > to having KVM_CREATE_PLANE create vCPUs? IIRC, we talked about being able to > > provide the new FD, but that would be easy enough to handle in KVM_CREATE_PLANE, > > e.g. with an array of fds. > > IIRC we mentioned that there is nothing in the VSM spec preventing > higher VTLs from enabling a subset of vCPUs. That said, even the TLFS > mentions that doing so is not such a great idea (15.4 VTL Enablement): > > "Enable the target VTL on one or more virtual processors. [...] It is > recommended that all VPs have the same enabled VTLs. Having a VTL > enabled on some VPs (but not all) can lead to unexpected behavior." > > One thing I've been meaning to research is moving device emulation into > guest execution context by using VTLs. In that context, it might make > sense to only enable VTLs on specific vCPUs. But I'm only speculating. Creating vCPUs for a VTL in KVM doesn't need to _enable_ that VTL, and AIUI shouldn't enable the VTL, because HvCallEnablePartitionVtl "only" enables the VTL for the VM, HvCallEnableVpVtl is what fully enables the VTL for a given vCPU. What I am proposing is to create the KVM vCPU object(s) at KVM_CREATE_PLANE, purely to help avoid NULL pointer dereferences. Actually, since KVM will likely need uAPI to let userspace enable a VTL for a vCPU even if the vCPU object is auto-created, we could have KVM auto-create the objects transparently, i.e. still provide KVM_CREATE_VCPU_PLANE, but under the hood it would simply enable a flag and install the vCPU's file descriptor. > Otherwise, I cannot think of real world scenarios where this property is > needed. > > > k.g. is the expectation that userspace will create all planes before creating > > any vCPUs? > > The opposite really, VTLs can be initiated anytime during runtime. Oh, right. > > My concern with relying on userspace to create vCPUs is that it will mean KVM > > will need to support, or at least not blow up on, VMs with multiple planes, but > > only a subset of vCPUs at planes > 0. Given the snafus with vcpus_array, it's > > not at all hard to imagine scenarios where KVM tries to access a NULL vCPU in > > a different plane.