From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from foss.arm.com (foss.arm.com [217.140.110.172]) by smtp.subspace.kernel.org (Postfix) with ESMTP id 080E012B60; Wed, 19 Jul 2023 09:28:35 +0000 (UTC) Received: from usa-sjc-imap-foss1.foss.arm.com (unknown [10.121.207.14]) by usa-sjc-mx-foss1.foss.arm.com (Postfix) with ESMTP id 47D4E2F4; Wed, 19 Jul 2023 02:29:18 -0700 (PDT) Received: from [10.57.33.122] (unknown [10.57.33.122]) by usa-sjc-imap-foss1.foss.arm.com (Postfix) with ESMTPSA id 794C13F67D; Wed, 19 Jul 2023 02:28:31 -0700 (PDT) Message-ID: <7da93c6e-1cbf-8840-282e-f115197b80c4@arm.com> Date: Wed, 19 Jul 2023 10:28:29 +0100 Precedence: bulk X-Mailing-List: linux-coco@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:102.0) Gecko/20100101 Thunderbird/102.12.0 Subject: Re: [Question - ARM CCA] vCPU Hotplug Support in ARM Realm world might require ARM spec change? To: Salil Mehta , "steven.price@arm.com" , Jean-Philippe Brucker , "james.morse@arm.com" , "linux-coco@lists.linux.dev" , "kvmarm@lists.linux.dev" Cc: Catalin Marinas , Jonathan Cameron , Marc Zyngier , Will Deacon , "christoffer.dall@arm.com" , "oliver.upton@linux.dev" , "mark.rutland@arm.com" , "pbonzini@redhat.com" , Salil Mehta , "andrew.jones@linux.dev" , yuzenghui , "kvmarm@lists.cs.columbia.edu" , "linux-arm-kernel@lists.infradead.org" , "linux-kernel@vger.kernel.org" , "kvm@vger.kernel.org" , Gareth Stockwell References: <9cb24131a09a48e9a622e92bf8346c9d@huawei.com> From: Suzuki K Poulose In-Reply-To: <9cb24131a09a48e9a622e92bf8346c9d@huawei.com> Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit Hi Salil Thanks for raising this. On 19/07/2023 03:35, Salil Mehta wrote: > [Reposting it here from Linaro Open Discussion List for more eyes to look at] > > Hello, > I have recently started to dabble with ARM CCA stuff and check if our > recent changes to support vCPU Hotplug in ARM64 can work in the realm > world. I have realized that in the RMM specification[1] PSCI_CPU_ON > command(B5.3.3) does not handles the PSCI_DENIED return code(B5.4.2), > from the host. This might be required to support vCPU Hotplug feature > in the realm world in future. vCPU Hotplug is an important feature to > support kata-containers in realm world as it reduces the VM boot time > and facilitates dynamic adjustment of vCPUs (which I think should be > true even with Realm world as current implementation only makes use > of the PSCI_ON/OFF to realize the Hotplug look-like effect?) > > > As per our recent changes [2], [3] related to support vCPU Hotplug on > ARM64, we handle the guest exits due to SMC/HVC Hypercall in the > user-space i.e. VMM/Qemu. In realm world, REC Exits to host due to > PSCI_CPU_ON should undergo similar policy checks and I think, > > 1. Host should *deny* to online the target vCPUs which are NOT plugged > 2. This means target REC should be denied by host. Can host call > RMI_PSCI_COMPETE in such s case? > 3. The *return* value (B5.3.3.1.3 Output values) should be PSCI_DENIED The Realm exit with EXIT_PSCI already provides the parameters passed onto the PSCI request. This happens for all PSCI calls except (PSCI_VERSION and PSCI_FEAUTRES). The hyp could forward these exits to the VMM and could invoke the RMI_PSCI_COMPLETE only when the VMM blesses the request (wherever applicable). However, the RMM spec currently doesn't allow denying the request. i.e., without RMI_PSCI_COMPLETE, the REC cannot be scheduled back in. We will address this in the RMM spec and get back to you. Kind regards Suzuki > 4. Failure condition (B5.3.3.2) should be amended with > runnable pre: target_rec.flags.runnable == NOT_RUNNABLE (?) > post: result == PSCI_DENIED (?) > 5. Change would also be required in the flow (D1.4 PSCI flows) depicting > PSCI_CPU_ON flow (D1.4.1) > > > I do understand that ARM CCA support is in its infancy stage and > discussing about vCPU Hotplug in realm world seem to be a far-fetched > idea right now. But specification changes require lot of time and if > this change is really required then it should be further discussed > within ARM. > > Many thanks! > > > Bes regards > Salil > > > References: > > [1] https://developer.arm.com/documentation/den0137/latest/ > [2] https://github.com/salil-mehta/qemu.git virt-cpuhp-armv8/rfc-v1-port11052023.dev-1 > [3] https://git.gitlab.arm.com/linux-arm/linux-jm.git virtual_cpu_hotplug/rfc/v2 >