From mboxrd@z Thu Jan 1 00:00:00 1970 Received: from us-smtp-delivery-124.mimecast.com (us-smtp-delivery-124.mimecast.com [170.10.129.124]) (using TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)) (No client certificate requested) by smtp.subspace.kernel.org (Postfix) with ESMTPS id AE969DDBB for ; Sun, 2 Feb 2025 02:06:57 +0000 (UTC) Authentication-Results: smtp.subspace.kernel.org; arc=none smtp.client-ip=170.10.129.124 ARC-Seal:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738462019; cv=none; b=pvPK+mDrSh4k4MeWOIVpQWMoMisSSkkAvXyyUy8L58Vb6JyfkizF6AAi10PSLHqVBLL6MWc946NXd+ZEWsxRGA67Mu2BcYhH9CjHvphfD7H0iztMrNMlyAVTqRk0LJ+JPffGSbh3zFiHgnBkCJUEvcTbnMatAvJ2xnHp13Z5VTk= ARC-Message-Signature:i=1; a=rsa-sha256; d=subspace.kernel.org; s=arc-20240116; t=1738462019; c=relaxed/simple; bh=Cv+rYeVYUucgFkd9j+IahXxjPtJ2f4F7GoUBqwxuulM=; h=Message-ID:Date:MIME-Version:Subject:To:Cc:References:From: In-Reply-To:Content-Type; b=O+QcAGCmOp/bvo1gb03AXLjk7QQZcVFcvVm4JoQIs/OIrq1C38euv5mgZ3kpavv22nLOepqI1MfBPcUobjJyIXeUXyBrLpGPL1L6z2g9kF7C25+r9RBMN+OWasdzaYdwUN7/NCvmw6QD65HIBcXxVzGybVMrRX2VebRupvkcYGI= ARC-Authentication-Results:i=1; smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com; spf=pass smtp.mailfrom=redhat.com; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b=Svt9dxlB; arc=none smtp.client-ip=170.10.129.124 Authentication-Results: smtp.subspace.kernel.org; dmarc=pass (p=none dis=none) header.from=redhat.com Authentication-Results: smtp.subspace.kernel.org; spf=pass smtp.mailfrom=redhat.com Authentication-Results: smtp.subspace.kernel.org; dkim=pass (1024-bit key) header.d=redhat.com header.i=@redhat.com header.b="Svt9dxlB" DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=redhat.com; s=mimecast20190719; t=1738462016; h=from:from:reply-to:subject:subject:date:date:message-id:message-id: to:to:cc:cc:mime-version:mime-version:content-type:content-type: content-transfer-encoding:content-transfer-encoding: in-reply-to:in-reply-to:references:references; bh=sSlKml/+kejUBtq+rKq08KSBVq8nnU94jjP3M+wY8SY=; b=Svt9dxlB1OTozFWZn6cwH9d2ra/FbtLYpwIgjJCv3Lie7ZGkOv98Ku77Yf6UMrrD8Eu2dj QTeK1JnsFDD8Igii5zKdHM16Nz506pbiU7TYeURMDDh31G5VsN9ig7WdtChwSs6iWJmeCG Ox3HeGp9Rn2c1n9f4/wj3GktwuTeByQ= Received: from mail-pl1-f200.google.com (mail-pl1-f200.google.com [209.85.214.200]) by relay.mimecast.com with ESMTP with STARTTLS (version=TLSv1.3, cipher=TLS_AES_256_GCM_SHA384) id us-mta-439-8_nrJykXOa2S4Yr7cIVbFQ-1; Sat, 01 Feb 2025 21:06:55 -0500 X-MC-Unique: 8_nrJykXOa2S4Yr7cIVbFQ-1 X-Mimecast-MFC-AGG-ID: 8_nrJykXOa2S4Yr7cIVbFQ Received: by mail-pl1-f200.google.com with SMTP id d9443c01a7336-218ad674181so89588615ad.1 for ; Sat, 01 Feb 2025 18:06:55 -0800 (PST) X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1738462014; x=1739066814; h=content-transfer-encoding:in-reply-to:from:content-language :references:cc:to:subject:user-agent:mime-version:date:message-id :x-gm-message-state:from:to:cc:subject:date:message-id:reply-to; bh=sSlKml/+kejUBtq+rKq08KSBVq8nnU94jjP3M+wY8SY=; b=FeqgZsDJtu8t9cNFeYJv+LUUVmrjQNIXnrSi6M+oKy4CnBFB3dpPVKqhtsraskf4DG GPTQNVL50UJb+X0eN2A63D4uAAnwb1mQs/ToOfyWRJ5BBY+iHvPFi1cUWhQFm8FDQCZc GcAkr1Q7taeNe1/uqCec0/Va9i8yEKOuBE3f5cO2OmVeEUGdiwR+JBCcLNuABDGYAYE2 CX/+38cZ+ftrEi2oJx1d949GpPQ5RJ/T4H9c3gKsn0xq69iPIaPWmECX2MzFPQDhIMT2 S/zTF8uLrHK+UCxuDtKfjkcn0vDeds7h1M8TCGvW6tkzCC0xRG/ClrOig9hwBxHHZ3sh muqw== X-Forwarded-Encrypted: i=1; AJvYcCUks7CZcu512QPKIOB6WnH4DjzjXl2/O7ngiEbuNhWd8KxJqLe0h4IHLvB5L5MNQnLw1JYHse0TqI3/@lists.linux.dev X-Gm-Message-State: AOJu0Yxa1YPAsHHFB9TjLgQN1ACcK3ZzvaSqtZuVLpA5yiMJU7yS5l4O qp51k9IHFal/xyoUcQ/ATL41iXJSPBphAAnuNvWUuMnBPvHn4uRWLwcY1Eyyy1lw+7PDZPN0TV9 eg669jS163yNwuCTiGK0EW4fxY5Nvor2Yf94zTEEcEZFeeCTf0WPrtrekeNY= X-Gm-Gg: ASbGncvU2EN/eKV9/E6/dqOQjPNor534eyP5f1yz9VsWf9q4jz/Gx+8CTXmLgvWfJ2N zJrhDZtVCC9jZ6SwWoKkJR6b5snx4Cxnz/afZIC91QLWgw4IfsHwwyJhpvq4MOTkPulbGmCmkMk maWKdldj9uB61zs1BLLlMWG9Q4e8fOBPfVOacSVGx8zGGnrWywf/ElOMYHBHT+pDCphxk0Vbh6P 2YzV92brrp0dPeXu/5Zwe15BWQXI5I2If9hQGaLQwfc6ZPqmVR5vOYmi3lIjFeovKdYpN1LKGJY I+Lj7w== X-Received: by 2002:a05:6300:7111:b0:1ed:a4b1:9124 with SMTP id adf61e73a8af0-1eda4b195bcmr6482947637.8.1738462014134; Sat, 01 Feb 2025 18:06:54 -0800 (PST) X-Google-Smtp-Source: AGHT+IEuAmakbdVZEYfdxlzFY/5lw1yKumpAmcjFfn5o/6n2SOgtQbAy8hTNJJ7loIVoDWvimSAuIQ== X-Received: by 2002:a05:6300:7111:b0:1ed:a4b1:9124 with SMTP id adf61e73a8af0-1eda4b195bcmr6482907637.8.1738462013708; Sat, 01 Feb 2025 18:06:53 -0800 (PST) Received: from [192.168.68.55] ([180.233.125.64]) by smtp.gmail.com with ESMTPSA id d2e1a72fcca58-72fe631bf29sm5625326b3a.20.2025.02.01.18.06.46 (version=TLS1_3 cipher=TLS_AES_128_GCM_SHA256 bits=128/128); Sat, 01 Feb 2025 18:06:53 -0800 (PST) Message-ID: <489025af-a0c3-449f-a746-f898b77fc660@redhat.com> Date: Sun, 2 Feb 2025 12:06:44 +1000 Precedence: bulk X-Mailing-List: linux-coco@lists.linux.dev List-Id: List-Subscribe: List-Unsubscribe: MIME-Version: 1.0 User-Agent: Mozilla Thunderbird Subject: Re: [PATCH v6 23/43] KVM: arm64: Handle Realm PSCI requests To: Steven Price , kvm@vger.kernel.org, kvmarm@lists.linux.dev Cc: Catalin Marinas , Marc Zyngier , Will Deacon , James Morse , Oliver Upton , Suzuki K Poulose , Zenghui Yu , linux-arm-kernel@lists.infradead.org, linux-kernel@vger.kernel.org, Joey Gouly , Alexandru Elisei , Christoffer Dall , Fuad Tabba , linux-coco@lists.linux.dev, Ganapatrao Kulkarni , Shanker Donthineni , Alper Gun , "Aneesh Kumar K . V" References: <20241212155610.76522-1-steven.price@arm.com> <20241212155610.76522-24-steven.price@arm.com> From: Gavin Shan In-Reply-To: <20241212155610.76522-24-steven.price@arm.com> X-Mimecast-Spam-Score: 0 X-Mimecast-MFC-PROC-ID: vaVm1Bfzmizhxsh2Qr-291bOJPzJFQhI-uWQS6ub-2Y_1738462014 X-Mimecast-Originator: redhat.com Content-Language: en-US Content-Type: text/plain; charset=UTF-8; format=flowed Content-Transfer-Encoding: 7bit On 12/13/24 1:55 AM, Steven Price wrote: > The RMM needs to be informed of the target REC when a PSCI call is made > with an MPIDR argument. Expose an ioctl to the userspace in case the PSCI > is handled by it. > > Co-developed-by: Suzuki K Poulose > Signed-off-by: Suzuki K Poulose > Signed-off-by: Steven Price > --- > arch/arm64/include/asm/kvm_rme.h | 3 +++ > arch/arm64/kvm/arm.c | 25 +++++++++++++++++++++++++ > arch/arm64/kvm/psci.c | 29 +++++++++++++++++++++++++++++ > arch/arm64/kvm/rme.c | 15 +++++++++++++++ > 4 files changed, 72 insertions(+) > > diff --git a/arch/arm64/include/asm/kvm_rme.h b/arch/arm64/include/asm/kvm_rme.h > index 158f77e24a26..90a4537ad38d 100644 > --- a/arch/arm64/include/asm/kvm_rme.h > +++ b/arch/arm64/include/asm/kvm_rme.h > @@ -113,6 +113,9 @@ int realm_set_ipa_state(struct kvm_vcpu *vcpu, > unsigned long addr, unsigned long end, > unsigned long ripas, > unsigned long *top_ipa); > +int realm_psci_complete(struct kvm_vcpu *calling, > + struct kvm_vcpu *target, > + unsigned long status); > The 'calling' may be renamed to 'source', consistent to the names in arch/arm64/kvm/psci.c. Other nitpicks can be found below. > #define RMM_RTT_BLOCK_LEVEL 2 > #define RMM_RTT_MAX_LEVEL 3 > diff --git a/arch/arm64/kvm/arm.c b/arch/arm64/kvm/arm.c > index f588b528c3f9..eff1a4ec892b 100644 > --- a/arch/arm64/kvm/arm.c > +++ b/arch/arm64/kvm/arm.c > @@ -1745,6 +1745,22 @@ static int kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu, > return __kvm_arm_vcpu_set_events(vcpu, events); > } > > +static int kvm_arm_vcpu_rmm_psci_complete(struct kvm_vcpu *vcpu, > + struct kvm_arm_rmm_psci_complete *arg) > +{ > + struct kvm_vcpu *target = kvm_mpidr_to_vcpu(vcpu->kvm, arg->target_mpidr); > + > + if (!target) > + return -EINVAL; > + > + /* > + * RMM v1.0 only supports PSCI_RET_SUCCESS or PSCI_RET_DENIED > + * for the status. But, let us leave it to the RMM to filter > + * for making this future proof. > + */ > + return realm_psci_complete(vcpu, target, arg->psci_status); > +} > + > long kvm_arch_vcpu_ioctl(struct file *filp, > unsigned int ioctl, unsigned long arg) > { > @@ -1867,6 +1883,15 @@ long kvm_arch_vcpu_ioctl(struct file *filp, > > return kvm_arm_vcpu_finalize(vcpu, what); > } > + case KVM_ARM_VCPU_RMM_PSCI_COMPLETE: { > + struct kvm_arm_rmm_psci_complete req; > + > + if (!kvm_is_realm(vcpu->kvm)) > + return -EINVAL; We probably need to check with vcpu_is_rec(). -EPERM seems more precise than -EINVAL. > + if (copy_from_user(&req, argp, sizeof(req))) > + return -EFAULT; > + return kvm_arm_vcpu_rmm_psci_complete(vcpu, &req); > + } > default: > r = -EINVAL; > } > diff --git a/arch/arm64/kvm/psci.c b/arch/arm64/kvm/psci.c > index 3b5dbe9a0a0e..9dc161abc30c 100644 > --- a/arch/arm64/kvm/psci.c > +++ b/arch/arm64/kvm/psci.c > @@ -103,6 +103,12 @@ static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu) > > reset_state->reset = true; > kvm_make_request(KVM_REQ_VCPU_RESET, vcpu); > + /* > + * Make sure we issue PSCI_COMPLETE before the VCPU can be > + * scheduled. > + */ > + if (vcpu_is_rec(vcpu)) > + realm_psci_complete(source_vcpu, vcpu, PSCI_RET_SUCCESS); > > /* > * Make sure the reset request is observed if the RUNNABLE mp_state is > @@ -115,6 +121,10 @@ static unsigned long kvm_psci_vcpu_on(struct kvm_vcpu *source_vcpu) > > out_unlock: > spin_unlock(&vcpu->arch.mp_state_lock); > + if (vcpu_is_rec(vcpu) && ret != PSCI_RET_SUCCESS) > + realm_psci_complete(source_vcpu, vcpu, > + ret == PSCI_RET_ALREADY_ON ? > + PSCI_RET_SUCCESS : PSCI_RET_DENIED); {} is needed here. > return ret; > } > > @@ -142,6 +152,25 @@ static unsigned long kvm_psci_vcpu_affinity_info(struct kvm_vcpu *vcpu) > /* Ignore other bits of target affinity */ > target_affinity &= target_affinity_mask; > > + if (vcpu_is_rec(vcpu)) { > + struct kvm_vcpu *target_vcpu; > + > + /* RMM supports only zero affinity level */ > + if (lowest_affinity_level != 0) > + return PSCI_RET_INVALID_PARAMS; > + > + target_vcpu = kvm_mpidr_to_vcpu(kvm, target_affinity); > + if (!target_vcpu) > + return PSCI_RET_INVALID_PARAMS; > + > + /* > + * Provide the references of running and target RECs to the RMM ^^^^^^^ the source > + * so that the RMM can complete the PSCI request. > + */ > + realm_psci_complete(vcpu, target_vcpu, PSCI_RET_SUCCESS); > + return PSCI_RET_SUCCESS; > + } > + > /* > * If one or more VCPU matching target affinity are running > * then ON else OFF > diff --git a/arch/arm64/kvm/rme.c b/arch/arm64/kvm/rme.c > index 146ef598a581..5831d379760a 100644 > --- a/arch/arm64/kvm/rme.c > +++ b/arch/arm64/kvm/rme.c > @@ -118,6 +118,21 @@ static void free_delegated_granule(phys_addr_t phys) > free_page((unsigned long)phys_to_virt(phys)); > } > > +int realm_psci_complete(struct kvm_vcpu *calling, struct kvm_vcpu *target, > + unsigned long status) > +{ > + int ret; > + > + ret = rmi_psci_complete(virt_to_phys(calling->arch.rec.rec_page), > + virt_to_phys(target->arch.rec.rec_page), > + status); > + ^^^^^ Unnecessary blank line. > + if (ret) > + return -EINVAL; > + > + return 0; > +} > + > static int realm_rtt_create(struct realm *realm, > unsigned long addr, > int level, Thanks, Gavin