Re: [PATCH v16 00/45] arm64: Support for Arm CCA in KVM

[Suzuki K Poulose <suzuki.poulose@arm.com>]

On 12/08/2026 11:35, Gavin Shan wrote:
> Hi Alper and Suzuki,
> 
> On 8/12/26 4:04 PM, Suzuki K Poulose wrote:
>> Hi Alper, Gavin
>>
>> On 12/08/2026 04:25, Alper Gun wrote:
>>> On Tue, Aug 11, 2026 at 8:08 PM Gavin Shan <gshan@redhat.com> wrote:
>>>> As the following calltrace indicates, -EAGAIN is returned from tf- 
>>>> rmm::update_ripas()
>>>> because true is returned from s2tte_drain_pending() for the S2TTE 
>>>> corresponding to
>>>> IPA 0x80000000. Linux host received error (RMI_ERROR_RTT, level=3) 
>>>> in ripas_change().
>>>> Upon this specific error and the IPA range [0x80000000 0x90000000], 
>>>> find_map_level()
>>>> returns level of 2, and realm_create_rtt_levels() returns 0 without 
>>>> populating any
>>>> RTTs. After that, rmi_rtt_set_ripas() is re-executed and the above 
>>>> loop starts over
>>>> again.
>>>>
>>>>     Linux host
>>>>     ==========
>>>>     kvm_arch_vcpu_ioctl_run                     // cca/host-v16
>>>>       check_vcpu_requests
>>>>         kvm_check_request
>>>>           kvm_rec_handle_request
>>>>             kvm_complete_ripas_change
>>>>               realm_set_ipa_state
>>>>                 ripas_change
>>>>                   rmi_rtt_set_ripas
>>>>                     SMC_RMI_RTT_SET_RIPAS
>>>>
>>>>     TF-RMM
>>>>     ======
>>>>     SMC_RMI_RTT_SET_RIPAS                      // tf-rmm/topics/rmm- 
>>>> v2.0-poc_3
>>>>       smc_rtt_set_ripas
>>>>         s2tt_walk_lock_unlock
>>>>         rtt_set_ripas_range
>>>>           update_ripas
>>>>             s2tte_drain_pending                // true, returns -EAGAIN
>>>>
>>>> The problem is the pending-bit for RTE corresponding IPA address 
>>>> 0x80000000 isn't cleared
>>>> when SMC_RMI_RTT_SET_RIPAS is invoked. I didn't figure out how this 
>>>> bit is set and why
>>>> it's not cleared in time.
>>
>> Thanks for the details.
>>
>>>>
>>>
>>> Hi Gavin, Suzuki,
>>>
>>> I think I ran into a similar issue on rmm-v2.0-poc_3 last week.
>>> This looks like a potential RMM bug: could bit 32 be part of the 
>>> physical
>>> Address (if PA >= 4 GiB)?
>>>
>>> It seems s2tte_drain_pending() in lib/s2tt/src/s2tt.c checks bit 32 
>>> without
>>> checking whether the descriptor is valid or invalid.
>>>
>>> In my testing, guarding the drain checks with a check for 
>>> S2TTE_INVALID seemed
>>> to resolve the boot hang:
>>> --- a/lib/s2tt/src/s2tt.c
>>> +++ b/lib/s2tt/src/s2tt.c
>>> @@ -1701,6 +1701,10 @@ unsigned long
>>> s2tte_clear_drain_pending(unsigned long s2tte)
>>>
>>>   bool s2tte_drain_pending(unsigned long s2tte)
>>>   {
>>> + if ((s2tte & S2TT_DESC_VALID_MASK) != S2TTE_INVALID) {
>>> + return false;
>>
>> We should use also consider cases where the entry is INVALID, but
>> has HIPAS=ASSIGNED/ASSIGNED_DEV to make it tighter. So, I think
>> it is better to use :
>>
>> s2tte_is_unassigned() or in the library stick to :
>>
>> if (!s2tte_has_hipas(s2tte, S2TTE_INVALID_HIPAS_UNASSIGNED))
>>      return false;
>>
>> May be we should assert this and make the caller responsible for
>> checking the bit. I will leave it to the tf-RMM team to fix.
>>
>> But for now, please use the above fix.
>>
> 
> Both worked for me. With the extra check in place, the realm guest can boot
> up successfully.
> 
> FYI, The below additional checks in s2tte_tlbi_pending() and 
> s2tte_drain_handle()
> aren't needed because they're always guarded by s2tte_drain_pending() in 
> all
> calling sites.

fyi, the tf-RMM patch is out for review here : (Thanks Javier)

https://review.trustedfirmware.org/c/TF-RMM/tf-rmm/+/53531

Please feel free to cherry-pick that one

Cheers
Suzuki