Re: [PATCH v16 00/45] arm64: Support for Arm CCA in KVM

[Gavin Shan <gshan@redhat.com>]

Hi Alper and Suzuki,

On 8/12/26 4:04 PM, Suzuki K Poulose wrote:
> Hi Alper, Gavin
> 
> On 12/08/2026 04:25, Alper Gun wrote:
>> On Tue, Aug 11, 2026 at 8:08 PM Gavin Shan <gshan@redhat.com> wrote:
>>> As the following calltrace indicates, -EAGAIN is returned from tf-rmm::update_ripas()
>>> because true is returned from s2tte_drain_pending() for the S2TTE corresponding to
>>> IPA 0x80000000. Linux host received error (RMI_ERROR_RTT, level=3) in ripas_change().
>>> Upon this specific error and the IPA range [0x80000000 0x90000000], find_map_level()
>>> returns level of 2, and realm_create_rtt_levels() returns 0 without populating any
>>> RTTs. After that, rmi_rtt_set_ripas() is re-executed and the above loop starts over
>>> again.
>>>
>>>     Linux host
>>>     ==========
>>>     kvm_arch_vcpu_ioctl_run                     // cca/host-v16
>>>       check_vcpu_requests
>>>         kvm_check_request
>>>           kvm_rec_handle_request
>>>             kvm_complete_ripas_change
>>>               realm_set_ipa_state
>>>                 ripas_change
>>>                   rmi_rtt_set_ripas
>>>                     SMC_RMI_RTT_SET_RIPAS
>>>
>>>     TF-RMM
>>>     ======
>>>     SMC_RMI_RTT_SET_RIPAS                      // tf-rmm/topics/rmm-v2.0-poc_3
>>>       smc_rtt_set_ripas
>>>         s2tt_walk_lock_unlock
>>>         rtt_set_ripas_range
>>>           update_ripas
>>>             s2tte_drain_pending                // true, returns -EAGAIN
>>>
>>> The problem is the pending-bit for RTE corresponding IPA address 0x80000000 isn't cleared
>>> when SMC_RMI_RTT_SET_RIPAS is invoked. I didn't figure out how this bit is set and why
>>> it's not cleared in time.
> 
> Thanks for the details.
> 
>>>
>>
>> Hi Gavin, Suzuki,
>>
>> I think I ran into a similar issue on rmm-v2.0-poc_3 last week.
>> This looks like a potential RMM bug: could bit 32 be part of the physical
>> Address (if PA >= 4 GiB)?
>>
>> It seems s2tte_drain_pending() in lib/s2tt/src/s2tt.c checks bit 32 without
>> checking whether the descriptor is valid or invalid.
>>
>> In my testing, guarding the drain checks with a check for S2TTE_INVALID seemed
>> to resolve the boot hang:
>> --- a/lib/s2tt/src/s2tt.c
>> +++ b/lib/s2tt/src/s2tt.c
>> @@ -1701,6 +1701,10 @@ unsigned long
>> s2tte_clear_drain_pending(unsigned long s2tte)
>>
>>   bool s2tte_drain_pending(unsigned long s2tte)
>>   {
>> + if ((s2tte & S2TT_DESC_VALID_MASK) != S2TTE_INVALID) {
>> + return false;
> 
> We should use also consider cases where the entry is INVALID, but
> has HIPAS=ASSIGNED/ASSIGNED_DEV to make it tighter. So, I think
> it is better to use :
> 
> s2tte_is_unassigned() or in the library stick to :
> 
> if (!s2tte_has_hipas(s2tte, S2TTE_INVALID_HIPAS_UNASSIGNED))
>      return false;
> 
> May be we should assert this and make the caller responsible for
> checking the bit. I will leave it to the tf-RMM team to fix.
> 
> But for now, please use the above fix.
> 

Both worked for me. With the extra check in place, the realm guest can boot
up successfully.

FYI, The below additional checks in s2tte_tlbi_pending() and s2tte_drain_handle()
aren't needed because they're always guarded by s2tte_drain_pending() in all
calling sites.

Thanks,
Gavin


> Cheers
> Suzuki
> 
> 
>> + }
>> +
>>    return (s2tte & S2TTE_SW_DRAIN_PENDING_BIT) != 0UL;
>>   }
>>
>> @@ -1730,11 +1734,19 @@ unsigned long
>> s2tte_clear_tlbi_pending(unsigned long s2tte)
>>
>>   bool s2tte_tlbi_pending(unsigned long s2tte)
>>   {
>> + if ((s2tte & S2TT_DESC_VALID_MASK) != S2TTE_INVALID) {
>> + return false;
>> + }
>> +
>>    return (s2tte & S2TTE_SW_TLBI_PENDING_BIT) != 0UL;
>>   }
>>
>>   unsigned int s2tte_drain_handle(unsigned long s2tte)
>>   {
>> + if ((s2tte & S2TT_DESC_VALID_MASK) != S2TTE_INVALID) {
>> + return 0U;
>> + }
>> +
>>    return (unsigned int)EXTRACT(S2TTE_SW_HANDLE, s2tte);
>>   }
>>
> 
> 
> 
> 
>> Sharing in case it helps.
>> Thanks,
>> Alper
>