Re: [PATCH v3] tests/functional/aarch64: add tests for FEAT_RME

[Pierrick Bouvier <pierrick.bouvier@linaro.org>]

Hi Jean-Philippe,
(adding Daniel and Thomas as well)

On 12/3/24 07:54, Jean-Philippe Brucker wrote:
> On Mon, Dec 02, 2024 at 04:17:41PM -0800, Pierrick Bouvier wrote:
>> This boot an OP-TEE environment, and launch a nested guest VM inside it
>> using the Realms feature. We do it for virt and sbsa-ref platforms.
>>
>> Signed-off-by: Pierrick Bouvier <pierrick.bouvier@linaro.org>
>>
>> -----
>>
>> v2:
>> - move test to its own file
>> - add sbsa test
>> - check output of `cca-workload-attestation report`
>>
>> v3:
>> - build and run test with cca-v4 images
>> - factorize nested guest test between both tests
>> - remove accel tcg option as it is the default when running tests
>> Note: It's a long test and there is a work in progress to understand why
>> debug build is so slow (x12 vs optimized).
> 
> Can the functional tests take as long as we want, or are they regularly
> run by a CI?
> 

It is run by the CI and developers as part of functional tests (make 
check-functional). The shorter the better thus.
Adding more (shorter) tests would be acceptable though, as they can be 
parallelized.

> The `cca-workload-attestation report` command is great for making sure the
> RME feature is working and the guest is running in a Realm, and it's very
> light. In my opinion it's sufficient here.
> 
> We could also run kvm-unit-tests on the host, various VM payloads and
> configs (in particular edk2 boot), and performing local attestation, but
> those would all be testing the other software components, so don't seem
> relevant to this functional test of the QEMU emulator. That said we do
> need regression tests for the whole software stack (including QEMU VMM)
> and this test automation is really convenient so I'd like to extend it
> with extra tests but outside the QEMU codebase.
>

I saw that kvm-unit-tests are mentioned in our documentation, but I'm 
not sure if we have any functional test image running them.

I'm not familiar enough with this topic to know what would be the value 
to add such tests, or edk2 boot.
That said, if we can have extra (short) tests upstream, I don't think it 
would hurt anyone.

For your need when working on an RME stack, it may be easier to keep 
those specific tests on your side.

Someone on this thread will probably have a better answer than mine :).

> Thanks,
> Jean
>