KVM selftest for L2 guest execution fails

KVM selftest for L2 guest execution fails

[Itaru Kitayama]

Hi,
My small kvm sefltest tested on the kvmarm/next with NV2+VHE on FVP fails:

# ./arm64/test5
Random seed: 0x6b8b4567
==== Test Assertion Failure ====
  lib/kvm_util.c:1746: !ret
  pid=257 tid=257 errno=11 - Resource temporarily unavailable
sh: line 1: addr2line: command not found
  KVM_RUN failed, rc: -1 errno: 11 (Resource temporarily unavailable)

Below is the test code, am I doing still wrong?

Thanks,
Itaru.

 // SPDX-License-Identifier: GPL-2.0-only

#include "test_util.h"
#include "kvm_util.h"
#include "processor.h"
#include "ucall.h"

#define GUEST_SYNC_VAL 0x1234
#define UCALL_GPA 0x500000

static void l1_guest_code(void)
{
        /*
        * Set up HCR_EL2 to enable virtualization and nested support:
        *   - HCR_EL2.VM  = 1 (enable stage-2 translation)
        *   - HCR_EL2.NV  = 1 (trap to EL2 on guest HVC, SMC, WFI, etc.)
        *   - HCR_EL2.TGE = 1 (trap general exceptions in EL1 to EL2)
        */
       uint64_t hcr = (1UL << 0)   |  // VM
                      (1UL << 27)  |  // TGE
                      (1UL << 31);    // NV

       uint64_t spsr = 0x3c5;  // EL1h, AArch64, interrupts masked
       uint64_t l2_pc = 0x500000;     // L2 guest entry address
       uint64_t l2_sp = 0x510000;     // L2 guest stack

       __asm__ volatile(
               // Set up EL2 to EL1 return
               "msr hcr_el2, %[hcr]\n"
               "msr elr_el2, %[el1_pc]\n"
               "msr spsr_el2, %[spsr]\n"
               "mov sp, %[el1_sp]\n"
               "eret\n"
               :
               : [hcr] "r"(hcr),
                 [el1_pc] "r"(l2_pc),
                 [spsr] "r"(spsr),
                 [el1_sp] "r"(l2_sp)
               : "memory"
       );
}

static void l2_guest(void)
{
        asm volatile(
                "mov x0, #0x2a\n\t"
                "mov x1, %[ucall_sync]\n\t"
                "hvc #0\n\t"
                :
                : [ucall_sync] "i"(UCALL_SYNC)
                : "x0", "x1"
        );
}

static void guest_code()
{
       GUEST_SYNC(42);
}

int main(void)
{
        struct kvm_vm *vm;
        struct kvm_vcpu *vcpu;
        struct kvm_vcpu_init init = {};
        u64 feat = 0;

        /* NV2 must be supported or we skip */
        if (!kvm_check_cap(KVM_CAP_ARM_EL2))
                exit(KSFT_SKIP);

        feat |= BIT(KVM_ARM_VCPU_HAS_EL2); // Ask for virtual EL2

        vm = vm_create(1);

        vm_ioctl(vm, KVM_ARM_PREFERRED_TARGET, &init);
        init.features[0] = feat;

        vcpu = aarch64_vcpu_add(vm, 0, &init, l1_guest_code);

        // Backing memory for ucalls
        virt_pg_map(vm, UCALL_GPA, UCALL_GPA);
        ucall_init(vm, UCALL_GPA);

#define L2_MEM_START  0x500000
#define L2_MEM_SIZE   0x20000  // 128KB is enough

        vm_userspace_mem_region_add(vm, VM_MEM_SRC_ANONYMOUS, L2_MEM_START, 12, L2_MEM_SIZE / vm->page_size, KVM_MEM_LOG_DIRTY_PAGES);
        virt_pg_map(vm, 0x500000, 0x500000); // L2 code
        virt_pg_map(vm, 0x510000, 0x510000); // L2 stack
        memcpy(addr_gpa2hva(vm, 0x500000), l2_guest, vm->page_size);

        vcpu_run(vcpu);

        printf("vcpu_run exited: reason = %u\n", vcpu->run->exit_reason);

        struct ucall uc;
        switch (get_ucall(vcpu, &uc)) {
                case UCALL_SYNC:
                printf("Guest sync: val = 0x%lx\n", uc.args[1]);
                break;
                case UCALL_DONE:
                printf("Guest done\n");
                break;
                default:
                TEST_FAIL("Unknown ucall %lu\n", uc.cmd);
        }

        return 0;

}

Re: KVM selftest for L2 guest execution fails

[Marc Zyngier]

On Wed, 30 Jul 2025 07:44:48 +0100,
Itaru Kitayama <itaru.kitayama@linux.dev> wrote:
> 
> Hi,
> My small kvm sefltest tested on the kvmarm/next with NV2+VHE on FVP fails:
> 
> # ./arm64/test5
> Random seed: 0x6b8b4567
> ==== Test Assertion Failure ====
>   lib/kvm_util.c:1746: !ret
>   pid=257 tid=257 errno=11 - Resource temporarily unavailable
> sh: line 1: addr2line: command not found
>   KVM_RUN failed, rc: -1 errno: 11 (Resource temporarily unavailable)
> 
> Below is the test code, am I doing still wrong?

Quite a lot is wrong there.

> 
> Thanks,
> Itaru.
> 
>  // SPDX-License-Identifier: GPL-2.0-only
> 
> #include "test_util.h"
> #include "kvm_util.h"
> #include "processor.h"
> #include "ucall.h"
> 
> #define GUEST_SYNC_VAL 0x1234
> #define UCALL_GPA 0x500000
> 
> static void l1_guest_code(void)
> {
>         /*
>         * Set up HCR_EL2 to enable virtualization and nested support:
>         *   - HCR_EL2.VM  = 1 (enable stage-2 translation)
>         *   - HCR_EL2.NV  = 1 (trap to EL2 on guest HVC, SMC, WFI, etc.)
>         *   - HCR_EL2.TGE = 1 (trap general exceptions in EL1 to EL2)

You really want to read the spec again -- this is not what this means.

>         */
>        uint64_t hcr = (1UL << 0)   |  // VM
>                       (1UL << 27)  |  // TGE
>                       (1UL << 31);    // NV

- You enable S2 translation, and yet don't setup anything related to it

- You have TGE set, and yet (try to) return to EL1

- You set NV, and yet don't have a VNCR_EL2 populated

I don't know what you are trying to do here, but this is pretty badly
thought out.

I'd suggest you start by having a simple L2, without making use of NV
at all in the guest. You will still have to have a stage-2 and a
correct HCR_EL2 configuration -- none of which is as trivial as this
test suggests.

Once that is actually working, you can go wild and enable NV. But not
before.

	M.

-- 
Without deviation from the norm, progress is not possible.

Re: KVM selftest for L2 guest execution fails

[Itaru Kitayama]

Hi Marc,

> On Jul 30, 2025, at 21:24, Marc Zyngier <maz@kernel.org> wrote:
> 
> On Wed, 30 Jul 2025 07:44:48 +0100,
> Itaru Kitayama <itaru.kitayama@linux.dev> wrote:
>> 
>> Hi,
>> My small kvm sefltest tested on the kvmarm/next with NV2+VHE on FVP fails:
>> 
>> # ./arm64/test5
>> Random seed: 0x6b8b4567
>> ==== Test Assertion Failure ====
>>  lib/kvm_util.c:1746: !ret
>>  pid=257 tid=257 errno=11 - Resource temporarily unavailable
>> sh: line 1: addr2line: command not found
>>  KVM_RUN failed, rc: -1 errno: 11 (Resource temporarily unavailable)
>> 
>> Below is the test code, am I doing still wrong?
> 
> Quite a lot is wrong there.
> 
>> 
>> Thanks,
>> Itaru.
>> 
>> // SPDX-License-Identifier: GPL-2.0-only
>> 
>> #include "test_util.h"
>> #include "kvm_util.h"
>> #include "processor.h"
>> #include "ucall.h"
>> 
>> #define GUEST_SYNC_VAL 0x1234
>> #define UCALL_GPA 0x500000
>> 
>> static void l1_guest_code(void)
>> {
>>        /*
>>        * Set up HCR_EL2 to enable virtualization and nested support:
>>        *   - HCR_EL2.VM  = 1 (enable stage-2 translation)
>>        *   - HCR_EL2.NV  = 1 (trap to EL2 on guest HVC, SMC, WFI, etc.)
>>        *   - HCR_EL2.TGE = 1 (trap general exceptions in EL1 to EL2)
> 
> You really want to read the spec again -- this is not what this means.
> 
>>        */
>>       uint64_t hcr = (1UL << 0)   |  // VM
>>                      (1UL << 27)  |  // TGE
>>                      (1UL << 31);    // NV
> 
> - You enable S2 translation, and yet don't setup anything related to it
> 
> - You have TGE set, and yet (try to) return to EL1
> 
> - You set NV, and yet don't have a VNCR_EL2 populated
> 
> I don't know what you are trying to do here, but this is pretty badly
> thought out.
> 
> I'd suggest you start by having a simple L2, without making use of NV
> at all in the guest. You will still have to have a stage-2 and a
> correct HCR_EL2 configuration -- none of which is as trivial as this
> test suggests.
> 
> Once that is actually working, you can go wild and enable NV. But not
> before.

Thanks for your comments. Will follow your suggestions.

Itaru. 

> 
> M.
> 
> -- 
> Without deviation from the norm, progress is not possible.