[PATCH] target/arm/kvm: fall back if nested unsupported

[PATCH] target/arm/kvm: fall back if nested unsupported

[Alyssa Ross]

If I create a machine with more CPUs than KVM supports, but specify
multiple accelerator options, QEMU will fall back to the next
accelerator.  This is great, because if I've explicitly specified
multiple accelerators, I've told QEMU I'm fine with any of them being
used to provide the machine I want.

When I create a machine with nested virtualization enabled, though,
this doesn't happen.  KVM often doesn't support it, but TCG always
does.  The nice thing to do would be for QEMU to fall back to TCG if
KVM can't provide, like it does when too many CPUs are requested.
This patch adjusts the behaviour to do that.

This is very helpful for OS development scripts that run an OS in QEMU
— I want everybody to be able to run the script, always with
virtualization enabled because the OS requires it, but for it to take
advantage of KVM acceleration when available.

Signed-off-by: Alyssa Ross <hi@alyssa.is>
---
 hw/arm/virt.c    | 6 ------
 target/arm/kvm.c | 8 ++++++++
 2 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 7456614d05..0b63b2eac3 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -2372,12 +2372,6 @@ static void machvirt_init(MachineState *machine)
         exit(1);
     }
 
-    if (vms->virt && kvm_enabled() && !kvm_arm_el2_supported()) {
-        error_report("mach-virt: host kernel KVM does not support providing "
-                     "Virtualization extensions to the guest CPU");
-        exit(1);
-    }
-
     if (vms->virt && !kvm_enabled() && !tcg_enabled() && !qtest_enabled()) {
         error_report("mach-virt: %s does not support providing "
                      "Virtualization extensions to the guest CPU",
diff --git a/target/arm/kvm.c b/target/arm/kvm.c
index d4a68874b8..20dcc6a820 100644
--- a/target/arm/kvm.c
+++ b/target/arm/kvm.c
@@ -615,6 +615,14 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
         ret = -EINVAL;
     }
 
+    if (object_property_find(OBJECT(ms), "virtualization") &&
+        object_property_get_bool(OBJECT(ms), "virtualization", NULL) &&
+        !kvm_arm_el2_supported()) {
+        error_report("Using ARM nested virtualization with KVM requires "
+                     "a host kernel with KVM_CAP_ARM_EL2");
+        ret = -EINVAL;
+    }
+
     if (kvm_check_extension(s, KVM_CAP_ARM_NISV_TO_USER)) {
         if (kvm_vm_enable_cap(s, KVM_CAP_ARM_NISV_TO_USER, 0)) {
             error_report("Failed to enable KVM_CAP_ARM_NISV_TO_USER cap");

base-commit: ae56950eac7b61b1abf42003329ee0f3ce111711
-- 
2.53.0

Re: [PATCH] target/arm/kvm: fall back if nested unsupported

[Paolo Bonzini]

Queued, thanks.

Paolo

Re: [PATCH] target/arm/kvm: fall back if nested unsupported

[Peter Maydell]

On Wed, 11 Mar 2026 at 09:54, Alyssa Ross <hi@alyssa.is> wrote:
>
> If I create a machine with more CPUs than KVM supports, but specify
> multiple accelerator options, QEMU will fall back to the next
> accelerator.  This is great, because if I've explicitly specified
> multiple accelerators, I've told QEMU I'm fine with any of them being
> used to provide the machine I want.
>
> When I create a machine with nested virtualization enabled, though,
> this doesn't happen.  KVM often doesn't support it, but TCG always
> does.  The nice thing to do would be for QEMU to fall back to TCG if
> KVM can't provide, like it does when too many CPUs are requested.
> This patch adjusts the behaviour to do that.
>
> This is very helpful for OS development scripts that run an OS in QEMU
> — I want everybody to be able to run the script, always with
> virtualization enabled because the OS requires it, but for it to take
> advantage of KVM acceleration when available.
>
> Signed-off-by: Alyssa Ross <hi@alyssa.is>
> ---
>  hw/arm/virt.c    | 6 ------
>  target/arm/kvm.c | 8 ++++++++
>  2 files changed, 8 insertions(+), 6 deletions(-)
>
> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> index 7456614d05..0b63b2eac3 100644
> --- a/hw/arm/virt.c
> +++ b/hw/arm/virt.c
> @@ -2372,12 +2372,6 @@ static void machvirt_init(MachineState *machine)
>          exit(1);
>      }
>
> -    if (vms->virt && kvm_enabled() && !kvm_arm_el2_supported()) {
> -        error_report("mach-virt: host kernel KVM does not support providing "
> -                     "Virtualization extensions to the guest CPU");
> -        exit(1);
> -    }
> -
>      if (vms->virt && !kvm_enabled() && !tcg_enabled() && !qtest_enabled()) {
>          error_report("mach-virt: %s does not support providing "
>                       "Virtualization extensions to the guest CPU",

I think there is a bigger problem here. The code in the virt
board init assumes that the accelerator has already been
selected: based on whether kvm_enabled() or tcg_enabled()
it decides things like which GIC version can be used, whether
"-machine gic-version=host" is valid or not, and so on.

This bit we're deleting here is just one of multiple checks
we do that assume that we know the accelerator already. If
we actually don't know if we're going to be using TCG or KVM
then all this code needs to be rethought.

Paolo, can you unqueue this for a bit while we think about it?

thanks
-- PMM

Re: [PATCH] target/arm/kvm: fall back if nested unsupported

[Paolo Bonzini]

On Wed, Mar 11, 2026 at 6:03 PM Peter Maydell <peter.maydell@linaro.org> wrote:
>
> On Wed, 11 Mar 2026 at 09:54, Alyssa Ross <hi@alyssa.is> wrote:
> >
> > If I create a machine with more CPUs than KVM supports, but specify
> > multiple accelerator options, QEMU will fall back to the next
> > accelerator.  This is great, because if I've explicitly specified
> > multiple accelerators, I've told QEMU I'm fine with any of them being
> > used to provide the machine I want.
> >
> > When I create a machine with nested virtualization enabled, though,
> > this doesn't happen.  KVM often doesn't support it, but TCG always
> > does.  The nice thing to do would be for QEMU to fall back to TCG if
> > KVM can't provide, like it does when too many CPUs are requested.
> > This patch adjusts the behaviour to do that.
> >
> > This is very helpful for OS development scripts that run an OS in QEMU
> > — I want everybody to be able to run the script, always with
> > virtualization enabled because the OS requires it, but for it to take
> > advantage of KVM acceleration when available.
> >
> > Signed-off-by: Alyssa Ross <hi@alyssa.is>
> > ---
> >  hw/arm/virt.c    | 6 ------
> >  target/arm/kvm.c | 8 ++++++++
> >  2 files changed, 8 insertions(+), 6 deletions(-)
> >
> > diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> > index 7456614d05..0b63b2eac3 100644
> > --- a/hw/arm/virt.c
> > +++ b/hw/arm/virt.c
> > @@ -2372,12 +2372,6 @@ static void machvirt_init(MachineState *machine)
> >          exit(1);
> >      }
> >
> > -    if (vms->virt && kvm_enabled() && !kvm_arm_el2_supported()) {
> > -        error_report("mach-virt: host kernel KVM does not support providing "
> > -                     "Virtualization extensions to the guest CPU");
> > -        exit(1);
> > -    }
> > -
> >      if (vms->virt && !kvm_enabled() && !tcg_enabled() && !qtest_enabled()) {
> >          error_report("mach-virt: %s does not support providing "
> >                       "Virtualization extensions to the guest CPU",
>
> I think there is a bigger problem here. The code in the virt
> board init assumes that the accelerator has already been
> selected: based on whether kvm_enabled() or tcg_enabled()
> it decides things like which GIC version can be used, whether
> "-machine gic-version=host" is valid or not, and so on.
>
> This bit we're deleting here is just one of multiple checks
> we do that assume that we know the accelerator already. If
> we actually don't know if we're going to be using TCG or KVM
> then all this code needs to be rethought.

We do. This code runs at the very end of qemu_init()
(qmp_x_init_preconfig->qemu_init_board->machine_run_board_init).

The bug is on the KVM side, as it lets the configuration slip even
though it's not valid; the above KVM check should really be more of an
abort() if anything. (This is also why I picked it despite it touching
hw/arm/virt.c - from my point of view the KVM fix made the above code
go dead).

> Paolo, can you unqueue this for a bit while we think about it?

Yes, of course.

Paolo

Re: [PATCH] target/arm/kvm: fall back if nested unsupported

[Alyssa Ross]

[-- Attachment #1: Type: text/plain, Size: 3403 bytes --]

Paolo Bonzini <pbonzini@redhat.com> writes:

> On Wed, Mar 11, 2026 at 6:03 PM Peter Maydell <peter.maydell@linaro.org> wrote:
>>
>> On Wed, 11 Mar 2026 at 09:54, Alyssa Ross <hi@alyssa.is> wrote:
>> >
>> > If I create a machine with more CPUs than KVM supports, but specify
>> > multiple accelerator options, QEMU will fall back to the next
>> > accelerator.  This is great, because if I've explicitly specified
>> > multiple accelerators, I've told QEMU I'm fine with any of them being
>> > used to provide the machine I want.
>> >
>> > When I create a machine with nested virtualization enabled, though,
>> > this doesn't happen.  KVM often doesn't support it, but TCG always
>> > does.  The nice thing to do would be for QEMU to fall back to TCG if
>> > KVM can't provide, like it does when too many CPUs are requested.
>> > This patch adjusts the behaviour to do that.
>> >
>> > This is very helpful for OS development scripts that run an OS in QEMU
>> > — I want everybody to be able to run the script, always with
>> > virtualization enabled because the OS requires it, but for it to take
>> > advantage of KVM acceleration when available.
>> >
>> > Signed-off-by: Alyssa Ross <hi@alyssa.is>
>> > ---
>> >  hw/arm/virt.c    | 6 ------
>> >  target/arm/kvm.c | 8 ++++++++
>> >  2 files changed, 8 insertions(+), 6 deletions(-)
>> >
>> > diff --git a/hw/arm/virt.c b/hw/arm/virt.c
>> > index 7456614d05..0b63b2eac3 100644
>> > --- a/hw/arm/virt.c
>> > +++ b/hw/arm/virt.c
>> > @@ -2372,12 +2372,6 @@ static void machvirt_init(MachineState *machine)
>> >          exit(1);
>> >      }
>> >
>> > -    if (vms->virt && kvm_enabled() && !kvm_arm_el2_supported()) {
>> > -        error_report("mach-virt: host kernel KVM does not support providing "
>> > -                     "Virtualization extensions to the guest CPU");
>> > -        exit(1);
>> > -    }
>> > -
>> >      if (vms->virt && !kvm_enabled() && !tcg_enabled() && !qtest_enabled()) {
>> >          error_report("mach-virt: %s does not support providing "
>> >                       "Virtualization extensions to the guest CPU",
>>
>> I think there is a bigger problem here. The code in the virt
>> board init assumes that the accelerator has already been
>> selected: based on whether kvm_enabled() or tcg_enabled()
>> it decides things like which GIC version can be used, whether
>> "-machine gic-version=host" is valid or not, and so on.
>>
>> This bit we're deleting here is just one of multiple checks
>> we do that assume that we know the accelerator already. If
>> we actually don't know if we're going to be using TCG or KVM
>> then all this code needs to be rethought.
>
> We do. This code runs at the very end of qemu_init()
> (qmp_x_init_preconfig->qemu_init_board->machine_run_board_init).
>
> The bug is on the KVM side, as it lets the configuration slip even
> though it's not valid; the above KVM check should really be more of an
> abort() if anything. (This is also why I picked it despite it touching
> hw/arm/virt.c - from my point of view the KVM fix made the above code
> go dead).

Yeah, the problem isn't that we don't know which accelerator is in use
in the virt board — we do.  It's that by that point it's too late to
fall back to the next acceptable accelerator if KVM can't provide
nested virtualization, so we need to check earlier.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: [PATCH] target/arm/kvm: fall back if nested unsupported

[Peter Maydell]

On Wed, 11 Mar 2026 at 09:54, Alyssa Ross <hi@alyssa.is> wrote:
>
> If I create a machine with more CPUs than KVM supports, but specify
> multiple accelerator options, QEMU will fall back to the next
> accelerator.  This is great, because if I've explicitly specified
> multiple accelerators, I've told QEMU I'm fine with any of them being
> used to provide the machine I want.
>
> When I create a machine with nested virtualization enabled, though,
> this doesn't happen.  KVM often doesn't support it, but TCG always
> does.  The nice thing to do would be for QEMU to fall back to TCG if
> KVM can't provide, like it does when too many CPUs are requested.
> This patch adjusts the behaviour to do that.
>
> This is very helpful for OS development scripts that run an OS in QEMU
> — I want everybody to be able to run the script, always with
> virtualization enabled because the OS requires it, but for it to take
> advantage of KVM acceleration when available.
>
> Signed-off-by: Alyssa Ross <hi@alyssa.is>
> ---
>  hw/arm/virt.c    | 6 ------
>  target/arm/kvm.c | 8 ++++++++
>  2 files changed, 8 insertions(+), 6 deletions(-)
>
> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> index 7456614d05..0b63b2eac3 100644
> --- a/hw/arm/virt.c
> +++ b/hw/arm/virt.c
> @@ -2372,12 +2372,6 @@ static void machvirt_init(MachineState *machine)
>          exit(1);
>      }
>
> -    if (vms->virt && kvm_enabled() && !kvm_arm_el2_supported()) {
> -        error_report("mach-virt: host kernel KVM does not support providing "
> -                     "Virtualization extensions to the guest CPU");
> -        exit(1);
> -    }
> -
>      if (vms->virt && !kvm_enabled() && !tcg_enabled() && !qtest_enabled()) {
>          error_report("mach-virt: %s does not support providing "
>                       "Virtualization extensions to the guest CPU",
> diff --git a/target/arm/kvm.c b/target/arm/kvm.c
> index d4a68874b8..20dcc6a820 100644
> --- a/target/arm/kvm.c
> +++ b/target/arm/kvm.c
> @@ -615,6 +615,14 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
>          ret = -EINVAL;
>      }
>
> +    if (object_property_find(OBJECT(ms), "virtualization") &&
> +        object_property_get_bool(OBJECT(ms), "virtualization", NULL) &&
> +        !kvm_arm_el2_supported()) {
> +        error_report("Using ARM nested virtualization with KVM requires "
> +                     "a host kernel with KVM_CAP_ARM_EL2");
> +        ret = -EINVAL;
> +    }

Looking a bit closer at this, it's a bit awkward that we're
looking at a machine property in generic target/arm code.
There is no guarantee that the machine is "virt" or that every
KVM-supporting machine has a "virtualization" property, and
the target/ code isn't really supposed to do board-specific stuff.

The board-independent way to say "are we trying to enable EL2" is
to look at the CPU property has_el2. But the CPU isn't created at
this point, so it's too early to do that here.

Similar things where the early accelerator code wants information
from the board we have handled with a method in MachineClass,
like get_physical_address_range. We could do that here, but
maybe it's a bit over-engineered? IDK.

There are other cases also where the virt board will error
out for "KVM and some config we could do with TCG":
 * KVM + EL2 + GICv2
 * KVM + EL2 + kernel-irqchip=off
 * KVM + GICv2 but host hardware doesn't support GICv2 emulation
 * KVM + GICv3 but host hardware doesn't support GICv3 emulation
 * KVM + MTE but host hardware/kernel doesn't support MTE

The first two are "always fails", but if we're going to have
a mechanism for "figure out in kvm_arch_init what KVM features
the board is going to want to enable" then it might be nice
to also be able to use it for the last 3.

-- PMM

Re: [PATCH] target/arm/kvm: fall back if nested unsupported

[Alyssa Ross]

[-- Attachment #1: Type: text/plain, Size: 4552 bytes --]

Peter Maydell <peter.maydell@linaro.org> writes:

> On Wed, 11 Mar 2026 at 09:54, Alyssa Ross <hi@alyssa.is> wrote:
>>
>> If I create a machine with more CPUs than KVM supports, but specify
>> multiple accelerator options, QEMU will fall back to the next
>> accelerator.  This is great, because if I've explicitly specified
>> multiple accelerators, I've told QEMU I'm fine with any of them being
>> used to provide the machine I want.
>>
>> When I create a machine with nested virtualization enabled, though,
>> this doesn't happen.  KVM often doesn't support it, but TCG always
>> does.  The nice thing to do would be for QEMU to fall back to TCG if
>> KVM can't provide, like it does when too many CPUs are requested.
>> This patch adjusts the behaviour to do that.
>>
>> This is very helpful for OS development scripts that run an OS in QEMU
>> — I want everybody to be able to run the script, always with
>> virtualization enabled because the OS requires it, but for it to take
>> advantage of KVM acceleration when available.
>>
>> Signed-off-by: Alyssa Ross <hi@alyssa.is>
>> ---
>>  hw/arm/virt.c    | 6 ------
>>  target/arm/kvm.c | 8 ++++++++
>>  2 files changed, 8 insertions(+), 6 deletions(-)
>>
>> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
>> index 7456614d05..0b63b2eac3 100644
>> --- a/hw/arm/virt.c
>> +++ b/hw/arm/virt.c
>> @@ -2372,12 +2372,6 @@ static void machvirt_init(MachineState *machine)
>>          exit(1);
>>      }
>>
>> -    if (vms->virt && kvm_enabled() && !kvm_arm_el2_supported()) {
>> -        error_report("mach-virt: host kernel KVM does not support providing "
>> -                     "Virtualization extensions to the guest CPU");
>> -        exit(1);
>> -    }
>> -
>>      if (vms->virt && !kvm_enabled() && !tcg_enabled() && !qtest_enabled()) {
>>          error_report("mach-virt: %s does not support providing "
>>                       "Virtualization extensions to the guest CPU",
>> diff --git a/target/arm/kvm.c b/target/arm/kvm.c
>> index d4a68874b8..20dcc6a820 100644
>> --- a/target/arm/kvm.c
>> +++ b/target/arm/kvm.c
>> @@ -615,6 +615,14 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
>>          ret = -EINVAL;
>>      }
>>
>> +    if (object_property_find(OBJECT(ms), "virtualization") &&
>> +        object_property_get_bool(OBJECT(ms), "virtualization", NULL) &&
>> +        !kvm_arm_el2_supported()) {
>> +        error_report("Using ARM nested virtualization with KVM requires "
>> +                     "a host kernel with KVM_CAP_ARM_EL2");
>> +        ret = -EINVAL;
>> +    }
>
> Looking a bit closer at this, it's a bit awkward that we're
> looking at a machine property in generic target/arm code.

Isn't SMP a machine property that we're looking at just above?

> There is no guarantee that the machine is "virt" or that every
> KVM-supporting machine has a "virtualization" property, and
> the target/ code isn't really supposed to do board-specific stuff.

FWIW: there's another arm board with the same property, which I think
would have the same requirements.  That's why I chose to just check the
property rather than checking machine type — the idea being to establish
the convention that machines /do/ have a virtualization property where
it makes sense.

> The board-independent way to say "are we trying to enable EL2" is
> to look at the CPU property has_el2. But the CPU isn't created at
> this point, so it's too early to do that here.

I spent a day or so trying to make something involving has_el2 work, but
ultimately couldn't get anywhere with it.

> Similar things where the early accelerator code wants information
> from the board we have handled with a method in MachineClass,
> like get_physical_address_range. We could do that here, but
> maybe it's a bit over-engineered? IDK.

I'll give it a go and see if I can make it work.

> There are other cases also where the virt board will error
> out for "KVM and some config we could do with TCG":
>  * KVM + EL2 + GICv2
>  * KVM + EL2 + kernel-irqchip=off
>  * KVM + GICv2 but host hardware doesn't support GICv2 emulation
>  * KVM + GICv3 but host hardware doesn't support GICv3 emulation
>  * KVM + MTE but host hardware/kernel doesn't support MTE
>
> The first two are "always fails", but if we're going to have
> a mechanism for "figure out in kvm_arch_init what KVM features
> the board is going to want to enable" then it might be nice
> to also be able to use it for the last 3.

I can try that.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]

Re: [PATCH] target/arm/kvm: fall back if nested unsupported

[Peter Maydell]

On Fri, 13 Mar 2026 at 11:47, Alyssa Ross <hi@alyssa.is> wrote:
>
> Peter Maydell <peter.maydell@linaro.org> writes:
> > Looking a bit closer at this, it's a bit awkward that we're
> > looking at a machine property in generic target/arm code.
>
> Isn't SMP a machine property that we're looking at just above?

It's one of the ones in the base MachineState that exist for
all boards.

> > There is no guarantee that the machine is "virt" or that every
> > KVM-supporting machine has a "virtualization" property, and
> > the target/ code isn't really supposed to do board-specific stuff.
>
> FWIW: there's another arm board with the same property, which I think
> would have the same requirements.  That's why I chose to just check the
> property rather than checking machine type — the idea being to establish
> the convention that machines /do/ have a virtualization property where
> it makes sense.
>
> > The board-independent way to say "are we trying to enable EL2" is
> > to look at the CPU property has_el2. But the CPU isn't created at
> > this point, so it's too early to do that here.
>
> I spent a day or so trying to make something involving has_el2 work, but
> ultimately couldn't get anywhere with it.
>
> > Similar things where the early accelerator code wants information
> > from the board we have handled with a method in MachineClass,
> > like get_physical_address_range. We could do that here, but
> > maybe it's a bit over-engineered? IDK.
>
> I'll give it a go and see if I can make it work.
>
> > There are other cases also where the virt board will error
> > out for "KVM and some config we could do with TCG":
> >  * KVM + EL2 + GICv2
> >  * KVM + EL2 + kernel-irqchip=off
> >  * KVM + GICv2 but host hardware doesn't support GICv2 emulation
> >  * KVM + GICv3 but host hardware doesn't support GICv3 emulation
> >  * KVM + MTE but host hardware/kernel doesn't support MTE
> >
> > The first two are "always fails", but if we're going to have
> > a mechanism for "figure out in kvm_arch_init what KVM features
> > the board is going to want to enable" then it might be nice
> > to also be able to use it for the last 3.
>
> I can try that.

Thanks, that would be helpful. I don't really have any
specific ideas about what kind of API might be best here.

There are probably similar issues with the other accelerators,
e.g. -accel hvf,tcg -machine virtualization=on
also ought to fall back to TCG, and similarly with "picked
a GIC version we can't handle with this accelerator".

-- PMM

Re: [PATCH] target/arm/kvm: fall back if nested unsupported

[Mohamed Mediouni]

> On 13. Mar 2026, at 13:33, Peter Maydell <peter.maydell@linaro.org> wrote:
> 
> On Fri, 13 Mar 2026 at 11:47, Alyssa Ross <hi@alyssa.is> wrote:
>> 
>> Peter Maydell <peter.maydell@linaro.org> writes:
>>> Looking a bit closer at this, it's a bit awkward that we're
>>> looking at a machine property in generic target/arm code.
>> 
>> Isn't SMP a machine property that we're looking at just above?
> 
> It's one of the ones in the base MachineState that exist for
> all boards.
> 
>>> There is no guarantee that the machine is "virt" or that every
>>> KVM-supporting machine has a "virtualization" property, and
>>> the target/ code isn't really supposed to do board-specific stuff.
>> 
>> FWIW: there's another arm board with the same property, which I think
>> would have the same requirements.  That's why I chose to just check the
>> property rather than checking machine type — the idea being to establish
>> the convention that machines /do/ have a virtualization property where
>> it makes sense.
>> 
>>> The board-independent way to say "are we trying to enable EL2" is
>>> to look at the CPU property has_el2. But the CPU isn't created at
>>> this point, so it's too early to do that here.
>> 
>> I spent a day or so trying to make something involving has_el2 work, but
>> ultimately couldn't get anywhere with it.
>> 
>>> Similar things where the early accelerator code wants information
>>> from the board we have handled with a method in MachineClass,
>>> like get_physical_address_range. We could do that here, but
>>> maybe it's a bit over-engineered? IDK.
>> 
>> I'll give it a go and see if I can make it work.
>> 
>>> There are other cases also where the virt board will error
>>> out for "KVM and some config we could do with TCG":
>>> * KVM + EL2 + GICv2
>>> * KVM + EL2 + kernel-irqchip=off
>>> * KVM + GICv2 but host hardware doesn't support GICv2 emulation
>>> * KVM + GICv3 but host hardware doesn't support GICv3 emulation
>>> * KVM + MTE but host hardware/kernel doesn't support MTE
>>> 
>>> The first two are "always fails", but if we're going to have
>>> a mechanism for "figure out in kvm_arch_init what KVM features
>>> the board is going to want to enable" then it might be nice
>>> to also be able to use it for the last 3.
>> 
>> I can try that.
> 
> Thanks, that would be helpful. I don't really have any
> specific ideas about what kind of API might be best here.
> 
> There are probably similar issues with the other accelerators,
> e.g. -accel hvf,tcg -machine virtualization=on
> also ought to fall back to TCG, and similarly with "picked
> a GIC version we can't handle with this accelerator”.
Hello,

Something to note about that (with HVF and KVM) is that there’s
also kernel-irqchip=off/on to take into mind. For example on a vGICv3-only
system you can still run GICv2 VMs with kernel-irqchip=off on KVM.

And for HVF once the hardware irqchip is merged kernel-irqchip=off
will remain there for (primarily) GICv2 usage.

Re: [PATCH] target/arm/kvm: fall back if nested unsupported

[Alyssa Ross]

[-- Attachment #1: Type: text/plain, Size: 3710 bytes --]

Peter Maydell <peter.maydell@linaro.org> writes:

> On Wed, 11 Mar 2026 at 09:54, Alyssa Ross <hi@alyssa.is> wrote:
>>
>> If I create a machine with more CPUs than KVM supports, but specify
>> multiple accelerator options, QEMU will fall back to the next
>> accelerator.  This is great, because if I've explicitly specified
>> multiple accelerators, I've told QEMU I'm fine with any of them being
>> used to provide the machine I want.
>>
>> When I create a machine with nested virtualization enabled, though,
>> this doesn't happen.  KVM often doesn't support it, but TCG always
>> does.  The nice thing to do would be for QEMU to fall back to TCG if
>> KVM can't provide, like it does when too many CPUs are requested.
>> This patch adjusts the behaviour to do that.
>>
>> This is very helpful for OS development scripts that run an OS in QEMU
>> — I want everybody to be able to run the script, always with
>> virtualization enabled because the OS requires it, but for it to take
>> advantage of KVM acceleration when available.
>>
>> Signed-off-by: Alyssa Ross <hi@alyssa.is>
>> ---
>>  hw/arm/virt.c    | 6 ------
>>  target/arm/kvm.c | 8 ++++++++
>>  2 files changed, 8 insertions(+), 6 deletions(-)
>>
>> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
>> index 7456614d05..0b63b2eac3 100644
>> --- a/hw/arm/virt.c
>> +++ b/hw/arm/virt.c
>> @@ -2372,12 +2372,6 @@ static void machvirt_init(MachineState *machine)
>>          exit(1);
>>      }
>>
>> -    if (vms->virt && kvm_enabled() && !kvm_arm_el2_supported()) {
>> -        error_report("mach-virt: host kernel KVM does not support providing "
>> -                     "Virtualization extensions to the guest CPU");
>> -        exit(1);
>> -    }
>> -
>>      if (vms->virt && !kvm_enabled() && !tcg_enabled() && !qtest_enabled()) {
>>          error_report("mach-virt: %s does not support providing "
>>                       "Virtualization extensions to the guest CPU",
>> diff --git a/target/arm/kvm.c b/target/arm/kvm.c
>> index d4a68874b8..20dcc6a820 100644
>> --- a/target/arm/kvm.c
>> +++ b/target/arm/kvm.c
>> @@ -615,6 +615,14 @@ int kvm_arch_init(MachineState *ms, KVMState *s)
>>          ret = -EINVAL;
>>      }
>>
>> +    if (object_property_find(OBJECT(ms), "virtualization") &&
>> +        object_property_get_bool(OBJECT(ms), "virtualization", NULL) &&
>> +        !kvm_arm_el2_supported()) {
>> +        error_report("Using ARM nested virtualization with KVM requires "
>> +                     "a host kernel with KVM_CAP_ARM_EL2");
>> +        ret = -EINVAL;
>> +    }
>
> Looking a bit closer at this, it's a bit awkward that we're
> looking at a machine property in generic target/arm code.
> There is no guarantee that the machine is "virt" or that every
> KVM-supporting machine has a "virtualization" property, and
> the target/ code isn't really supposed to do board-specific stuff.
>
> The board-independent way to say "are we trying to enable EL2" is
> to look at the CPU property has_el2. But the CPU isn't created at
> this point, so it's too early to do that here.
>
> Similar things where the early accelerator code wants information
> from the board we have handled with a method in MachineClass,
> like get_physical_address_range. We could do that here, but
> maybe it's a bit over-engineered? IDK.

What do you envisage this looking like for other platforms?  Something I
considered when working on this patch was moving "virtualization" from a
machine to an accelerator property, but I ran into the problem that such
a property isn't exposed on e.g. x86_64 as far as I can tell.  This
MachineState method would have the same problem.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 227 bytes --]