* [PATCH v6 01/13] hw/arm: virt: add GICv2m for the case when ITS is not available
2025-08-08 7:01 [PATCH v6 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
@ 2025-08-08 7:01 ` Mohamed Mediouni
2025-08-08 7:01 ` [PATCH v6 02/13] hvf: switch hvf_arm_get_host_cpu_features to not create a vCPU Mohamed Mediouni
` (11 subsequent siblings)
12 siblings, 0 replies; 29+ messages in thread
From: Mohamed Mediouni @ 2025-08-08 7:01 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Shannon Zhao, Michael S. Tsirkin, Ani Sinha,
qemu-arm, Paolo Bonzini, Roman Bolshakov, Igor Mammedov,
Phil Dennis-Jordan, Alexander Graf, Mads Ynddal, Cameron Esfahani,
Mohamed Mediouni
On Hypervisor.framework for macOS and WHPX for Windows, the provided environment is a GICv3 without ITS.
As such, support a GICv3 w/ GICv2m for that scenario.
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
---
hw/arm/virt-acpi-build.c | 4 +++-
hw/arm/virt.c | 8 ++++++++
include/hw/arm/virt.h | 2 ++
3 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
index b01fc4f8ef..969fa3f686 100644
--- a/hw/arm/virt-acpi-build.c
+++ b/hw/arm/virt-acpi-build.c
@@ -848,7 +848,9 @@ build_madt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
build_append_int_noprefix(table_data, memmap[VIRT_GIC_ITS].base, 8);
build_append_int_noprefix(table_data, 0, 4); /* Reserved */
}
- } else {
+ }
+
+ if (!vms->its && !vms->no_gicv3_with_gicv2m) {
const uint16_t spi_base = vms->irqmap[VIRT_GIC_V2M] + ARM_SPI_BASE;
/* 5.2.12.16 GIC MSI Frame Structure */
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index ef6be3660f..5951b331f3 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -953,6 +953,8 @@ static void create_gic(VirtMachineState *vms, MemoryRegion *mem)
if (vms->gic_version != VIRT_GIC_VERSION_2 && vms->its) {
create_its(vms);
+ } else if (vms->gic_version != VIRT_GIC_VERSION_2 && !vms->no_gicv3_with_gicv2m) {
+ create_v2m(vms);
} else if (vms->gic_version == VIRT_GIC_VERSION_2) {
create_v2m(vms);
}
@@ -2402,6 +2404,8 @@ static void machvirt_init(MachineState *machine)
vms->ns_el2_virt_timer_irq = ns_el2_virt_timer_present() &&
!vmc->no_ns_el2_virt_timer_irq;
+ vms->no_gicv3_with_gicv2m = vmc->no_gicv3_with_gicv2m;
+
fdt_add_timer_nodes(vms);
fdt_add_cpu_nodes(vms);
@@ -3410,6 +3414,7 @@ static void virt_instance_init(Object *obj)
vms->its = true;
/* Allow ITS emulation if the machine version supports it */
vms->tcg_its = !vmc->no_tcg_its;
+ vms->no_gicv3_with_gicv2m = false;
/* Default disallows iommu instantiation */
vms->iommu = VIRT_IOMMU_NONE;
@@ -3462,8 +3467,11 @@ DEFINE_VIRT_MACHINE_AS_LATEST(10, 1)
static void virt_machine_10_0_options(MachineClass *mc)
{
+ VirtMachineClass *vmc = VIRT_MACHINE_CLASS(OBJECT_CLASS(mc));
+
virt_machine_10_1_options(mc);
compat_props_add(mc->compat_props, hw_compat_10_0, hw_compat_10_0_len);
+ vmc->no_gicv3_with_gicv2m = true;
}
DEFINE_VIRT_MACHINE(10, 0)
diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h
index 365a28b082..725ec18fd2 100644
--- a/include/hw/arm/virt.h
+++ b/include/hw/arm/virt.h
@@ -131,6 +131,7 @@ struct VirtMachineClass {
bool no_cpu_topology;
bool no_tcg_lpa2;
bool no_ns_el2_virt_timer_irq;
+ bool no_gicv3_with_gicv2m;
bool no_nested_smmu;
};
@@ -178,6 +179,7 @@ struct VirtMachineState {
char *oem_id;
char *oem_table_id;
bool ns_el2_virt_timer_irq;
+ bool no_gicv3_with_gicv2m;
CXLState cxl_devices_state;
};
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH v6 02/13] hvf: switch hvf_arm_get_host_cpu_features to not create a vCPU
2025-08-08 7:01 [PATCH v6 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
2025-08-08 7:01 ` [PATCH v6 01/13] hw/arm: virt: add GICv2m for the case when ITS is not available Mohamed Mediouni
@ 2025-08-08 7:01 ` Mohamed Mediouni
2025-08-20 12:29 ` Mads Ynddal
2025-08-08 7:01 ` [PATCH v6 03/13] accel, hw/arm, include/system/hvf: infrastructure changes for HVF vGIC Mohamed Mediouni
` (10 subsequent siblings)
12 siblings, 1 reply; 29+ messages in thread
From: Mohamed Mediouni @ 2025-08-08 7:01 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Shannon Zhao, Michael S. Tsirkin, Ani Sinha,
qemu-arm, Paolo Bonzini, Roman Bolshakov, Igor Mammedov,
Phil Dennis-Jordan, Alexander Graf, Mads Ynddal, Cameron Esfahani,
Mohamed Mediouni, Philippe Mathieu-Daudé
Creating a vCPU locks out APIs such as hv_gic_create().
As a result, switch to using the hv_vcpu_config_get_feature_reg interface.
Hardcode MIDR because Apple deliberately doesn't expose a divergent MIDR across systems.
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
target/arm/hvf/hvf.c | 36 +++++++++++++++---------------------
1 file changed, 15 insertions(+), 21 deletions(-)
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index 47b0cd3a35..460782dbc0 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -864,24 +864,24 @@ static bool hvf_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf)
{
ARMISARegisters host_isar = {};
const struct isar_regs {
- int reg;
+ hv_feature_reg_t reg;
uint64_t *val;
} regs[] = {
- { HV_SYS_REG_ID_AA64PFR0_EL1, &host_isar.idregs[ID_AA64PFR0_EL1_IDX] },
- { HV_SYS_REG_ID_AA64PFR1_EL1, &host_isar.idregs[ID_AA64PFR1_EL1_IDX] },
- { HV_SYS_REG_ID_AA64DFR0_EL1, &host_isar.idregs[ID_AA64DFR0_EL1_IDX] },
- { HV_SYS_REG_ID_AA64DFR1_EL1, &host_isar.idregs[ID_AA64DFR1_EL1_IDX] },
- { HV_SYS_REG_ID_AA64ISAR0_EL1, &host_isar.idregs[ID_AA64ISAR0_EL1_IDX] },
- { HV_SYS_REG_ID_AA64ISAR1_EL1, &host_isar.idregs[ID_AA64ISAR1_EL1_IDX] },
+ { HV_FEATURE_REG_ID_AA64PFR0_EL1, &host_isar.idregs[ID_AA64PFR0_EL1_IDX] },
+ { HV_FEATURE_REG_ID_AA64PFR1_EL1, &host_isar.idregs[ID_AA64PFR1_EL1_IDX] },
+ { HV_FEATURE_REG_ID_AA64DFR0_EL1, &host_isar.idregs[ID_AA64DFR0_EL1_IDX] },
+ { HV_FEATURE_REG_ID_AA64DFR1_EL1, &host_isar.idregs[ID_AA64DFR1_EL1_IDX] },
+ { HV_FEATURE_REG_ID_AA64ISAR0_EL1, &host_isar.idregs[ID_AA64ISAR0_EL1_IDX] },
+ { HV_FEATURE_REG_ID_AA64ISAR1_EL1, &host_isar.idregs[ID_AA64ISAR1_EL1_IDX] },
/* Add ID_AA64ISAR2_EL1 here when HVF supports it */
- { HV_SYS_REG_ID_AA64MMFR0_EL1, &host_isar.idregs[ID_AA64MMFR0_EL1_IDX] },
- { HV_SYS_REG_ID_AA64MMFR1_EL1, &host_isar.idregs[ID_AA64MMFR1_EL1_IDX] },
- { HV_SYS_REG_ID_AA64MMFR2_EL1, &host_isar.idregs[ID_AA64MMFR2_EL1_IDX] },
+ { HV_FEATURE_REG_ID_AA64MMFR0_EL1, &host_isar.idregs[ID_AA64MMFR0_EL1_IDX] },
+ { HV_FEATURE_REG_ID_AA64MMFR1_EL1, &host_isar.idregs[ID_AA64MMFR1_EL1_IDX] },
+ { HV_FEATURE_REG_ID_AA64MMFR2_EL1, &host_isar.idregs[ID_AA64MMFR2_EL1_IDX] },
/* Add ID_AA64MMFR3_EL1 here when HVF supports it */
};
- hv_vcpu_t fd;
+
hv_return_t r = HV_SUCCESS;
- hv_vcpu_exit_t *exit;
+ hv_vcpu_config_t hv_vcpu_config = hv_vcpu_config_create();
int i;
ahcf->dtb_compatible = "arm,armv8";
@@ -891,17 +891,11 @@ static bool hvf_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf)
(1ULL << ARM_FEATURE_PMU) |
(1ULL << ARM_FEATURE_GENERIC_TIMER);
- /* We set up a small vcpu to extract host registers */
-
- if (hv_vcpu_create(&fd, &exit, NULL) != HV_SUCCESS) {
- return false;
- }
-
for (i = 0; i < ARRAY_SIZE(regs); i++) {
- r |= hv_vcpu_get_sys_reg(fd, regs[i].reg, regs[i].val);
+ r |= hv_vcpu_config_get_feature_reg(hv_vcpu_config, regs[i].reg, regs[i].val);
}
- r |= hv_vcpu_get_sys_reg(fd, HV_SYS_REG_MIDR_EL1, &ahcf->midr);
- r |= hv_vcpu_destroy(fd);
+ /* post-Armv6, Vendor: Apple (0x61), model and revision not set (all zeroes) */
+ ahcf->midr = 0x610f0000;
clamp_id_aa64mmfr0_parange_to_ipa_size(&host_isar);
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 29+ messages in thread* Re: [PATCH v6 02/13] hvf: switch hvf_arm_get_host_cpu_features to not create a vCPU
2025-08-08 7:01 ` [PATCH v6 02/13] hvf: switch hvf_arm_get_host_cpu_features to not create a vCPU Mohamed Mediouni
@ 2025-08-20 12:29 ` Mads Ynddal
0 siblings, 0 replies; 29+ messages in thread
From: Mads Ynddal @ 2025-08-20 12:29 UTC (permalink / raw)
To: Mohamed Mediouni
Cc: qemu-devel, Peter Maydell, Shannon Zhao, Michael S. Tsirkin,
Ani Sinha, qemu-arm, Paolo Bonzini, Roman Bolshakov,
Igor Mammedov, Phil Dennis-Jordan, Alexander Graf,
Cameron Esfahani, Philippe Mathieu-Daudé
> On 8 Aug 2025, at 09.01, Mohamed Mediouni <mohamed@unpredictable.fr> wrote:
>
> Creating a vCPU locks out APIs such as hv_gic_create().
>
> As a result, switch to using the hv_vcpu_config_get_feature_reg interface.
>
> Hardcode MIDR because Apple deliberately doesn't expose a divergent MIDR across systems.
>
> Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
>
> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
> target/arm/hvf/hvf.c | 36 +++++++++++++++---------------------
> 1 file changed, 15 insertions(+), 21 deletions(-)
>
Good way to avoid the limitations of hv_gic_create.
Reviewed-by: Mads Ynddal <mads@ynddal.dk>
^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH v6 03/13] accel, hw/arm, include/system/hvf: infrastructure changes for HVF vGIC
2025-08-08 7:01 [PATCH v6 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
2025-08-08 7:01 ` [PATCH v6 01/13] hw/arm: virt: add GICv2m for the case when ITS is not available Mohamed Mediouni
2025-08-08 7:01 ` [PATCH v6 02/13] hvf: switch hvf_arm_get_host_cpu_features to not create a vCPU Mohamed Mediouni
@ 2025-08-08 7:01 ` Mohamed Mediouni
2025-08-12 13:05 ` Philippe Mathieu-Daudé
2025-08-08 7:01 ` [PATCH v6 04/13] hw/intc: Add hvf vGIC interrupt controller support Mohamed Mediouni
` (9 subsequent siblings)
12 siblings, 1 reply; 29+ messages in thread
From: Mohamed Mediouni @ 2025-08-08 7:01 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Shannon Zhao, Michael S. Tsirkin, Ani Sinha,
qemu-arm, Paolo Bonzini, Roman Bolshakov, Igor Mammedov,
Phil Dennis-Jordan, Alexander Graf, Mads Ynddal, Cameron Esfahani,
Mohamed Mediouni
Misc changes needed for HVF vGIC enablement.
Note: x86_64 macOS exposes interrupt controller virtualisation since macOS 12.
Keeping an #ifdef here in case we end up supporting that...
However, given that x86_64 macOS is on its way out, it'll probably (?) not be supported in Qemu.
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
---
accel/hvf/hvf-all.c | 50 ++++++++++++++++++++++++++++++++++++++
accel/stubs/hvf-stub.c | 1 +
hw/arm/virt.c | 16 ++++++++----
hw/intc/arm_gicv3_common.c | 3 +++
include/system/hvf.h | 3 +++
system/vl.c | 2 ++
6 files changed, 70 insertions(+), 5 deletions(-)
diff --git a/accel/hvf/hvf-all.c b/accel/hvf/hvf-all.c
index 0a4b498e83..abd95f977c 100644
--- a/accel/hvf/hvf-all.c
+++ b/accel/hvf/hvf-all.c
@@ -10,6 +10,8 @@
#include "qemu/osdep.h"
#include "qemu/error-report.h"
+#include "qapi/error.h"
+#include "qapi/qapi-visit-common.h"
#include "accel/accel-ops.h"
#include "system/address-spaces.h"
#include "system/memory.h"
@@ -20,6 +22,7 @@
#include "trace.h"
bool hvf_allowed;
+bool hvf_kernel_irqchip;
struct mac_slot {
int present;
@@ -290,6 +293,43 @@ static int hvf_gdbstub_sstep_flags(AccelState *as)
return SSTEP_ENABLE | SSTEP_NOIRQ;
}
+static void hvf_set_kernel_irqchip(Object *obj, Visitor *v,
+ const char *name, void *opaque,
+ Error **errp)
+{
+ OnOffSplit mode;
+ if (!visit_type_OnOffSplit(v, name, &mode, errp)) {
+ return;
+ }
+
+ switch (mode) {
+ case ON_OFF_SPLIT_ON:
+#ifdef __x86_64__
+ /* macOS 12 onwards exposes an HVF virtual APIC. */
+ error_setg(errp, "HVF: kernel irqchip is not currently implemented for x86.");
+ break;
+#else
+ hvf_kernel_irqchip = true;
+ break;
+#endif
+
+ case ON_OFF_SPLIT_OFF:
+ hvf_kernel_irqchip = false;
+ break;
+
+ case ON_OFF_SPLIT_SPLIT:
+ error_setg(errp, "HVF: split irqchip is not supported on HVF.");
+ break;
+
+ default:
+ /*
+ * The value was checked in visit_type_OnOffSplit() above. If
+ * we get here, then something is wrong in QEMU.
+ */
+ abort();
+ }
+}
+
static void hvf_accel_class_init(ObjectClass *oc, const void *data)
{
AccelClass *ac = ACCEL_CLASS(oc);
@@ -297,6 +337,16 @@ static void hvf_accel_class_init(ObjectClass *oc, const void *data)
ac->init_machine = hvf_accel_init;
ac->allowed = &hvf_allowed;
ac->gdbstub_supported_sstep_flags = hvf_gdbstub_sstep_flags;
+#ifdef __x86_64__
+ hvf_kernel_irqchip = false;
+#else
+ hvf_kernel_irqchip = true;
+#endif
+ object_class_property_add(oc, "kernel-irqchip", "on|off|split",
+ NULL, hvf_set_kernel_irqchip,
+ NULL, NULL);
+ object_class_property_set_description(oc, "kernel-irqchip",
+ "Configure HVF irqchip");
}
static const TypeInfo hvf_accel_type = {
diff --git a/accel/stubs/hvf-stub.c b/accel/stubs/hvf-stub.c
index 42eadc5ca9..6bd08759ba 100644
--- a/accel/stubs/hvf-stub.c
+++ b/accel/stubs/hvf-stub.c
@@ -10,3 +10,4 @@
#include "system/hvf.h"
bool hvf_allowed;
+bool hvf_kernel_irqchip;
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 5951b331f3..59e6c2b9df 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -830,7 +830,7 @@ static void create_gic(VirtMachineState *vms, MemoryRegion *mem)
* interrupts; there are always 32 of the former (mandated by GIC spec).
*/
qdev_prop_set_uint32(vms->gic, "num-irq", NUM_IRQS + 32);
- if (!kvm_irqchip_in_kernel()) {
+ if (!kvm_irqchip_in_kernel() && !hvf_irqchip_in_kernel()) {
qdev_prop_set_bit(vms->gic, "has-security-extensions", vms->secure);
}
@@ -853,8 +853,9 @@ static void create_gic(VirtMachineState *vms, MemoryRegion *mem)
qdev_prop_set_array(vms->gic, "redist-region-count",
redist_region_count);
- if (!kvm_irqchip_in_kernel()) {
- if (vms->tcg_its) {
+ if (!kvm_irqchip_in_kernel() &&
+ !(hvf_enabled() && hvf_irqchip_in_kernel())) {
+ if (vms->its && vms->tcg_its) {
object_property_set_link(OBJECT(vms->gic), "sysmem",
OBJECT(mem), &error_fatal);
qdev_prop_set_bit(vms->gic, "has-lpi", true);
@@ -864,7 +865,7 @@ static void create_gic(VirtMachineState *vms, MemoryRegion *mem)
ARCH_GIC_MAINT_IRQ);
}
} else {
- if (!kvm_irqchip_in_kernel()) {
+ if (!kvm_irqchip_in_kernel() && !hvf_irqchip_in_kernel()) {
qdev_prop_set_bit(vms->gic, "has-virtualization-extensions",
vms->virt);
}
@@ -2060,7 +2061,12 @@ static void finalize_gic_version(VirtMachineState *vms)
/* KVM w/o kernel irqchip can only deal with GICv2 */
gics_supported |= VIRT_GIC_VERSION_2_MASK;
accel_name = "KVM with kernel-irqchip=off";
- } else if (tcg_enabled() || hvf_enabled() || qtest_enabled()) {
+ } else if (hvf_enabled()) {
+ if (!hvf_irqchip_in_kernel()) {
+ gics_supported |= VIRT_GIC_VERSION_2_MASK;
+ }
+ gics_supported |= VIRT_GIC_VERSION_3_MASK;
+ } else if (tcg_enabled() || qtest_enabled()) {
gics_supported |= VIRT_GIC_VERSION_2_MASK;
if (module_object_class_by_name("arm-gicv3")) {
gics_supported |= VIRT_GIC_VERSION_3_MASK;
diff --git a/hw/intc/arm_gicv3_common.c b/hw/intc/arm_gicv3_common.c
index e438d8c042..b8eee27260 100644
--- a/hw/intc/arm_gicv3_common.c
+++ b/hw/intc/arm_gicv3_common.c
@@ -32,6 +32,7 @@
#include "gicv3_internal.h"
#include "hw/arm/linux-boot-if.h"
#include "system/kvm.h"
+#include "system/hvf.h"
static void gicv3_gicd_no_migration_shift_bug_post_load(GICv3State *cs)
@@ -662,6 +663,8 @@ const char *gicv3_class_name(void)
{
if (kvm_irqchip_in_kernel()) {
return "kvm-arm-gicv3";
+ } else if (hvf_enabled() && hvf_irqchip_in_kernel()) {
+ return "hvf-arm-gicv3";
} else {
if (kvm_enabled()) {
error_report("Userspace GICv3 is not supported with KVM");
diff --git a/include/system/hvf.h b/include/system/hvf.h
index d3dcf088b3..dc8da85979 100644
--- a/include/system/hvf.h
+++ b/include/system/hvf.h
@@ -26,8 +26,11 @@
#ifdef CONFIG_HVF_IS_POSSIBLE
extern bool hvf_allowed;
#define hvf_enabled() (hvf_allowed)
+extern bool hvf_kernel_irqchip;
+#define hvf_irqchip_in_kernel() (hvf_kernel_irqchip)
#else /* !CONFIG_HVF_IS_POSSIBLE */
#define hvf_enabled() 0
+#define hvf_irqchip_in_kernel() 0
#endif /* !CONFIG_HVF_IS_POSSIBLE */
#define TYPE_HVF_ACCEL ACCEL_CLASS_NAME("hvf")
diff --git a/system/vl.c b/system/vl.c
index 3b7057e6c6..1c072d15a4 100644
--- a/system/vl.c
+++ b/system/vl.c
@@ -1773,6 +1773,8 @@ static void qemu_apply_legacy_machine_options(QDict *qdict)
false);
object_register_sugar_prop(ACCEL_CLASS_NAME("whpx"), "kernel-irqchip", value,
false);
+ object_register_sugar_prop(ACCEL_CLASS_NAME("hvf"), "kernel-irqchip", value,
+ false);
qdict_del(qdict, "kernel-irqchip");
}
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 29+ messages in thread* Re: [PATCH v6 03/13] accel, hw/arm, include/system/hvf: infrastructure changes for HVF vGIC
2025-08-08 7:01 ` [PATCH v6 03/13] accel, hw/arm, include/system/hvf: infrastructure changes for HVF vGIC Mohamed Mediouni
@ 2025-08-12 13:05 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2025-08-12 13:05 UTC (permalink / raw)
To: Mohamed Mediouni, qemu-devel
Cc: Peter Maydell, Shannon Zhao, Michael S. Tsirkin, Ani Sinha,
qemu-arm, Paolo Bonzini, Roman Bolshakov, Igor Mammedov,
Phil Dennis-Jordan, Alexander Graf, Mads Ynddal, Cameron Esfahani
On 8/8/25 09:01, Mohamed Mediouni wrote:
> Misc changes needed for HVF vGIC enablement.
>
> Note: x86_64 macOS exposes interrupt controller virtualisation since macOS 12.
> Keeping an #ifdef here in case we end up supporting that...
>
> However, given that x86_64 macOS is on its way out, it'll probably (?) not be supported in Qemu.
>
> Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
> ---
> accel/hvf/hvf-all.c | 50 ++++++++++++++++++++++++++++++++++++++
> accel/stubs/hvf-stub.c | 1 +
> hw/arm/virt.c | 16 ++++++++----
> hw/intc/arm_gicv3_common.c | 3 +++
> include/system/hvf.h | 3 +++
> system/vl.c | 2 ++
> 6 files changed, 70 insertions(+), 5 deletions(-)
> @@ -290,6 +293,43 @@ static int hvf_gdbstub_sstep_flags(AccelState *as)
> return SSTEP_ENABLE | SSTEP_NOIRQ;
> }
>
> +static void hvf_set_kernel_irqchip(Object *obj, Visitor *v,
> + const char *name, void *opaque,
> + Error **errp)
> +{
> + OnOffSplit mode;
> + if (!visit_type_OnOffSplit(v, name, &mode, errp)) {
> + return;
> + }
> +
> + switch (mode) {
> + case ON_OFF_SPLIT_ON:
> +#ifdef __x86_64__
> + /* macOS 12 onwards exposes an HVF virtual APIC. */
> + error_setg(errp, "HVF: kernel irqchip is not currently implemented for x86.");
> + break;
> +#else
> + hvf_kernel_irqchip = true;
> + break;
> +#endif
> +
> + case ON_OFF_SPLIT_OFF:
> + hvf_kernel_irqchip = false;
> + break;
> +
> + case ON_OFF_SPLIT_SPLIT:
> + error_setg(errp, "HVF: split irqchip is not supported on HVF.");
> + break;
> +
> + default:
> + /*
> + * The value was checked in visit_type_OnOffSplit() above. If
> + * we get here, then something is wrong in QEMU.
> + */
> + abort();
> + }
> +}
> +
> static void hvf_accel_class_init(ObjectClass *oc, const void *data)
> {
> AccelClass *ac = ACCEL_CLASS(oc);
> @@ -297,6 +337,16 @@ static void hvf_accel_class_init(ObjectClass *oc, const void *data)
> ac->init_machine = hvf_accel_init;
> ac->allowed = &hvf_allowed;
> ac->gdbstub_supported_sstep_flags = hvf_gdbstub_sstep_flags;
> +#ifdef __x86_64__
> + hvf_kernel_irqchip = false;
> +#else
> + hvf_kernel_irqchip = true;
I'm getting on Silicon m1:
qemu-system-aarch64: unknown type 'hvf-arm-gicv3'
> +#endif
> + object_class_property_add(oc, "kernel-irqchip", "on|off|split",
> + NULL, hvf_set_kernel_irqchip,
> + NULL, NULL);
> + object_class_property_set_description(oc, "kernel-irqchip",
> + "Configure HVF irqchip");
> }
>
> static const TypeInfo hvf_accel_type = {
> diff --git a/accel/stubs/hvf-stub.c b/accel/stubs/hvf-stub.c
> index 42eadc5ca9..6bd08759ba 100644
> --- a/accel/stubs/hvf-stub.c
> +++ b/accel/stubs/hvf-stub.c
> @@ -10,3 +10,4 @@
> #include "system/hvf.h"
>
> bool hvf_allowed;
> +bool hvf_kernel_irqchip;
> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> index 5951b331f3..59e6c2b9df 100644
> --- a/hw/arm/virt.c
> +++ b/hw/arm/virt.c
> @@ -830,7 +830,7 @@ static void create_gic(VirtMachineState *vms, MemoryRegion *mem)
> * interrupts; there are always 32 of the former (mandated by GIC spec).
> */
> qdev_prop_set_uint32(vms->gic, "num-irq", NUM_IRQS + 32);
> - if (!kvm_irqchip_in_kernel()) {
> + if (!kvm_irqchip_in_kernel() && !hvf_irqchip_in_kernel()) {
> qdev_prop_set_bit(vms->gic, "has-security-extensions", vms->secure);
> }
>
> @@ -853,8 +853,9 @@ static void create_gic(VirtMachineState *vms, MemoryRegion *mem)
> qdev_prop_set_array(vms->gic, "redist-region-count",
> redist_region_count);
>
> - if (!kvm_irqchip_in_kernel()) {
> - if (vms->tcg_its) {
> + if (!kvm_irqchip_in_kernel() &&
> + !(hvf_enabled() && hvf_irqchip_in_kernel())) {
> + if (vms->its && vms->tcg_its) {
> object_property_set_link(OBJECT(vms->gic), "sysmem",
> OBJECT(mem), &error_fatal);
> qdev_prop_set_bit(vms->gic, "has-lpi", true);
> @@ -864,7 +865,7 @@ static void create_gic(VirtMachineState *vms, MemoryRegion *mem)
> ARCH_GIC_MAINT_IRQ);
> }
> } else {
> - if (!kvm_irqchip_in_kernel()) {
> + if (!kvm_irqchip_in_kernel() && !hvf_irqchip_in_kernel()) {
> qdev_prop_set_bit(vms->gic, "has-virtualization-extensions",
> vms->virt);
> }
> @@ -2060,7 +2061,12 @@ static void finalize_gic_version(VirtMachineState *vms)
> /* KVM w/o kernel irqchip can only deal with GICv2 */
> gics_supported |= VIRT_GIC_VERSION_2_MASK;
> accel_name = "KVM with kernel-irqchip=off";
> - } else if (tcg_enabled() || hvf_enabled() || qtest_enabled()) {
> + } else if (hvf_enabled()) {
> + if (!hvf_irqchip_in_kernel()) {
> + gics_supported |= VIRT_GIC_VERSION_2_MASK;
> + }
> + gics_supported |= VIRT_GIC_VERSION_3_MASK;
> + } else if (tcg_enabled() || qtest_enabled()) {
> gics_supported |= VIRT_GIC_VERSION_2_MASK;
> if (module_object_class_by_name("arm-gicv3")) {
> gics_supported |= VIRT_GIC_VERSION_3_MASK;
> diff --git a/hw/intc/arm_gicv3_common.c b/hw/intc/arm_gicv3_common.c
> index e438d8c042..b8eee27260 100644
> --- a/hw/intc/arm_gicv3_common.c
> +++ b/hw/intc/arm_gicv3_common.c
> @@ -32,6 +32,7 @@
> #include "gicv3_internal.h"
> #include "hw/arm/linux-boot-if.h"
> #include "system/kvm.h"
> +#include "system/hvf.h"
>
>
> static void gicv3_gicd_no_migration_shift_bug_post_load(GICv3State *cs)
> @@ -662,6 +663,8 @@ const char *gicv3_class_name(void)
> {
> if (kvm_irqchip_in_kernel()) {
> return "kvm-arm-gicv3";
> + } else if (hvf_enabled() && hvf_irqchip_in_kernel()) {
> + return "hvf-arm-gicv3";
> } else {
> if (kvm_enabled()) {
> error_report("Userspace GICv3 is not supported with KVM");
> diff --git a/include/system/hvf.h b/include/system/hvf.h
> index d3dcf088b3..dc8da85979 100644
> --- a/include/system/hvf.h
> +++ b/include/system/hvf.h
> @@ -26,8 +26,11 @@
> #ifdef CONFIG_HVF_IS_POSSIBLE
> extern bool hvf_allowed;
> #define hvf_enabled() (hvf_allowed)
> +extern bool hvf_kernel_irqchip;
> +#define hvf_irqchip_in_kernel() (hvf_kernel_irqchip)
> #else /* !CONFIG_HVF_IS_POSSIBLE */
> #define hvf_enabled() 0
> +#define hvf_irqchip_in_kernel() 0
> #endif /* !CONFIG_HVF_IS_POSSIBLE */
>
> #define TYPE_HVF_ACCEL ACCEL_CLASS_NAME("hvf")
> diff --git a/system/vl.c b/system/vl.c
> index 3b7057e6c6..1c072d15a4 100644
> --- a/system/vl.c
> +++ b/system/vl.c
> @@ -1773,6 +1773,8 @@ static void qemu_apply_legacy_machine_options(QDict *qdict)
> false);
> object_register_sugar_prop(ACCEL_CLASS_NAME("whpx"), "kernel-irqchip", value,
> false);
> + object_register_sugar_prop(ACCEL_CLASS_NAME("hvf"), "kernel-irqchip", value,
> + false);
> qdict_del(qdict, "kernel-irqchip");
> }
>
^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH v6 04/13] hw/intc: Add hvf vGIC interrupt controller support
2025-08-08 7:01 [PATCH v6 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
` (2 preceding siblings ...)
2025-08-08 7:01 ` [PATCH v6 03/13] accel, hw/arm, include/system/hvf: infrastructure changes for HVF vGIC Mohamed Mediouni
@ 2025-08-08 7:01 ` Mohamed Mediouni
2025-08-12 16:19 ` Philippe Mathieu-Daudé
2025-08-14 12:57 ` Mads Ynddal
2025-08-08 7:01 ` [PATCH v6 05/13] hw/arm, target/arm: nested virtualisation on HVF Mohamed Mediouni
` (8 subsequent siblings)
12 siblings, 2 replies; 29+ messages in thread
From: Mohamed Mediouni @ 2025-08-08 7:01 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Shannon Zhao, Michael S. Tsirkin, Ani Sinha,
qemu-arm, Paolo Bonzini, Roman Bolshakov, Igor Mammedov,
Phil Dennis-Jordan, Alexander Graf, Mads Ynddal, Cameron Esfahani,
Mohamed Mediouni
This opens up the door to nested virtualisation support.
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
---
hw/intc/arm_gicv3_hvf.c | 723 ++++++++++++++++++++++++++++++++++++++++
hw/intc/meson.build | 1 +
2 files changed, 724 insertions(+)
create mode 100644 hw/intc/arm_gicv3_hvf.c
diff --git a/hw/intc/arm_gicv3_hvf.c b/hw/intc/arm_gicv3_hvf.c
new file mode 100644
index 0000000000..df86038972
--- /dev/null
+++ b/hw/intc/arm_gicv3_hvf.c
@@ -0,0 +1,723 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * ARM Generic Interrupt Controller using HVF platform support
+ *
+ * Copyright (c) 2025 Mohamed Mediouni
+ * Based on vGICv3 KVM code by Pavel Fedin
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "hw/intc/arm_gicv3_common.h"
+#include "qemu/error-report.h"
+#include "qemu/module.h"
+#include "system/runstate.h"
+#include "system/hvf.h"
+#include "system/hvf_int.h"
+#include "hvf_arm.h"
+#include "gicv3_internal.h"
+#include "vgic_common.h"
+#include "qom/object.h"
+#include "target/arm/cpregs.h"
+#include <Hypervisor/Hypervisor.h>
+
+struct HVFARMGICv3Class {
+ ARMGICv3CommonClass parent_class;
+ DeviceRealize parent_realize;
+ ResettablePhases parent_phases;
+};
+
+#define TYPE_HVF_GICV3 "hvf-arm-gicv3"
+typedef struct HVFARMGICv3Class HVFARMGICv3Class;
+
+/* This is reusing the GICv3State typedef from ARM_GICV3_ITS_COMMON */
+DECLARE_OBJ_CHECKERS(GICv3State, HVFARMGICv3Class,
+ HVF_GICV3, TYPE_HVF_GICV3);
+
+/*
+ * Loop through each distributor IRQ related register; since bits
+ * corresponding to SPIs and PPIs are RAZ/WI when affinity routing
+ * is enabled, we skip those.
+ */
+#define for_each_dist_irq_reg(_irq, _max, _field_width) \
+ for (_irq = GIC_INTERNAL; _irq < _max; _irq += (32 / _field_width))
+
+static void hvf_dist_get_priority(GICv3State *s, hv_gic_distributor_reg_t offset
+ , uint8_t *bmp)
+{
+ uint64_t reg;
+ uint32_t *field;
+ int irq;
+ field = (uint32_t *)(bmp);
+
+ for_each_dist_irq_reg(irq, s->num_irq, 8) {
+ hv_gic_get_distributor_reg(offset, ®);
+ *field = reg;
+ offset += 4;
+ field++;
+ }
+}
+
+static void hvf_dist_put_priority(GICv3State *s, hv_gic_distributor_reg_t offset
+ , uint8_t *bmp)
+{
+ uint32_t reg, *field;
+ int irq;
+ field = (uint32_t *)(bmp);
+
+ for_each_dist_irq_reg(irq, s->num_irq, 8) {
+ reg = *field;
+ hv_gic_set_distributor_reg(offset, reg);
+ offset += 4;
+ field++;
+ }
+}
+
+static void hvf_dist_get_edge_trigger(GICv3State *s, hv_gic_distributor_reg_t offset,
+ uint32_t *bmp)
+{
+ uint64_t reg;
+ int irq;
+
+ for_each_dist_irq_reg(irq, s->num_irq, 2) {
+ hv_gic_get_distributor_reg(offset, ®);
+ reg = half_unshuffle32(reg >> 1);
+ if (irq % 32 != 0) {
+ reg = (reg << 16);
+ }
+ *gic_bmp_ptr32(bmp, irq) |= reg;
+ offset += 4;
+ }
+}
+
+static void hvf_dist_put_edge_trigger(GICv3State *s, hv_gic_distributor_reg_t offset,
+ uint32_t *bmp)
+{
+ uint32_t reg;
+ int irq;
+
+ for_each_dist_irq_reg(irq, s->num_irq, 2) {
+ reg = *gic_bmp_ptr32(bmp, irq);
+ if (irq % 32 != 0) {
+ reg = (reg & 0xffff0000) >> 16;
+ } else {
+ reg = reg & 0xffff;
+ }
+ reg = half_shuffle32(reg) << 1;
+ hv_gic_set_distributor_reg(offset, reg);
+ offset += 4;
+ }
+}
+
+/* Read a bitmap register group from the kernel VGIC. */
+static void hvf_dist_getbmp(GICv3State *s, hv_gic_distributor_reg_t offset, uint32_t *bmp)
+{
+ uint64_t reg;
+ int irq;
+
+ for_each_dist_irq_reg(irq, s->num_irq, 1) {
+
+ hv_gic_get_distributor_reg(offset, ®);
+ *gic_bmp_ptr32(bmp, irq) = reg;
+ offset += 4;
+ }
+}
+
+static void hvf_dist_putbmp(GICv3State *s, hv_gic_distributor_reg_t offset,
+ hv_gic_distributor_reg_t clroffset, uint32_t *bmp)
+{
+ uint32_t reg;
+ int irq;
+
+ for_each_dist_irq_reg(irq, s->num_irq, 1) {
+ /*
+ * If this bitmap is a set/clear register pair, first write to the
+ * clear-reg to clear all bits before using the set-reg to write
+ * the 1 bits.
+ */
+ if (clroffset != 0) {
+ reg = 0;
+ hv_gic_set_distributor_reg(clroffset, reg);
+ clroffset += 4;
+ }
+ reg = *gic_bmp_ptr32(bmp, irq);
+ hv_gic_set_distributor_reg(offset, reg);
+ offset += 4;
+ }
+}
+
+static void hvf_gicv3_check(GICv3State *s)
+{
+ uint64_t reg;
+ uint32_t num_irq;
+
+ /* Sanity checking s->num_irq */
+ hv_gic_get_distributor_reg(HV_GIC_DISTRIBUTOR_REG_GICD_TYPER, ®);
+ num_irq = ((reg & 0x1f) + 1) * 32;
+
+ if (num_irq < s->num_irq) {
+ error_report("Model requests %u IRQs, but HVF supports max %u",
+ s->num_irq, num_irq);
+ abort();
+ }
+}
+
+static void hvf_gicv3_put_cpu(CPUState *cpu_state, run_on_cpu_data arg)
+{
+ uint32_t reg;
+ uint64_t reg64;
+ int i, num_pri_bits;
+
+ /* Redistributor state */
+ GICv3CPUState *c = arg.host_ptr;
+ hv_vcpu_t vcpu = c->cpu->accel->fd;
+
+ reg = c->gicr_waker;
+ hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_IGROUPR0, reg);
+
+ reg = c->gicr_igroupr0;
+ hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_IGROUPR0, reg);
+
+ reg = ~0;
+ hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ICENABLER0, reg);
+ reg = c->gicr_ienabler0;
+ hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ISENABLER0, reg);
+
+ /* Restore config before pending so we treat level/edge correctly */
+ reg = half_shuffle32(c->edge_trigger >> 16) << 1;
+ hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ICFGR1, reg);
+
+ reg = ~0;
+ hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ICPENDR0, reg);
+ reg = c->gicr_ipendr0;
+ hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ISPENDR0, reg);
+
+ reg = ~0;
+ hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ICACTIVER0, reg);
+ reg = c->gicr_iactiver0;
+ hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ISACTIVER0, reg);
+
+ for (i = 0; i < GIC_INTERNAL; i += 4) {
+ reg = c->gicr_ipriorityr[i] |
+ (c->gicr_ipriorityr[i + 1] << 8) |
+ (c->gicr_ipriorityr[i + 2] << 16) |
+ (c->gicr_ipriorityr[i + 3] << 24);
+ hv_gic_set_redistributor_reg(vcpu,
+ HV_GIC_REDISTRIBUTOR_REG_GICR_IPRIORITYR0 + i, reg);
+ }
+
+ /* CPU interface state */
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_SRE_EL1, c->icc_sre_el1);
+
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_CTLR_EL1,
+ c->icc_ctlr_el1[GICV3_NS]);
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_IGRPEN0_EL1,
+ c->icc_igrpen[GICV3_G0]);
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_IGRPEN1_EL1,
+ c->icc_igrpen[GICV3_G1NS]);
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_PMR_EL1, c->icc_pmr_el1);
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_BPR0_EL1, c->icc_bpr[GICV3_G0]);
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_BPR1_EL1, c->icc_bpr[GICV3_G1NS]);
+
+ num_pri_bits = ((c->icc_ctlr_el1[GICV3_NS] &
+ ICC_CTLR_EL1_PRIBITS_MASK) >>
+ ICC_CTLR_EL1_PRIBITS_SHIFT) + 1;
+
+ switch (num_pri_bits) {
+ case 7:
+ reg64 = c->icc_apr[GICV3_G0][3];
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1 + 3, reg64);
+ reg64 = c->icc_apr[GICV3_G0][2];
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1 + 2, reg64);
+ /* fall through */
+ case 6:
+ reg64 = c->icc_apr[GICV3_G0][1];
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1 + 1, reg64);
+ /* fall through */
+ default:
+ reg64 = c->icc_apr[GICV3_G0][0];
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1, reg64);
+ }
+
+ switch (num_pri_bits) {
+ case 7:
+ reg64 = c->icc_apr[GICV3_G1NS][3];
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1 + 3, reg64);
+ reg64 = c->icc_apr[GICV3_G1NS][2];
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1 + 2, reg64);
+ /* fall through */
+ case 6:
+ reg64 = c->icc_apr[GICV3_G1NS][1];
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1 + 1, reg64);
+ /* fall through */
+ default:
+ reg64 = c->icc_apr[GICV3_G1NS][0];
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1, reg64);
+ }
+
+ /* Registers beyond this point are with nested virt only */
+ if (!c->gic->maint_irq) {
+ return;
+ }
+
+ hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_VMCR_EL2, c->ich_vmcr_el2);
+ hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_HCR_EL2, c->ich_hcr_el2);
+
+ for (int i = 0; i < GICV3_LR_MAX; i++) {
+ hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_LR0_EL2, c->ich_lr_el2[i]);
+ }
+
+ num_pri_bits = c->vpribits;
+
+ switch (num_pri_bits) {
+ case 7:
+ hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2 + 3,
+ c->ich_apr[GICV3_G0][3]);
+ hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2 + 2,
+ c->ich_apr[GICV3_G0][2]);
+ /* fall through */
+ case 6:
+ hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2 + 1,
+ c->ich_apr[GICV3_G0][1]);
+ /* fall through */
+ default:
+ hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2,
+ c->ich_apr[GICV3_G0][0]);
+ }
+
+ switch (num_pri_bits) {
+ case 7:
+ hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2 + 3,
+ c->ich_apr[GICV3_G1NS][3]);
+ hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2 + 2,
+ c->ich_apr[GICV3_G1NS][2]);
+ /* fall through */
+ case 6:
+ hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2 + 1,
+ c->ich_apr[GICV3_G1NS][1]);
+ /* fall through */
+ default:
+ hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2,
+ c->ich_apr[GICV3_G1NS][0]);
+ }
+}
+
+static void hvf_gicv3_put(GICv3State *s)
+{
+ uint32_t reg;
+ uint64_t redist_typer;
+ int ncpu, i;
+
+ hvf_gicv3_check(s);
+
+ hv_vcpu_t vcpu0 = s->cpu[0].cpu->accel->fd;
+ hv_gic_get_redistributor_reg(vcpu0, HV_GIC_REDISTRIBUTOR_REG_GICR_TYPER
+ , &redist_typer);
+
+ reg = s->gicd_ctlr;
+ hv_gic_set_distributor_reg(HV_GIC_DISTRIBUTOR_REG_GICD_CTLR, reg);
+
+ if (redist_typer & GICR_TYPER_PLPIS) {
+ error_report("ITS is not supported on HVF.");
+ abort();
+ }
+
+ /* per-CPU state */
+
+ for (ncpu = 0; ncpu < s->num_cpu; ncpu++) {
+ run_on_cpu_data data;
+ data.host_ptr = &s->cpu[ncpu];
+ run_on_cpu(s->cpu[ncpu].cpu, hvf_gicv3_put_cpu, data);
+ }
+
+ /* s->enable bitmap -> GICD_ISENABLERn */
+ hvf_dist_putbmp(s, HV_GIC_DISTRIBUTOR_REG_GICD_ISENABLER0
+ , HV_GIC_DISTRIBUTOR_REG_GICD_ICENABLER0, s->enabled);
+
+ /* s->group bitmap -> GICD_IGROUPRn */
+ hvf_dist_putbmp(s, HV_GIC_DISTRIBUTOR_REG_GICD_IGROUPR0
+ , 0, s->group);
+
+ /* Restore targets before pending to ensure the pending state is set on
+ * the appropriate CPU interfaces in the kernel
+ */
+
+ /* s->gicd_irouter[irq] -> GICD_IROUTERn */
+ for (i = GIC_INTERNAL; i < s->num_irq; i++) {
+ uint32_t offset = HV_GIC_DISTRIBUTOR_REG_GICD_IROUTER32 + (8 * i)
+ - (8 * GIC_INTERNAL);
+ hv_gic_set_distributor_reg(offset, s->gicd_irouter[i]);
+ }
+
+ /*
+ * s->trigger bitmap -> GICD_ICFGRn
+ * (restore configuration registers before pending IRQs so we treat
+ * level/edge correctly)
+ */
+ hvf_dist_put_edge_trigger(s, HV_GIC_DISTRIBUTOR_REG_GICD_ICFGR0, s->edge_trigger);
+
+ /* s->pending bitmap -> GICD_ISPENDRn */
+ hvf_dist_putbmp(s, HV_GIC_DISTRIBUTOR_REG_GICD_ISPENDR0,
+ HV_GIC_DISTRIBUTOR_REG_GICD_ICPENDR0, s->pending);
+
+ /* s->active bitmap -> GICD_ISACTIVERn */
+ hvf_dist_putbmp(s, HV_GIC_DISTRIBUTOR_REG_GICD_ISACTIVER0,
+ HV_GIC_DISTRIBUTOR_REG_GICD_ICACTIVER0, s->active);
+
+ /* s->gicd_ipriority[] -> GICD_IPRIORITYRn */
+ hvf_dist_put_priority(s, HV_GIC_DISTRIBUTOR_REG_GICD_IPRIORITYR0, s->gicd_ipriority);
+}
+
+static void hvf_gicv3_get_cpu(CPUState *cpu_state, run_on_cpu_data arg)
+{
+ uint64_t reg;
+ int i, num_pri_bits;
+
+ /* Redistributor state */
+ GICv3CPUState *c = arg.host_ptr;
+ hv_vcpu_t vcpu = c->cpu->accel->fd;
+
+ hv_gic_get_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_IGROUPR0,
+ ®);
+ c->gicr_igroupr0 = reg;
+ hv_gic_get_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ISENABLER0,
+ ®);
+ c->gicr_ienabler0 = reg;
+ hv_gic_get_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ICFGR1,
+ ®);
+ c->edge_trigger = half_unshuffle32(reg >> 1) << 16;
+ hv_gic_get_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ISPENDR0,
+ ®);
+ c->gicr_ipendr0 = reg;
+ hv_gic_get_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ISACTIVER0,
+ ®);
+ c->gicr_iactiver0 = reg;
+
+ for (i = 0; i < GIC_INTERNAL; i += 4) {
+ hv_gic_get_redistributor_reg(
+ vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_IPRIORITYR0 + i, ®);
+ c->gicr_ipriorityr[i] = extract32(reg, 0, 8);
+ c->gicr_ipriorityr[i + 1] = extract32(reg, 8, 8);
+ c->gicr_ipriorityr[i + 2] = extract32(reg, 16, 8);
+ c->gicr_ipriorityr[i + 3] = extract32(reg, 24, 8);
+ }
+
+ /* CPU interface */
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_SRE_EL1, &c->icc_sre_el1);
+
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_CTLR_EL1,
+ &c->icc_ctlr_el1[GICV3_NS]);
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_IGRPEN0_EL1,
+ &c->icc_igrpen[GICV3_G0]);
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_IGRPEN1_EL1,
+ &c->icc_igrpen[GICV3_G1NS]);
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_PMR_EL1, &c->icc_pmr_el1);
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_BPR0_EL1, &c->icc_bpr[GICV3_G0]);
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_BPR1_EL1, &c->icc_bpr[GICV3_G1NS]);
+ num_pri_bits = ((c->icc_ctlr_el1[GICV3_NS] & ICC_CTLR_EL1_PRIBITS_MASK) >>
+ ICC_CTLR_EL1_PRIBITS_SHIFT) +
+ 1;
+
+ switch (num_pri_bits) {
+ case 7:
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1 + 3,
+ &c->icc_apr[GICV3_G0][3]);
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1 + 2,
+ &c->icc_apr[GICV3_G0][2]);
+ /* fall through */
+ case 6:
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1 + 1,
+ &c->icc_apr[GICV3_G0][1]);
+ /* fall through */
+ default:
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1,
+ &c->icc_apr[GICV3_G0][0]);
+ }
+
+ switch (num_pri_bits) {
+ case 7:
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1 + 3,
+ &c->icc_apr[GICV3_G1NS][3]);
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1 + 2,
+ &c->icc_apr[GICV3_G1NS][2]);
+ /* fall through */
+ case 6:
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1 + 1,
+ &c->icc_apr[GICV3_G1NS][1]);
+ /* fall through */
+ default:
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1,
+ &c->icc_apr[GICV3_G1NS][0]);
+ }
+
+ /* Registers beyond this point are with nested virt only */
+ if (!c->gic->maint_irq) {
+ return;
+ }
+
+ hv_gic_get_ich_reg(vcpu, HV_GIC_ICH_REG_VMCR_EL2, &c->ich_vmcr_el2);
+ hv_gic_get_ich_reg(vcpu, HV_GIC_ICH_REG_HCR_EL2, &c->ich_hcr_el2);
+
+ for (int i = 0; i < GICV3_LR_MAX; i++) {
+ hv_gic_get_ich_reg(vcpu, HV_GIC_ICH_REG_LR0_EL2, &c->ich_lr_el2[i]);
+ }
+
+ num_pri_bits = c->vpribits;
+
+ switch (num_pri_bits) {
+ case 7:
+ hv_gic_get_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2 + 3,
+ &c->ich_apr[GICV3_G0][3]);
+ hv_gic_get_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2 + 2,
+ &c->ich_apr[GICV3_G0][2]);
+ /* fall through */
+ case 6:
+ hv_gic_get_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2 + 1,
+ &c->ich_apr[GICV3_G0][1]);
+ /* fall through */
+ default:
+ hv_gic_get_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2,
+ &c->ich_apr[GICV3_G0][0]);
+ }
+
+ switch (num_pri_bits) {
+ case 7:
+ hv_gic_get_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2 + 3,
+ &c->ich_apr[GICV3_G1NS][3]);
+ hv_gic_get_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2 + 2,
+ &c->ich_apr[GICV3_G1NS][2]);
+ /* fall through */
+ case 6:
+ hv_gic_get_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2 + 1,
+ &c->ich_apr[GICV3_G1NS][1]);
+ /* fall through */
+ default:
+ hv_gic_get_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2,
+ &c->ich_apr[GICV3_G1NS][0]);
+ }
+}
+
+static void hvf_gicv3_get(GICv3State *s)
+{
+ uint64_t reg, redist_typer;
+ int ncpu, i;
+
+ hvf_gicv3_check(s);
+
+ hv_vcpu_t vcpu0 = s->cpu[0].cpu->accel->fd;
+ hv_gic_get_redistributor_reg(vcpu0,
+ HV_GIC_REDISTRIBUTOR_REG_GICR_TYPER, &redist_typer);
+
+ hv_gic_get_distributor_reg(HV_GIC_DISTRIBUTOR_REG_GICD_CTLR, ®);
+ s->gicd_ctlr = reg;
+
+ /* Redistributor state (one per CPU) */
+
+ for (ncpu = 0; ncpu < s->num_cpu; ncpu++) {
+ run_on_cpu_data data;
+ data.host_ptr = &s->cpu[ncpu];
+ run_on_cpu(s->cpu[ncpu].cpu, hvf_gicv3_get_cpu, data);
+ }
+
+ if (redist_typer & GICR_TYPER_PLPIS) {
+ error_report("ITS is not supported on HVF.");
+ abort();
+ }
+
+ /* GICD_IGROUPRn -> s->group bitmap */
+ hvf_dist_getbmp(s, HV_GIC_DISTRIBUTOR_REG_GICD_IGROUPR0, s->group);
+
+ /* GICD_ISENABLERn -> s->enabled bitmap */
+ hvf_dist_getbmp(s, HV_GIC_DISTRIBUTOR_REG_GICD_ISENABLER0, s->enabled);
+
+ /* GICD_ISPENDRn -> s->pending bitmap */
+ hvf_dist_getbmp(s, HV_GIC_DISTRIBUTOR_REG_GICD_ISPENDR0, s->pending);
+
+ /* GICD_ISACTIVERn -> s->active bitmap */
+ hvf_dist_getbmp(s, HV_GIC_DISTRIBUTOR_REG_GICD_ISACTIVER0, s->active);
+
+ /* GICD_ICFGRn -> s->trigger bitmap */
+ hvf_dist_get_edge_trigger(s, HV_GIC_DISTRIBUTOR_REG_GICD_ICFGR0
+ , s->edge_trigger);
+
+ /* GICD_IPRIORITYRn -> s->gicd_ipriority[] */
+ hvf_dist_get_priority(s, HV_GIC_DISTRIBUTOR_REG_GICD_IPRIORITYR0
+ , s->gicd_ipriority);
+
+ /* GICD_IROUTERn -> s->gicd_irouter[irq] */
+ for (i = GIC_INTERNAL; i < s->num_irq; i++) {
+ uint32_t offset = HV_GIC_DISTRIBUTOR_REG_GICD_IROUTER32
+ + (8 * i) - (8 * GIC_INTERNAL);
+ hv_gic_get_distributor_reg(offset, &s->gicd_irouter[i]);
+ }
+}
+
+static void hvf_gicv3_set_irq(void *opaque, int irq, int level)
+{
+ GICv3State *s = (GICv3State *)opaque;
+ if (irq > s->num_irq) {
+ return;
+ }
+ hv_gic_set_spi(GIC_INTERNAL + irq, !!level);
+}
+
+static void hvf_gicv3_icc_reset(CPUARMState *env, const ARMCPRegInfo *ri)
+{
+ GICv3State *s;
+ GICv3CPUState *c;
+
+ c = (GICv3CPUState *)env->gicv3state;
+ s = c->gic;
+
+ c->icc_pmr_el1 = 0;
+ /*
+ * Architecturally the reset value of the ICC_BPR registers
+ * is UNKNOWN. We set them all to 0 here; when the kernel
+ * uses these values to program the ICH_VMCR_EL2 fields that
+ * determine the guest-visible ICC_BPR register values, the
+ * hardware's "writing a value less than the minimum sets
+ * the field to the minimum value" behaviour will result in
+ * them effectively resetting to the correct minimum value
+ * for the host GIC.
+ */
+ c->icc_bpr[GICV3_G0] = 0;
+ c->icc_bpr[GICV3_G1] = 0;
+ c->icc_bpr[GICV3_G1NS] = 0;
+
+ c->icc_sre_el1 = 0x7;
+ memset(c->icc_apr, 0, sizeof(c->icc_apr));
+ memset(c->icc_igrpen, 0, sizeof(c->icc_igrpen));
+
+ if (s->migration_blocker) {
+ return;
+ }
+
+ /* Initialize to actual HW supported configuration */
+ hv_gic_get_icc_reg(c->cpu->accel->fd,
+ HV_GIC_ICC_REG_CTLR_EL1, &c->icc_ctlr_el1[GICV3_NS]);
+
+ c->icc_ctlr_el1[GICV3_S] = c->icc_ctlr_el1[GICV3_NS];
+}
+
+static void hvf_gicv3_reset_hold(Object *obj, ResetType type)
+{
+ GICv3State *s = ARM_GICV3_COMMON(obj);
+ HVFARMGICv3Class *kgc = HVF_GICV3_GET_CLASS(s);
+
+ if (kgc->parent_phases.hold) {
+ kgc->parent_phases.hold(obj, type);
+ }
+
+ hvf_gicv3_put(s);
+}
+
+
+/*
+ * CPU interface registers of GIC needs to be reset on CPU reset.
+ * For the calling arm_gicv3_icc_reset() on CPU reset, we register
+ * below ARMCPRegInfo. As we reset the whole cpu interface under single
+ * register reset, we define only one register of CPU interface instead
+ * of defining all the registers.
+ */
+static const ARMCPRegInfo gicv3_cpuif_reginfo[] = {
+ { .name = "ICC_CTLR_EL1", .state = ARM_CP_STATE_BOTH,
+ .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 12, .opc2 = 4,
+ /*
+ * If ARM_CP_NOP is used, resetfn is not called,
+ * So ARM_CP_NO_RAW is appropriate type.
+ */
+ .type = ARM_CP_NO_RAW,
+ .access = PL1_RW,
+ .readfn = arm_cp_read_zero,
+ .writefn = arm_cp_write_ignore,
+ /*
+ * We hang the whole cpu interface reset routine off here
+ * rather than parcelling it out into one little function
+ * per register
+ */
+ .resetfn = hvf_gicv3_icc_reset,
+ },
+};
+
+static void hvf_gicv3_realize(DeviceState *dev, Error **errp)
+{
+ GICv3State *s = HVF_GICV3(dev);
+ HVFARMGICv3Class *kgc = HVF_GICV3_GET_CLASS(s);
+ Error *local_err = NULL;
+ int i;
+
+ kgc->parent_realize(dev, &local_err);
+ if (local_err) {
+ error_propagate(errp, local_err);
+ return;
+ }
+
+ if (s->revision != 3) {
+ error_setg(errp, "unsupported GIC revision %d for platform GIC",
+ s->revision);
+ }
+
+ if (s->security_extn) {
+ error_setg(errp, "the platform vGICv3 does not implement the "
+ "security extensions");
+ return;
+ }
+
+ if (s->nmi_support) {
+ error_setg(errp, "NMI is not supported with the platform GIC");
+ return;
+ }
+
+ if (s->nb_redist_regions > 1) {
+ error_setg(errp, "Multiple VGICv3 redistributor regions are not "
+ "supported by HVF");
+ error_append_hint(errp, "A maximum of %d VCPUs can be used",
+ s->redist_region_count[0]);
+ return;
+ }
+
+ gicv3_init_irqs_and_mmio(s, hvf_gicv3_set_irq, NULL);
+
+ for (i = 0; i < s->num_cpu; i++) {
+ ARMCPU *cpu = ARM_CPU(qemu_get_cpu(i));
+
+ define_arm_cp_regs(cpu, gicv3_cpuif_reginfo);
+ }
+
+ if (s->maint_irq && s->maint_irq != HV_GIC_INT_MAINTENANCE) {
+ error_setg(errp, "vGIC maintenance IRQ mismatch with the hardcoded one in HVF.");
+ return;
+ }
+}
+
+static void hvf_gicv3_class_init(ObjectClass *klass, const void *data)
+{
+ DeviceClass *dc = DEVICE_CLASS(klass);
+ ResettableClass *rc = RESETTABLE_CLASS(klass);
+ ARMGICv3CommonClass *agcc = ARM_GICV3_COMMON_CLASS(klass);
+ HVFARMGICv3Class *kgc = HVF_GICV3_CLASS(klass);
+
+ agcc->pre_save = hvf_gicv3_get;
+ agcc->post_load = hvf_gicv3_put;
+
+ device_class_set_parent_realize(dc, hvf_gicv3_realize,
+ &kgc->parent_realize);
+ resettable_class_set_parent_phases(rc, NULL, hvf_gicv3_reset_hold, NULL,
+ &kgc->parent_phases);
+}
+
+static const TypeInfo hvf_arm_gicv3_info = {
+ .name = TYPE_HVF_GICV3,
+ .parent = TYPE_ARM_GICV3_COMMON,
+ .instance_size = sizeof(GICv3State),
+ .class_init = hvf_gicv3_class_init,
+ .class_size = sizeof(HVFARMGICv3Class),
+};
+
+static void hvf_gicv3_register_types(void)
+{
+ type_register_static(&hvf_arm_gicv3_info);
+}
+
+type_init(hvf_gicv3_register_types)
diff --git a/hw/intc/meson.build b/hw/intc/meson.build
index 3137521a4a..f446e966e3 100644
--- a/hw/intc/meson.build
+++ b/hw/intc/meson.build
@@ -42,6 +42,7 @@ specific_ss.add(when: 'CONFIG_ARM_GIC', if_true: files('arm_gicv3_cpuif_common.c
specific_ss.add(when: 'CONFIG_ARM_GICV3', if_true: files('arm_gicv3_cpuif.c'))
specific_ss.add(when: 'CONFIG_ARM_GIC_KVM', if_true: files('arm_gic_kvm.c'))
specific_ss.add(when: ['CONFIG_ARM_GIC_KVM', 'TARGET_AARCH64'], if_true: files('arm_gicv3_kvm.c', 'arm_gicv3_its_kvm.c'))
+specific_ss.add(when: ['CONFIG_HVF', 'CONFIG_ARM_GICV3'], if_true: files('arm_gicv3_hvf.c'))
specific_ss.add(when: 'CONFIG_ARM_V7M', if_true: files('armv7m_nvic.c'))
specific_ss.add(when: 'CONFIG_GRLIB', if_true: files('grlib_irqmp.c'))
specific_ss.add(when: 'CONFIG_IOAPIC', if_true: files('ioapic.c'))
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 29+ messages in thread* Re: [PATCH v6 04/13] hw/intc: Add hvf vGIC interrupt controller support
2025-08-08 7:01 ` [PATCH v6 04/13] hw/intc: Add hvf vGIC interrupt controller support Mohamed Mediouni
@ 2025-08-12 16:19 ` Philippe Mathieu-Daudé
2025-08-14 12:57 ` Mads Ynddal
1 sibling, 0 replies; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2025-08-12 16:19 UTC (permalink / raw)
To: Mohamed Mediouni, qemu-devel
Cc: Peter Maydell, Shannon Zhao, Michael S. Tsirkin, Ani Sinha,
qemu-arm, Paolo Bonzini, Roman Bolshakov, Igor Mammedov,
Phil Dennis-Jordan, Alexander Graf, Mads Ynddal, Cameron Esfahani,
Michael Tokarev
Hi,
On 8/8/25 09:01, Mohamed Mediouni wrote:
> This opens up the door to nested virtualisation support.
>
> Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
> ---
> hw/intc/arm_gicv3_hvf.c | 723 ++++++++++++++++++++++++++++++++++++++++
> hw/intc/meson.build | 1 +
> 2 files changed, 724 insertions(+)
> create mode 100644 hw/intc/arm_gicv3_hvf.c
> +static void hvf_gicv3_put_cpu(CPUState *cpu_state, run_on_cpu_data arg)
> +{
> + uint32_t reg;
> + uint64_t reg64;
> + int i, num_pri_bits;
> +
> + /* Redistributor state */
> + GICv3CPUState *c = arg.host_ptr;
> + hv_vcpu_t vcpu = c->cpu->accel->fd;
> +
> + reg = c->gicr_waker;
> + hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_IGROUPR0, reg);
> +
> + reg = c->gicr_igroupr0;
> + hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_IGROUPR0, reg);
> +
> + reg = ~0;
> + hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ICENABLER0, reg);
> + reg = c->gicr_ienabler0;
> + hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ISENABLER0, reg);
> +
> + /* Restore config before pending so we treat level/edge correctly */
> + reg = half_shuffle32(c->edge_trigger >> 16) << 1;
> + hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ICFGR1, reg);
> +
> + reg = ~0;
> + hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ICPENDR0, reg);
> + reg = c->gicr_ipendr0;
> + hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ISPENDR0, reg);
> +
> + reg = ~0;
> + hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ICACTIVER0, reg);
> + reg = c->gicr_iactiver0;
> + hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ISACTIVER0, reg);
> +
> + for (i = 0; i < GIC_INTERNAL; i += 4) {
> + reg = c->gicr_ipriorityr[i] |
> + (c->gicr_ipriorityr[i + 1] << 8) |
> + (c->gicr_ipriorityr[i + 2] << 16) |
> + (c->gicr_ipriorityr[i + 3] << 24);
> + hv_gic_set_redistributor_reg(vcpu,
> + HV_GIC_REDISTRIBUTOR_REG_GICR_IPRIORITYR0 + i, reg);
> + }
> +
> + /* CPU interface state */
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_SRE_EL1, c->icc_sre_el1);
> +
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_CTLR_EL1,
> + c->icc_ctlr_el1[GICV3_NS]);
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_IGRPEN0_EL1,
> + c->icc_igrpen[GICV3_G0]);
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_IGRPEN1_EL1,
> + c->icc_igrpen[GICV3_G1NS]);
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_PMR_EL1, c->icc_pmr_el1);
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_BPR0_EL1, c->icc_bpr[GICV3_G0]);
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_BPR1_EL1, c->icc_bpr[GICV3_G1NS]);
> +
> + num_pri_bits = ((c->icc_ctlr_el1[GICV3_NS] &
> + ICC_CTLR_EL1_PRIBITS_MASK) >>
> + ICC_CTLR_EL1_PRIBITS_SHIFT) + 1;
> +
> + switch (num_pri_bits) {
> + case 7:
> + reg64 = c->icc_apr[GICV3_G0][3];
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1 + 3, reg64);
> + reg64 = c->icc_apr[GICV3_G0][2];
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1 + 2, reg64);
> + /* fall through */
> + case 6:
> + reg64 = c->icc_apr[GICV3_G0][1];
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1 + 1, reg64);
> + /* fall through */
> + default:
> + reg64 = c->icc_apr[GICV3_G0][0];
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1, reg64);
> + }
> +
> + switch (num_pri_bits) {
> + case 7:
> + reg64 = c->icc_apr[GICV3_G1NS][3];
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1 + 3, reg64);
> + reg64 = c->icc_apr[GICV3_G1NS][2];
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1 + 2, reg64);
> + /* fall through */
> + case 6:
> + reg64 = c->icc_apr[GICV3_G1NS][1];
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1 + 1, reg64);
> + /* fall through */
> + default:
> + reg64 = c->icc_apr[GICV3_G1NS][0];
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1, reg64);
> + }
> +
> + /* Registers beyond this point are with nested virt only */
> + if (!c->gic->maint_irq) {
> + return;
FYI Michael reported a bug produced by this similar pattern:
https://lore.kernel.org/qemu-devel/3458e5b7-b53e-4057-baea-03a729452255@tls.msk.ru/
and Peter said on IRC "in a long realize function that's a bit
bug-prone. I think I would vote for factoring it out so you could write
if (mch->has_smm_ranges) { set_up_smm_ranges(); }".
Maybe we can extract the rest as hvf_gicv3_put_cpu_nested(),
hvf_gicv3_nested_put_cpu() -- or better name --?
> + }
> +
> + hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_VMCR_EL2, c->ich_vmcr_el2);
> + hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_HCR_EL2, c->ich_hcr_el2);
> +
> + for (int i = 0; i < GICV3_LR_MAX; i++) {
> + hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_LR0_EL2, c->ich_lr_el2[i]);
> + }
> +
> + num_pri_bits = c->vpribits;
> +
> + switch (num_pri_bits) {
> + case 7:
> + hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2 + 3,
> + c->ich_apr[GICV3_G0][3]);
> + hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2 + 2,
> + c->ich_apr[GICV3_G0][2]);
> + /* fall through */
> + case 6:
> + hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2 + 1,
> + c->ich_apr[GICV3_G0][1]);
> + /* fall through */
> + default:
> + hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2,
> + c->ich_apr[GICV3_G0][0]);
> + }
> +
> + switch (num_pri_bits) {
> + case 7:
> + hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2 + 3,
> + c->ich_apr[GICV3_G1NS][3]);
> + hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2 + 2,
> + c->ich_apr[GICV3_G1NS][2]);
> + /* fall through */
> + case 6:
> + hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2 + 1,
> + c->ich_apr[GICV3_G1NS][1]);
> + /* fall through */
> + default:
> + hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2,
> + c->ich_apr[GICV3_G1NS][0]);
> + }
> +}
^ permalink raw reply [flat|nested] 29+ messages in thread* Re: [PATCH v6 04/13] hw/intc: Add hvf vGIC interrupt controller support
2025-08-08 7:01 ` [PATCH v6 04/13] hw/intc: Add hvf vGIC interrupt controller support Mohamed Mediouni
2025-08-12 16:19 ` Philippe Mathieu-Daudé
@ 2025-08-14 12:57 ` Mads Ynddal
2025-08-15 5:55 ` Mohamed Mediouni
1 sibling, 1 reply; 29+ messages in thread
From: Mads Ynddal @ 2025-08-14 12:57 UTC (permalink / raw)
To: Mohamed Mediouni
Cc: qemu-devel, Peter Maydell, Shannon Zhao, Michael S. Tsirkin,
Ani Sinha, qemu-arm, Paolo Bonzini, Roman Bolshakov,
Igor Mammedov, Phil Dennis-Jordan, Alexander Graf,
Cameron Esfahani
> +static void hvf_gicv3_put(GICv3State *s)
> +{
> + uint32_t reg;
> + uint64_t redist_typer;
> + int ncpu, i;
> +
> + hvf_gicv3_check(s);
> +
> + hv_vcpu_t vcpu0 = s->cpu[0].cpu->accel->fd;
> + hv_gic_get_redistributor_reg(vcpu0, HV_GIC_REDISTRIBUTOR_REG_GICR_TYPER
> + , &redist_typer);
> +
I think you should go through all your "hv_" calls and do error handling
where it's possible, and if not, at least check it with assert_hvf_ok.
This call to hv_gic_get_redistributor_reg is failing on my setup. I
don't know if there could be others.
I've configured my VM like this:
-machine virt,kernel-irqchip=on,its=off
-boot menu=on,splash-time=0
^ permalink raw reply [flat|nested] 29+ messages in thread* Re: [PATCH v6 04/13] hw/intc: Add hvf vGIC interrupt controller support
2025-08-14 12:57 ` Mads Ynddal
@ 2025-08-15 5:55 ` Mohamed Mediouni
2025-08-19 13:21 ` Mads Ynddal
0 siblings, 1 reply; 29+ messages in thread
From: Mohamed Mediouni @ 2025-08-15 5:55 UTC (permalink / raw)
To: Mads Ynddal
Cc: qemu-devel, Peter Maydell, Shannon Zhao, Michael S. Tsirkin,
Ani Sinha, qemu-arm, Paolo Bonzini, Roman Bolshakov,
Igor Mammedov, Phil Dennis-Jordan, Alexander Graf,
Cameron Esfahani
> On 14. Aug 2025, at 14:57, Mads Ynddal <mads@ynddal.dk> wrote:
>
>
>> +static void hvf_gicv3_put(GICv3State *s)
>> +{
>> + uint32_t reg;
>> + uint64_t redist_typer;
>> + int ncpu, i;
>> +
>> + hvf_gicv3_check(s);
>> +
>> + hv_vcpu_t vcpu0 = s->cpu[0].cpu->accel->fd;
>> + hv_gic_get_redistributor_reg(vcpu0, HV_GIC_REDISTRIBUTOR_REG_GICR_TYPER
>> + , &redist_typer);
>> +
>
> I think you should go through all your "hv_" calls and do error handling
> where it's possible, and if not, at least check it with assert_hvf_ok.
> This call to hv_gic_get_redistributor_reg is failing on my setup. I
> don't know if there could be others.
>
> I've configured my VM like this:
> -machine virt,kernel-irqchip=on,its=off
> -boot menu=on,splash-time=0
Will do this for my next revision. On which chip/macOS release did you see this failure?
-m
^ permalink raw reply [flat|nested] 29+ messages in thread* Re: [PATCH v6 04/13] hw/intc: Add hvf vGIC interrupt controller support
2025-08-15 5:55 ` Mohamed Mediouni
@ 2025-08-19 13:21 ` Mads Ynddal
0 siblings, 0 replies; 29+ messages in thread
From: Mads Ynddal @ 2025-08-19 13:21 UTC (permalink / raw)
To: Mohamed Mediouni
Cc: qemu-devel, Peter Maydell, Shannon Zhao, Michael S. Tsirkin,
Ani Sinha, qemu-arm, Paolo Bonzini, Roman Bolshakov,
Igor Mammedov, Phil Dennis-Jordan, Alexander Graf,
Cameron Esfahani
> On which chip/macOS release did you see this failure?
This was on Apple M3 Pro. Must have been macOS 15.6.
^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH v6 05/13] hw/arm, target/arm: nested virtualisation on HVF
2025-08-08 7:01 [PATCH v6 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
` (3 preceding siblings ...)
2025-08-08 7:01 ` [PATCH v6 04/13] hw/intc: Add hvf vGIC interrupt controller support Mohamed Mediouni
@ 2025-08-08 7:01 ` Mohamed Mediouni
2025-08-11 12:47 ` Philippe Mathieu-Daudé
2025-08-11 12:56 ` Philippe Mathieu-Daudé
2025-08-08 7:01 ` [PATCH v6 06/13] hvf: save/restore Apple GIC state Mohamed Mediouni
` (7 subsequent siblings)
12 siblings, 2 replies; 29+ messages in thread
From: Mohamed Mediouni @ 2025-08-08 7:01 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Shannon Zhao, Michael S. Tsirkin, Ani Sinha,
qemu-arm, Paolo Bonzini, Roman Bolshakov, Igor Mammedov,
Phil Dennis-Jordan, Alexander Graf, Mads Ynddal, Cameron Esfahani,
Mohamed Mediouni
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
---
hw/arm/virt.c | 9 ++++++---
target/arm/hvf-stub.c | 15 +++++++++++++++
target/arm/hvf/hvf.c | 41 +++++++++++++++++++++++++++++++++++++++--
target/arm/hvf_arm.h | 3 +++
4 files changed, 63 insertions(+), 5 deletions(-)
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 59e6c2b9df..91d8cd9363 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -817,8 +817,7 @@ static void create_gic(VirtMachineState *vms, MemoryRegion *mem)
g_assert_not_reached();
}
- if (kvm_enabled() && vms->virt &&
- (revision != 3 || !kvm_irqchip_in_kernel())) {
+ if (kvm_enabled() && vms->virt && (revision != 3 || !kvm_irqchip_in_kernel())) {
error_report("KVM EL2 is only supported with in-kernel GICv3");
exit(1);
}
@@ -2281,7 +2280,8 @@ static void machvirt_init(MachineState *machine)
exit(1);
}
- if (vms->virt && !kvm_enabled() && !tcg_enabled() && !qtest_enabled()) {
+ if (vms->virt && !kvm_enabled() && !tcg_enabled()
+ && !hvf_enabled() && !qtest_enabled()) {
error_report("mach-virt: %s does not support providing "
"Virtualization extensions to the guest CPU",
current_accel_name());
@@ -2553,6 +2553,9 @@ static void virt_set_virt(Object *obj, bool value, Error **errp)
VirtMachineState *vms = VIRT_MACHINE(obj);
vms->virt = value;
+#if defined(CONFIG_HVF) && defined(__aarch64__)
+ hvf_arm_el2_enable(value);
+#endif
}
static bool virt_get_highmem(Object *obj, Error **errp)
diff --git a/target/arm/hvf-stub.c b/target/arm/hvf-stub.c
index ff137267a0..95ec4ea62f 100644
--- a/target/arm/hvf-stub.c
+++ b/target/arm/hvf-stub.c
@@ -18,3 +18,18 @@ uint32_t hvf_arm_get_max_ipa_bit_size(void)
{
g_assert_not_reached();
}
+
+bool hvf_arm_el2_supported(void)
+{
+ g_assert_not_reached();
+}
+
+bool hvf_arm_el2_enabled(void)
+{
+ g_assert_not_reached();
+}
+
+void hvf_arm_el2_enable(bool)
+{
+ g_assert_not_reached();
+}
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index 460782dbc0..483a50329b 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -26,6 +26,7 @@
#include "system/address-spaces.h"
#include "system/memory.h"
#include "hw/boards.h"
+#include "hw/arm/virt.h"
#include "hw/irq.h"
#include "qemu/main-loop.h"
#include "system/cpus.h"
@@ -891,6 +892,10 @@ static bool hvf_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf)
(1ULL << ARM_FEATURE_PMU) |
(1ULL << ARM_FEATURE_GENERIC_TIMER);
+ if (hvf_arm_el2_enabled()) {
+ ahcf->features |= 1ULL << ARM_FEATURE_EL2;
+ }
+
for (i = 0; i < ARRAY_SIZE(regs); i++) {
r |= hv_vcpu_config_get_feature_reg(hv_vcpu_config, regs[i].reg, regs[i].val);
}
@@ -958,6 +963,25 @@ uint32_t hvf_arm_get_max_ipa_bit_size(void)
return round_down_to_parange_bit_size(max_ipa_size);
}
+bool hvf_arm_el2_supported(void)
+{
+ bool is_nested_virt_supported;
+ hv_return_t ret = hv_vm_config_get_el2_supported(&is_nested_virt_supported);
+ assert_hvf_ok(ret);
+ return is_nested_virt_supported;
+}
+
+static bool is_nested_virt_enabled = false;
+bool hvf_arm_el2_enabled(void)
+{
+ return is_nested_virt_enabled;
+}
+
+void hvf_arm_el2_enable(bool enable)
+{
+ is_nested_virt_enabled = enable;
+}
+
void hvf_arm_set_cpu_features_from_host(ARMCPU *cpu)
{
if (!arm_host_cpu_features.dtb_compatible) {
@@ -994,6 +1018,13 @@ hv_return_t hvf_arch_vm_create(MachineState *ms, uint32_t pa_range)
}
chosen_ipa_bit_size = pa_range;
+ if (hvf_arm_el2_enabled()) {
+ ret = hv_vm_config_set_el2_enabled(config, true);
+ if (ret != HV_SUCCESS) {
+ goto cleanup;
+ }
+ }
+
ret = hv_vm_create(config);
cleanup:
@@ -1101,6 +1132,13 @@ static void hvf_psci_cpu_off(ARMCPU *arm_cpu)
assert(ret == QEMU_ARM_POWERCTL_RET_SUCCESS);
}
+static int hvf_psci_get_target_el(void)
+{
+ if (hvf_arm_el2_enabled()) {
+ return 2;
+ }
+ return 1;
+}
/*
* Handle a PSCI call.
*
@@ -1122,7 +1160,6 @@ static bool hvf_handle_psci_call(CPUState *cpu)
CPUState *target_cpu_state;
ARMCPU *target_cpu;
target_ulong entry;
- int target_el = 1;
int32_t ret = 0;
trace_hvf_psci_call(param[0], param[1], param[2], param[3],
@@ -1176,7 +1213,7 @@ static bool hvf_handle_psci_call(CPUState *cpu)
entry = param[2];
context_id = param[3];
ret = arm_set_cpu_on(mpidr, entry, context_id,
- target_el, target_aarch64);
+ hvf_psci_get_target_el(), target_aarch64);
break;
case QEMU_PSCI_0_1_FN_CPU_OFF:
case QEMU_PSCI_0_2_FN_CPU_OFF:
diff --git a/target/arm/hvf_arm.h b/target/arm/hvf_arm.h
index ea82f2691d..bf55e7ae28 100644
--- a/target/arm/hvf_arm.h
+++ b/target/arm/hvf_arm.h
@@ -24,5 +24,8 @@ void hvf_arm_set_cpu_features_from_host(ARMCPU *cpu);
uint32_t hvf_arm_get_default_ipa_bit_size(void);
uint32_t hvf_arm_get_max_ipa_bit_size(void);
+bool hvf_arm_el2_supported(void);
+bool hvf_arm_el2_enabled(void);
+void hvf_arm_el2_enable(bool);
#endif
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 29+ messages in thread* Re: [PATCH v6 05/13] hw/arm, target/arm: nested virtualisation on HVF
2025-08-08 7:01 ` [PATCH v6 05/13] hw/arm, target/arm: nested virtualisation on HVF Mohamed Mediouni
@ 2025-08-11 12:47 ` Philippe Mathieu-Daudé
2025-08-11 13:35 ` Mohamed Mediouni
2025-08-11 12:56 ` Philippe Mathieu-Daudé
1 sibling, 1 reply; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2025-08-11 12:47 UTC (permalink / raw)
To: Mohamed Mediouni, qemu-devel
Cc: Peter Maydell, Shannon Zhao, Michael S. Tsirkin, Ani Sinha,
qemu-arm, Paolo Bonzini, Roman Bolshakov, Igor Mammedov,
Phil Dennis-Jordan, Alexander Graf, Mads Ynddal, Cameron Esfahani
On 8/8/25 09:01, Mohamed Mediouni wrote:
> Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
> ---
> hw/arm/virt.c | 9 ++++++---
> target/arm/hvf-stub.c | 15 +++++++++++++++
> target/arm/hvf/hvf.c | 41 +++++++++++++++++++++++++++++++++++++++--
> target/arm/hvf_arm.h | 3 +++
> 4 files changed, 63 insertions(+), 5 deletions(-)
I'm getting HV_UNSUPPORTED on Silicon M1, I suppose because this
method is not being called in this series (is there a patch missing?):
> +bool hvf_arm_el2_supported(void)
> +{
> + bool is_nested_virt_supported;
> + hv_return_t ret = hv_vm_config_get_el2_supported(&is_nested_virt_supported);
> + assert_hvf_ok(ret);
> + return is_nested_virt_supported;
> +}
^ permalink raw reply [flat|nested] 29+ messages in thread* Re: [PATCH v6 05/13] hw/arm, target/arm: nested virtualisation on HVF
2025-08-11 12:47 ` Philippe Mathieu-Daudé
@ 2025-08-11 13:35 ` Mohamed Mediouni
2025-08-11 13:40 ` Philippe Mathieu-Daudé
0 siblings, 1 reply; 29+ messages in thread
From: Mohamed Mediouni @ 2025-08-11 13:35 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: qemu-devel, Peter Maydell, Shannon Zhao, Michael S. Tsirkin,
Ani Sinha, qemu-arm, Paolo Bonzini, Roman Bolshakov,
Igor Mammedov, Phil Dennis-Jordan, Alexander Graf, Mads Ynddal,
Cameron Esfahani
[-- Attachment #1: Type: text/plain, Size: 295 bytes --]
> On 11. Aug 2025, at 14:47, Philippe Mathieu-Daudé <philmd@linaro.org> wrote:
>
> I'm getting HV_UNSUPPORTED on Silicon M1, I suppose because this
> method is not being called in this series (is there a patch missing?):
Nested virt is only supported by macOS on Apple M3 and later
[-- Attachment #2: Type: text/html, Size: 2059 bytes --]
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v6 05/13] hw/arm, target/arm: nested virtualisation on HVF
2025-08-11 13:35 ` Mohamed Mediouni
@ 2025-08-11 13:40 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2025-08-11 13:40 UTC (permalink / raw)
To: Mohamed Mediouni
Cc: qemu-devel, Peter Maydell, Shannon Zhao, Michael S. Tsirkin,
Ani Sinha, qemu-arm, Paolo Bonzini, Roman Bolshakov,
Igor Mammedov, Phil Dennis-Jordan, Alexander Graf, Mads Ynddal,
Cameron Esfahani
On 11/8/25 15:35, Mohamed Mediouni wrote:
>
>
>> On 11. Aug 2025, at 14:47, Philippe Mathieu-Daudé <philmd@linaro.org>
>> wrote:
>>
>> I'm getting HV_UNSUPPORTED on Silicon M1, I suppose because this
>> method is not being called in this series (is there a patch missing?):
> Nested virt is only supported by macOS on Apple M3 and later
I know, but maybe this is what we need to report to users trying to
enable it, rather than aborting ;)
^ permalink raw reply [flat|nested] 29+ messages in thread
* Re: [PATCH v6 05/13] hw/arm, target/arm: nested virtualisation on HVF
2025-08-08 7:01 ` [PATCH v6 05/13] hw/arm, target/arm: nested virtualisation on HVF Mohamed Mediouni
2025-08-11 12:47 ` Philippe Mathieu-Daudé
@ 2025-08-11 12:56 ` Philippe Mathieu-Daudé
2025-08-11 13:13 ` Philippe Mathieu-Daudé
2025-08-11 13:35 ` Mohamed Mediouni
1 sibling, 2 replies; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2025-08-11 12:56 UTC (permalink / raw)
To: Mohamed Mediouni, qemu-devel
Cc: Peter Maydell, Shannon Zhao, Michael S. Tsirkin, Ani Sinha,
qemu-arm, Paolo Bonzini, Roman Bolshakov, Igor Mammedov,
Phil Dennis-Jordan, Alexander Graf, Mads Ynddal, Cameron Esfahani
On 8/8/25 09:01, Mohamed Mediouni wrote:
> Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
> ---
> hw/arm/virt.c | 9 ++++++---
> target/arm/hvf-stub.c | 15 +++++++++++++++
> target/arm/hvf/hvf.c | 41 +++++++++++++++++++++++++++++++++++++++--
> target/arm/hvf_arm.h | 3 +++
> 4 files changed, 63 insertions(+), 5 deletions(-)
>
> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> index 59e6c2b9df..91d8cd9363 100644
> --- a/hw/arm/virt.c
> +++ b/hw/arm/virt.c
> @@ -817,8 +817,7 @@ static void create_gic(VirtMachineState *vms, MemoryRegion *mem)
> g_assert_not_reached();
> }
>
> - if (kvm_enabled() && vms->virt &&
> - (revision != 3 || !kvm_irqchip_in_kernel())) {
> + if (kvm_enabled() && vms->virt && (revision != 3 || !kvm_irqchip_in_kernel())) {
> error_report("KVM EL2 is only supported with in-kernel GICv3");
> exit(1);
> }
> @@ -2281,7 +2280,8 @@ static void machvirt_init(MachineState *machine)
> exit(1);
> }
>
> - if (vms->virt && !kvm_enabled() && !tcg_enabled() && !qtest_enabled()) {
> + if (vms->virt && !kvm_enabled() && !tcg_enabled()
> + && !hvf_enabled() && !qtest_enabled()) {
> error_report("mach-virt: %s does not support providing "
> "Virtualization extensions to the guest CPU",
> current_accel_name());
> @@ -2553,6 +2553,9 @@ static void virt_set_virt(Object *obj, bool value, Error **errp)
> VirtMachineState *vms = VIRT_MACHINE(obj);
>
> vms->virt = value;
> +#if defined(CONFIG_HVF) && defined(__aarch64__)
> + hvf_arm_el2_enable(value);
> +#endif
> }
We don't have hvf_arm_el1_enable(). I think we just want to always
provide the most performant configuration to users.
IOW, we don't need hvf_arm_el2_enable(). I'd just always enable EL2
(if supported) in hvf_arm_get_host_cpu_features().
^ permalink raw reply [flat|nested] 29+ messages in thread* Re: [PATCH v6 05/13] hw/arm, target/arm: nested virtualisation on HVF
2025-08-11 12:56 ` Philippe Mathieu-Daudé
@ 2025-08-11 13:13 ` Philippe Mathieu-Daudé
2025-08-11 13:35 ` Mohamed Mediouni
1 sibling, 0 replies; 29+ messages in thread
From: Philippe Mathieu-Daudé @ 2025-08-11 13:13 UTC (permalink / raw)
To: Mohamed Mediouni, qemu-devel
Cc: Peter Maydell, Shannon Zhao, Michael S. Tsirkin, Ani Sinha,
qemu-arm, Paolo Bonzini, Roman Bolshakov, Igor Mammedov,
Phil Dennis-Jordan, Alexander Graf, Mads Ynddal, Cameron Esfahani
On 11/8/25 14:56, Philippe Mathieu-Daudé wrote:
> On 8/8/25 09:01, Mohamed Mediouni wrote:
>> Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
>> ---
>> hw/arm/virt.c | 9 ++++++---
>> target/arm/hvf-stub.c | 15 +++++++++++++++
>> target/arm/hvf/hvf.c | 41 +++++++++++++++++++++++++++++++++++++++--
>> target/arm/hvf_arm.h | 3 +++
>> 4 files changed, 63 insertions(+), 5 deletions(-)
>>
>> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
>> index 59e6c2b9df..91d8cd9363 100644
>> --- a/hw/arm/virt.c
>> +++ b/hw/arm/virt.c
>> @@ -817,8 +817,7 @@ static void create_gic(VirtMachineState *vms,
>> MemoryRegion *mem)
>> g_assert_not_reached();
>> }
>> - if (kvm_enabled() && vms->virt &&
>> - (revision != 3 || !kvm_irqchip_in_kernel())) {
>> + if (kvm_enabled() && vms->virt && (revision != 3 || !
>> kvm_irqchip_in_kernel())) {
>> error_report("KVM EL2 is only supported with in-kernel GICv3");
>> exit(1);
>> }
>> @@ -2281,7 +2280,8 @@ static void machvirt_init(MachineState *machine)
>> exit(1);
>> }
>> - if (vms->virt && !kvm_enabled() && !tcg_enabled() && !
>> qtest_enabled()) {
>> + if (vms->virt && !kvm_enabled() && !tcg_enabled()
>> + && !hvf_enabled() && !qtest_enabled()) {
>> error_report("mach-virt: %s does not support providing "
>> "Virtualization extensions to the guest CPU",
>> current_accel_name());
>> @@ -2553,6 +2553,9 @@ static void virt_set_virt(Object *obj, bool
>> value, Error **errp)
>> VirtMachineState *vms = VIRT_MACHINE(obj);
>> vms->virt = value;
>> +#if defined(CONFIG_HVF) && defined(__aarch64__)
>> + hvf_arm_el2_enable(value);
>> +#endif
>> }
> We don't have hvf_arm_el1_enable(). I think we just want to always
> provide the most performant configuration to users.
> IOW, we don't need hvf_arm_el2_enable(). I'd just always enable EL2
> (if supported) in hvf_arm_get_host_cpu_features().
We create vCPUs in machvirt_init(). Upon vCPU INITialization, we check
host features. Then during REALIZation, we check the QOM properties.
If 'virt' flag is disabled, machvirt_init() sets "has_el2" property
to false, effectively setting ARMCPU::has_el2 field to %false.
In arm_cpu_realizefn(), if ARMCPU::has_el2 we unset ARM_FEATURE_EL2.
^ permalink raw reply [flat|nested] 29+ messages in thread* Re: [PATCH v6 05/13] hw/arm, target/arm: nested virtualisation on HVF
2025-08-11 12:56 ` Philippe Mathieu-Daudé
2025-08-11 13:13 ` Philippe Mathieu-Daudé
@ 2025-08-11 13:35 ` Mohamed Mediouni
1 sibling, 0 replies; 29+ messages in thread
From: Mohamed Mediouni @ 2025-08-11 13:35 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: qemu-devel, Peter Maydell, Shannon Zhao, Michael S. Tsirkin,
Ani Sinha, qemu-arm, Paolo Bonzini, Roman Bolshakov,
Igor Mammedov, Phil Dennis-Jordan, Alexander Graf, Mads Ynddal,
Cameron Esfahani
> On 11. Aug 2025, at 14:56, Philippe Mathieu-Daudé <philmd@linaro.org> wrote:
>
> On 8/8/25 09:01, Mohamed Mediouni wrote:
>> Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
>> ---
>> hw/arm/virt.c | 9 ++++++---
>> target/arm/hvf-stub.c | 15 +++++++++++++++
>> target/arm/hvf/hvf.c | 41 +++++++++++++++++++++++++++++++++++++++--
>> target/arm/hvf_arm.h | 3 +++
>> 4 files changed, 63 insertions(+), 5 deletions(-)
>> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
>> index 59e6c2b9df..91d8cd9363 100644
>> --- a/hw/arm/virt.c
>> +++ b/hw/arm/virt.c
>> @@ -817,8 +817,7 @@ static void create_gic(VirtMachineState *vms, MemoryRegion *mem)
>> g_assert_not_reached();
>> }
>> - if (kvm_enabled() && vms->virt &&
>> - (revision != 3 || !kvm_irqchip_in_kernel())) {
>> + if (kvm_enabled() && vms->virt && (revision != 3 || !kvm_irqchip_in_kernel())) {
>> error_report("KVM EL2 is only supported with in-kernel GICv3");
>> exit(1);
>> }
>> @@ -2281,7 +2280,8 @@ static void machvirt_init(MachineState *machine)
>> exit(1);
>> }
>> - if (vms->virt && !kvm_enabled() && !tcg_enabled() && !qtest_enabled()) {
>> + if (vms->virt && !kvm_enabled() && !tcg_enabled()
>> + && !hvf_enabled() && !qtest_enabled()) {
>> error_report("mach-virt: %s does not support providing "
>> "Virtualization extensions to the guest CPU",
>> current_accel_name());
>> @@ -2553,6 +2553,9 @@ static void virt_set_virt(Object *obj, bool value, Error **errp)
>> VirtMachineState *vms = VIRT_MACHINE(obj);
>> vms->virt = value;
>> +#if defined(CONFIG_HVF) && defined(__aarch64__)
>> + hvf_arm_el2_enable(value);
>> +#endif
>> }
> We don't have hvf_arm_el1_enable(). I think we just want to always
> provide the most performant configuration to users.
> IOW, we don't need hvf_arm_el2_enable(). I'd just always enable EL2
> (if supported) in hvf_arm_get_host_cpu_features().
A different feature set is exposed when EL2 is enabled. For example, SME is not currently exposed in that mode.
(We currently disable SME in Qemu anyway but that’s something I plan to take care of)
^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH v6 06/13] hvf: save/restore Apple GIC state
2025-08-08 7:01 [PATCH v6 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
` (4 preceding siblings ...)
2025-08-08 7:01 ` [PATCH v6 05/13] hw/arm, target/arm: nested virtualisation on HVF Mohamed Mediouni
@ 2025-08-08 7:01 ` Mohamed Mediouni
2025-08-08 7:01 ` [PATCH v6 07/13] target/arm: hvf: pass through CNTHCTL_EL2 and MDCCINT_EL1 Mohamed Mediouni
` (6 subsequent siblings)
12 siblings, 0 replies; 29+ messages in thread
From: Mohamed Mediouni @ 2025-08-08 7:01 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Shannon Zhao, Michael S. Tsirkin, Ani Sinha,
qemu-arm, Paolo Bonzini, Roman Bolshakov, Igor Mammedov,
Phil Dennis-Jordan, Alexander Graf, Mads Ynddal, Cameron Esfahani,
Mohamed Mediouni
On HVF, some of the GIC state is in an opaque Apple-provided structure.
Save/restore that state to be able to save/restore VMs.
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
---
target/arm/hvf/hvf.c | 73 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 73 insertions(+)
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index 483a50329b..3cc6f1e159 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -22,6 +22,7 @@
#include "cpu-sysregs.h"
#include <mach/mach_time.h>
+#include <stdint.h>
#include "system/address-spaces.h"
#include "system/memory.h"
@@ -2175,15 +2176,83 @@ static const VMStateDescription vmstate_hvf_vtimer = {
},
};
+/* Apple specific opaque state for the vGIC */
+
+typedef struct HVGICState {
+ void *state;
+ uint32_t size;
+} HVGICState;
+
+static HVGICState gic;
+
+static int hvf_gic_opaque_state_get(void)
+{
+ hv_gic_state_t gic_state;
+ hv_return_t err;
+ size_t size;
+
+ gic_state = hv_gic_state_create();
+ if (gic_state == NULL) {
+ error_report("hvf: vgic: failed to create hv_gic_state_create.");
+ return 1;
+ }
+ err = hv_gic_state_get_size(gic_state, &size);
+ gic.size = size;
+ if (err != HV_SUCCESS) {
+ error_report("hvf: vgic: failed to get GIC state size.");
+ return 1;
+ }
+ gic.state = malloc(gic.size);
+ err = hv_gic_state_get_data(gic_state, gic.state);
+ if (err != HV_SUCCESS) {
+ error_report("hvf: vgic: failed to get GIC state.");
+ return 1;
+ }
+ return 0;
+}
+
+static int hvf_gic_opaque_state_set(void)
+{
+ hv_return_t err;
+ if (!gic.size) {
+ return 0;
+ }
+ err = hv_gic_set_state(gic.state, gic.size);
+ if (err != HV_SUCCESS) {
+ error_report("hvf: vgic: failed to restore GIC state.");
+ return 1;
+ }
+ return 0;
+}
+
+static const VMStateDescription vmstate_hvf_gic = {
+ .name = "hvf-gic",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .fields = (const VMStateField[]) {
+ VMSTATE_UINT32(size, HVGICState),
+ VMSTATE_VBUFFER_UINT32(state,
+ HVGICState, 0, 0,
+ size),
+ VMSTATE_END_OF_LIST()
+ },
+};
+
static void hvf_vm_state_change(void *opaque, bool running, RunState state)
{
HVFVTimer *s = opaque;
if (running) {
+ if (hvf_irqchip_in_kernel()) {
+ hvf_gic_opaque_state_set();
+ }
/* Update vtimer offset on all CPUs */
hvf_state->vtimer_offset = mach_absolute_time() - s->vtimer_val;
cpu_synchronize_all_states();
} else {
+ if (hvf_irqchip_in_kernel()) {
+ hvf_gic_opaque_state_get();
+ }
/* Remember vtimer value on every pause */
s->vtimer_val = hvf_vtimer_val_raw();
}
@@ -2193,6 +2262,10 @@ int hvf_arch_init(void)
{
hvf_state->vtimer_offset = mach_absolute_time();
vmstate_register(NULL, 0, &vmstate_hvf_vtimer, &vtimer);
+ if (hvf_irqchip_in_kernel()) {
+ gic.size = 0;
+ vmstate_register(NULL, 0, &vmstate_hvf_gic, &gic);
+ }
qemu_add_vm_change_state_handler(hvf_vm_state_change, &vtimer);
hvf_arm_init_debug();
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH v6 07/13] target/arm: hvf: pass through CNTHCTL_EL2 and MDCCINT_EL1
2025-08-08 7:01 [PATCH v6 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
` (5 preceding siblings ...)
2025-08-08 7:01 ` [PATCH v6 06/13] hvf: save/restore Apple GIC state Mohamed Mediouni
@ 2025-08-08 7:01 ` Mohamed Mediouni
2025-08-08 7:01 ` [PATCH v6 08/13] hw/arm: virt: cleanly fail on attempt to use the platform vGIC together with ITS Mohamed Mediouni
` (5 subsequent siblings)
12 siblings, 0 replies; 29+ messages in thread
From: Mohamed Mediouni @ 2025-08-08 7:01 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Shannon Zhao, Michael S. Tsirkin, Ani Sinha,
qemu-arm, Paolo Bonzini, Roman Bolshakov, Igor Mammedov,
Phil Dennis-Jordan, Alexander Graf, Mads Ynddal, Cameron Esfahani,
Mohamed Mediouni, Philippe Mathieu-Daudé
HVF traps accesses to CNTHCTL_EL2. For nested guests, HVF traps accesses to MDCCINT_EL1.
Pass through those accesses to the Hypervisor.framework library.
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
target/arm/hvf/hvf.c | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index 3cc6f1e159..788d0d41f4 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -297,6 +297,10 @@ void hvf_arm_init_debug(void)
#define SYSREG_DBGWVR15_EL1 SYSREG(2, 0, 0, 15, 6)
#define SYSREG_DBGWCR15_EL1 SYSREG(2, 0, 0, 15, 7)
+/* EL2 registers */
+#define SYSREG_CNTHCTL_EL2 SYSREG(3, 4, 14, 1, 0)
+#define SYSREG_MDCCINT_EL1 SYSREG(2, 0, 0, 2, 0)
+
#define WFX_IS_WFE (1 << 0)
#define TMR_CTL_ENABLE (1 << 0)
@@ -1373,6 +1377,12 @@ static int hvf_sysreg_read(CPUState *cpu, uint32_t reg, uint64_t *val)
case SYSREG_OSDLR_EL1:
/* Dummy register */
return 0;
+ case SYSREG_CNTHCTL_EL2:
+ assert_hvf_ok(hv_vcpu_get_sys_reg(cpu->accel->fd, HV_SYS_REG_CNTHCTL_EL2, val));
+ return 0;
+ case SYSREG_MDCCINT_EL1:
+ assert_hvf_ok(hv_vcpu_get_sys_reg(cpu->accel->fd, HV_SYS_REG_MDCCINT_EL1, val));
+ return 0;
case SYSREG_ICC_AP0R0_EL1:
case SYSREG_ICC_AP0R1_EL1:
case SYSREG_ICC_AP0R2_EL1:
@@ -1690,6 +1700,12 @@ static int hvf_sysreg_write(CPUState *cpu, uint32_t reg, uint64_t val)
case SYSREG_OSDLR_EL1:
/* Dummy register */
return 0;
+ case SYSREG_CNTHCTL_EL2:
+ assert_hvf_ok(hv_vcpu_set_sys_reg(cpu->accel->fd, HV_SYS_REG_CNTHCTL_EL2, val));
+ return 0;
+ case SYSREG_MDCCINT_EL1:
+ assert_hvf_ok(hv_vcpu_set_sys_reg(cpu->accel->fd, HV_SYS_REG_MDCCINT_EL1, val));
+ return 0;
case SYSREG_LORC_EL1:
/* Dummy register */
return 0;
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH v6 08/13] hw/arm: virt: cleanly fail on attempt to use the platform vGIC together with ITS
2025-08-08 7:01 [PATCH v6 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
` (6 preceding siblings ...)
2025-08-08 7:01 ` [PATCH v6 07/13] target/arm: hvf: pass through CNTHCTL_EL2 and MDCCINT_EL1 Mohamed Mediouni
@ 2025-08-08 7:01 ` Mohamed Mediouni
2025-08-08 7:01 ` [PATCH v6 09/13] hvf: only call hvf_sync_vtimer() when running without the platform vGIC Mohamed Mediouni
` (4 subsequent siblings)
12 siblings, 0 replies; 29+ messages in thread
From: Mohamed Mediouni @ 2025-08-08 7:01 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Shannon Zhao, Michael S. Tsirkin, Ani Sinha,
qemu-arm, Paolo Bonzini, Roman Bolshakov, Igor Mammedov,
Phil Dennis-Jordan, Alexander Graf, Mads Ynddal, Cameron Esfahani,
Mohamed Mediouni
Apple's platform vGIC doesn't support ITS. Deal with this by reporting to the user and not creating the ITS device.
Regular configuration: GICv3 + ITS
Resulting configuration here: GICv3 with no MSIs
And its=off explicitly for the newest machine version: GICv3 + GICv2m
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
---
hw/arm/virt.c | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 91d8cd9363..005e923a22 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -740,6 +740,16 @@ static void create_its(VirtMachineState *vms)
return;
}
+ if (hvf_enabled() && hvf_irqchip_in_kernel() && vms->tcg_its) {
+ /*
+ * In the HVF case, inform the user that they can use the
+ * user-mode GIC if they want to have an ITS.
+ */
+ info_report("ITS not supported without kernel-irqchip=off on HVF");
+ info_report("Disabling ITS");
+ return;
+ }
+
dev = qdev_new(its_class_name());
object_property_set_link(OBJECT(dev), "parent-gicv3", OBJECT(vms->gic),
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH v6 09/13] hvf: only call hvf_sync_vtimer() when running without the platform vGIC
2025-08-08 7:01 [PATCH v6 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
` (7 preceding siblings ...)
2025-08-08 7:01 ` [PATCH v6 08/13] hw/arm: virt: cleanly fail on attempt to use the platform vGIC together with ITS Mohamed Mediouni
@ 2025-08-08 7:01 ` Mohamed Mediouni
2025-08-19 12:37 ` Mads Ynddal
2025-08-08 7:01 ` [PATCH v6 10/13] hvf: sync registers used at EL2 Mohamed Mediouni
` (3 subsequent siblings)
12 siblings, 1 reply; 29+ messages in thread
From: Mohamed Mediouni @ 2025-08-08 7:01 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Shannon Zhao, Michael S. Tsirkin, Ani Sinha,
qemu-arm, Paolo Bonzini, Roman Bolshakov, Igor Mammedov,
Phil Dennis-Jordan, Alexander Graf, Mads Ynddal, Cameron Esfahani,
Mohamed Mediouni
When running with the Apple vGIC, the EL1 vtimer is handled by the platform.
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
---
target/arm/hvf/hvf.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index 788d0d41f4..d8741f942d 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -2008,7 +2008,8 @@ int hvf_vcpu_exec(CPUState *cpu)
g_assert_not_reached();
}
- hvf_sync_vtimer(cpu);
+ if (!hvf_irqchip_in_kernel())
+ hvf_sync_vtimer(cpu);
switch (ec) {
case EC_SOFTWARESTEP: {
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 29+ messages in thread* Re: [PATCH v6 09/13] hvf: only call hvf_sync_vtimer() when running without the platform vGIC
2025-08-08 7:01 ` [PATCH v6 09/13] hvf: only call hvf_sync_vtimer() when running without the platform vGIC Mohamed Mediouni
@ 2025-08-19 12:37 ` Mads Ynddal
0 siblings, 0 replies; 29+ messages in thread
From: Mads Ynddal @ 2025-08-19 12:37 UTC (permalink / raw)
To: Mohamed Mediouni
Cc: qemu-devel, Peter Maydell, Shannon Zhao, Michael S. Tsirkin,
Ani Sinha, qemu-arm, Paolo Bonzini, Roman Bolshakov,
Igor Mammedov, Phil Dennis-Jordan, Alexander Graf,
Cameron Esfahani
> On 8 Aug 2025, at 09.01, Mohamed Mediouni <mohamed@unpredictable.fr> wrote:
>
> When running with the Apple vGIC, the EL1 vtimer is handled by the platform.
>
> Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
> ---
> target/arm/hvf/hvf.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
Reviewed-by: Mads Ynddal <mads@ynddal.dk>
^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH v6 10/13] hvf: sync registers used at EL2
2025-08-08 7:01 [PATCH v6 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
` (8 preceding siblings ...)
2025-08-08 7:01 ` [PATCH v6 09/13] hvf: only call hvf_sync_vtimer() when running without the platform vGIC Mohamed Mediouni
@ 2025-08-08 7:01 ` Mohamed Mediouni
2025-08-08 7:01 ` [PATCH v6 11/13] hvf: gate ARM_FEATURE_PMU register emulation behind not being " Mohamed Mediouni
` (2 subsequent siblings)
12 siblings, 0 replies; 29+ messages in thread
From: Mohamed Mediouni @ 2025-08-08 7:01 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Shannon Zhao, Michael S. Tsirkin, Ani Sinha,
qemu-arm, Paolo Bonzini, Roman Bolshakov, Igor Mammedov,
Phil Dennis-Jordan, Alexander Graf, Mads Ynddal, Cameron Esfahani,
Mohamed Mediouni
When starting up the VM at EL2, more sysregs are available. Sync the state of those.
In addition, sync the state of the EL1 physical timer when the vGIC is used, even
if running at EL1. However, no OS running at EL1 is expected to use those registers.
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
---
target/arm/hvf/hvf.c | 53 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 53 insertions(+)
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index d8741f942d..41b4321b0b 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -406,6 +406,8 @@ static const struct hvf_reg_match hvf_fpreg_match[] = {
struct hvf_sreg_match {
int reg;
uint32_t key;
+ bool vgic;
+ bool el2;
uint32_t cp_idx;
};
@@ -551,6 +553,41 @@ static struct hvf_sreg_match hvf_sreg_match[] = {
{ HV_SYS_REG_CNTV_CTL_EL0, HVF_SYSREG(14, 3, 3, 3, 1) },
{ HV_SYS_REG_CNTV_CVAL_EL0, HVF_SYSREG(14, 3, 3, 3, 2) },
{ HV_SYS_REG_SP_EL1, HVF_SYSREG(4, 1, 3, 4, 0) },
+ /* vGIC */
+ { HV_SYS_REG_CNTP_CTL_EL0, HVF_SYSREG(14, 2, 3, 3, 1), true },
+ { HV_SYS_REG_CNTP_CVAL_EL0, HVF_SYSREG(14, 2, 3, 3, 2), true },
+#ifdef SYNC_NO_RAW_REGS
+ { HV_SYS_REG_CNTP_TVAL_EL0, HVF_SYSREG(14, 2, 3, 3, 0), true},
+#endif
+ /* vGIC + EL2 */
+ { HV_SYS_REG_CNTHCTL_EL2, HVF_SYSREG(14, 1, 3, 4, 0), true, true },
+ { HV_SYS_REG_CNTHP_CVAL_EL2, HVF_SYSREG(14, 2, 3, 4, 2), true, true },
+ { HV_SYS_REG_CNTHP_CTL_EL2, HVF_SYSREG(14, 2, 3, 4, 1), true, true },
+#ifdef SYNC_NO_RAW_REGS
+ { HV_SYS_REG_CNTHP_TVAL_EL2, HVF_SYSREG(14, 2, 3, 4, 0), true, true },
+#endif
+ { HV_SYS_REG_CNTVOFF_EL2, HVF_SYSREG(14, 0, 3, 4, 3), true, true },
+ /* EL2 */
+ { HV_SYS_REG_CPTR_EL2, HVF_SYSREG(1, 1, 3, 4, 2), .el2 = true },
+ { HV_SYS_REG_ELR_EL2, HVF_SYSREG(4, 0, 3, 4, 1), .el2 = true },
+ { HV_SYS_REG_ESR_EL2, HVF_SYSREG(5, 2, 3, 4, 0), .el2 = true },
+ { HV_SYS_REG_FAR_EL2, HVF_SYSREG(6, 0, 3, 4, 0), .el2 = true },
+ { HV_SYS_REG_HCR_EL2, HVF_SYSREG(1, 1, 3, 4, 0), .el2 = true },
+ { HV_SYS_REG_HPFAR_EL2, HVF_SYSREG(6, 0, 3, 4, 4), .el2 = true },
+ { HV_SYS_REG_MAIR_EL2, HVF_SYSREG(10, 2, 3, 4, 0), .el2 = true },
+ { HV_SYS_REG_MDCR_EL2, HVF_SYSREG(1, 1, 3, 4, 1), .el2 = true },
+ { HV_SYS_REG_SCTLR_EL2, HVF_SYSREG(1, 0, 3, 4, 0), .el2 = true },
+ { HV_SYS_REG_SPSR_EL2, HVF_SYSREG(4, 0, 3, 4, 0), .el2 = true },
+ { HV_SYS_REG_SP_EL2, HVF_SYSREG(4, 1, 3, 6, 0), .el2 = true},
+ { HV_SYS_REG_TCR_EL2, HVF_SYSREG(2, 0, 3, 4, 2), .el2 = true },
+ { HV_SYS_REG_TPIDR_EL2, HVF_SYSREG(13, 0, 3, 4, 2), .el2 = true },
+ { HV_SYS_REG_TTBR0_EL2, HVF_SYSREG(2, 0, 3, 4, 0), .el2 = true },
+ { HV_SYS_REG_TTBR1_EL2, HVF_SYSREG(2, 0, 3, 4, 1), .el2 = true },
+ { HV_SYS_REG_VBAR_EL2, HVF_SYSREG(12, 0, 3, 4, 0), .el2 = true },
+ { HV_SYS_REG_VMPIDR_EL2, HVF_SYSREG(0, 0, 3, 4, 5), .el2 = true },
+ { HV_SYS_REG_VPIDR_EL2, HVF_SYSREG(0, 0, 3, 4, 0), .el2 = true },
+ { HV_SYS_REG_VTCR_EL2, HVF_SYSREG(2, 1, 3, 4, 2), .el2 = true },
+ { HV_SYS_REG_VTTBR_EL2, HVF_SYSREG(2, 1, 3, 4, 0), .el2 = true },
};
int hvf_get_registers(CPUState *cpu)
@@ -594,6 +631,14 @@ int hvf_get_registers(CPUState *cpu)
continue;
}
+ if (hvf_sreg_match[i].vgic && !hvf_irqchip_in_kernel()) {
+ continue;
+ }
+
+ if (hvf_sreg_match[i].el2 && !hvf_arm_el2_enabled()) {
+ continue;
+ }
+
if (cpu->accel->guest_debug_enabled) {
/* Handle debug registers */
switch (hvf_sreg_match[i].reg) {
@@ -731,6 +776,14 @@ int hvf_put_registers(CPUState *cpu)
continue;
}
+ if (hvf_sreg_match[i].vgic && !hvf_irqchip_in_kernel()) {
+ continue;
+ }
+
+ if (hvf_sreg_match[i].el2 && !hvf_arm_el2_enabled()) {
+ continue;
+ }
+
if (cpu->accel->guest_debug_enabled) {
/* Handle debug registers */
switch (hvf_sreg_match[i].reg) {
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH v6 11/13] hvf: gate ARM_FEATURE_PMU register emulation behind not being at EL2
2025-08-08 7:01 [PATCH v6 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
` (9 preceding siblings ...)
2025-08-08 7:01 ` [PATCH v6 10/13] hvf: sync registers used at EL2 Mohamed Mediouni
@ 2025-08-08 7:01 ` Mohamed Mediouni
2025-08-08 7:01 ` [PATCH v6 12/13] target/arm: hvf: instantiate GIC early Mohamed Mediouni
2025-08-08 7:01 ` [PATCH v6 13/13] target/arm: hvf: add asserts for code paths not leveraged when using the vGIC Mohamed Mediouni
12 siblings, 0 replies; 29+ messages in thread
From: Mohamed Mediouni @ 2025-08-08 7:01 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Shannon Zhao, Michael S. Tsirkin, Ani Sinha,
qemu-arm, Paolo Bonzini, Roman Bolshakov, Igor Mammedov,
Phil Dennis-Jordan, Alexander Graf, Mads Ynddal, Cameron Esfahani,
Mohamed Mediouni, Philippe Mathieu-Daudé
From Apple documentation:
> When EL2 is disabled, PMU register accesses trigger "Trapped MSR, MRS, or
> System Instruction" exceptions. When this happens, hv_vcpu_run() returns, and the
> hv_vcpu_exit_t object contains the information about this exception.
> When EL2 is enabled, the handling of PMU register accesses is determined by the PMUVer
> field of ID_AA64DFR0_EL1 register.
> If the PMUVer field value is zero or is invalid, PMU register accesses generate "Undefined"
> exceptions, which are sent to the guest.
> If the PMUVer field value is non-zero and valid, PMU register accesses are emulated by the framework.
> The ID_AA64DFR0_EL1 register can be modified via hv_vcpu_set_sys_reg API.
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
target/arm/hvf/hvf.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index 41b4321b0b..6da636724b 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -1380,7 +1380,7 @@ static int hvf_sysreg_read(CPUState *cpu, uint32_t reg, uint64_t *val)
ARMCPU *arm_cpu = ARM_CPU(cpu);
CPUARMState *env = &arm_cpu->env;
- if (arm_feature(env, ARM_FEATURE_PMU)) {
+ if (!hvf_arm_el2_enabled() && arm_feature(env, ARM_FEATURE_PMU)) {
switch (reg) {
case SYSREG_PMCR_EL0:
*val = env->cp15.c9_pmcr;
@@ -1677,7 +1677,7 @@ static int hvf_sysreg_write(CPUState *cpu, uint32_t reg, uint64_t val)
SYSREG_OP2(reg),
val);
- if (arm_feature(env, ARM_FEATURE_PMU)) {
+ if (!hvf_arm_el2_enabled() && arm_feature(env, ARM_FEATURE_PMU)) {
switch (reg) {
case SYSREG_PMCCNTR_EL0:
pmu_op_start(env);
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 29+ messages in thread* [PATCH v6 12/13] target/arm: hvf: instantiate GIC early
2025-08-08 7:01 [PATCH v6 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
` (10 preceding siblings ...)
2025-08-08 7:01 ` [PATCH v6 11/13] hvf: gate ARM_FEATURE_PMU register emulation behind not being " Mohamed Mediouni
@ 2025-08-08 7:01 ` Mohamed Mediouni
2025-08-14 13:06 ` Mads Ynddal
2025-08-08 7:01 ` [PATCH v6 13/13] target/arm: hvf: add asserts for code paths not leveraged when using the vGIC Mohamed Mediouni
12 siblings, 1 reply; 29+ messages in thread
From: Mohamed Mediouni @ 2025-08-08 7:01 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Shannon Zhao, Michael S. Tsirkin, Ani Sinha,
qemu-arm, Paolo Bonzini, Roman Bolshakov, Igor Mammedov,
Phil Dennis-Jordan, Alexander Graf, Mads Ynddal, Cameron Esfahani,
Mohamed Mediouni
While figuring out a better spot for it, put it in hv_arch_vm_create().
After hv_vcpu_create is documented as too late, and deferring
vCPU initialization isn't enough either.
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
---
target/arm/hvf/hvf.c | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index 6da636724b..bb7b84ff35 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -1084,6 +1084,21 @@ hv_return_t hvf_arch_vm_create(MachineState *ms, uint32_t pa_range)
}
ret = hv_vm_create(config);
+ if (hvf_irqchip_in_kernel()) {
+ /*
+ * Instantiate GIC.
+ * This must be done prior to the creation of any vCPU
+ * but past hv_vm_create()
+ */
+ hv_gic_config_t cfg = hv_gic_config_create();
+ hv_gic_config_set_distributor_base(cfg, 0x08000000);
+ hv_gic_config_set_redistributor_base(cfg, 0x080A0000);
+ hv_return_t err = hv_gic_create(cfg);
+ if (err != HV_SUCCESS) {
+ error_report("error creating platform VGIC");
+ goto cleanup;
+ }
+ }
cleanup:
os_release(config);
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 29+ messages in thread* Re: [PATCH v6 12/13] target/arm: hvf: instantiate GIC early
2025-08-08 7:01 ` [PATCH v6 12/13] target/arm: hvf: instantiate GIC early Mohamed Mediouni
@ 2025-08-14 13:06 ` Mads Ynddal
0 siblings, 0 replies; 29+ messages in thread
From: Mads Ynddal @ 2025-08-14 13:06 UTC (permalink / raw)
To: Mohamed Mediouni
Cc: qemu-devel, Peter Maydell, Shannon Zhao, Michael S. Tsirkin,
Ani Sinha, qemu-arm, Paolo Bonzini, Roman Bolshakov,
Igor Mammedov, Phil Dennis-Jordan, Alexander Graf,
Cameron Esfahani
> On 8 Aug 2025, at 09.01, Mohamed Mediouni <mohamed@unpredictable.fr> wrote:
>
> While figuring out a better spot for it, put it in hv_arch_vm_create().
>
> After hv_vcpu_create is documented as too late, and deferring
> vCPU initialization isn't enough either.
>
> Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
> ---
> target/arm/hvf/hvf.c | 15 +++++++++++++++
> 1 file changed, 15 insertions(+)
>
> diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
> index 6da636724b..bb7b84ff35 100644
> --- a/target/arm/hvf/hvf.c
> +++ b/target/arm/hvf/hvf.c
> @@ -1084,6 +1084,21 @@ hv_return_t hvf_arch_vm_create(MachineState *ms, uint32_t pa_range)
> }
>
> ret = hv_vm_create(config);
> + if (hvf_irqchip_in_kernel()) {
> + /*
> + * Instantiate GIC.
> + * This must be done prior to the creation of any vCPU
> + * but past hv_vm_create()
> + */
> + hv_gic_config_t cfg = hv_gic_config_create();
> + hv_gic_config_set_distributor_base(cfg, 0x08000000);
> + hv_gic_config_set_redistributor_base(cfg, 0x080A0000);
> + hv_return_t err = hv_gic_create(cfg);
> + if (err != HV_SUCCESS) {
> + error_report("error creating platform VGIC");
> + goto cleanup;
> + }
> + }
>
> cleanup:
> os_release(config);
> --
> 2.39.5 (Apple Git-154)
>
It is difficult to find a place to initialize the GIC config, so I don't
know if it gets better than this.
Should the values 0x08000000 and 0x080A0000 be defined somewhere, or
found through a look up? I see hw/arm/virt.c has them in the
base_memmap.
You can do os_release(cfg) after hv_gic_create.
^ permalink raw reply [flat|nested] 29+ messages in thread
* [PATCH v6 13/13] target/arm: hvf: add asserts for code paths not leveraged when using the vGIC
2025-08-08 7:01 [PATCH v6 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
` (11 preceding siblings ...)
2025-08-08 7:01 ` [PATCH v6 12/13] target/arm: hvf: instantiate GIC early Mohamed Mediouni
@ 2025-08-08 7:01 ` Mohamed Mediouni
2025-08-19 12:22 ` Mads Ynddal
12 siblings, 1 reply; 29+ messages in thread
From: Mohamed Mediouni @ 2025-08-08 7:01 UTC (permalink / raw)
To: qemu-devel
Cc: Peter Maydell, Shannon Zhao, Michael S. Tsirkin, Ani Sinha,
qemu-arm, Paolo Bonzini, Roman Bolshakov, Igor Mammedov,
Phil Dennis-Jordan, Alexander Graf, Mads Ynddal, Cameron Esfahani,
Mohamed Mediouni
When using the vGIC, timers are directly handled by the platform.
No vmexits ought to happen in that case. Abort if reaching those code paths.
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
---
target/arm/hvf/hvf.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index bb7b84ff35..1d285c70d8 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -1477,6 +1477,7 @@ static int hvf_sysreg_read(CPUState *cpu, uint32_t reg, uint64_t *val)
case SYSREG_ICC_SGI1R_EL1:
case SYSREG_ICC_SRE_EL1:
case SYSREG_ICC_CTLR_EL1:
+ assert(!hvf_irqchip_in_kernel());
/* Call the TCG sysreg handler. This is only safe for GICv3 regs. */
if (hvf_sysreg_read_cp(cpu, reg, val)) {
return 0;
@@ -1803,6 +1804,7 @@ static int hvf_sysreg_write(CPUState *cpu, uint32_t reg, uint64_t val)
case SYSREG_ICC_SGI0R_EL1:
case SYSREG_ICC_SGI1R_EL1:
case SYSREG_ICC_SRE_EL1:
+ assert(!hvf_irqchip_in_kernel());
/* Call the TCG sysreg handler. This is only safe for GICv3 regs. */
if (hvf_sysreg_write_cp(cpu, reg, val)) {
return 0;
@@ -2066,6 +2068,7 @@ int hvf_vcpu_exec(CPUState *cpu)
/* This is the main one, handle below. */
break;
case HV_EXIT_REASON_VTIMER_ACTIVATED:
+ assert(!hvf_irqchip_in_kernel());
qemu_set_irq(arm_cpu->gt_timer_outputs[GTIMER_VIRT], 1);
cpu->accel->vtimer_masked = true;
return 0;
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 29+ messages in thread* Re: [PATCH v6 13/13] target/arm: hvf: add asserts for code paths not leveraged when using the vGIC
2025-08-08 7:01 ` [PATCH v6 13/13] target/arm: hvf: add asserts for code paths not leveraged when using the vGIC Mohamed Mediouni
@ 2025-08-19 12:22 ` Mads Ynddal
0 siblings, 0 replies; 29+ messages in thread
From: Mads Ynddal @ 2025-08-19 12:22 UTC (permalink / raw)
To: Mohamed Mediouni
Cc: qemu-devel, Peter Maydell, Shannon Zhao, Michael S. Tsirkin,
Ani Sinha, qemu-arm, Paolo Bonzini, Roman Bolshakov,
Igor Mammedov, Phil Dennis-Jordan, Alexander Graf,
Cameron Esfahani
> On 8 Aug 2025, at 09.01, Mohamed Mediouni <mohamed@unpredictable.fr> wrote:
>
> When using the vGIC, timers are directly handled by the platform.
> No vmexits ought to happen in that case. Abort if reaching those code paths.
>
> Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
> ---
> target/arm/hvf/hvf.c | 3 +++
> 1 file changed, 3 insertions(+)
Makes good sense to verify this.
Reviewed-by: Mads Ynddal <mads@ynddal.dk>
^ permalink raw reply [flat|nested] 29+ messages in thread