* [PATCH v5 00/13] HVF: Add support for platform vGIC and nested virtualisation
@ 2025-07-28 13:41 Mohamed Mediouni
2025-07-28 13:41 ` [PATCH v5 01/13] hvf: switch hvf_arm_get_host_cpu_features to not create a vCPU Mohamed Mediouni
` (12 more replies)
0 siblings, 13 replies; 24+ messages in thread
From: Mohamed Mediouni @ 2025-07-28 13:41 UTC (permalink / raw)
To: qemu-devel
Cc: Mads Ynddal, Paolo Bonzini, Shannon Zhao, Phil Dennis-Jordan,
Igor Mammedov, qemu-arm, Alexander Graf, Roman Bolshakov,
Peter Maydell, Michael S. Tsirkin, Ani Sinha, Cameron Esfahani,
Mohamed Mediouni
This series adds supports for nested virtualisation when using HVF on arm64 Macs.
It has three parts:
- Apple vGICv3 support and necessary infrastructure changes for it
- support for MSI interrupts in GICv3 + GICv2m configurations, which is independent
from Apple platforms and can be merged independently.
- Nested virtualisation support. Note that the nested virtualisation implementation
shipping as of macOS 26.0 is nVHE only, and does not leverage VNCR (FEAT_NV1-style).
Known issues:
- when nested virt is enabled, no UI response within EDK2
and a permanent wait. Workaround: -boot menu=on,splash-time=0. Interrupts do
work later on in Linux.
- This series doesn't contain EL2 physical timer emulation, which is
needed if not leveraging the Apple vGIC.
However, this is usable and there seems to be some interest for it. Hopefully
it'll be at least somewhat useful.
PS: I can step up to maintain HVF support if needed. WHPX arm64 patch series
is coming soon too.
v1->v2:
Oops. I did a mistake when preparing my patches.
- Add hvf_arm_el2_enable(_) call to virt_set_virt
- Fix nested virt support check to add HVF
v2->v3:
- LORC_EL1 patch was merged separately, remove from this series.
- fix LPIs when kernel-irqchip disabled and using TCG
- remove spurious if case in vGIC supported version detection (inapplicable now)
- Add hvf_enabled() check in combination with hvf kernel-irqchip checks
- cleanly fail on attempt to use the platform vGIC together with ITS
v3->v4:
- GIC state save improvements, including saving the opaque Apple-specific state
- Saving HVF system register state when using the vGIC and/or EL2
v5:
- oops, fixed up save/restore to be functional
- misc changes otherwise
Mohamed Mediouni (13):
hvf: switch hvf_arm_get_host_cpu_features to not create a vCPU
accel, hw/arm, include/system/hvf: infrastructure changes for HVF vGIC
hw/intc: Add hvf vGIC interrupt controller support
hw/arm, target/arm: nested virtualisation on HVF
hvf: save/restore Apple GIC state
target/arm: hvf: pass through CNTHCTL_EL2 and MDCCINT_EL1
hw/arm: virt: add GICv2m for the case when ITS is not available
hw/arm: virt: cleanly fail on attempt to use the platform vGIC
together with ITS
hvf: only call hvf_sync_vtimer() when running without the platform
vGIC
hvf: sync registers used at EL2
hvf: gate ARM_FEATURE_PMU register emulation behind not being at EL2
target/arm: hvf: instantiate GIC early
target/arm: hvf: add asserts for code paths not leveraged when using
the vGIC
accel/hvf/hvf-all.c | 44 +++
accel/stubs/hvf-stub.c | 1 +
hw/arm/virt-acpi-build.c | 4 +-
hw/arm/virt.c | 43 ++-
hw/intc/arm_gicv3_common.c | 3 +
hw/intc/arm_gicv3_hvf.c | 722 +++++++++++++++++++++++++++++++++++++
hw/intc/meson.build | 1 +
include/hw/arm/virt.h | 2 +
include/system/hvf.h | 3 +
system/vl.c | 2 +
target/arm/hvf-stub.c | 15 +
target/arm/hvf/hvf.c | 239 ++++++++++--
target/arm/hvf_arm.h | 3 +
13 files changed, 1049 insertions(+), 33 deletions(-)
create mode 100644 hw/intc/arm_gicv3_hvf.c
--
2.39.5 (Apple Git-154)
^ permalink raw reply [flat|nested] 24+ messages in thread
* [PATCH v5 01/13] hvf: switch hvf_arm_get_host_cpu_features to not create a vCPU
2025-07-28 13:41 [PATCH v5 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
@ 2025-07-28 13:41 ` Mohamed Mediouni
2025-08-06 11:12 ` Philippe Mathieu-Daudé
2025-07-28 13:41 ` [PATCH v5 02/13] accel, hw/arm, include/system/hvf: infrastructure changes for HVF vGIC Mohamed Mediouni
` (11 subsequent siblings)
12 siblings, 1 reply; 24+ messages in thread
From: Mohamed Mediouni @ 2025-07-28 13:41 UTC (permalink / raw)
To: qemu-devel
Cc: Mads Ynddal, Paolo Bonzini, Shannon Zhao, Phil Dennis-Jordan,
Igor Mammedov, qemu-arm, Alexander Graf, Roman Bolshakov,
Peter Maydell, Michael S. Tsirkin, Ani Sinha, Cameron Esfahani,
Mohamed Mediouni
Creating a vCPU locks out APIs such as hv_gic_create().
As a result, switch to using the hv_vcpu_config_get_feature_reg interface.
Hardcode MIDR because Apple deliberately doesn't expose a divergent MIDR across systems.
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
---
target/arm/hvf/hvf.c | 35 ++++++++++++++---------------------
1 file changed, 14 insertions(+), 21 deletions(-)
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index 47b0cd3a35..5ee0df17e3 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -864,24 +864,24 @@ static bool hvf_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf)
{
ARMISARegisters host_isar = {};
const struct isar_regs {
- int reg;
+ hv_feature_reg_t reg;
uint64_t *val;
} regs[] = {
- { HV_SYS_REG_ID_AA64PFR0_EL1, &host_isar.idregs[ID_AA64PFR0_EL1_IDX] },
- { HV_SYS_REG_ID_AA64PFR1_EL1, &host_isar.idregs[ID_AA64PFR1_EL1_IDX] },
- { HV_SYS_REG_ID_AA64DFR0_EL1, &host_isar.idregs[ID_AA64DFR0_EL1_IDX] },
- { HV_SYS_REG_ID_AA64DFR1_EL1, &host_isar.idregs[ID_AA64DFR1_EL1_IDX] },
- { HV_SYS_REG_ID_AA64ISAR0_EL1, &host_isar.idregs[ID_AA64ISAR0_EL1_IDX] },
- { HV_SYS_REG_ID_AA64ISAR1_EL1, &host_isar.idregs[ID_AA64ISAR1_EL1_IDX] },
+ { HV_FEATURE_REG_ID_AA64PFR0_EL1, &host_isar.idregs[ID_AA64PFR0_EL1_IDX] },
+ { HV_FEATURE_REG_ID_AA64PFR1_EL1, &host_isar.idregs[ID_AA64PFR1_EL1_IDX] },
+ { HV_FEATURE_REG_ID_AA64DFR0_EL1, &host_isar.idregs[ID_AA64DFR0_EL1_IDX] },
+ { HV_FEATURE_REG_ID_AA64DFR1_EL1, &host_isar.idregs[ID_AA64DFR1_EL1_IDX] },
+ { HV_FEATURE_REG_ID_AA64ISAR0_EL1, &host_isar.idregs[ID_AA64ISAR0_EL1_IDX] },
+ { HV_FEATURE_REG_ID_AA64ISAR1_EL1, &host_isar.idregs[ID_AA64ISAR1_EL1_IDX] },
/* Add ID_AA64ISAR2_EL1 here when HVF supports it */
- { HV_SYS_REG_ID_AA64MMFR0_EL1, &host_isar.idregs[ID_AA64MMFR0_EL1_IDX] },
- { HV_SYS_REG_ID_AA64MMFR1_EL1, &host_isar.idregs[ID_AA64MMFR1_EL1_IDX] },
- { HV_SYS_REG_ID_AA64MMFR2_EL1, &host_isar.idregs[ID_AA64MMFR2_EL1_IDX] },
+ { HV_FEATURE_REG_ID_AA64MMFR0_EL1, &host_isar.idregs[ID_AA64MMFR0_EL1_IDX] },
+ { HV_FEATURE_REG_ID_AA64MMFR1_EL1, &host_isar.idregs[ID_AA64MMFR1_EL1_IDX] },
+ { HV_FEATURE_REG_ID_AA64MMFR2_EL1, &host_isar.idregs[ID_AA64MMFR2_EL1_IDX] },
/* Add ID_AA64MMFR3_EL1 here when HVF supports it */
};
- hv_vcpu_t fd;
+
hv_return_t r = HV_SUCCESS;
- hv_vcpu_exit_t *exit;
+ hv_vcpu_config_t hv_vcpu_config = hv_vcpu_config_create();
int i;
ahcf->dtb_compatible = "arm,armv8";
@@ -891,17 +891,10 @@ static bool hvf_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf)
(1ULL << ARM_FEATURE_PMU) |
(1ULL << ARM_FEATURE_GENERIC_TIMER);
- /* We set up a small vcpu to extract host registers */
-
- if (hv_vcpu_create(&fd, &exit, NULL) != HV_SUCCESS) {
- return false;
- }
-
for (i = 0; i < ARRAY_SIZE(regs); i++) {
- r |= hv_vcpu_get_sys_reg(fd, regs[i].reg, regs[i].val);
+ r |= hv_vcpu_config_get_feature_reg(hv_vcpu_config, regs[i].reg, regs[i].val);
}
- r |= hv_vcpu_get_sys_reg(fd, HV_SYS_REG_MIDR_EL1, &ahcf->midr);
- r |= hv_vcpu_destroy(fd);
+ ahcf->midr = 0x610f0000;
clamp_id_aa64mmfr0_parange_to_ipa_size(&host_isar);
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v5 02/13] accel, hw/arm, include/system/hvf: infrastructure changes for HVF vGIC
2025-07-28 13:41 [PATCH v5 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
2025-07-28 13:41 ` [PATCH v5 01/13] hvf: switch hvf_arm_get_host_cpu_features to not create a vCPU Mohamed Mediouni
@ 2025-07-28 13:41 ` Mohamed Mediouni
2025-08-06 11:32 ` Philippe Mathieu-Daudé
2025-07-28 13:41 ` [PATCH v5 03/13] hw/intc: Add hvf vGIC interrupt controller support Mohamed Mediouni
` (10 subsequent siblings)
12 siblings, 1 reply; 24+ messages in thread
From: Mohamed Mediouni @ 2025-07-28 13:41 UTC (permalink / raw)
To: qemu-devel
Cc: Mads Ynddal, Paolo Bonzini, Shannon Zhao, Phil Dennis-Jordan,
Igor Mammedov, qemu-arm, Alexander Graf, Roman Bolshakov,
Peter Maydell, Michael S. Tsirkin, Ani Sinha, Cameron Esfahani,
Mohamed Mediouni
Misc changes needed for HVF vGIC enablement.
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
---
accel/hvf/hvf-all.c | 44 ++++++++++++++++++++++++++++++++++++++
accel/stubs/hvf-stub.c | 1 +
hw/arm/virt.c | 16 +++++++++-----
hw/intc/arm_gicv3_common.c | 3 +++
include/system/hvf.h | 3 +++
system/vl.c | 2 ++
6 files changed, 64 insertions(+), 5 deletions(-)
diff --git a/accel/hvf/hvf-all.c b/accel/hvf/hvf-all.c
index 0a4b498e83..5af76ba7a6 100644
--- a/accel/hvf/hvf-all.c
+++ b/accel/hvf/hvf-all.c
@@ -10,6 +10,8 @@
#include "qemu/osdep.h"
#include "qemu/error-report.h"
+#include "qapi/error.h"
+#include "qapi/qapi-visit-common.h"
#include "accel/accel-ops.h"
#include "system/address-spaces.h"
#include "system/memory.h"
@@ -20,6 +22,7 @@
#include "trace.h"
bool hvf_allowed;
+bool hvf_kernel_irqchip;
struct mac_slot {
int present;
@@ -290,6 +293,41 @@ static int hvf_gdbstub_sstep_flags(AccelState *as)
return SSTEP_ENABLE | SSTEP_NOIRQ;
}
+static void hvf_set_kernel_irqchip(Object *obj, Visitor *v,
+ const char *name, void *opaque,
+ Error **errp)
+{
+#ifdef __aarch64__
+ OnOffSplit mode;
+ if (!visit_type_OnOffSplit(v, name, &mode, errp)) {
+ return;
+ }
+
+ switch (mode) {
+ case ON_OFF_SPLIT_ON:
+ hvf_kernel_irqchip = true;
+ break;
+
+ case ON_OFF_SPLIT_OFF:
+ hvf_kernel_irqchip = false;
+ break;
+
+ case ON_OFF_SPLIT_SPLIT:
+ error_setg(errp, "HVF: split irqchip is not supported on Arm.");
+ break;
+
+ default:
+ /*
+ * The value was checked in visit_type_OnOffSplit() above. If
+ * we get here, then something is wrong in QEMU.
+ */
+ abort();
+ }
+#else
+ error_setg(errp, "HVF: setting irqchip configuration not supported on x86_64.");
+#endif
+}
+
static void hvf_accel_class_init(ObjectClass *oc, const void *data)
{
AccelClass *ac = ACCEL_CLASS(oc);
@@ -297,6 +335,12 @@ static void hvf_accel_class_init(ObjectClass *oc, const void *data)
ac->init_machine = hvf_accel_init;
ac->allowed = &hvf_allowed;
ac->gdbstub_supported_sstep_flags = hvf_gdbstub_sstep_flags;
+ hvf_kernel_irqchip = true;
+ object_class_property_add(oc, "kernel-irqchip", "on|off|split",
+ NULL, hvf_set_kernel_irqchip,
+ NULL, NULL);
+ object_class_property_set_description(oc, "kernel-irqchip",
+ "Configure HVF irqchip");
}
static const TypeInfo hvf_accel_type = {
diff --git a/accel/stubs/hvf-stub.c b/accel/stubs/hvf-stub.c
index 42eadc5ca9..6bd08759ba 100644
--- a/accel/stubs/hvf-stub.c
+++ b/accel/stubs/hvf-stub.c
@@ -10,3 +10,4 @@
#include "system/hvf.h"
bool hvf_allowed;
+bool hvf_kernel_irqchip;
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index ef6be3660f..7da1176cda 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -830,7 +830,7 @@ static void create_gic(VirtMachineState *vms, MemoryRegion *mem)
* interrupts; there are always 32 of the former (mandated by GIC spec).
*/
qdev_prop_set_uint32(vms->gic, "num-irq", NUM_IRQS + 32);
- if (!kvm_irqchip_in_kernel()) {
+ if (!kvm_irqchip_in_kernel() && !hvf_irqchip_in_kernel()) {
qdev_prop_set_bit(vms->gic, "has-security-extensions", vms->secure);
}
@@ -853,8 +853,9 @@ static void create_gic(VirtMachineState *vms, MemoryRegion *mem)
qdev_prop_set_array(vms->gic, "redist-region-count",
redist_region_count);
- if (!kvm_irqchip_in_kernel()) {
- if (vms->tcg_its) {
+ if (!kvm_irqchip_in_kernel() &&
+ !(hvf_enabled() && hvf_irqchip_in_kernel())) {
+ if (vms->its && vms->tcg_its) {
object_property_set_link(OBJECT(vms->gic), "sysmem",
OBJECT(mem), &error_fatal);
qdev_prop_set_bit(vms->gic, "has-lpi", true);
@@ -864,7 +865,7 @@ static void create_gic(VirtMachineState *vms, MemoryRegion *mem)
ARCH_GIC_MAINT_IRQ);
}
} else {
- if (!kvm_irqchip_in_kernel()) {
+ if (!kvm_irqchip_in_kernel() && !hvf_irqchip_in_kernel()) {
qdev_prop_set_bit(vms->gic, "has-virtualization-extensions",
vms->virt);
}
@@ -2058,7 +2059,12 @@ static void finalize_gic_version(VirtMachineState *vms)
/* KVM w/o kernel irqchip can only deal with GICv2 */
gics_supported |= VIRT_GIC_VERSION_2_MASK;
accel_name = "KVM with kernel-irqchip=off";
- } else if (tcg_enabled() || hvf_enabled() || qtest_enabled()) {
+ } else if (hvf_enabled()) {
+ if (!hvf_irqchip_in_kernel()) {
+ gics_supported |= VIRT_GIC_VERSION_2_MASK;
+ }
+ gics_supported |= VIRT_GIC_VERSION_3_MASK;
+ } else if (tcg_enabled() || qtest_enabled()) {
gics_supported |= VIRT_GIC_VERSION_2_MASK;
if (module_object_class_by_name("arm-gicv3")) {
gics_supported |= VIRT_GIC_VERSION_3_MASK;
diff --git a/hw/intc/arm_gicv3_common.c b/hw/intc/arm_gicv3_common.c
index e438d8c042..b8eee27260 100644
--- a/hw/intc/arm_gicv3_common.c
+++ b/hw/intc/arm_gicv3_common.c
@@ -32,6 +32,7 @@
#include "gicv3_internal.h"
#include "hw/arm/linux-boot-if.h"
#include "system/kvm.h"
+#include "system/hvf.h"
static void gicv3_gicd_no_migration_shift_bug_post_load(GICv3State *cs)
@@ -662,6 +663,8 @@ const char *gicv3_class_name(void)
{
if (kvm_irqchip_in_kernel()) {
return "kvm-arm-gicv3";
+ } else if (hvf_enabled() && hvf_irqchip_in_kernel()) {
+ return "hvf-arm-gicv3";
} else {
if (kvm_enabled()) {
error_report("Userspace GICv3 is not supported with KVM");
diff --git a/include/system/hvf.h b/include/system/hvf.h
index d3dcf088b3..dc8da85979 100644
--- a/include/system/hvf.h
+++ b/include/system/hvf.h
@@ -26,8 +26,11 @@
#ifdef CONFIG_HVF_IS_POSSIBLE
extern bool hvf_allowed;
#define hvf_enabled() (hvf_allowed)
+extern bool hvf_kernel_irqchip;
+#define hvf_irqchip_in_kernel() (hvf_kernel_irqchip)
#else /* !CONFIG_HVF_IS_POSSIBLE */
#define hvf_enabled() 0
+#define hvf_irqchip_in_kernel() 0
#endif /* !CONFIG_HVF_IS_POSSIBLE */
#define TYPE_HVF_ACCEL ACCEL_CLASS_NAME("hvf")
diff --git a/system/vl.c b/system/vl.c
index 3b7057e6c6..1c072d15a4 100644
--- a/system/vl.c
+++ b/system/vl.c
@@ -1773,6 +1773,8 @@ static void qemu_apply_legacy_machine_options(QDict *qdict)
false);
object_register_sugar_prop(ACCEL_CLASS_NAME("whpx"), "kernel-irqchip", value,
false);
+ object_register_sugar_prop(ACCEL_CLASS_NAME("hvf"), "kernel-irqchip", value,
+ false);
qdict_del(qdict, "kernel-irqchip");
}
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v5 03/13] hw/intc: Add hvf vGIC interrupt controller support
2025-07-28 13:41 [PATCH v5 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
2025-07-28 13:41 ` [PATCH v5 01/13] hvf: switch hvf_arm_get_host_cpu_features to not create a vCPU Mohamed Mediouni
2025-07-28 13:41 ` [PATCH v5 02/13] accel, hw/arm, include/system/hvf: infrastructure changes for HVF vGIC Mohamed Mediouni
@ 2025-07-28 13:41 ` Mohamed Mediouni
2025-08-06 10:32 ` Philippe Mathieu-Daudé
2025-07-28 13:41 ` [PATCH v5 04/13] hw/arm, target/arm: nested virtualisation on HVF Mohamed Mediouni
` (9 subsequent siblings)
12 siblings, 1 reply; 24+ messages in thread
From: Mohamed Mediouni @ 2025-07-28 13:41 UTC (permalink / raw)
To: qemu-devel
Cc: Mads Ynddal, Paolo Bonzini, Shannon Zhao, Phil Dennis-Jordan,
Igor Mammedov, qemu-arm, Alexander Graf, Roman Bolshakov,
Peter Maydell, Michael S. Tsirkin, Ani Sinha, Cameron Esfahani,
Mohamed Mediouni
This opens up the door to nested virtualisation support.
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
---
hw/intc/arm_gicv3_hvf.c | 722 ++++++++++++++++++++++++++++++++++++++++
hw/intc/meson.build | 1 +
2 files changed, 723 insertions(+)
create mode 100644 hw/intc/arm_gicv3_hvf.c
diff --git a/hw/intc/arm_gicv3_hvf.c b/hw/intc/arm_gicv3_hvf.c
new file mode 100644
index 0000000000..1e9662230d
--- /dev/null
+++ b/hw/intc/arm_gicv3_hvf.c
@@ -0,0 +1,722 @@
+/* SPDX-License-Identifier: GPL-2.0-or-later */
+/*
+ * ARM Generic Interrupt Controller using HVF platform support
+ *
+ * Copyright (c) 2025 Mohamed Mediouni
+ * Based on vGICv3 KVM code by Pavel Fedin
+ *
+ */
+
+#include "qemu/osdep.h"
+#include "qapi/error.h"
+#include "hw/intc/arm_gicv3_common.h"
+#include "qemu/error-report.h"
+#include "qemu/module.h"
+#include "system/runstate.h"
+#include "system/hvf.h"
+#include "system/hvf_int.h"
+#include "hvf_arm.h"
+#include "gicv3_internal.h"
+#include "vgic_common.h"
+#include "qom/object.h"
+#include "target/arm/cpregs.h"
+#include <Hypervisor/Hypervisor.h>
+
+struct HVFARMGICv3Class {
+ ARMGICv3CommonClass parent_class;
+ DeviceRealize parent_realize;
+ ResettablePhases parent_phases;
+};
+
+#define TYPE_HVF_GICV3 "hvf-arm-gicv3"
+typedef struct HVFARMGICv3Class HVFARMGICv3Class;
+
+/* This is reusing the GICv3State typedef from ARM_GICV3_ITS_COMMON */
+DECLARE_OBJ_CHECKERS(GICv3State, HVFARMGICv3Class,
+ HVF_GICV3, TYPE_HVF_GICV3);
+
+/*
+ * Loop through each distributor IRQ related register; since bits
+ * corresponding to SPIs and PPIs are RAZ/WI when affinity routing
+ * is enabled, we skip those.
+ */
+#define for_each_dist_irq_reg(_irq, _max, _field_width) \
+ for (_irq = GIC_INTERNAL; _irq < _max; _irq += (32 / _field_width))
+
+static void hvf_dist_get_priority(GICv3State *s, hv_gic_distributor_reg_t offset
+ , uint8_t *bmp)
+{
+ uint64_t reg;
+ uint32_t *field;
+ int irq;
+ field = (uint32_t *)(bmp);
+
+ for_each_dist_irq_reg(irq, s->num_irq, 8) {
+ hv_gic_get_distributor_reg(offset, ®);
+ *field = reg;
+ offset += 4;
+ field++;
+ }
+}
+
+static void hvf_dist_put_priority(GICv3State *s, hv_gic_distributor_reg_t offset
+ , uint8_t *bmp)
+{
+ uint32_t reg, *field;
+ int irq;
+ field = (uint32_t *)(bmp);
+
+ for_each_dist_irq_reg(irq, s->num_irq, 8) {
+ reg = *field;
+ hv_gic_set_distributor_reg(offset, reg);
+ offset += 4;
+ field++;
+ }
+}
+
+static void hvf_dist_get_edge_trigger(GICv3State *s, hv_gic_distributor_reg_t offset,
+ uint32_t *bmp)
+{
+ uint64_t reg;
+ int irq;
+
+ for_each_dist_irq_reg(irq, s->num_irq, 2) {
+ hv_gic_get_distributor_reg(offset, ®);
+ reg = half_unshuffle32(reg >> 1);
+ if (irq % 32 != 0) {
+ reg = (reg << 16);
+ }
+ *gic_bmp_ptr32(bmp, irq) |= reg;
+ offset += 4;
+ }
+}
+
+static void hvf_dist_put_edge_trigger(GICv3State *s, hv_gic_distributor_reg_t offset,
+ uint32_t *bmp)
+{
+ uint32_t reg;
+ int irq;
+
+ for_each_dist_irq_reg(irq, s->num_irq, 2) {
+ reg = *gic_bmp_ptr32(bmp, irq);
+ if (irq % 32 != 0) {
+ reg = (reg & 0xffff0000) >> 16;
+ } else {
+ reg = reg & 0xffff;
+ }
+ reg = half_shuffle32(reg) << 1;
+ hv_gic_set_distributor_reg(offset, reg);
+ offset += 4;
+ }
+}
+
+/* Read a bitmap register group from the kernel VGIC. */
+static void hvf_dist_getbmp(GICv3State *s, hv_gic_distributor_reg_t offset, uint32_t *bmp)
+{
+ uint64_t reg;
+ int irq;
+
+ for_each_dist_irq_reg(irq, s->num_irq, 1) {
+
+ hv_gic_get_distributor_reg(offset, ®);
+ *gic_bmp_ptr32(bmp, irq) = reg;
+ offset += 4;
+ }
+}
+
+static void hvf_dist_putbmp(GICv3State *s, hv_gic_distributor_reg_t offset,
+ hv_gic_distributor_reg_t clroffset, uint32_t *bmp)
+{
+ uint32_t reg;
+ int irq;
+
+ for_each_dist_irq_reg(irq, s->num_irq, 1) {
+ /*
+ * If this bitmap is a set/clear register pair, first write to the
+ * clear-reg to clear all bits before using the set-reg to write
+ * the 1 bits.
+ */
+ if (clroffset != 0) {
+ reg = 0;
+ hv_gic_set_distributor_reg(clroffset, reg);
+ clroffset += 4;
+ }
+ reg = *gic_bmp_ptr32(bmp, irq);
+ hv_gic_set_distributor_reg(offset, reg);
+ offset += 4;
+ }
+}
+
+static void hvf_gicv3_check(GICv3State *s)
+{
+ uint64_t reg;
+ uint32_t num_irq;
+
+ /* Sanity checking s->num_irq */
+ hv_gic_get_distributor_reg(HV_GIC_DISTRIBUTOR_REG_GICD_TYPER, ®);
+ num_irq = ((reg & 0x1f) + 1) * 32;
+
+ if (num_irq < s->num_irq) {
+ error_report("Model requests %u IRQs, but HVF supports max %u",
+ s->num_irq, num_irq);
+ abort();
+ }
+}
+
+static void hvf_gicv3_put_cpu(CPUState *cpu_state, run_on_cpu_data arg)
+{
+ uint32_t reg;
+ uint64_t reg64;
+ int i, num_pri_bits;
+
+ /* Redistributor state */
+ GICv3CPUState *c = arg.host_ptr;
+ hv_vcpu_t vcpu = c->cpu->accel->fd;
+
+ reg = c->gicr_waker;
+ hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_IGROUPR0, reg);
+
+ reg = c->gicr_igroupr0;
+ hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_IGROUPR0, reg);
+
+ reg = ~0;
+ hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ICENABLER0, reg);
+ reg = c->gicr_ienabler0;
+ hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ISENABLER0, reg);
+
+ /* Restore config before pending so we treat level/edge correctly */
+ reg = half_shuffle32(c->edge_trigger >> 16) << 1;
+ hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ICFGR1, reg);
+
+ reg = ~0;
+ hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ICPENDR0, reg);
+ reg = c->gicr_ipendr0;
+ hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ISPENDR0, reg);
+
+ reg = ~0;
+ hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ICACTIVER0, reg);
+ reg = c->gicr_iactiver0;
+ hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ISACTIVER0, reg);
+
+ for (i = 0; i < GIC_INTERNAL; i += 4) {
+ reg = c->gicr_ipriorityr[i] |
+ (c->gicr_ipriorityr[i + 1] << 8) |
+ (c->gicr_ipriorityr[i + 2] << 16) |
+ (c->gicr_ipriorityr[i + 3] << 24);
+ hv_gic_set_redistributor_reg(vcpu,
+ HV_GIC_REDISTRIBUTOR_REG_GICR_IPRIORITYR0 + i, reg);
+ }
+
+ /* CPU interface state */
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_SRE_EL1, c->icc_sre_el1);
+
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_CTLR_EL1,
+ c->icc_ctlr_el1[GICV3_NS]);
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_IGRPEN0_EL1,
+ c->icc_igrpen[GICV3_G0]);
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_IGRPEN1_EL1,
+ c->icc_igrpen[GICV3_G1NS]);
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_PMR_EL1, c->icc_pmr_el1);
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_BPR0_EL1, c->icc_bpr[GICV3_G0]);
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_BPR1_EL1, c->icc_bpr[GICV3_G1NS]);
+
+ num_pri_bits = ((c->icc_ctlr_el1[GICV3_NS] &
+ ICC_CTLR_EL1_PRIBITS_MASK) >>
+ ICC_CTLR_EL1_PRIBITS_SHIFT) + 1;
+
+ switch (num_pri_bits) {
+ case 7:
+ reg64 = c->icc_apr[GICV3_G0][3];
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1 + 3, reg64);
+ reg64 = c->icc_apr[GICV3_G0][2];
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1 + 2, reg64);
+ /* fall through */
+ case 6:
+ reg64 = c->icc_apr[GICV3_G0][1];
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1 + 1, reg64);
+ /* fall through */
+ default:
+ reg64 = c->icc_apr[GICV3_G0][0];
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1, reg64);
+ }
+
+ switch (num_pri_bits) {
+ case 7:
+ reg64 = c->icc_apr[GICV3_G1NS][3];
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1 + 3, reg64);
+ reg64 = c->icc_apr[GICV3_G1NS][2];
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1 + 2, reg64);
+ /* fall through */
+ case 6:
+ reg64 = c->icc_apr[GICV3_G1NS][1];
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1 + 1, reg64);
+ /* fall through */
+ default:
+ reg64 = c->icc_apr[GICV3_G1NS][0];
+ hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1, reg64);
+ }
+
+ if (!hvf_arm_el2_enabled()) {
+ return;
+ }
+
+ hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_VMCR_EL2, c->ich_vmcr_el2);
+ hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_HCR_EL2, c->ich_hcr_el2);
+
+ for (int i = 0; i < GICV3_LR_MAX; i++) {
+ hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_LR0_EL2, c->ich_lr_el2[i]);
+ }
+
+ num_pri_bits = c->vpribits;
+
+ switch (num_pri_bits) {
+ case 7:
+ hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2 + 3,
+ c->ich_apr[GICV3_G0][3]);
+ hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2 + 2,
+ c->ich_apr[GICV3_G0][2]);
+ /* fall through */
+ case 6:
+ hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2 + 1,
+ c->ich_apr[GICV3_G0][1]);
+ /* fall through */
+ default:
+ hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2,
+ c->ich_apr[GICV3_G0][0]);
+ }
+
+ switch (num_pri_bits) {
+ case 7:
+ hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2 + 3,
+ c->ich_apr[GICV3_G1NS][3]);
+ hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2 + 2,
+ c->ich_apr[GICV3_G1NS][2]);
+ /* fall through */
+ case 6:
+ hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2 + 1,
+ c->ich_apr[GICV3_G1NS][1]);
+ /* fall through */
+ default:
+ hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2,
+ c->ich_apr[GICV3_G1NS][0]);
+ }
+}
+
+static void hvf_gicv3_put(GICv3State *s)
+{
+ uint32_t reg;
+ uint64_t redist_typer;
+ int ncpu, i;
+
+ hvf_gicv3_check(s);
+
+ hv_vcpu_t vcpu0 = s->cpu[0].cpu->accel->fd;
+ hv_gic_get_redistributor_reg(vcpu0, HV_GIC_REDISTRIBUTOR_REG_GICR_TYPER
+ , &redist_typer);
+
+ reg = s->gicd_ctlr;
+ hv_gic_set_distributor_reg(HV_GIC_DISTRIBUTOR_REG_GICD_CTLR, reg);
+
+ if (redist_typer & GICR_TYPER_PLPIS) {
+ error_report("ITS is not supported on HVF.");
+ abort();
+ }
+
+ /* per-CPU state */
+
+ for (ncpu = 0; ncpu < s->num_cpu; ncpu++) {
+ run_on_cpu_data data;
+ data.host_ptr = &s->cpu[ncpu];
+ run_on_cpu(s->cpu[ncpu].cpu, hvf_gicv3_put_cpu, data);
+ }
+
+ /* s->enable bitmap -> GICD_ISENABLERn */
+ hvf_dist_putbmp(s, HV_GIC_DISTRIBUTOR_REG_GICD_ISENABLER0
+ , HV_GIC_DISTRIBUTOR_REG_GICD_ICENABLER0, s->enabled);
+
+ /* s->group bitmap -> GICD_IGROUPRn */
+ hvf_dist_putbmp(s, HV_GIC_DISTRIBUTOR_REG_GICD_IGROUPR0
+ , 0, s->group);
+
+ /* Restore targets before pending to ensure the pending state is set on
+ * the appropriate CPU interfaces in the kernel
+ */
+
+ /* s->gicd_irouter[irq] -> GICD_IROUTERn */
+ for (i = GIC_INTERNAL; i < s->num_irq; i++) {
+ uint32_t offset = HV_GIC_DISTRIBUTOR_REG_GICD_IROUTER32 + (8 * i)
+ - (8 * GIC_INTERNAL);
+ hv_gic_set_distributor_reg(offset, s->gicd_irouter[i]);
+ }
+
+ /*
+ * s->trigger bitmap -> GICD_ICFGRn
+ * (restore configuration registers before pending IRQs so we treat
+ * level/edge correctly)
+ */
+ hvf_dist_put_edge_trigger(s, HV_GIC_DISTRIBUTOR_REG_GICD_ICFGR0, s->edge_trigger);
+
+ /* s->pending bitmap -> GICD_ISPENDRn */
+ hvf_dist_putbmp(s, HV_GIC_DISTRIBUTOR_REG_GICD_ISPENDR0,
+ HV_GIC_DISTRIBUTOR_REG_GICD_ICPENDR0, s->pending);
+
+ /* s->active bitmap -> GICD_ISACTIVERn */
+ hvf_dist_putbmp(s, HV_GIC_DISTRIBUTOR_REG_GICD_ISACTIVER0,
+ HV_GIC_DISTRIBUTOR_REG_GICD_ICACTIVER0, s->active);
+
+ /* s->gicd_ipriority[] -> GICD_IPRIORITYRn */
+ hvf_dist_put_priority(s, HV_GIC_DISTRIBUTOR_REG_GICD_IPRIORITYR0, s->gicd_ipriority);
+}
+
+static void hvf_gicv3_get_cpu(CPUState *cpu_state, run_on_cpu_data arg)
+{
+ uint64_t reg;
+ int i, num_pri_bits;
+
+ /* Redistributor state */
+ GICv3CPUState *c = arg.host_ptr;
+ hv_vcpu_t vcpu = c->cpu->accel->fd;
+
+ hv_gic_get_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_IGROUPR0,
+ ®);
+ c->gicr_igroupr0 = reg;
+ hv_gic_get_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ISENABLER0,
+ ®);
+ c->gicr_ienabler0 = reg;
+ hv_gic_get_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ICFGR1,
+ ®);
+ c->edge_trigger = half_unshuffle32(reg >> 1) << 16;
+ hv_gic_get_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ISPENDR0,
+ ®);
+ c->gicr_ipendr0 = reg;
+ hv_gic_get_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ISACTIVER0,
+ ®);
+ c->gicr_iactiver0 = reg;
+
+ for (i = 0; i < GIC_INTERNAL; i += 4) {
+ hv_gic_get_redistributor_reg(
+ vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_IPRIORITYR0 + i, ®);
+ c->gicr_ipriorityr[i] = extract32(reg, 0, 8);
+ c->gicr_ipriorityr[i + 1] = extract32(reg, 8, 8);
+ c->gicr_ipriorityr[i + 2] = extract32(reg, 16, 8);
+ c->gicr_ipriorityr[i + 3] = extract32(reg, 24, 8);
+ }
+
+ /* CPU interface */
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_SRE_EL1, &c->icc_sre_el1);
+
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_CTLR_EL1,
+ &c->icc_ctlr_el1[GICV3_NS]);
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_IGRPEN0_EL1,
+ &c->icc_igrpen[GICV3_G0]);
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_IGRPEN1_EL1,
+ &c->icc_igrpen[GICV3_G1NS]);
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_PMR_EL1, &c->icc_pmr_el1);
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_BPR0_EL1, &c->icc_bpr[GICV3_G0]);
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_BPR1_EL1, &c->icc_bpr[GICV3_G1NS]);
+ num_pri_bits = ((c->icc_ctlr_el1[GICV3_NS] & ICC_CTLR_EL1_PRIBITS_MASK) >>
+ ICC_CTLR_EL1_PRIBITS_SHIFT) +
+ 1;
+
+ switch (num_pri_bits) {
+ case 7:
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1 + 3,
+ &c->icc_apr[GICV3_G0][3]);
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1 + 2,
+ &c->icc_apr[GICV3_G0][2]);
+ /* fall through */
+ case 6:
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1 + 1,
+ &c->icc_apr[GICV3_G0][1]);
+ /* fall through */
+ default:
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1,
+ &c->icc_apr[GICV3_G0][0]);
+ }
+
+ switch (num_pri_bits) {
+ case 7:
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1 + 3,
+ &c->icc_apr[GICV3_G1NS][3]);
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1 + 2,
+ &c->icc_apr[GICV3_G1NS][2]);
+ /* fall through */
+ case 6:
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1 + 1,
+ &c->icc_apr[GICV3_G1NS][1]);
+ /* fall through */
+ default:
+ hv_gic_get_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1,
+ &c->icc_apr[GICV3_G1NS][0]);
+ }
+
+ /* Registers beyond this are with nested virt only */
+ if (!hvf_arm_el2_enabled()) {
+ return;
+ }
+
+ hv_gic_get_ich_reg(vcpu, HV_GIC_ICH_REG_VMCR_EL2, &c->ich_vmcr_el2);
+ hv_gic_get_ich_reg(vcpu, HV_GIC_ICH_REG_HCR_EL2, &c->ich_hcr_el2);
+
+ for (int i = 0; i < GICV3_LR_MAX; i++) {
+ hv_gic_get_ich_reg(vcpu, HV_GIC_ICH_REG_LR0_EL2, &c->ich_lr_el2[i]);
+ }
+
+ num_pri_bits = c->vpribits;
+
+ switch (num_pri_bits) {
+ case 7:
+ hv_gic_get_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2 + 3,
+ &c->ich_apr[GICV3_G0][3]);
+ hv_gic_get_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2 + 2,
+ &c->ich_apr[GICV3_G0][2]);
+ /* fall through */
+ case 6:
+ hv_gic_get_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2 + 1,
+ &c->ich_apr[GICV3_G0][1]);
+ /* fall through */
+ default:
+ hv_gic_get_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2,
+ &c->ich_apr[GICV3_G0][0]);
+ }
+
+ switch (num_pri_bits) {
+ case 7:
+ hv_gic_get_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2 + 3,
+ &c->ich_apr[GICV3_G1NS][3]);
+ hv_gic_get_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2 + 2,
+ &c->ich_apr[GICV3_G1NS][2]);
+ /* fall through */
+ case 6:
+ hv_gic_get_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2 + 1,
+ &c->ich_apr[GICV3_G1NS][1]);
+ /* fall through */
+ default:
+ hv_gic_get_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2,
+ &c->ich_apr[GICV3_G1NS][0]);
+ }
+}
+
+static void hvf_gicv3_get(GICv3State *s)
+{
+ uint64_t reg, redist_typer;
+ int ncpu, i;
+
+ hvf_gicv3_check(s);
+
+ hv_vcpu_t vcpu0 = s->cpu[0].cpu->accel->fd;
+ hv_gic_get_redistributor_reg(vcpu0,
+ HV_GIC_REDISTRIBUTOR_REG_GICR_TYPER, &redist_typer);
+
+ hv_gic_get_distributor_reg(HV_GIC_DISTRIBUTOR_REG_GICD_CTLR, ®);
+ s->gicd_ctlr = reg;
+
+ /* Redistributor state (one per CPU) */
+
+ for (ncpu = 0; ncpu < s->num_cpu; ncpu++) {
+ run_on_cpu_data data;
+ data.host_ptr = &s->cpu[ncpu];
+ run_on_cpu(s->cpu[ncpu].cpu, hvf_gicv3_get_cpu, data);
+ }
+
+ if (redist_typer & GICR_TYPER_PLPIS) {
+ error_report("ITS is not supported on HVF.");
+ abort();
+ }
+
+ /* GICD_IGROUPRn -> s->group bitmap */
+ hvf_dist_getbmp(s, HV_GIC_DISTRIBUTOR_REG_GICD_IGROUPR0, s->group);
+
+ /* GICD_ISENABLERn -> s->enabled bitmap */
+ hvf_dist_getbmp(s, HV_GIC_DISTRIBUTOR_REG_GICD_ISENABLER0, s->enabled);
+
+ /* GICD_ISPENDRn -> s->pending bitmap */
+ hvf_dist_getbmp(s, HV_GIC_DISTRIBUTOR_REG_GICD_ISPENDR0, s->pending);
+
+ /* GICD_ISACTIVERn -> s->active bitmap */
+ hvf_dist_getbmp(s, HV_GIC_DISTRIBUTOR_REG_GICD_ISACTIVER0, s->active);
+
+ /* GICD_ICFGRn -> s->trigger bitmap */
+ hvf_dist_get_edge_trigger(s, HV_GIC_DISTRIBUTOR_REG_GICD_ICFGR0
+ , s->edge_trigger);
+
+ /* GICD_IPRIORITYRn -> s->gicd_ipriority[] */
+ hvf_dist_get_priority(s, HV_GIC_DISTRIBUTOR_REG_GICD_IPRIORITYR0
+ , s->gicd_ipriority);
+
+ /* GICD_IROUTERn -> s->gicd_irouter[irq] */
+ for (i = GIC_INTERNAL; i < s->num_irq; i++) {
+ uint32_t offset = HV_GIC_DISTRIBUTOR_REG_GICD_IROUTER32
+ + (8 * i) - (8 * GIC_INTERNAL);
+ hv_gic_get_distributor_reg(offset, &s->gicd_irouter[i]);
+ }
+}
+
+static void hvf_gicv3_set_irq(void *opaque, int irq, int level)
+{
+ GICv3State *s = (GICv3State *)opaque;
+ if (irq > s->num_irq) {
+ return;
+ }
+ hv_gic_set_spi(GIC_INTERNAL + irq, !!level);
+}
+
+static void hvf_gicv3_icc_reset(CPUARMState *env, const ARMCPRegInfo *ri)
+{
+ GICv3State *s;
+ GICv3CPUState *c;
+
+ c = (GICv3CPUState *)env->gicv3state;
+ s = c->gic;
+
+ c->icc_pmr_el1 = 0;
+ /*
+ * Architecturally the reset value of the ICC_BPR registers
+ * is UNKNOWN. We set them all to 0 here; when the kernel
+ * uses these values to program the ICH_VMCR_EL2 fields that
+ * determine the guest-visible ICC_BPR register values, the
+ * hardware's "writing a value less than the minimum sets
+ * the field to the minimum value" behaviour will result in
+ * them effectively resetting to the correct minimum value
+ * for the host GIC.
+ */
+ c->icc_bpr[GICV3_G0] = 0;
+ c->icc_bpr[GICV3_G1] = 0;
+ c->icc_bpr[GICV3_G1NS] = 0;
+
+ c->icc_sre_el1 = 0x7;
+ memset(c->icc_apr, 0, sizeof(c->icc_apr));
+ memset(c->icc_igrpen, 0, sizeof(c->icc_igrpen));
+
+ if (s->migration_blocker) {
+ return;
+ }
+
+ /* Initialize to actual HW supported configuration */
+ hv_gic_get_icc_reg(c->cpu->accel->fd,
+ HV_GIC_ICC_REG_CTLR_EL1, &c->icc_ctlr_el1[GICV3_NS]);
+
+ c->icc_ctlr_el1[GICV3_S] = c->icc_ctlr_el1[GICV3_NS];
+}
+
+static void hvf_gicv3_reset_hold(Object *obj, ResetType type)
+{
+ GICv3State *s = ARM_GICV3_COMMON(obj);
+ HVFARMGICv3Class *kgc = HVF_GICV3_GET_CLASS(s);
+
+ if (kgc->parent_phases.hold) {
+ kgc->parent_phases.hold(obj, type);
+ }
+
+ hvf_gicv3_put(s);
+}
+
+
+/*
+ * CPU interface registers of GIC needs to be reset on CPU reset.
+ * For the calling arm_gicv3_icc_reset() on CPU reset, we register
+ * below ARMCPRegInfo. As we reset the whole cpu interface under single
+ * register reset, we define only one register of CPU interface instead
+ * of defining all the registers.
+ */
+static const ARMCPRegInfo gicv3_cpuif_reginfo[] = {
+ { .name = "ICC_CTLR_EL1", .state = ARM_CP_STATE_BOTH,
+ .opc0 = 3, .opc1 = 0, .crn = 12, .crm = 12, .opc2 = 4,
+ /*
+ * If ARM_CP_NOP is used, resetfn is not called,
+ * So ARM_CP_NO_RAW is appropriate type.
+ */
+ .type = ARM_CP_NO_RAW,
+ .access = PL1_RW,
+ .readfn = arm_cp_read_zero,
+ .writefn = arm_cp_write_ignore,
+ /*
+ * We hang the whole cpu interface reset routine off here
+ * rather than parcelling it out into one little function
+ * per register
+ */
+ .resetfn = hvf_gicv3_icc_reset,
+ },
+};
+
+static void hvf_gicv3_realize(DeviceState *dev, Error **errp)
+{
+ GICv3State *s = HVF_GICV3(dev);
+ HVFARMGICv3Class *kgc = HVF_GICV3_GET_CLASS(s);
+ Error *local_err = NULL;
+ int i;
+
+ kgc->parent_realize(dev, &local_err);
+ if (local_err) {
+ error_propagate(errp, local_err);
+ return;
+ }
+
+ if (s->revision != 3) {
+ error_setg(errp, "unsupported GIC revision %d for platform GIC",
+ s->revision);
+ }
+
+ if (s->security_extn) {
+ error_setg(errp, "the platform vGICv3 does not implement the "
+ "security extensions");
+ return;
+ }
+
+ if (s->nmi_support) {
+ error_setg(errp, "NMI is not supported with the platform GIC");
+ return;
+ }
+
+ if (s->nb_redist_regions > 1) {
+ error_setg(errp, "Multiple VGICv3 redistributor regions are not "
+ "supported by HVF");
+ error_append_hint(errp, "A maximum of %d VCPUs can be used",
+ s->redist_region_count[0]);
+ return;
+ }
+
+ gicv3_init_irqs_and_mmio(s, hvf_gicv3_set_irq, NULL);
+
+ for (i = 0; i < s->num_cpu; i++) {
+ ARMCPU *cpu = ARM_CPU(qemu_get_cpu(i));
+
+ define_arm_cp_regs(cpu, gicv3_cpuif_reginfo);
+ }
+
+ if (s->maint_irq && s->maint_irq != HV_GIC_INT_MAINTENANCE) {
+ error_setg(errp, "vGIC maintenance IRQ mismatch with the hardcoded one in HVF.");
+ return;
+ }
+}
+
+static void hvf_gicv3_class_init(ObjectClass *klass, const void *data)
+{
+ DeviceClass *dc = DEVICE_CLASS(klass);
+ ResettableClass *rc = RESETTABLE_CLASS(klass);
+ ARMGICv3CommonClass *agcc = ARM_GICV3_COMMON_CLASS(klass);
+ HVFARMGICv3Class *kgc = HVF_GICV3_CLASS(klass);
+
+ agcc->pre_save = hvf_gicv3_get;
+ agcc->post_load = hvf_gicv3_put;
+
+ device_class_set_parent_realize(dc, hvf_gicv3_realize,
+ &kgc->parent_realize);
+ resettable_class_set_parent_phases(rc, NULL, hvf_gicv3_reset_hold, NULL,
+ &kgc->parent_phases);
+}
+
+static const TypeInfo hvf_arm_gicv3_info = {
+ .name = TYPE_HVF_GICV3,
+ .parent = TYPE_ARM_GICV3_COMMON,
+ .instance_size = sizeof(GICv3State),
+ .class_init = hvf_gicv3_class_init,
+ .class_size = sizeof(HVFARMGICv3Class),
+};
+
+static void hvf_gicv3_register_types(void)
+{
+ type_register_static(&hvf_arm_gicv3_info);
+}
+
+type_init(hvf_gicv3_register_types)
diff --git a/hw/intc/meson.build b/hw/intc/meson.build
index 3137521a4a..f446e966e3 100644
--- a/hw/intc/meson.build
+++ b/hw/intc/meson.build
@@ -42,6 +42,7 @@ specific_ss.add(when: 'CONFIG_ARM_GIC', if_true: files('arm_gicv3_cpuif_common.c
specific_ss.add(when: 'CONFIG_ARM_GICV3', if_true: files('arm_gicv3_cpuif.c'))
specific_ss.add(when: 'CONFIG_ARM_GIC_KVM', if_true: files('arm_gic_kvm.c'))
specific_ss.add(when: ['CONFIG_ARM_GIC_KVM', 'TARGET_AARCH64'], if_true: files('arm_gicv3_kvm.c', 'arm_gicv3_its_kvm.c'))
+specific_ss.add(when: ['CONFIG_HVF', 'CONFIG_ARM_GICV3'], if_true: files('arm_gicv3_hvf.c'))
specific_ss.add(when: 'CONFIG_ARM_V7M', if_true: files('armv7m_nvic.c'))
specific_ss.add(when: 'CONFIG_GRLIB', if_true: files('grlib_irqmp.c'))
specific_ss.add(when: 'CONFIG_IOAPIC', if_true: files('ioapic.c'))
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v5 04/13] hw/arm, target/arm: nested virtualisation on HVF
2025-07-28 13:41 [PATCH v5 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
` (2 preceding siblings ...)
2025-07-28 13:41 ` [PATCH v5 03/13] hw/intc: Add hvf vGIC interrupt controller support Mohamed Mediouni
@ 2025-07-28 13:41 ` Mohamed Mediouni
2025-07-28 13:41 ` [PATCH v5 05/13] hvf: save/restore Apple GIC state Mohamed Mediouni
` (8 subsequent siblings)
12 siblings, 0 replies; 24+ messages in thread
From: Mohamed Mediouni @ 2025-07-28 13:41 UTC (permalink / raw)
To: qemu-devel
Cc: Mads Ynddal, Paolo Bonzini, Shannon Zhao, Phil Dennis-Jordan,
Igor Mammedov, qemu-arm, Alexander Graf, Roman Bolshakov,
Peter Maydell, Michael S. Tsirkin, Ani Sinha, Cameron Esfahani,
Mohamed Mediouni
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
---
hw/arm/virt.c | 9 ++++++---
target/arm/hvf-stub.c | 15 +++++++++++++++
target/arm/hvf/hvf.c | 41 +++++++++++++++++++++++++++++++++++++++--
target/arm/hvf_arm.h | 3 +++
4 files changed, 63 insertions(+), 5 deletions(-)
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 7da1176cda..7348d55104 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -817,8 +817,7 @@ static void create_gic(VirtMachineState *vms, MemoryRegion *mem)
g_assert_not_reached();
}
- if (kvm_enabled() && vms->virt &&
- (revision != 3 || !kvm_irqchip_in_kernel())) {
+ if (kvm_enabled() && vms->virt && (revision != 3 || !kvm_irqchip_in_kernel())) {
error_report("KVM EL2 is only supported with in-kernel GICv3");
exit(1);
}
@@ -2279,7 +2278,8 @@ static void machvirt_init(MachineState *machine)
exit(1);
}
- if (vms->virt && !kvm_enabled() && !tcg_enabled() && !qtest_enabled()) {
+ if (vms->virt && !kvm_enabled() && !tcg_enabled()
+ && !hvf_enabled() && !qtest_enabled()) {
error_report("mach-virt: %s does not support providing "
"Virtualization extensions to the guest CPU",
current_accel_name());
@@ -2549,6 +2549,9 @@ static void virt_set_virt(Object *obj, bool value, Error **errp)
VirtMachineState *vms = VIRT_MACHINE(obj);
vms->virt = value;
+#if defined(CONFIG_HVF) && defined(__aarch64__)
+ hvf_arm_el2_enable(value);
+#endif
}
static bool virt_get_highmem(Object *obj, Error **errp)
diff --git a/target/arm/hvf-stub.c b/target/arm/hvf-stub.c
index ff137267a0..95ec4ea62f 100644
--- a/target/arm/hvf-stub.c
+++ b/target/arm/hvf-stub.c
@@ -18,3 +18,18 @@ uint32_t hvf_arm_get_max_ipa_bit_size(void)
{
g_assert_not_reached();
}
+
+bool hvf_arm_el2_supported(void)
+{
+ g_assert_not_reached();
+}
+
+bool hvf_arm_el2_enabled(void)
+{
+ g_assert_not_reached();
+}
+
+void hvf_arm_el2_enable(bool)
+{
+ g_assert_not_reached();
+}
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index 5ee0df17e3..4ac42bda32 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -26,6 +26,7 @@
#include "system/address-spaces.h"
#include "system/memory.h"
#include "hw/boards.h"
+#include "hw/arm/virt.h"
#include "hw/irq.h"
#include "qemu/main-loop.h"
#include "system/cpus.h"
@@ -891,6 +892,10 @@ static bool hvf_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf)
(1ULL << ARM_FEATURE_PMU) |
(1ULL << ARM_FEATURE_GENERIC_TIMER);
+ if (hvf_arm_el2_enabled()) {
+ ahcf->features |= 1ULL << ARM_FEATURE_EL2;
+ }
+
for (i = 0; i < ARRAY_SIZE(regs); i++) {
r |= hv_vcpu_config_get_feature_reg(hv_vcpu_config, regs[i].reg, regs[i].val);
}
@@ -957,6 +962,25 @@ uint32_t hvf_arm_get_max_ipa_bit_size(void)
return round_down_to_parange_bit_size(max_ipa_size);
}
+bool hvf_arm_el2_supported(void)
+{
+ bool is_nested_virt_supported;
+ hv_return_t ret = hv_vm_config_get_el2_supported(&is_nested_virt_supported);
+ assert_hvf_ok(ret);
+ return is_nested_virt_supported;
+}
+
+static bool is_nested_virt_enabled = false;
+bool hvf_arm_el2_enabled(void)
+{
+ return is_nested_virt_enabled;
+}
+
+void hvf_arm_el2_enable(bool enable)
+{
+ is_nested_virt_enabled = enable;
+}
+
void hvf_arm_set_cpu_features_from_host(ARMCPU *cpu)
{
if (!arm_host_cpu_features.dtb_compatible) {
@@ -993,6 +1017,13 @@ hv_return_t hvf_arch_vm_create(MachineState *ms, uint32_t pa_range)
}
chosen_ipa_bit_size = pa_range;
+ if (hvf_arm_el2_enabled()) {
+ ret = hv_vm_config_set_el2_enabled(config, true);
+ if (ret != HV_SUCCESS) {
+ goto cleanup;
+ }
+ }
+
ret = hv_vm_create(config);
cleanup:
@@ -1100,6 +1131,13 @@ static void hvf_psci_cpu_off(ARMCPU *arm_cpu)
assert(ret == QEMU_ARM_POWERCTL_RET_SUCCESS);
}
+static int hvf_psci_get_target_el(void)
+{
+ if (hvf_arm_el2_enabled()) {
+ return 2;
+ }
+ return 1;
+}
/*
* Handle a PSCI call.
*
@@ -1121,7 +1159,6 @@ static bool hvf_handle_psci_call(CPUState *cpu)
CPUState *target_cpu_state;
ARMCPU *target_cpu;
target_ulong entry;
- int target_el = 1;
int32_t ret = 0;
trace_hvf_psci_call(param[0], param[1], param[2], param[3],
@@ -1175,7 +1212,7 @@ static bool hvf_handle_psci_call(CPUState *cpu)
entry = param[2];
context_id = param[3];
ret = arm_set_cpu_on(mpidr, entry, context_id,
- target_el, target_aarch64);
+ hvf_psci_get_target_el(), target_aarch64);
break;
case QEMU_PSCI_0_1_FN_CPU_OFF:
case QEMU_PSCI_0_2_FN_CPU_OFF:
diff --git a/target/arm/hvf_arm.h b/target/arm/hvf_arm.h
index ea82f2691d..bf55e7ae28 100644
--- a/target/arm/hvf_arm.h
+++ b/target/arm/hvf_arm.h
@@ -24,5 +24,8 @@ void hvf_arm_set_cpu_features_from_host(ARMCPU *cpu);
uint32_t hvf_arm_get_default_ipa_bit_size(void);
uint32_t hvf_arm_get_max_ipa_bit_size(void);
+bool hvf_arm_el2_supported(void);
+bool hvf_arm_el2_enabled(void);
+void hvf_arm_el2_enable(bool);
#endif
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v5 05/13] hvf: save/restore Apple GIC state
2025-07-28 13:41 [PATCH v5 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
` (3 preceding siblings ...)
2025-07-28 13:41 ` [PATCH v5 04/13] hw/arm, target/arm: nested virtualisation on HVF Mohamed Mediouni
@ 2025-07-28 13:41 ` Mohamed Mediouni
2025-07-28 13:41 ` [PATCH v5 06/13] target/arm: hvf: pass through CNTHCTL_EL2 and MDCCINT_EL1 Mohamed Mediouni
` (7 subsequent siblings)
12 siblings, 0 replies; 24+ messages in thread
From: Mohamed Mediouni @ 2025-07-28 13:41 UTC (permalink / raw)
To: qemu-devel
Cc: Mads Ynddal, Paolo Bonzini, Shannon Zhao, Phil Dennis-Jordan,
Igor Mammedov, qemu-arm, Alexander Graf, Roman Bolshakov,
Peter Maydell, Michael S. Tsirkin, Ani Sinha, Cameron Esfahani,
Mohamed Mediouni
On HVF, some of the GIC state is in an opaque Apple-provided structure.
Save/restore that state to be able to save/restore VMs.
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
---
target/arm/hvf/hvf.c | 73 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 73 insertions(+)
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index 4ac42bda32..f14a3a3cbd 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -22,6 +22,7 @@
#include "cpu-sysregs.h"
#include <mach/mach_time.h>
+#include <stdint.h>
#include "system/address-spaces.h"
#include "system/memory.h"
@@ -2174,15 +2175,83 @@ static const VMStateDescription vmstate_hvf_vtimer = {
},
};
+/* Apple specific opaque state for the vGIC */
+
+typedef struct HVGICState {
+ void *state;
+ uint32_t size;
+} HVGICState;
+
+static HVGICState gic;
+
+static int hvf_gic_opaque_state_get(void)
+{
+ hv_gic_state_t gic_state;
+ hv_return_t err;
+ size_t size;
+
+ gic_state = hv_gic_state_create();
+ if (gic_state == NULL) {
+ error_report("hvf: vgic: failed to create hv_gic_state_create.");
+ return 1;
+ }
+ err = hv_gic_state_get_size(gic_state, &size);
+ gic.size = size;
+ if (err != HV_SUCCESS) {
+ error_report("hvf: vgic: failed to get GIC state size.");
+ return 1;
+ }
+ gic.state = malloc(gic.size);
+ err = hv_gic_state_get_data(gic_state, gic.state);
+ if (err != HV_SUCCESS) {
+ error_report("hvf: vgic: failed to get GIC state.");
+ return 1;
+ }
+ return 0;
+}
+
+static int hvf_gic_opaque_state_set(void)
+{
+ hv_return_t err;
+ if (!gic.size) {
+ return 0;
+ }
+ err = hv_gic_set_state(gic.state, gic.size);
+ if (err != HV_SUCCESS) {
+ error_report("hvf: vgic: failed to restore GIC state.");
+ return 1;
+ }
+ return 0;
+}
+
+static const VMStateDescription vmstate_hvf_gic = {
+ .name = "hvf-gic",
+ .version_id = 1,
+ .minimum_version_id = 1,
+ .fields = (const VMStateField[]) {
+ VMSTATE_UINT32(size, HVGICState),
+ VMSTATE_VBUFFER_UINT32(state,
+ HVGICState, 0, 0,
+ size),
+ VMSTATE_END_OF_LIST()
+ },
+};
+
static void hvf_vm_state_change(void *opaque, bool running, RunState state)
{
HVFVTimer *s = opaque;
if (running) {
+ if (hvf_irqchip_in_kernel()) {
+ hvf_gic_opaque_state_set();
+ }
/* Update vtimer offset on all CPUs */
hvf_state->vtimer_offset = mach_absolute_time() - s->vtimer_val;
cpu_synchronize_all_states();
} else {
+ if (hvf_irqchip_in_kernel()) {
+ hvf_gic_opaque_state_get();
+ }
/* Remember vtimer value on every pause */
s->vtimer_val = hvf_vtimer_val_raw();
}
@@ -2192,6 +2261,10 @@ int hvf_arch_init(void)
{
hvf_state->vtimer_offset = mach_absolute_time();
vmstate_register(NULL, 0, &vmstate_hvf_vtimer, &vtimer);
+ if (hvf_irqchip_in_kernel()) {
+ gic.size = 0;
+ vmstate_register(NULL, 0, &vmstate_hvf_gic, &gic);
+ }
qemu_add_vm_change_state_handler(hvf_vm_state_change, &vtimer);
hvf_arm_init_debug();
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v5 06/13] target/arm: hvf: pass through CNTHCTL_EL2 and MDCCINT_EL1
2025-07-28 13:41 [PATCH v5 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
` (4 preceding siblings ...)
2025-07-28 13:41 ` [PATCH v5 05/13] hvf: save/restore Apple GIC state Mohamed Mediouni
@ 2025-07-28 13:41 ` Mohamed Mediouni
2025-08-06 11:20 ` Philippe Mathieu-Daudé
2025-07-28 13:41 ` [PATCH v5 07/13] hw/arm: virt: add GICv2m for the case when ITS is not available Mohamed Mediouni
` (6 subsequent siblings)
12 siblings, 1 reply; 24+ messages in thread
From: Mohamed Mediouni @ 2025-07-28 13:41 UTC (permalink / raw)
To: qemu-devel
Cc: Mads Ynddal, Paolo Bonzini, Shannon Zhao, Phil Dennis-Jordan,
Igor Mammedov, qemu-arm, Alexander Graf, Roman Bolshakov,
Peter Maydell, Michael S. Tsirkin, Ani Sinha, Cameron Esfahani,
Mohamed Mediouni
HVF traps accesses to CNTHCTL_EL2. For nested guests, HVF traps accesses to MDCCINT_EL1.
Pass through those accesses to the Hypervisor.framework library.
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
---
target/arm/hvf/hvf.c | 16 ++++++++++++++++
1 file changed, 16 insertions(+)
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index f14a3a3cbd..eefae3069f 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -297,6 +297,10 @@ void hvf_arm_init_debug(void)
#define SYSREG_DBGWVR15_EL1 SYSREG(2, 0, 0, 15, 6)
#define SYSREG_DBGWCR15_EL1 SYSREG(2, 0, 0, 15, 7)
+/* EL2 registers */
+#define SYSREG_CNTHCTL_EL2 SYSREG(3, 4, 14, 1, 0)
+#define SYSREG_MDCCINT_EL1 SYSREG(2, 0, 0, 2, 0)
+
#define WFX_IS_WFE (1 << 0)
#define TMR_CTL_ENABLE (1 << 0)
@@ -1372,6 +1376,12 @@ static int hvf_sysreg_read(CPUState *cpu, uint32_t reg, uint64_t *val)
case SYSREG_OSDLR_EL1:
/* Dummy register */
return 0;
+ case SYSREG_CNTHCTL_EL2:
+ assert_hvf_ok(hv_vcpu_get_sys_reg(cpu->accel->fd, HV_SYS_REG_CNTHCTL_EL2, val));
+ return 0;
+ case SYSREG_MDCCINT_EL1:
+ assert_hvf_ok(hv_vcpu_get_sys_reg(cpu->accel->fd, HV_SYS_REG_MDCCINT_EL1, val));
+ return 0;
case SYSREG_ICC_AP0R0_EL1:
case SYSREG_ICC_AP0R1_EL1:
case SYSREG_ICC_AP0R2_EL1:
@@ -1689,6 +1699,12 @@ static int hvf_sysreg_write(CPUState *cpu, uint32_t reg, uint64_t val)
case SYSREG_OSDLR_EL1:
/* Dummy register */
return 0;
+ case SYSREG_CNTHCTL_EL2:
+ assert_hvf_ok(hv_vcpu_set_sys_reg(cpu->accel->fd, HV_SYS_REG_CNTHCTL_EL2, val));
+ return 0;
+ case SYSREG_MDCCINT_EL1:
+ assert_hvf_ok(hv_vcpu_set_sys_reg(cpu->accel->fd, HV_SYS_REG_MDCCINT_EL1, val));
+ return 0;
case SYSREG_LORC_EL1:
/* Dummy register */
return 0;
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v5 07/13] hw/arm: virt: add GICv2m for the case when ITS is not available
2025-07-28 13:41 [PATCH v5 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
` (5 preceding siblings ...)
2025-07-28 13:41 ` [PATCH v5 06/13] target/arm: hvf: pass through CNTHCTL_EL2 and MDCCINT_EL1 Mohamed Mediouni
@ 2025-07-28 13:41 ` Mohamed Mediouni
2025-07-28 13:41 ` [PATCH v5 08/13] hw/arm: virt: cleanly fail on attempt to use the platform vGIC together with ITS Mohamed Mediouni
` (5 subsequent siblings)
12 siblings, 0 replies; 24+ messages in thread
From: Mohamed Mediouni @ 2025-07-28 13:41 UTC (permalink / raw)
To: qemu-devel
Cc: Mads Ynddal, Paolo Bonzini, Shannon Zhao, Phil Dennis-Jordan,
Igor Mammedov, qemu-arm, Alexander Graf, Roman Bolshakov,
Peter Maydell, Michael S. Tsirkin, Ani Sinha, Cameron Esfahani,
Mohamed Mediouni
On Hypervisor.framework for macOS and WHPX for Windows, the provided environment is a GICv3 without ITS.
As such, support a GICv3 w/ GICv2m for that scenario.
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
---
hw/arm/virt-acpi-build.c | 4 +++-
hw/arm/virt.c | 8 ++++++++
include/hw/arm/virt.h | 2 ++
3 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/hw/arm/virt-acpi-build.c b/hw/arm/virt-acpi-build.c
index b01fc4f8ef..969fa3f686 100644
--- a/hw/arm/virt-acpi-build.c
+++ b/hw/arm/virt-acpi-build.c
@@ -848,7 +848,9 @@ build_madt(GArray *table_data, BIOSLinker *linker, VirtMachineState *vms)
build_append_int_noprefix(table_data, memmap[VIRT_GIC_ITS].base, 8);
build_append_int_noprefix(table_data, 0, 4); /* Reserved */
}
- } else {
+ }
+
+ if (!vms->its && !vms->no_gicv3_with_gicv2m) {
const uint16_t spi_base = vms->irqmap[VIRT_GIC_V2M] + ARM_SPI_BASE;
/* 5.2.12.16 GIC MSI Frame Structure */
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 7348d55104..91d8cd9363 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -953,6 +953,8 @@ static void create_gic(VirtMachineState *vms, MemoryRegion *mem)
if (vms->gic_version != VIRT_GIC_VERSION_2 && vms->its) {
create_its(vms);
+ } else if (vms->gic_version != VIRT_GIC_VERSION_2 && !vms->no_gicv3_with_gicv2m) {
+ create_v2m(vms);
} else if (vms->gic_version == VIRT_GIC_VERSION_2) {
create_v2m(vms);
}
@@ -2408,6 +2410,8 @@ static void machvirt_init(MachineState *machine)
vms->ns_el2_virt_timer_irq = ns_el2_virt_timer_present() &&
!vmc->no_ns_el2_virt_timer_irq;
+ vms->no_gicv3_with_gicv2m = vmc->no_gicv3_with_gicv2m;
+
fdt_add_timer_nodes(vms);
fdt_add_cpu_nodes(vms);
@@ -3419,6 +3423,7 @@ static void virt_instance_init(Object *obj)
vms->its = true;
/* Allow ITS emulation if the machine version supports it */
vms->tcg_its = !vmc->no_tcg_its;
+ vms->no_gicv3_with_gicv2m = false;
/* Default disallows iommu instantiation */
vms->iommu = VIRT_IOMMU_NONE;
@@ -3471,8 +3476,11 @@ DEFINE_VIRT_MACHINE_AS_LATEST(10, 1)
static void virt_machine_10_0_options(MachineClass *mc)
{
+ VirtMachineClass *vmc = VIRT_MACHINE_CLASS(OBJECT_CLASS(mc));
+
virt_machine_10_1_options(mc);
compat_props_add(mc->compat_props, hw_compat_10_0, hw_compat_10_0_len);
+ vmc->no_gicv3_with_gicv2m = true;
}
DEFINE_VIRT_MACHINE(10, 0)
diff --git a/include/hw/arm/virt.h b/include/hw/arm/virt.h
index 365a28b082..725ec18fd2 100644
--- a/include/hw/arm/virt.h
+++ b/include/hw/arm/virt.h
@@ -131,6 +131,7 @@ struct VirtMachineClass {
bool no_cpu_topology;
bool no_tcg_lpa2;
bool no_ns_el2_virt_timer_irq;
+ bool no_gicv3_with_gicv2m;
bool no_nested_smmu;
};
@@ -178,6 +179,7 @@ struct VirtMachineState {
char *oem_id;
char *oem_table_id;
bool ns_el2_virt_timer_irq;
+ bool no_gicv3_with_gicv2m;
CXLState cxl_devices_state;
};
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v5 08/13] hw/arm: virt: cleanly fail on attempt to use the platform vGIC together with ITS
2025-07-28 13:41 [PATCH v5 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
` (6 preceding siblings ...)
2025-07-28 13:41 ` [PATCH v5 07/13] hw/arm: virt: add GICv2m for the case when ITS is not available Mohamed Mediouni
@ 2025-07-28 13:41 ` Mohamed Mediouni
2025-07-28 13:41 ` [PATCH v5 09/13] hvf: only call hvf_sync_vtimer() when running without the platform vGIC Mohamed Mediouni
` (4 subsequent siblings)
12 siblings, 0 replies; 24+ messages in thread
From: Mohamed Mediouni @ 2025-07-28 13:41 UTC (permalink / raw)
To: qemu-devel
Cc: Mads Ynddal, Paolo Bonzini, Shannon Zhao, Phil Dennis-Jordan,
Igor Mammedov, qemu-arm, Alexander Graf, Roman Bolshakov,
Peter Maydell, Michael S. Tsirkin, Ani Sinha, Cameron Esfahani,
Mohamed Mediouni
Apple's platform vGIC doesn't support ITS. Deal with this by reporting to the user and not creating the ITS device.
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
---
hw/arm/virt.c | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 91d8cd9363..005e923a22 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -740,6 +740,16 @@ static void create_its(VirtMachineState *vms)
return;
}
+ if (hvf_enabled() && hvf_irqchip_in_kernel() && vms->tcg_its) {
+ /*
+ * In the HVF case, inform the user that they can use the
+ * user-mode GIC if they want to have an ITS.
+ */
+ info_report("ITS not supported without kernel-irqchip=off on HVF");
+ info_report("Disabling ITS");
+ return;
+ }
+
dev = qdev_new(its_class_name());
object_property_set_link(OBJECT(dev), "parent-gicv3", OBJECT(vms->gic),
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v5 09/13] hvf: only call hvf_sync_vtimer() when running without the platform vGIC
2025-07-28 13:41 [PATCH v5 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
` (7 preceding siblings ...)
2025-07-28 13:41 ` [PATCH v5 08/13] hw/arm: virt: cleanly fail on attempt to use the platform vGIC together with ITS Mohamed Mediouni
@ 2025-07-28 13:41 ` Mohamed Mediouni
2025-07-28 13:41 ` [PATCH v5 10/13] hvf: sync registers used at EL2 Mohamed Mediouni
` (3 subsequent siblings)
12 siblings, 0 replies; 24+ messages in thread
From: Mohamed Mediouni @ 2025-07-28 13:41 UTC (permalink / raw)
To: qemu-devel
Cc: Mads Ynddal, Paolo Bonzini, Shannon Zhao, Phil Dennis-Jordan,
Igor Mammedov, qemu-arm, Alexander Graf, Roman Bolshakov,
Peter Maydell, Michael S. Tsirkin, Ani Sinha, Cameron Esfahani,
Mohamed Mediouni
When running with the Apple vGIC, the EL1 vtimer is handled by the platform.
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
---
target/arm/hvf/hvf.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index eefae3069f..7699669e73 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -2007,7 +2007,8 @@ int hvf_vcpu_exec(CPUState *cpu)
g_assert_not_reached();
}
- hvf_sync_vtimer(cpu);
+ if (!hvf_irqchip_in_kernel())
+ hvf_sync_vtimer(cpu);
switch (ec) {
case EC_SOFTWARESTEP: {
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v5 10/13] hvf: sync registers used at EL2
2025-07-28 13:41 [PATCH v5 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
` (8 preceding siblings ...)
2025-07-28 13:41 ` [PATCH v5 09/13] hvf: only call hvf_sync_vtimer() when running without the platform vGIC Mohamed Mediouni
@ 2025-07-28 13:41 ` Mohamed Mediouni
2025-08-06 11:10 ` Philippe Mathieu-Daudé
2025-07-28 13:41 ` [PATCH v5 11/13] hvf: gate ARM_FEATURE_PMU register emulation behind not being " Mohamed Mediouni
` (2 subsequent siblings)
12 siblings, 1 reply; 24+ messages in thread
From: Mohamed Mediouni @ 2025-07-28 13:41 UTC (permalink / raw)
To: qemu-devel
Cc: Mads Ynddal, Paolo Bonzini, Shannon Zhao, Phil Dennis-Jordan,
Igor Mammedov, qemu-arm, Alexander Graf, Roman Bolshakov,
Peter Maydell, Michael S. Tsirkin, Ani Sinha, Cameron Esfahani,
Mohamed Mediouni
When starting up the VM at EL2, more sysregs are available. Sync the state of those.
In addition, sync the state of the EL1 physical timer when the vGIC is used, even
if running at EL1. However, no OS running at EL1 is expected to use those registers.
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
---
target/arm/hvf/hvf.c | 53 ++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 53 insertions(+)
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index 7699669e73..a1e928ddfa 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -406,6 +406,8 @@ static const struct hvf_reg_match hvf_fpreg_match[] = {
struct hvf_sreg_match {
int reg;
uint32_t key;
+ bool vgic;
+ bool el2;
uint32_t cp_idx;
};
@@ -551,6 +553,41 @@ static struct hvf_sreg_match hvf_sreg_match[] = {
{ HV_SYS_REG_CNTV_CTL_EL0, HVF_SYSREG(14, 3, 3, 3, 1) },
{ HV_SYS_REG_CNTV_CVAL_EL0, HVF_SYSREG(14, 3, 3, 3, 2) },
{ HV_SYS_REG_SP_EL1, HVF_SYSREG(4, 1, 3, 4, 0) },
+ /* vGIC */
+ { HV_SYS_REG_CNTP_CTL_EL0, HVF_SYSREG(14, 2, 3, 3, 1), true },
+ { HV_SYS_REG_CNTP_CVAL_EL0, HVF_SYSREG(14, 2, 3, 3, 2), true },
+#ifdef SYNC_NO_RAW_REGS
+ { HV_SYS_REG_CNTP_TVAL_EL0, HVF_SYSREG(14, 2, 3, 3, 0), true},
+#endif
+ /* vGIC + EL2 */
+ { HV_SYS_REG_CNTHCTL_EL2, HVF_SYSREG(14, 1, 3, 4, 0), true, true },
+ { HV_SYS_REG_CNTHP_CVAL_EL2, HVF_SYSREG(14, 2, 3, 4, 2), true, true },
+ { HV_SYS_REG_CNTHP_CTL_EL2, HVF_SYSREG(14, 2, 3, 4, 1), true, true },
+#ifdef SYNC_NO_RAW_REGS
+ { HV_SYS_REG_CNTHP_TVAL_EL2, HVF_SYSREG(14, 2, 3, 4, 0), true, true },
+#endif
+ { HV_SYS_REG_CNTVOFF_EL2, HVF_SYSREG(14, 0, 3, 4, 3), true, true },
+ /* EL2 */
+ { HV_SYS_REG_CPTR_EL2, HVF_SYSREG(1, 1, 3, 4, 2), .el2 = true },
+ { HV_SYS_REG_ELR_EL2, HVF_SYSREG(4, 0, 3, 4, 1), .el2 = true },
+ { HV_SYS_REG_ESR_EL2, HVF_SYSREG(5, 2, 3, 4, 0), .el2 = true },
+ { HV_SYS_REG_FAR_EL2, HVF_SYSREG(6, 0, 3, 4, 0), .el2 = true },
+ { HV_SYS_REG_HCR_EL2, HVF_SYSREG(1, 1, 3, 4, 0), .el2 = true },
+ { HV_SYS_REG_HPFAR_EL2, HVF_SYSREG(6, 0, 3, 4, 4), .el2 = true },
+ { HV_SYS_REG_MAIR_EL2, HVF_SYSREG(10, 2, 3, 4, 0), .el2 = true },
+ { HV_SYS_REG_MDCR_EL2, HVF_SYSREG(1, 1, 3, 4, 1), .el2 = true },
+ { HV_SYS_REG_SCTLR_EL2, HVF_SYSREG(1, 0, 3, 4, 0), .el2 = true },
+ { HV_SYS_REG_SPSR_EL2, HVF_SYSREG(4, 0, 3, 4, 0), .el2 = true },
+ { HV_SYS_REG_SP_EL2, HVF_SYSREG(4, 1, 3, 6, 0), .el2 = true},
+ { HV_SYS_REG_TCR_EL2, HVF_SYSREG(2, 0, 3, 4, 2), .el2 = true },
+ { HV_SYS_REG_TPIDR_EL2, HVF_SYSREG(13, 0, 3, 4, 2), .el2 = true },
+ { HV_SYS_REG_TTBR0_EL2, HVF_SYSREG(2, 0, 3, 4, 0), .el2 = true },
+ { HV_SYS_REG_TTBR1_EL2, HVF_SYSREG(2, 0, 3, 4, 1), .el2 = true },
+ { HV_SYS_REG_VBAR_EL2, HVF_SYSREG(12, 0, 3, 4, 0), .el2 = true },
+ { HV_SYS_REG_VMPIDR_EL2, HVF_SYSREG(0, 0, 3, 4, 5), .el2 = true },
+ { HV_SYS_REG_VPIDR_EL2, HVF_SYSREG(0, 0, 3, 4, 0), .el2 = true },
+ { HV_SYS_REG_VTCR_EL2, HVF_SYSREG(2, 1, 3, 4, 2), .el2 = true },
+ { HV_SYS_REG_VTTBR_EL2, HVF_SYSREG(2, 1, 3, 4, 0), .el2 = true },
};
int hvf_get_registers(CPUState *cpu)
@@ -594,6 +631,14 @@ int hvf_get_registers(CPUState *cpu)
continue;
}
+ if (hvf_sreg_match[i].vgic && !hvf_irqchip_in_kernel()) {
+ continue;
+ }
+
+ if (hvf_sreg_match[i].el2 && !hvf_arm_el2_enabled()) {
+ continue;
+ }
+
if (cpu->accel->guest_debug_enabled) {
/* Handle debug registers */
switch (hvf_sreg_match[i].reg) {
@@ -731,6 +776,14 @@ int hvf_put_registers(CPUState *cpu)
continue;
}
+ if (hvf_sreg_match[i].vgic && !hvf_irqchip_in_kernel()) {
+ continue;
+ }
+
+ if (hvf_sreg_match[i].el2 && !hvf_arm_el2_enabled()) {
+ continue;
+ }
+
if (cpu->accel->guest_debug_enabled) {
/* Handle debug registers */
switch (hvf_sreg_match[i].reg) {
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v5 11/13] hvf: gate ARM_FEATURE_PMU register emulation behind not being at EL2
2025-07-28 13:41 [PATCH v5 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
` (9 preceding siblings ...)
2025-07-28 13:41 ` [PATCH v5 10/13] hvf: sync registers used at EL2 Mohamed Mediouni
@ 2025-07-28 13:41 ` Mohamed Mediouni
2025-08-06 11:21 ` Philippe Mathieu-Daudé
2025-07-28 13:41 ` [PATCH v5 12/13] target/arm: hvf: instantiate GIC early Mohamed Mediouni
2025-07-28 13:41 ` [PATCH v5 13/13] target/arm: hvf: add asserts for code paths not leveraged when using the vGIC Mohamed Mediouni
12 siblings, 1 reply; 24+ messages in thread
From: Mohamed Mediouni @ 2025-07-28 13:41 UTC (permalink / raw)
To: qemu-devel
Cc: Mads Ynddal, Paolo Bonzini, Shannon Zhao, Phil Dennis-Jordan,
Igor Mammedov, qemu-arm, Alexander Graf, Roman Bolshakov,
Peter Maydell, Michael S. Tsirkin, Ani Sinha, Cameron Esfahani,
Mohamed Mediouni
From Apple documentation:
> When EL2 is disabled, PMU register accesses trigger "Trapped MSR, MRS, or
> System Instruction" exceptions. When this happens, hv_vcpu_run() returns, and the
> hv_vcpu_exit_t object contains the information about this exception.
> When EL2 is enabled, the handling of PMU register accesses is determined by the PMUVer
> field of ID_AA64DFR0_EL1 register.
> If the PMUVer field value is zero or is invalid, PMU register accesses generate "Undefined"
> exceptions, which are sent to the guest.
> If the PMUVer field value is non-zero and valid, PMU register accesses are emulated by the framework.
> The ID_AA64DFR0_EL1 register can be modified via hv_vcpu_set_sys_reg API.
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
---
target/arm/hvf/hvf.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index a1e928ddfa..f70870fb62 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -1379,7 +1379,7 @@ static int hvf_sysreg_read(CPUState *cpu, uint32_t reg, uint64_t *val)
ARMCPU *arm_cpu = ARM_CPU(cpu);
CPUARMState *env = &arm_cpu->env;
- if (arm_feature(env, ARM_FEATURE_PMU)) {
+ if (!hvf_arm_el2_enabled() && arm_feature(env, ARM_FEATURE_PMU)) {
switch (reg) {
case SYSREG_PMCR_EL0:
*val = env->cp15.c9_pmcr;
@@ -1676,7 +1676,7 @@ static int hvf_sysreg_write(CPUState *cpu, uint32_t reg, uint64_t val)
SYSREG_OP2(reg),
val);
- if (arm_feature(env, ARM_FEATURE_PMU)) {
+ if (!hvf_arm_el2_enabled() && arm_feature(env, ARM_FEATURE_PMU)) {
switch (reg) {
case SYSREG_PMCCNTR_EL0:
pmu_op_start(env);
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v5 12/13] target/arm: hvf: instantiate GIC early
2025-07-28 13:41 [PATCH v5 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
` (10 preceding siblings ...)
2025-07-28 13:41 ` [PATCH v5 11/13] hvf: gate ARM_FEATURE_PMU register emulation behind not being " Mohamed Mediouni
@ 2025-07-28 13:41 ` Mohamed Mediouni
2025-07-28 13:41 ` [PATCH v5 13/13] target/arm: hvf: add asserts for code paths not leveraged when using the vGIC Mohamed Mediouni
12 siblings, 0 replies; 24+ messages in thread
From: Mohamed Mediouni @ 2025-07-28 13:41 UTC (permalink / raw)
To: qemu-devel
Cc: Mads Ynddal, Paolo Bonzini, Shannon Zhao, Phil Dennis-Jordan,
Igor Mammedov, qemu-arm, Alexander Graf, Roman Bolshakov,
Peter Maydell, Michael S. Tsirkin, Ani Sinha, Cameron Esfahani,
Mohamed Mediouni
While figuring out a better spot for it, put it in hv_arch_vm_create().
After hv_vcpu_create is documented as too late, and deferring
vCPU initialization isn't enough either.
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
---
target/arm/hvf/hvf.c | 15 +++++++++++++++
1 file changed, 15 insertions(+)
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index f70870fb62..1fd9517f3e 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -1083,6 +1083,21 @@ hv_return_t hvf_arch_vm_create(MachineState *ms, uint32_t pa_range)
}
ret = hv_vm_create(config);
+ if (hvf_irqchip_in_kernel()) {
+ /*
+ * Instantiate GIC.
+ * This must be done prior to the creation of any vCPU
+ * but past hv_vm_create()
+ */
+ hv_gic_config_t cfg = hv_gic_config_create();
+ hv_gic_config_set_distributor_base(cfg, 0x08000000);
+ hv_gic_config_set_redistributor_base(cfg, 0x080A0000);
+ hv_return_t err = hv_gic_create(cfg);
+ if (err != HV_SUCCESS) {
+ error_report("error creating platform VGIC");
+ goto cleanup;
+ }
+ }
cleanup:
os_release(config);
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 24+ messages in thread
* [PATCH v5 13/13] target/arm: hvf: add asserts for code paths not leveraged when using the vGIC
2025-07-28 13:41 [PATCH v5 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
` (11 preceding siblings ...)
2025-07-28 13:41 ` [PATCH v5 12/13] target/arm: hvf: instantiate GIC early Mohamed Mediouni
@ 2025-07-28 13:41 ` Mohamed Mediouni
12 siblings, 0 replies; 24+ messages in thread
From: Mohamed Mediouni @ 2025-07-28 13:41 UTC (permalink / raw)
To: qemu-devel
Cc: Mads Ynddal, Paolo Bonzini, Shannon Zhao, Phil Dennis-Jordan,
Igor Mammedov, qemu-arm, Alexander Graf, Roman Bolshakov,
Peter Maydell, Michael S. Tsirkin, Ani Sinha, Cameron Esfahani,
Mohamed Mediouni
When using the vGIC, timers are directly handled by the platform.
No vmexits ought to happen in that case. Abort if reaching those code paths.
Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
---
target/arm/hvf/hvf.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
index 1fd9517f3e..12cdcb3f2c 100644
--- a/target/arm/hvf/hvf.c
+++ b/target/arm/hvf/hvf.c
@@ -1476,6 +1476,7 @@ static int hvf_sysreg_read(CPUState *cpu, uint32_t reg, uint64_t *val)
case SYSREG_ICC_SGI1R_EL1:
case SYSREG_ICC_SRE_EL1:
case SYSREG_ICC_CTLR_EL1:
+ assert(!hvf_irqchip_in_kernel());
/* Call the TCG sysreg handler. This is only safe for GICv3 regs. */
if (hvf_sysreg_read_cp(cpu, reg, val)) {
return 0;
@@ -1802,6 +1803,7 @@ static int hvf_sysreg_write(CPUState *cpu, uint32_t reg, uint64_t val)
case SYSREG_ICC_SGI0R_EL1:
case SYSREG_ICC_SGI1R_EL1:
case SYSREG_ICC_SRE_EL1:
+ assert(!hvf_irqchip_in_kernel());
/* Call the TCG sysreg handler. This is only safe for GICv3 regs. */
if (hvf_sysreg_write_cp(cpu, reg, val)) {
return 0;
@@ -2065,6 +2067,7 @@ int hvf_vcpu_exec(CPUState *cpu)
/* This is the main one, handle below. */
break;
case HV_EXIT_REASON_VTIMER_ACTIVATED:
+ assert(!hvf_irqchip_in_kernel());
qemu_set_irq(arm_cpu->gt_timer_outputs[GTIMER_VIRT], 1);
cpu->accel->vtimer_masked = true;
return 0;
--
2.39.5 (Apple Git-154)
^ permalink raw reply related [flat|nested] 24+ messages in thread
* Re: [PATCH v5 03/13] hw/intc: Add hvf vGIC interrupt controller support
2025-07-28 13:41 ` [PATCH v5 03/13] hw/intc: Add hvf vGIC interrupt controller support Mohamed Mediouni
@ 2025-08-06 10:32 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 24+ messages in thread
From: Philippe Mathieu-Daudé @ 2025-08-06 10:32 UTC (permalink / raw)
To: Mohamed Mediouni, qemu-devel
Cc: Mads Ynddal, Paolo Bonzini, Shannon Zhao, Phil Dennis-Jordan,
Igor Mammedov, qemu-arm, Alexander Graf, Roman Bolshakov,
Peter Maydell, Michael S. Tsirkin, Ani Sinha, Cameron Esfahani
On 28/7/25 15:41, Mohamed Mediouni wrote:
> This opens up the door to nested virtualisation support.
>
> Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
> ---
> hw/intc/arm_gicv3_hvf.c | 722 ++++++++++++++++++++++++++++++++++++++++
> hw/intc/meson.build | 1 +
> 2 files changed, 723 insertions(+)
> create mode 100644 hw/intc/arm_gicv3_hvf.c
> +static void hvf_gicv3_put_cpu(CPUState *cpu_state, run_on_cpu_data arg)
> +{
> + uint32_t reg;
> + uint64_t reg64;
> + int i, num_pri_bits;
> +
> + /* Redistributor state */
> + GICv3CPUState *c = arg.host_ptr;
> + hv_vcpu_t vcpu = c->cpu->accel->fd;
> +
> + reg = c->gicr_waker;
> + hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_IGROUPR0, reg);
> +
> + reg = c->gicr_igroupr0;
> + hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_IGROUPR0, reg);
> +
> + reg = ~0;
> + hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ICENABLER0, reg);
> + reg = c->gicr_ienabler0;
> + hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ISENABLER0, reg);
> +
> + /* Restore config before pending so we treat level/edge correctly */
> + reg = half_shuffle32(c->edge_trigger >> 16) << 1;
> + hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ICFGR1, reg);
> +
> + reg = ~0;
> + hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ICPENDR0, reg);
> + reg = c->gicr_ipendr0;
> + hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ISPENDR0, reg);
> +
> + reg = ~0;
> + hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ICACTIVER0, reg);
> + reg = c->gicr_iactiver0;
> + hv_gic_set_redistributor_reg(vcpu, HV_GIC_REDISTRIBUTOR_REG_GICR_ISACTIVER0, reg);
> +
> + for (i = 0; i < GIC_INTERNAL; i += 4) {
> + reg = c->gicr_ipriorityr[i] |
> + (c->gicr_ipriorityr[i + 1] << 8) |
> + (c->gicr_ipriorityr[i + 2] << 16) |
> + (c->gicr_ipriorityr[i + 3] << 24);
> + hv_gic_set_redistributor_reg(vcpu,
> + HV_GIC_REDISTRIBUTOR_REG_GICR_IPRIORITYR0 + i, reg);
> + }
> +
> + /* CPU interface state */
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_SRE_EL1, c->icc_sre_el1);
> +
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_CTLR_EL1,
> + c->icc_ctlr_el1[GICV3_NS]);
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_IGRPEN0_EL1,
> + c->icc_igrpen[GICV3_G0]);
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_IGRPEN1_EL1,
> + c->icc_igrpen[GICV3_G1NS]);
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_PMR_EL1, c->icc_pmr_el1);
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_BPR0_EL1, c->icc_bpr[GICV3_G0]);
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_BPR1_EL1, c->icc_bpr[GICV3_G1NS]);
> +
> + num_pri_bits = ((c->icc_ctlr_el1[GICV3_NS] &
> + ICC_CTLR_EL1_PRIBITS_MASK) >>
> + ICC_CTLR_EL1_PRIBITS_SHIFT) + 1;
> +
> + switch (num_pri_bits) {
> + case 7:
> + reg64 = c->icc_apr[GICV3_G0][3];
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1 + 3, reg64);
> + reg64 = c->icc_apr[GICV3_G0][2];
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1 + 2, reg64);
> + /* fall through */
> + case 6:
> + reg64 = c->icc_apr[GICV3_G0][1];
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1 + 1, reg64);
> + /* fall through */
> + default:
> + reg64 = c->icc_apr[GICV3_G0][0];
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP0R0_EL1, reg64);
> + }
> +
> + switch (num_pri_bits) {
> + case 7:
> + reg64 = c->icc_apr[GICV3_G1NS][3];
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1 + 3, reg64);
> + reg64 = c->icc_apr[GICV3_G1NS][2];
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1 + 2, reg64);
> + /* fall through */
> + case 6:
> + reg64 = c->icc_apr[GICV3_G1NS][1];
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1 + 1, reg64);
> + /* fall through */
> + default:
> + reg64 = c->icc_apr[GICV3_G1NS][0];
> + hv_gic_set_icc_reg(vcpu, HV_GIC_ICC_REG_AP1R0_EL1, reg64);
> + }
> +
> + if (!hvf_arm_el2_enabled()) {
This method is added in the next patch.
> + return;
> + }
> +
> + hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_VMCR_EL2, c->ich_vmcr_el2);
> + hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_HCR_EL2, c->ich_hcr_el2);
> +
> + for (int i = 0; i < GICV3_LR_MAX; i++) {
> + hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_LR0_EL2, c->ich_lr_el2[i]);
> + }
> +
> + num_pri_bits = c->vpribits;
> +
> + switch (num_pri_bits) {
> + case 7:
> + hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2 + 3,
> + c->ich_apr[GICV3_G0][3]);
> + hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2 + 2,
> + c->ich_apr[GICV3_G0][2]);
> + /* fall through */
> + case 6:
> + hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2 + 1,
> + c->ich_apr[GICV3_G0][1]);
> + /* fall through */
> + default:
> + hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP0R0_EL2,
> + c->ich_apr[GICV3_G0][0]);
> + }
> +
> + switch (num_pri_bits) {
> + case 7:
> + hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2 + 3,
> + c->ich_apr[GICV3_G1NS][3]);
> + hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2 + 2,
> + c->ich_apr[GICV3_G1NS][2]);
> + /* fall through */
> + case 6:
> + hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2 + 1,
> + c->ich_apr[GICV3_G1NS][1]);
> + /* fall through */
> + default:
> + hv_gic_set_ich_reg(vcpu, HV_GIC_ICH_REG_AP1R0_EL2,
> + c->ich_apr[GICV3_G1NS][0]);
> + }
> +}
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v5 10/13] hvf: sync registers used at EL2
2025-07-28 13:41 ` [PATCH v5 10/13] hvf: sync registers used at EL2 Mohamed Mediouni
@ 2025-08-06 11:10 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 24+ messages in thread
From: Philippe Mathieu-Daudé @ 2025-08-06 11:10 UTC (permalink / raw)
To: Mohamed Mediouni, qemu-devel
Cc: Mads Ynddal, Paolo Bonzini, Shannon Zhao, Phil Dennis-Jordan,
Igor Mammedov, qemu-arm, Alexander Graf, Roman Bolshakov,
Peter Maydell, Michael S. Tsirkin, Ani Sinha, Cameron Esfahani
On 28/7/25 15:41, Mohamed Mediouni wrote:
> When starting up the VM at EL2, more sysregs are available. Sync the state of those.
>
> In addition, sync the state of the EL1 physical timer when the vGIC is used, even
> if running at EL1. However, no OS running at EL1 is expected to use those registers.
>
> Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
> ---
> target/arm/hvf/hvf.c | 53 ++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 53 insertions(+)
>
> diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
> index 7699669e73..a1e928ddfa 100644
> --- a/target/arm/hvf/hvf.c
> +++ b/target/arm/hvf/hvf.c
> @@ -406,6 +406,8 @@ static const struct hvf_reg_match hvf_fpreg_match[] = {
> struct hvf_sreg_match {
> int reg;
> uint32_t key;
> + bool vgic;
> + bool el2;
> uint32_t cp_idx;
> };
>
> @@ -551,6 +553,41 @@ static struct hvf_sreg_match hvf_sreg_match[] = {
> { HV_SYS_REG_CNTV_CTL_EL0, HVF_SYSREG(14, 3, 3, 3, 1) },
> { HV_SYS_REG_CNTV_CVAL_EL0, HVF_SYSREG(14, 3, 3, 3, 2) },
> { HV_SYS_REG_SP_EL1, HVF_SYSREG(4, 1, 3, 4, 0) },
> + /* vGIC */
> + { HV_SYS_REG_CNTP_CTL_EL0, HVF_SYSREG(14, 2, 3, 3, 1), true },
> + { HV_SYS_REG_CNTP_CVAL_EL0, HVF_SYSREG(14, 2, 3, 3, 2), true },
> +#ifdef SYNC_NO_RAW_REGS
> + { HV_SYS_REG_CNTP_TVAL_EL0, HVF_SYSREG(14, 2, 3, 3, 0), true},
> +#endif
> + /* vGIC + EL2 */
> + { HV_SYS_REG_CNTHCTL_EL2, HVF_SYSREG(14, 1, 3, 4, 0), true, true },
> + { HV_SYS_REG_CNTHP_CVAL_EL2, HVF_SYSREG(14, 2, 3, 4, 2), true, true },
> + { HV_SYS_REG_CNTHP_CTL_EL2, HVF_SYSREG(14, 2, 3, 4, 1), true, true },
> +#ifdef SYNC_NO_RAW_REGS
> + { HV_SYS_REG_CNTHP_TVAL_EL2, HVF_SYSREG(14, 2, 3, 4, 0), true, true },
> +#endif
> + { HV_SYS_REG_CNTVOFF_EL2, HVF_SYSREG(14, 0, 3, 4, 3), true, true },
> + /* EL2 */
> + { HV_SYS_REG_CPTR_EL2, HVF_SYSREG(1, 1, 3, 4, 2), .el2 = true },
> + { HV_SYS_REG_ELR_EL2, HVF_SYSREG(4, 0, 3, 4, 1), .el2 = true },
> + { HV_SYS_REG_ESR_EL2, HVF_SYSREG(5, 2, 3, 4, 0), .el2 = true },
> + { HV_SYS_REG_FAR_EL2, HVF_SYSREG(6, 0, 3, 4, 0), .el2 = true },
> + { HV_SYS_REG_HCR_EL2, HVF_SYSREG(1, 1, 3, 4, 0), .el2 = true },
> + { HV_SYS_REG_HPFAR_EL2, HVF_SYSREG(6, 0, 3, 4, 4), .el2 = true },
> + { HV_SYS_REG_MAIR_EL2, HVF_SYSREG(10, 2, 3, 4, 0), .el2 = true },
> + { HV_SYS_REG_MDCR_EL2, HVF_SYSREG(1, 1, 3, 4, 1), .el2 = true },
> + { HV_SYS_REG_SCTLR_EL2, HVF_SYSREG(1, 0, 3, 4, 0), .el2 = true },
> + { HV_SYS_REG_SPSR_EL2, HVF_SYSREG(4, 0, 3, 4, 0), .el2 = true },
> + { HV_SYS_REG_SP_EL2, HVF_SYSREG(4, 1, 3, 6, 0), .el2 = true},
> + { HV_SYS_REG_TCR_EL2, HVF_SYSREG(2, 0, 3, 4, 2), .el2 = true },
> + { HV_SYS_REG_TPIDR_EL2, HVF_SYSREG(13, 0, 3, 4, 2), .el2 = true },
> + { HV_SYS_REG_TTBR0_EL2, HVF_SYSREG(2, 0, 3, 4, 0), .el2 = true },
> + { HV_SYS_REG_TTBR1_EL2, HVF_SYSREG(2, 0, 3, 4, 1), .el2 = true },
> + { HV_SYS_REG_VBAR_EL2, HVF_SYSREG(12, 0, 3, 4, 0), .el2 = true },
> + { HV_SYS_REG_VMPIDR_EL2, HVF_SYSREG(0, 0, 3, 4, 5), .el2 = true },
> + { HV_SYS_REG_VPIDR_EL2, HVF_SYSREG(0, 0, 3, 4, 0), .el2 = true },
> + { HV_SYS_REG_VTCR_EL2, HVF_SYSREG(2, 1, 3, 4, 2), .el2 = true },
> + { HV_SYS_REG_VTTBR_EL2, HVF_SYSREG(2, 1, 3, 4, 0), .el2 = true },
> };
>
> int hvf_get_registers(CPUState *cpu)
> @@ -594,6 +631,14 @@ int hvf_get_registers(CPUState *cpu)
> continue;
> }
>
> + if (hvf_sreg_match[i].vgic && !hvf_irqchip_in_kernel()) {
> + continue;
> + }
> +
> + if (hvf_sreg_match[i].el2 && !hvf_arm_el2_enabled()) {
FYI this triggers an assertion when testing with my split-accel series
where we emulate EL2. Not your series issue, I need to distinct between
EL2 enabled by HW or SW.
> + continue;
> + }
> +
> if (cpu->accel->guest_debug_enabled) {
> /* Handle debug registers */
> switch (hvf_sreg_match[i].reg) {
> @@ -731,6 +776,14 @@ int hvf_put_registers(CPUState *cpu)
> continue;
> }
>
> + if (hvf_sreg_match[i].vgic && !hvf_irqchip_in_kernel()) {
> + continue;
> + }
> +
> + if (hvf_sreg_match[i].el2 && !hvf_arm_el2_enabled()) {
> + continue;
> + }
> +
> if (cpu->accel->guest_debug_enabled) {
> /* Handle debug registers */
> switch (hvf_sreg_match[i].reg) {
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v5 01/13] hvf: switch hvf_arm_get_host_cpu_features to not create a vCPU
2025-07-28 13:41 ` [PATCH v5 01/13] hvf: switch hvf_arm_get_host_cpu_features to not create a vCPU Mohamed Mediouni
@ 2025-08-06 11:12 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 24+ messages in thread
From: Philippe Mathieu-Daudé @ 2025-08-06 11:12 UTC (permalink / raw)
To: Mohamed Mediouni, qemu-devel
Cc: Mads Ynddal, Paolo Bonzini, Shannon Zhao, Phil Dennis-Jordan,
Igor Mammedov, qemu-arm, Alexander Graf, Roman Bolshakov,
Peter Maydell, Michael S. Tsirkin, Ani Sinha, Cameron Esfahani
Hi Mohamed,
On 28/7/25 15:41, Mohamed Mediouni wrote:
> Creating a vCPU locks out APIs such as hv_gic_create().
>
> As a result, switch to using the hv_vcpu_config_get_feature_reg interface.
>
> Hardcode MIDR because Apple deliberately doesn't expose a divergent MIDR across systems.
>
> Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
> ---
> target/arm/hvf/hvf.c | 35 ++++++++++++++---------------------
> 1 file changed, 14 insertions(+), 21 deletions(-)
> @@ -891,17 +891,10 @@ static bool hvf_arm_get_host_cpu_features(ARMHostCPUFeatures *ahcf)
> (1ULL << ARM_FEATURE_PMU) |
> (1ULL << ARM_FEATURE_GENERIC_TIMER);
>
> - /* We set up a small vcpu to extract host registers */
> -
> - if (hv_vcpu_create(&fd, &exit, NULL) != HV_SUCCESS) {
> - return false;
> - }
> -
> for (i = 0; i < ARRAY_SIZE(regs); i++) {
> - r |= hv_vcpu_get_sys_reg(fd, regs[i].reg, regs[i].val);
> + r |= hv_vcpu_config_get_feature_reg(hv_vcpu_config, regs[i].reg, regs[i].val);
> }
> - r |= hv_vcpu_get_sys_reg(fd, HV_SYS_REG_MIDR_EL1, &ahcf->midr);
> - r |= hv_vcpu_destroy(fd);
Could we add a comment explaining this magic value?
> + ahcf->midr = 0x610f0000;
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v5 06/13] target/arm: hvf: pass through CNTHCTL_EL2 and MDCCINT_EL1
2025-07-28 13:41 ` [PATCH v5 06/13] target/arm: hvf: pass through CNTHCTL_EL2 and MDCCINT_EL1 Mohamed Mediouni
@ 2025-08-06 11:20 ` Philippe Mathieu-Daudé
2025-08-11 10:08 ` Mads Ynddal
0 siblings, 1 reply; 24+ messages in thread
From: Philippe Mathieu-Daudé @ 2025-08-06 11:20 UTC (permalink / raw)
To: Mohamed Mediouni, qemu-devel
Cc: Mads Ynddal, Paolo Bonzini, Shannon Zhao, Phil Dennis-Jordan,
Igor Mammedov, qemu-arm, Alexander Graf, Roman Bolshakov,
Peter Maydell, Michael S. Tsirkin, Ani Sinha, Cameron Esfahani
On 28/7/25 15:41, Mohamed Mediouni wrote:
> HVF traps accesses to CNTHCTL_EL2. For nested guests, HVF traps accesses to MDCCINT_EL1.
> Pass through those accesses to the Hypervisor.framework library.
>
> Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
> ---
> target/arm/hvf/hvf.c | 16 ++++++++++++++++
> 1 file changed, 16 insertions(+)
>
> diff --git a/target/arm/hvf/hvf.c b/target/arm/hvf/hvf.c
> index f14a3a3cbd..eefae3069f 100644
> --- a/target/arm/hvf/hvf.c
> +++ b/target/arm/hvf/hvf.c
> @@ -297,6 +297,10 @@ void hvf_arm_init_debug(void)
> #define SYSREG_DBGWVR15_EL1 SYSREG(2, 0, 0, 15, 6)
> #define SYSREG_DBGWCR15_EL1 SYSREG(2, 0, 0, 15, 7)
>
> +/* EL2 registers */
> +#define SYSREG_CNTHCTL_EL2 SYSREG(3, 4, 14, 1, 0)
> +#define SYSREG_MDCCINT_EL1 SYSREG(2, 0, 0, 2, 0)
> +
> #define WFX_IS_WFE (1 << 0)
>
> #define TMR_CTL_ENABLE (1 << 0)
> @@ -1372,6 +1376,12 @@ static int hvf_sysreg_read(CPUState *cpu, uint32_t reg, uint64_t *val)
> case SYSREG_OSDLR_EL1:
> /* Dummy register */
> return 0;
> + case SYSREG_CNTHCTL_EL2:
> + assert_hvf_ok(hv_vcpu_get_sys_reg(cpu->accel->fd, HV_SYS_REG_CNTHCTL_EL2, val));
> + return 0;
We'd like to remove the assert_hvf_ok() calls, so adding more isn't
really helping. Anyhow,
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v5 11/13] hvf: gate ARM_FEATURE_PMU register emulation behind not being at EL2
2025-07-28 13:41 ` [PATCH v5 11/13] hvf: gate ARM_FEATURE_PMU register emulation behind not being " Mohamed Mediouni
@ 2025-08-06 11:21 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 24+ messages in thread
From: Philippe Mathieu-Daudé @ 2025-08-06 11:21 UTC (permalink / raw)
To: Mohamed Mediouni, qemu-devel
Cc: Mads Ynddal, Paolo Bonzini, Shannon Zhao, Phil Dennis-Jordan,
Igor Mammedov, qemu-arm, Alexander Graf, Roman Bolshakov,
Peter Maydell, Michael S. Tsirkin, Ani Sinha, Cameron Esfahani
On 28/7/25 15:41, Mohamed Mediouni wrote:
> From Apple documentation:
>
>> When EL2 is disabled, PMU register accesses trigger "Trapped MSR, MRS, or
>> System Instruction" exceptions. When this happens, hv_vcpu_run() returns, and the
>> hv_vcpu_exit_t object contains the information about this exception.
>
>> When EL2 is enabled, the handling of PMU register accesses is determined by the PMUVer
>> field of ID_AA64DFR0_EL1 register.
>> If the PMUVer field value is zero or is invalid, PMU register accesses generate "Undefined"
>> exceptions, which are sent to the guest.
>> If the PMUVer field value is non-zero and valid, PMU register accesses are emulated by the framework.
>> The ID_AA64DFR0_EL1 register can be modified via hv_vcpu_set_sys_reg API.
>
> Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
> ---
> target/arm/hvf/hvf.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v5 02/13] accel, hw/arm, include/system/hvf: infrastructure changes for HVF vGIC
2025-07-28 13:41 ` [PATCH v5 02/13] accel, hw/arm, include/system/hvf: infrastructure changes for HVF vGIC Mohamed Mediouni
@ 2025-08-06 11:32 ` Philippe Mathieu-Daudé
2025-08-07 2:03 ` Mohamed Mediouni
0 siblings, 1 reply; 24+ messages in thread
From: Philippe Mathieu-Daudé @ 2025-08-06 11:32 UTC (permalink / raw)
To: Mohamed Mediouni, qemu-devel
Cc: Mads Ynddal, Paolo Bonzini, Shannon Zhao, Phil Dennis-Jordan,
Igor Mammedov, qemu-arm, Alexander Graf, Roman Bolshakov,
Peter Maydell, Michael S. Tsirkin, Ani Sinha, Cameron Esfahani
On 28/7/25 15:41, Mohamed Mediouni wrote:
> Misc changes needed for HVF vGIC enablement.
>
> Signed-off-by: Mohamed Mediouni <mohamed@unpredictable.fr>
> ---
> accel/hvf/hvf-all.c | 44 ++++++++++++++++++++++++++++++++++++++
> accel/stubs/hvf-stub.c | 1 +
> hw/arm/virt.c | 16 +++++++++-----
> hw/intc/arm_gicv3_common.c | 3 +++
> include/system/hvf.h | 3 +++
> system/vl.c | 2 ++
> 6 files changed, 64 insertions(+), 5 deletions(-)
>
> diff --git a/accel/hvf/hvf-all.c b/accel/hvf/hvf-all.c
> index 0a4b498e83..5af76ba7a6 100644
> --- a/accel/hvf/hvf-all.c
> +++ b/accel/hvf/hvf-all.c
> @@ -10,6 +10,8 @@
>
> #include "qemu/osdep.h"
> #include "qemu/error-report.h"
> +#include "qapi/error.h"
> +#include "qapi/qapi-visit-common.h"
> #include "accel/accel-ops.h"
> #include "system/address-spaces.h"
> #include "system/memory.h"
> @@ -20,6 +22,7 @@
> #include "trace.h"
>
> bool hvf_allowed;
> +bool hvf_kernel_irqchip;
OK.
>
> struct mac_slot {
> int present;
> @@ -290,6 +293,41 @@ static int hvf_gdbstub_sstep_flags(AccelState *as)
> return SSTEP_ENABLE | SSTEP_NOIRQ;
> }
>
> +static void hvf_set_kernel_irqchip(Object *obj, Visitor *v,
> + const char *name, void *opaque,
> + Error **errp)
> +{
> +#ifdef __aarch64__
We try to avoid such #ifdef'ry...
> + OnOffSplit mode;
> + if (!visit_type_OnOffSplit(v, name, &mode, errp)) {
> + return;
> + }
> +
> + switch (mode) {
> + case ON_OFF_SPLIT_ON:
> + hvf_kernel_irqchip = true;
> + break;
> +
> + case ON_OFF_SPLIT_OFF:
> + hvf_kernel_irqchip = false;
> + break;
> +
> + case ON_OFF_SPLIT_SPLIT:
> + error_setg(errp, "HVF: split irqchip is not supported on Arm.");
> + break;
> +
> + default:
> + /*
> + * The value was checked in visit_type_OnOffSplit() above. If
> + * we get here, then something is wrong in QEMU.
> + */
> + abort();
> + }
> +#else
> + error_setg(errp, "HVF: setting irqchip configuration not supported on x86_64.");
> +#endif
> +}
> +
> static void hvf_accel_class_init(ObjectClass *oc, const void *data)
> {
> AccelClass *ac = ACCEL_CLASS(oc);
> @@ -297,6 +335,12 @@ static void hvf_accel_class_init(ObjectClass *oc, const void *data)
> ac->init_machine = hvf_accel_init;
> ac->allowed = &hvf_allowed;
> ac->gdbstub_supported_sstep_flags = hvf_gdbstub_sstep_flags;
... better add hvf_arch_register_class_properties(), empty stub
on x86 and filled in aarch64.
> + hvf_kernel_irqchip = true;
> + object_class_property_add(oc, "kernel-irqchip", "on|off|split",
> + NULL, hvf_set_kernel_irqchip,
> + NULL, NULL);
> + object_class_property_set_description(oc, "kernel-irqchip",
> + "Configure HVF irqchip");
> }
> diff --git a/system/vl.c b/system/vl.c
> index 3b7057e6c6..1c072d15a4 100644
> --- a/system/vl.c
> +++ b/system/vl.c
> @@ -1773,6 +1773,8 @@ static void qemu_apply_legacy_machine_options(QDict *qdict)
> false);
> object_register_sugar_prop(ACCEL_CLASS_NAME("whpx"), "kernel-irqchip", value,
> false);
> + object_register_sugar_prop(ACCEL_CLASS_NAME("hvf"), "kernel-irqchip", value,
> + false);
Discussing about this object_register_sugar_prop() call on IRC, Paolo
said:
> It's okay the property already exists for KVM so it's okay to add
> the synonym for HVF as well
> Maybe make sure it's not registered for x86?
> qdict_del(qdict, "kernel-irqchip");
> }
>
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v5 02/13] accel, hw/arm, include/system/hvf: infrastructure changes for HVF vGIC
2025-08-06 11:32 ` Philippe Mathieu-Daudé
@ 2025-08-07 2:03 ` Mohamed Mediouni
0 siblings, 0 replies; 24+ messages in thread
From: Mohamed Mediouni @ 2025-08-07 2:03 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: qemu-devel, Mads Ynddal, Paolo Bonzini, Shannon Zhao,
Phil Dennis-Jordan, Igor Mammedov, qemu-arm, Alexander Graf,
Roman Bolshakov, Peter Maydell, Michael S. Tsirkin, Ani Sinha,
Cameron Esfahani
[-- Attachment #1: Type: text/plain, Size: 460 bytes --]
> On 6. Aug 2025, at 13:32, Philippe Mathieu-Daudé <philmd@linaro.org> wrote:
>
> ... better add hvf_arch_register_class_properties(), empty stub
> on x86 and filled in aarch64.
Would it be fine to have the setting still without it doing anything right now?
(For reference, Apple added APIC APIs in macOS 12 on x86 - which we might possibly want to add later or not…)
Or might as well implement support for the HVF APIC in the same series (?)
[-- Attachment #2: Type: text/html, Size: 2257 bytes --]
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v5 06/13] target/arm: hvf: pass through CNTHCTL_EL2 and MDCCINT_EL1
2025-08-06 11:20 ` Philippe Mathieu-Daudé
@ 2025-08-11 10:08 ` Mads Ynddal
2025-08-11 10:22 ` Peter Maydell
0 siblings, 1 reply; 24+ messages in thread
From: Mads Ynddal @ 2025-08-11 10:08 UTC (permalink / raw)
To: Philippe Mathieu-Daudé
Cc: Mohamed Mediouni, qemu-devel, Paolo Bonzini, Shannon Zhao,
Phil Dennis-Jordan, Igor Mammedov, qemu-arm, Alexander Graf,
Roman Bolshakov, Peter Maydell, Michael S. Tsirkin, Ani Sinha,
Cameron Esfahani
> We'd like to remove the assert_hvf_ok() calls, so adding more isn't
> really helping. Anyhow,
>
> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
What is the preferred method going forward?
Apple designed the HV API to be able to fail at every function, and it's
rarely something that can be recovered from.
In [PATCH v4 04/15] of this series, we saw an issue that was hidden
because the return code was not properly checked (not calling from the
owning thread). Previously, I submitted a patch (d5bd8d8267) for the
same issue, because Apple had changed a function's behavior. This was
caught by an assert_hvf_ok.
I agree with you, that generally adding asserts is a bad design, but at
the same time, HV is designed in a way, that the code would be littered
with checks otherwise.
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v5 06/13] target/arm: hvf: pass through CNTHCTL_EL2 and MDCCINT_EL1
2025-08-11 10:08 ` Mads Ynddal
@ 2025-08-11 10:22 ` Peter Maydell
2025-08-12 11:29 ` Mads Ynddal
0 siblings, 1 reply; 24+ messages in thread
From: Peter Maydell @ 2025-08-11 10:22 UTC (permalink / raw)
To: Mads Ynddal
Cc: Philippe Mathieu-Daudé, Mohamed Mediouni, qemu-devel,
Paolo Bonzini, Shannon Zhao, Phil Dennis-Jordan, Igor Mammedov,
qemu-arm, Alexander Graf, Roman Bolshakov, Michael S. Tsirkin,
Ani Sinha, Cameron Esfahani
On Mon, 11 Aug 2025 at 11:08, Mads Ynddal <mads@ynddal.dk> wrote:
>
>
> > We'd like to remove the assert_hvf_ok() calls, so adding more isn't
> > really helping. Anyhow,
> >
> > Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> > Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
>
> What is the preferred method going forward?
>
> Apple designed the HV API to be able to fail at every function, and it's
> rarely something that can be recovered from.
>
> In [PATCH v4 04/15] of this series, we saw an issue that was hidden
> because the return code was not properly checked (not calling from the
> owning thread). Previously, I submitted a patch (d5bd8d8267) for the
> same issue, because Apple had changed a function's behavior. This was
> caught by an assert_hvf_ok.
>
> I agree with you, that generally adding asserts is a bad design, but at
> the same time, HV is designed in a way, that the code would be littered
> with checks otherwise.
This suggestion was I think mine, and I think partly I was
misled by the name of assert_hvf_ok(), because in fact it
isn't an assert(). (assert() should be specifically "if we
hit this this is a bug in QEMU", and "hvf returned an error"
is generally not that. It does call abort(), though, which
isn't much better.)
But also I think the existence of assert_hvf_ok() encourages
it to be used in cases where actually we should be doing
something more sensible with an error return. For instance,
in hvf_accel_init() if we can't init HVF then we should
return an error code to the caller, which might fall back
to the TCG accelerator -- but instead we have an assert_hvf_ok(),
so fallback won't work because we'll just bomb out.
The KVM API is also one where any API call (ioctl) can return an
error, but we don't generally assert() that they succeeded, except
in a few cases of "given where we are, for this to fail would
mean the kernel was broken". Instead we propagate error values
upwards when the function has a mechanism to do that,
and where appropriate we print an error message that's
hopefully reasonably comprehensible to the user and exit.
-- PMM
^ permalink raw reply [flat|nested] 24+ messages in thread
* Re: [PATCH v5 06/13] target/arm: hvf: pass through CNTHCTL_EL2 and MDCCINT_EL1
2025-08-11 10:22 ` Peter Maydell
@ 2025-08-12 11:29 ` Mads Ynddal
0 siblings, 0 replies; 24+ messages in thread
From: Mads Ynddal @ 2025-08-12 11:29 UTC (permalink / raw)
To: Peter Maydell
Cc: Philippe Mathieu-Daudé, Mohamed Mediouni, qemu-devel,
Paolo Bonzini, Shannon Zhao, Phil Dennis-Jordan, Igor Mammedov,
qemu-arm, Alexander Graf, Roman Bolshakov, Michael S. Tsirkin,
Ani Sinha, Cameron Esfahani
>>> We'd like to remove the assert_hvf_ok() calls, so adding more isn't
>>> really helping. Anyhow,
>>>
>>> Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
>>> Tested-by: Philippe Mathieu-Daudé <philmd@linaro.org>
>>
>> What is the preferred method going forward?
>>
>> Apple designed the HV API to be able to fail at every function, and it's
>> rarely something that can be recovered from.
>>
>> In [PATCH v4 04/15] of this series, we saw an issue that was hidden
>> because the return code was not properly checked (not calling from the
>> owning thread). Previously, I submitted a patch (d5bd8d8267) for the
>> same issue, because Apple had changed a function's behavior. This was
>> caught by an assert_hvf_ok.
>>
>> I agree with you, that generally adding asserts is a bad design, but at
>> the same time, HV is designed in a way, that the code would be littered
>> with checks otherwise.
>
> This suggestion was I think mine, and I think partly I was
> misled by the name of assert_hvf_ok(), because in fact it
> isn't an assert(). (assert() should be specifically "if we
> hit this this is a bug in QEMU", and "hvf returned an error"
> is generally not that. It does call abort(), though, which
> isn't much better.)
>
> But also I think the existence of assert_hvf_ok() encourages
> it to be used in cases where actually we should be doing
> something more sensible with an error return. For instance,
> in hvf_accel_init() if we can't init HVF then we should
> return an error code to the caller, which might fall back
> to the TCG accelerator -- but instead we have an assert_hvf_ok(),
> so fallback won't work because we'll just bomb out.
>
> The KVM API is also one where any API call (ioctl) can return an
> error, but we don't generally assert() that they succeeded, except
> in a few cases of "given where we are, for this to fail would
> mean the kernel was broken". Instead we propagate error values
> upwards when the function has a mechanism to do that,
> and where appropriate we print an error message that's
> hopefully reasonably comprehensible to the user and exit.
>
> -- PMM
Thanks for the clarification. I think I understand the sentiment.
Propagating the error codes when possible, is much preferred. But
letting the calls go unchecked is problematic too, so I don't know if we
can get around assert_hvf_ok in some shape or form.
You're right, that it might encourage use as an easy fix. I'll keep an
eye on it in my reviews.
^ permalink raw reply [flat|nested] 24+ messages in thread
end of thread, other threads:[~2025-08-12 11:31 UTC | newest]
Thread overview: 24+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-07-28 13:41 [PATCH v5 00/13] HVF: Add support for platform vGIC and nested virtualisation Mohamed Mediouni
2025-07-28 13:41 ` [PATCH v5 01/13] hvf: switch hvf_arm_get_host_cpu_features to not create a vCPU Mohamed Mediouni
2025-08-06 11:12 ` Philippe Mathieu-Daudé
2025-07-28 13:41 ` [PATCH v5 02/13] accel, hw/arm, include/system/hvf: infrastructure changes for HVF vGIC Mohamed Mediouni
2025-08-06 11:32 ` Philippe Mathieu-Daudé
2025-08-07 2:03 ` Mohamed Mediouni
2025-07-28 13:41 ` [PATCH v5 03/13] hw/intc: Add hvf vGIC interrupt controller support Mohamed Mediouni
2025-08-06 10:32 ` Philippe Mathieu-Daudé
2025-07-28 13:41 ` [PATCH v5 04/13] hw/arm, target/arm: nested virtualisation on HVF Mohamed Mediouni
2025-07-28 13:41 ` [PATCH v5 05/13] hvf: save/restore Apple GIC state Mohamed Mediouni
2025-07-28 13:41 ` [PATCH v5 06/13] target/arm: hvf: pass through CNTHCTL_EL2 and MDCCINT_EL1 Mohamed Mediouni
2025-08-06 11:20 ` Philippe Mathieu-Daudé
2025-08-11 10:08 ` Mads Ynddal
2025-08-11 10:22 ` Peter Maydell
2025-08-12 11:29 ` Mads Ynddal
2025-07-28 13:41 ` [PATCH v5 07/13] hw/arm: virt: add GICv2m for the case when ITS is not available Mohamed Mediouni
2025-07-28 13:41 ` [PATCH v5 08/13] hw/arm: virt: cleanly fail on attempt to use the platform vGIC together with ITS Mohamed Mediouni
2025-07-28 13:41 ` [PATCH v5 09/13] hvf: only call hvf_sync_vtimer() when running without the platform vGIC Mohamed Mediouni
2025-07-28 13:41 ` [PATCH v5 10/13] hvf: sync registers used at EL2 Mohamed Mediouni
2025-08-06 11:10 ` Philippe Mathieu-Daudé
2025-07-28 13:41 ` [PATCH v5 11/13] hvf: gate ARM_FEATURE_PMU register emulation behind not being " Mohamed Mediouni
2025-08-06 11:21 ` Philippe Mathieu-Daudé
2025-07-28 13:41 ` [PATCH v5 12/13] target/arm: hvf: instantiate GIC early Mohamed Mediouni
2025-07-28 13:41 ` [PATCH v5 13/13] target/arm: hvf: add asserts for code paths not leveraged when using the vGIC Mohamed Mediouni
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).