[RFC PATCH 00/19] accel: Preparatory cleanups for split-accel

[RFC PATCH 00/19] accel: Preparatory cleanups for split-accel

[Philippe Mathieu-Daudé]

Few changes needed before being able to add the
split acceleration. Mostly adapting few prototypes
so callees have all necessary information.

Philippe Mathieu-Daudé (19):
  hw/arm/virt: Only require TCG || QTest to use virtualization extension
  system/cpus: Only kick running vCPUs
  accel: Keep reference to AccelOpsClass in AccelClass
  accel: Propagate AccelState to AccelClass::init_machine()
  accel/kvm: Prefer local AccelState over global MachineState::accel
  accel/hvf: Fix TYPE_HVF_ACCEL instance size
  accel/hvf: Re-use QOM allocated state
  accel/tcg: Prefer local AccelState over global current_accel()
  accel: Factor accel_cpu_realize() out
  accel/dummy: Factor dummy_thread_precreate() out
  accel/dummy: Factor tcg_vcpu_thread_precreate() out
  accel: Factor accel_create_vcpu_thread() out
  accel: Introduce AccelOpsClass::cpu_thread_routine handler
  accel/dummy: Convert to AccelOpsClass::cpu_thread_routine
  accel/tcg: Convert to AccelOpsClass::cpu_thread_routine
  accel/hvf: Convert to AccelOpsClass::cpu_thread_routine
  accel/kvm: Convert to AccelOpsClass::cpu_thread_routine
  accel/nvmm: Convert to AccelOpsClass::cpu_thread_routine
  accel/whpx: Convert to AccelOpsClass::cpu_thread_routine

 accel/accel-internal.h            |  2 ++
 accel/tcg/tcg-accel-ops-mttcg.h   |  3 +-
 accel/tcg/tcg-accel-ops.h         |  1 +
 include/qemu/accel.h              |  8 ++++-
 include/system/accel-ops.h        |  5 +++-
 include/system/cpus.h             |  5 ++--
 include/system/hvf_int.h          |  1 +
 accel/accel-common.c              | 49 +++++++++++++++++++++++++++++--
 accel/accel-system.c              |  5 ++--
 accel/dummy-cpus.c                | 10 ++-----
 accel/hvf/hvf-accel-ops.c         | 25 +++-------------
 accel/kvm/kvm-accel-ops.c         | 12 +-------
 accel/kvm/kvm-all.c               |  6 ++--
 accel/qtest/qtest.c               |  5 ++--
 accel/tcg/tcg-accel-ops-mttcg.c   | 17 +----------
 accel/tcg/tcg-accel-ops-rr.c      |  3 +-
 accel/tcg/tcg-accel-ops.c         | 14 +++++++--
 accel/tcg/tcg-all.c               |  4 +--
 accel/xen/xen-all.c               |  3 +-
 bsd-user/main.c                   |  2 +-
 hw/arm/virt.c                     |  2 +-
 linux-user/main.c                 |  2 +-
 system/cpus.c                     |  9 ++++--
 target/i386/nvmm/nvmm-accel-ops.c | 12 +-------
 target/i386/nvmm/nvmm-all.c       |  2 +-
 target/i386/whpx/whpx-accel-ops.c | 12 +-------
 target/i386/whpx/whpx-all.c       |  2 +-
 27 files changed, 110 insertions(+), 111 deletions(-)

-- 
2.49.0

[RFC PATCH 01/19] hw/arm/virt: Only require TCG || QTest to use virtualization extension

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 hw/arm/virt.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/hw/arm/virt.c b/hw/arm/virt.c
index 9a6cd085a37..d55ce2c0f4e 100644
--- a/hw/arm/virt.c
+++ b/hw/arm/virt.c
@@ -2205,7 +2205,7 @@ static void machvirt_init(MachineState *machine)
         exit(1);
     }
 
-    if (vms->virt && (kvm_enabled() || hvf_enabled())) {
+    if (vms->virt && !tcg_enabled() && !qtest_enabled()) {
         error_report("mach-virt: %s does not support providing "
                      "Virtualization extensions to the guest CPU",
                      current_accel_name());
-- 
2.49.0

[RFC PATCH 02/19] system/cpus: Only kick running vCPUs

[Philippe Mathieu-Daudé]

As an optimization, avoid kicking stopped vCPUs.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 system/cpus.c | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/system/cpus.c b/system/cpus.c
index d16b0dff989..4835e5ced48 100644
--- a/system/cpus.c
+++ b/system/cpus.c
@@ -494,6 +494,11 @@ void cpus_kick_thread(CPUState *cpu)
 void qemu_cpu_kick(CPUState *cpu)
 {
     qemu_cond_broadcast(cpu->halt_cond);
+
+    if (!cpu_can_run(cpu)) {
+        return;
+    }
+
     if (cpus_accel->kick_vcpu_thread) {
         cpus_accel->kick_vcpu_thread(cpu);
     } else { /* default */
-- 
2.49.0

[RFC PATCH 03/19] accel: Keep reference to AccelOpsClass in AccelClass

[Philippe Mathieu-Daudé]

Allow dereferencing AccelOpsClass outside of accel/accel-system.c.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/qemu/accel.h       | 2 ++
 include/system/accel-ops.h | 3 ++-
 accel/accel-system.c       | 3 ++-
 accel/tcg/tcg-accel-ops.c  | 4 +++-
 4 files changed, 9 insertions(+), 3 deletions(-)

diff --git a/include/qemu/accel.h b/include/qemu/accel.h
index fbd3d897fef..23c62cbeb07 100644
--- a/include/qemu/accel.h
+++ b/include/qemu/accel.h
@@ -37,6 +37,8 @@ typedef struct AccelClass {
     /*< public >*/
 
     const char *name;
+    AccelOpsClass *ops;
+
     int (*init_machine)(MachineState *ms);
     bool (*cpu_common_realize)(CPUState *cpu, Error **errp);
     void (*cpu_common_unrealize)(CPUState *cpu);
diff --git a/include/system/accel-ops.h b/include/system/accel-ops.h
index 4c99d25aeff..44b37592d02 100644
--- a/include/system/accel-ops.h
+++ b/include/system/accel-ops.h
@@ -10,6 +10,7 @@
 #ifndef ACCEL_OPS_H
 #define ACCEL_OPS_H
 
+#include "qemu/accel.h"
 #include "exec/vaddr.h"
 #include "qom/object.h"
 
@@ -31,7 +32,7 @@ struct AccelOpsClass {
     /*< public >*/
 
     /* initialization function called when accel is chosen */
-    void (*ops_init)(AccelOpsClass *ops);
+    void (*ops_init)(AccelClass *ac);
 
     bool (*cpus_are_resettable)(void);
     void (*cpu_reset_hold)(CPUState *cpu);
diff --git a/accel/accel-system.c b/accel/accel-system.c
index a0f562ae9ff..64bc991b1ce 100644
--- a/accel/accel-system.c
+++ b/accel/accel-system.c
@@ -85,8 +85,9 @@ void accel_init_ops_interfaces(AccelClass *ac)
      * non-NULL create_vcpu_thread operation.
      */
     ops = ACCEL_OPS_CLASS(oc);
+    ac->ops = ops;
     if (ops->ops_init) {
-        ops->ops_init(ops);
+        ops->ops_init(ac);
     }
     cpus_register_accel(ops);
 }
diff --git a/accel/tcg/tcg-accel-ops.c b/accel/tcg/tcg-accel-ops.c
index b24d6a75625..da2e22a7dff 100644
--- a/accel/tcg/tcg-accel-ops.c
+++ b/accel/tcg/tcg-accel-ops.c
@@ -198,8 +198,10 @@ static inline void tcg_remove_all_breakpoints(CPUState *cpu)
     cpu_watchpoint_remove_all(cpu, BP_GDB);
 }
 
-static void tcg_accel_ops_init(AccelOpsClass *ops)
+static void tcg_accel_ops_init(AccelClass *ac)
 {
+    AccelOpsClass *ops = ac->ops;
+
     if (qemu_tcg_mttcg_enabled()) {
         ops->create_vcpu_thread = mttcg_start_vcpu_thread;
         ops->kick_vcpu_thread = mttcg_kick_vcpu_thread;
-- 
2.49.0

[RFC PATCH 04/19] accel: Propagate AccelState to AccelClass::init_machine()

[Philippe Mathieu-Daudé]

In order to avoid init_machine() to call current_accel(),
pass AccelState along.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/qemu/accel.h        | 2 +-
 accel/accel-system.c        | 2 +-
 accel/hvf/hvf-accel-ops.c   | 2 +-
 accel/kvm/kvm-all.c         | 2 +-
 accel/qtest/qtest.c         | 2 +-
 accel/tcg/tcg-all.c         | 2 +-
 bsd-user/main.c             | 2 +-
 linux-user/main.c           | 2 +-
 target/i386/nvmm/nvmm-all.c | 2 +-
 target/i386/whpx/whpx-all.c | 2 +-
 10 files changed, 10 insertions(+), 10 deletions(-)

diff --git a/include/qemu/accel.h b/include/qemu/accel.h
index 23c62cbeb07..c660c5f4b11 100644
--- a/include/qemu/accel.h
+++ b/include/qemu/accel.h
@@ -39,7 +39,7 @@ typedef struct AccelClass {
     const char *name;
     AccelOpsClass *ops;
 
-    int (*init_machine)(MachineState *ms);
+    int (*init_machine)(MachineState *ms, AccelState *as);
     bool (*cpu_common_realize)(CPUState *cpu, Error **errp);
     void (*cpu_common_unrealize)(CPUState *cpu);
 
diff --git a/accel/accel-system.c b/accel/accel-system.c
index 64bc991b1ce..68d2f28388b 100644
--- a/accel/accel-system.c
+++ b/accel/accel-system.c
@@ -37,7 +37,7 @@ int accel_init_machine(AccelState *accel, MachineState *ms)
     int ret;
     ms->accelerator = accel;
     *(acc->allowed) = true;
-    ret = acc->init_machine(ms);
+    ret = acc->init_machine(ms, accel);
     if (ret < 0) {
         ms->accelerator = NULL;
         *(acc->allowed) = false;
diff --git a/accel/hvf/hvf-accel-ops.c b/accel/hvf/hvf-accel-ops.c
index b8b6116bc84..3dd9de26dbb 100644
--- a/accel/hvf/hvf-accel-ops.c
+++ b/accel/hvf/hvf-accel-ops.c
@@ -315,7 +315,7 @@ static void dummy_signal(int sig)
 
 bool hvf_allowed;
 
-static int hvf_accel_init(MachineState *ms)
+static int hvf_accel_init(MachineState *ms, AccelState *as)
 {
     int x;
     hv_return_t ret;
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index a31778341c2..5b28b15ccec 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -2567,7 +2567,7 @@ static int kvm_setup_dirty_ring(KVMState *s)
     return 0;
 }
 
-static int kvm_init(MachineState *ms)
+static int kvm_init(MachineState *ms, AccelState *as)
 {
     MachineClass *mc = MACHINE_GET_CLASS(ms);
     static const char upgrade_note[] =
diff --git a/accel/qtest/qtest.c b/accel/qtest/qtest.c
index 92bed9264ce..c5a53ecb6dd 100644
--- a/accel/qtest/qtest.c
+++ b/accel/qtest/qtest.c
@@ -37,7 +37,7 @@ static void qtest_set_virtual_clock(int64_t count)
     qatomic_set_i64(&qtest_clock_counter, count);
 }
 
-static int qtest_init_accel(MachineState *ms)
+static int qtest_init_accel(MachineState *ms, AccelState *as)
 {
     return 0;
 }
diff --git a/accel/tcg/tcg-all.c b/accel/tcg/tcg-all.c
index 6e5dc333d59..6c5979861cf 100644
--- a/accel/tcg/tcg-all.c
+++ b/accel/tcg/tcg-all.c
@@ -80,7 +80,7 @@ static void tcg_accel_instance_init(Object *obj)
 
 bool one_insn_per_tb;
 
-static int tcg_init_machine(MachineState *ms)
+static int tcg_init_machine(MachineState *ms, AccelState *as)
 {
     TCGState *s = TCG_STATE(current_accel());
     unsigned max_threads = 1;
diff --git a/bsd-user/main.c b/bsd-user/main.c
index 7c0a059c3ba..ad2032a8b01 100644
--- a/bsd-user/main.c
+++ b/bsd-user/main.c
@@ -474,7 +474,7 @@ int main(int argc, char **argv)
                                  opt_one_insn_per_tb, &error_abort);
         object_property_set_int(OBJECT(accel), "tb-size",
                                 opt_tb_size, &error_abort);
-        ac->init_machine(NULL);
+        ac->init_machine(NULL, accel);
     }
 
     /*
diff --git a/linux-user/main.c b/linux-user/main.c
index 5ac5b55dc65..2441e6edc71 100644
--- a/linux-user/main.c
+++ b/linux-user/main.c
@@ -820,7 +820,7 @@ int main(int argc, char **argv, char **envp)
                                  opt_one_insn_per_tb, &error_abort);
         object_property_set_int(OBJECT(accel), "tb-size",
                                 opt_tb_size, &error_abort);
-        ac->init_machine(NULL);
+        ac->init_machine(NULL, accel);
     }
 
     /*
diff --git a/target/i386/nvmm/nvmm-all.c b/target/i386/nvmm/nvmm-all.c
index f1c6120ccf1..887534d4ca6 100644
--- a/target/i386/nvmm/nvmm-all.c
+++ b/target/i386/nvmm/nvmm-all.c
@@ -1153,7 +1153,7 @@ static struct RAMBlockNotifier nvmm_ram_notifier = {
 /* -------------------------------------------------------------------------- */
 
 static int
-nvmm_accel_init(MachineState *ms)
+nvmm_accel_init(MachineState *ms, AccelState *as)
 {
     int ret, err;
 
diff --git a/target/i386/whpx/whpx-all.c b/target/i386/whpx/whpx-all.c
index cf6d3e4cdd4..238c3b95c6f 100644
--- a/target/i386/whpx/whpx-all.c
+++ b/target/i386/whpx/whpx-all.c
@@ -2505,7 +2505,7 @@ static void whpx_set_kernel_irqchip(Object *obj, Visitor *v,
  * Partition support
  */
 
-static int whpx_accel_init(MachineState *ms)
+static int whpx_accel_init(MachineState *ms, AccelState *as)
 {
     struct whpx_state *whpx;
     int ret;
-- 
2.49.0

[RFC PATCH 05/19] accel/kvm: Prefer local AccelState over global MachineState::accel

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/kvm/kvm-all.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 5b28b15ccec..551a462926d 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -2582,15 +2582,13 @@ static int kvm_init(MachineState *ms, AccelState *as)
         { /* end of list */ }
     }, *nc = num_cpus;
     int soft_vcpus_limit, hard_vcpus_limit;
-    KVMState *s;
+    KVMState *s = KVM_STATE(as);
     const KVMCapabilityInfo *missing_cap;
     int ret;
     int type;
 
     qemu_mutex_init(&kml_slots_lock);
 
-    s = KVM_STATE(ms->accelerator);
-
     /*
      * On systems where the kernel can support different base page
      * sizes, host page size may be different from TARGET_PAGE_SIZE,
-- 
2.49.0

[RFC PATCH 06/19] accel/hvf: Fix TYPE_HVF_ACCEL instance size

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/system/hvf_int.h  | 1 +
 accel/hvf/hvf-accel-ops.c | 1 +
 2 files changed, 2 insertions(+)

diff --git a/include/system/hvf_int.h b/include/system/hvf_int.h
index 8c8b84012d9..d774e58df91 100644
--- a/include/system/hvf_int.h
+++ b/include/system/hvf_int.h
@@ -44,6 +44,7 @@ typedef struct hvf_vcpu_caps {
 
 struct HVFState {
     AccelState parent;
+
     hvf_slot slots[32];
     int num_slots;
 
diff --git a/accel/hvf/hvf-accel-ops.c b/accel/hvf/hvf-accel-ops.c
index 3dd9de26dbb..808ecea3816 100644
--- a/accel/hvf/hvf-accel-ops.c
+++ b/accel/hvf/hvf-accel-ops.c
@@ -366,6 +366,7 @@ static void hvf_accel_class_init(ObjectClass *oc, const void *data)
 static const TypeInfo hvf_accel_type = {
     .name = TYPE_HVF_ACCEL,
     .parent = TYPE_ACCEL,
+    .instance_size = sizeof(HVFState),
     .class_init = hvf_accel_class_init,
 };
 
-- 
2.49.0

[RFC PATCH 07/19] accel/hvf: Re-use QOM allocated state

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/hvf/hvf-accel-ops.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)

diff --git a/accel/hvf/hvf-accel-ops.c b/accel/hvf/hvf-accel-ops.c
index 808ecea3816..f4a983d1328 100644
--- a/accel/hvf/hvf-accel-ops.c
+++ b/accel/hvf/hvf-accel-ops.c
@@ -319,7 +319,7 @@ static int hvf_accel_init(MachineState *ms, AccelState *as)
 {
     int x;
     hv_return_t ret;
-    HVFState *s;
+    HVFState *s = HVF_STATE(as);
     int pa_range = 36;
     MachineClass *mc = MACHINE_GET_CLASS(ms);
 
@@ -333,8 +333,6 @@ static int hvf_accel_init(MachineState *ms, AccelState *as)
     ret = hvf_arch_vm_create(ms, (uint32_t)pa_range);
     assert_hvf_ok(ret);
 
-    s = g_new0(HVFState, 1);
-
     s->num_slots = ARRAY_SIZE(s->slots);
     for (x = 0; x < s->num_slots; ++x) {
         s->slots[x].size = 0;
-- 
2.49.0

[RFC PATCH 08/19] accel/tcg: Prefer local AccelState over global current_accel()

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/tcg/tcg-all.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/accel/tcg/tcg-all.c b/accel/tcg/tcg-all.c
index 6c5979861cf..4b1238ed345 100644
--- a/accel/tcg/tcg-all.c
+++ b/accel/tcg/tcg-all.c
@@ -82,7 +82,7 @@ bool one_insn_per_tb;
 
 static int tcg_init_machine(MachineState *ms, AccelState *as)
 {
-    TCGState *s = TCG_STATE(current_accel());
+    TCGState *s = TCG_STATE(as);
     unsigned max_threads = 1;
 
 #ifndef CONFIG_USER_ONLY
-- 
2.49.0

[RFC PATCH 09/19] accel: Factor accel_cpu_realize() out

[Philippe Mathieu-Daudé]

Factor accel_cpu_realize() out of accel_cpu_common_realize()
for re-use.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/accel-internal.h |  2 ++
 include/qemu/accel.h   |  2 ++
 accel/accel-common.c   | 15 ++++++++++++---
 3 files changed, 16 insertions(+), 3 deletions(-)

diff --git a/accel/accel-internal.h b/accel/accel-internal.h
index d3a4422cbf7..b541377c349 100644
--- a/accel/accel-internal.h
+++ b/accel/accel-internal.h
@@ -14,4 +14,6 @@
 
 void accel_init_ops_interfaces(AccelClass *ac);
 
+bool accel_cpu_realize(AccelState *accel, CPUState *cpu, Error **errp);
+
 #endif /* ACCEL_SYSTEM_H */
diff --git a/include/qemu/accel.h b/include/qemu/accel.h
index c660c5f4b11..3c9aaf9523c 100644
--- a/include/qemu/accel.h
+++ b/include/qemu/accel.h
@@ -91,6 +91,8 @@ void accel_setup_post(MachineState *ms);
  */
 void accel_cpu_instance_init(CPUState *cpu);
 
+bool accel_cpu_realize(AccelState *accel, CPUState *cpu, Error **errp);
+
 /**
  * accel_cpu_common_realize:
  * @cpu: The CPU that needs to call accel-specific cpu realization.
diff --git a/accel/accel-common.c b/accel/accel-common.c
index 4894b98d64a..4f3b42e7112 100644
--- a/accel/accel-common.c
+++ b/accel/accel-common.c
@@ -88,10 +88,14 @@ void accel_cpu_instance_init(CPUState *cpu)
     }
 }
 
-bool accel_cpu_common_realize(CPUState *cpu, Error **errp)
+bool accel_cpu_realize(AccelState *accel, CPUState *cpu, Error **errp)
 {
-    AccelState *accel = current_accel();
-    AccelClass *acc = ACCEL_GET_CLASS(accel);
+    AccelClass *acc;
+
+    if (!accel) {
+        accel = current_accel();
+    }
+    acc = ACCEL_GET_CLASS(accel);
 
     /* target specific realization */
     if (cpu->cc->accel_cpu
@@ -108,6 +112,11 @@ bool accel_cpu_common_realize(CPUState *cpu, Error **errp)
     return true;
 }
 
+bool accel_cpu_common_realize(CPUState *cpu, Error **errp)
+{
+    return accel_cpu_realize(NULL, cpu, errp);
+}
+
 void accel_cpu_common_unrealize(CPUState *cpu)
 {
     AccelState *accel = current_accel();
-- 
2.49.0

[RFC PATCH 10/19] accel/dummy: Factor dummy_thread_precreate() out

[Philippe Mathieu-Daudé]

Initialize the semaphore before creating the thread,
factor out as dummy_thread_precreate().

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/system/cpus.h |  1 +
 accel/dummy-cpus.c    | 12 +++++++++---
 2 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/include/system/cpus.h b/include/system/cpus.h
index 3226c765d01..bfaa339dd73 100644
--- a/include/system/cpus.h
+++ b/include/system/cpus.h
@@ -8,6 +8,7 @@ void cpus_register_accel(const AccelOpsClass *i);
 const AccelOpsClass *cpus_get_accel(void);
 
 /* accel/dummy-cpus.c */
+void dummy_thread_precreate(CPUState *cpu);
 
 /* Create a dummy vcpu for AccelOpsClass->create_vcpu_thread */
 void dummy_start_vcpu_thread(CPUState *);
diff --git a/accel/dummy-cpus.c b/accel/dummy-cpus.c
index 867276144fa..7c34e6c0fc5 100644
--- a/accel/dummy-cpus.c
+++ b/accel/dummy-cpus.c
@@ -64,15 +64,21 @@ static void *dummy_cpu_thread_fn(void *arg)
     return NULL;
 }
 
+void dummy_thread_precreate(CPUState *cpu)
+{
+#ifdef _WIN32
+    qemu_sem_init(&cpu->sem, 0);
+#endif
+}
+
 void dummy_start_vcpu_thread(CPUState *cpu)
 {
     char thread_name[VCPU_THREAD_NAME_SIZE];
 
+    dummy_thread_precreate(cpu);
+
     snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/DUMMY",
              cpu->cpu_index);
     qemu_thread_create(cpu->thread, thread_name, dummy_cpu_thread_fn, cpu,
                        QEMU_THREAD_JOINABLE);
-#ifdef _WIN32
-    qemu_sem_init(&cpu->sem, 0);
-#endif
 }
-- 
2.49.0

[RFC PATCH 11/19] accel/dummy: Factor tcg_vcpu_thread_precreate() out

[Philippe Mathieu-Daudé]

Factor tcg_vcpu_thread_precreate() out for re-use.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/tcg/tcg-accel-ops.h       | 1 +
 accel/tcg/tcg-accel-ops-mttcg.c | 3 +--
 accel/tcg/tcg-accel-ops-rr.c    | 3 +--
 accel/tcg/tcg-accel-ops.c       | 7 +++++++
 4 files changed, 10 insertions(+), 4 deletions(-)

diff --git a/accel/tcg/tcg-accel-ops.h b/accel/tcg/tcg-accel-ops.h
index 6feeb3f3e9b..129af89c3e7 100644
--- a/accel/tcg/tcg-accel-ops.h
+++ b/accel/tcg/tcg-accel-ops.h
@@ -14,6 +14,7 @@
 
 #include "system/cpus.h"
 
+void tcg_vcpu_thread_precreate(CPUState *cpu);
 void tcg_cpu_destroy(CPUState *cpu);
 int tcg_cpu_exec(CPUState *cpu);
 void tcg_handle_interrupt(CPUState *cpu, int mask);
diff --git a/accel/tcg/tcg-accel-ops-mttcg.c b/accel/tcg/tcg-accel-ops-mttcg.c
index dfcee30947e..462be7596b9 100644
--- a/accel/tcg/tcg-accel-ops-mttcg.c
+++ b/accel/tcg/tcg-accel-ops-mttcg.c
@@ -133,8 +133,7 @@ void mttcg_start_vcpu_thread(CPUState *cpu)
 {
     char thread_name[VCPU_THREAD_NAME_SIZE];
 
-    g_assert(tcg_enabled());
-    tcg_cpu_init_cflags(cpu, current_machine->smp.max_cpus > 1);
+    tcg_vcpu_thread_precreate(cpu);
 
     /* create a thread per vCPU with TCG (MTTCG) */
     snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/TCG",
diff --git a/accel/tcg/tcg-accel-ops-rr.c b/accel/tcg/tcg-accel-ops-rr.c
index 6eec5c9eee9..fc33a13e4e8 100644
--- a/accel/tcg/tcg-accel-ops-rr.c
+++ b/accel/tcg/tcg-accel-ops-rr.c
@@ -311,8 +311,7 @@ void rr_start_vcpu_thread(CPUState *cpu)
     static QemuCond *single_tcg_halt_cond;
     static QemuThread *single_tcg_cpu_thread;
 
-    g_assert(tcg_enabled());
-    tcg_cpu_init_cflags(cpu, false);
+    tcg_vcpu_thread_precreate(cpu);
 
     if (!single_tcg_cpu_thread) {
         single_tcg_halt_cond = cpu->halt_cond;
diff --git a/accel/tcg/tcg-accel-ops.c b/accel/tcg/tcg-accel-ops.c
index da2e22a7dff..e58e1c3738f 100644
--- a/accel/tcg/tcg-accel-ops.c
+++ b/accel/tcg/tcg-accel-ops.c
@@ -41,6 +41,7 @@
 #include "gdbstub/enums.h"
 
 #include "hw/core/cpu.h"
+#include "hw/boards.h"
 
 #include "tcg-accel-ops.h"
 #include "tcg-accel-ops-mttcg.h"
@@ -68,6 +69,12 @@ void tcg_cpu_init_cflags(CPUState *cpu, bool parallel)
     tcg_cflags_set(cpu, cflags);
 }
 
+void tcg_vcpu_thread_precreate(CPUState *cpu)
+{
+    g_assert(tcg_enabled());
+    tcg_cpu_init_cflags(cpu, current_machine->smp.max_cpus > 1);
+}
+
 void tcg_cpu_destroy(CPUState *cpu)
 {
     cpu_thread_signal_destroyed(cpu);
-- 
2.49.0

[RFC PATCH 12/19] accel: Factor accel_create_vcpu_thread() out

[Philippe Mathieu-Daudé]

Factor accel_create_vcpu_thread() out of system/cpus.c
to be able to access accel/ internal definitions.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/qemu/accel.h |  2 ++
 accel/accel-common.c | 20 ++++++++++++++++++++
 system/cpus.c        |  4 +---
 3 files changed, 23 insertions(+), 3 deletions(-)

diff --git a/include/qemu/accel.h b/include/qemu/accel.h
index 3c9aaf9523c..a351eebe567 100644
--- a/include/qemu/accel.h
+++ b/include/qemu/accel.h
@@ -93,6 +93,8 @@ void accel_cpu_instance_init(CPUState *cpu);
 
 bool accel_cpu_realize(AccelState *accel, CPUState *cpu, Error **errp);
 
+void accel_create_vcpu_thread(AccelState *accel, CPUState *cpu);
+
 /**
  * accel_cpu_common_realize:
  * @cpu: The CPU that needs to call accel-specific cpu realization.
diff --git a/accel/accel-common.c b/accel/accel-common.c
index 4f3b42e7112..6bd4ef47c2c 100644
--- a/accel/accel-common.c
+++ b/accel/accel-common.c
@@ -10,6 +10,7 @@
 #include "qemu/osdep.h"
 #include "qemu/accel.h"
 #include "qemu/target-info.h"
+#include "system/accel-ops.h"
 #include "accel/accel-cpu.h"
 #include "accel-internal.h"
 
@@ -88,6 +89,25 @@ void accel_cpu_instance_init(CPUState *cpu)
     }
 }
 
+void accel_create_vcpu_thread(AccelState *accel, CPUState *cpu)
+{
+    AccelClass *ac;
+
+    if (!accel) {
+        accel = current_accel();
+    }
+    ac = ACCEL_GET_CLASS(accel);
+
+    /* accelerators all implement the AccelOpsClass */
+    g_assert(ac->ops);
+
+    if (ac->ops->create_vcpu_thread != NULL) {
+        ac->ops->create_vcpu_thread(cpu);
+    } else {
+        g_assert_not_reached();
+    }
+}
+
 bool accel_cpu_realize(AccelState *accel, CPUState *cpu, Error **errp)
 {
     AccelClass *acc;
diff --git a/system/cpus.c b/system/cpus.c
index 4835e5ced48..b6dff01c7ea 100644
--- a/system/cpus.c
+++ b/system/cpus.c
@@ -709,9 +709,7 @@ void qemu_init_vcpu(CPUState *cpu)
         cpu_address_space_init(cpu, 0, "cpu-memory", cpu->memory);
     }
 
-    /* accelerators all implement the AccelOpsClass */
-    g_assert(cpus_accel != NULL && cpus_accel->create_vcpu_thread != NULL);
-    cpus_accel->create_vcpu_thread(cpu);
+    accel_create_vcpu_thread(NULL, cpu);
 
     while (!cpu->created) {
         qemu_cond_wait(&qemu_cpu_cond, &bql);
-- 
2.49.0

[RFC PATCH 13/19] accel: Introduce AccelOpsClass::cpu_thread_routine handler

[Philippe Mathieu-Daudé]

In order to have a generic function creating threads,
introduce the thread_precreate() and cpu_thread_routine()
handlers.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/system/accel-ops.h |  2 ++
 accel/accel-common.c       | 16 +++++++++++++++-
 2 files changed, 17 insertions(+), 1 deletion(-)

diff --git a/include/system/accel-ops.h b/include/system/accel-ops.h
index 44b37592d02..a6535a07343 100644
--- a/include/system/accel-ops.h
+++ b/include/system/accel-ops.h
@@ -37,6 +37,8 @@ struct AccelOpsClass {
     bool (*cpus_are_resettable)(void);
     void (*cpu_reset_hold)(CPUState *cpu);
 
+    void *(*cpu_thread_routine)(void *);
+    void (*thread_precreate)(CPUState *cpu);
     void (*create_vcpu_thread)(CPUState *cpu); /* MANDATORY NON-NULL */
     void (*kick_vcpu_thread)(CPUState *cpu);
     bool (*cpu_thread_is_idle)(CPUState *cpu);
diff --git a/accel/accel-common.c b/accel/accel-common.c
index 6bd4ef47c2c..39eb2b2c1be 100644
--- a/accel/accel-common.c
+++ b/accel/accel-common.c
@@ -11,6 +11,7 @@
 #include "qemu/accel.h"
 #include "qemu/target-info.h"
 #include "system/accel-ops.h"
+#include "system/cpus.h"
 #include "accel/accel-cpu.h"
 #include "accel-internal.h"
 
@@ -104,7 +105,20 @@ void accel_create_vcpu_thread(AccelState *accel, CPUState *cpu)
     if (ac->ops->create_vcpu_thread != NULL) {
         ac->ops->create_vcpu_thread(cpu);
     } else {
-        g_assert_not_reached();
+        char thread_name[VCPU_THREAD_NAME_SIZE];
+
+        assert(ac->name);
+        assert(ac->ops->cpu_thread_routine);
+
+        if (ac->ops->thread_precreate) {
+            ac->ops->thread_precreate(cpu);
+        }
+
+        snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/%s",
+                 cpu->cpu_index, ac->name);
+        qemu_thread_create(cpu->thread, thread_name,
+                           ac->ops->cpu_thread_routine,
+                           cpu, QEMU_THREAD_JOINABLE);
     }
 }
 
-- 
2.49.0

[RFC PATCH 14/19] accel/dummy: Convert to AccelOpsClass::cpu_thread_routine

[Philippe Mathieu-Daudé]

By converting to AccelOpsClass::cpu_thread_routine we can
let the common accel_create_vcpu_thread() create the thread.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/system/cpus.h |  4 +---
 accel/dummy-cpus.c    | 14 +-------------
 accel/qtest/qtest.c   |  3 ++-
 accel/xen/xen-all.c   |  3 ++-
 4 files changed, 6 insertions(+), 18 deletions(-)

diff --git a/include/system/cpus.h b/include/system/cpus.h
index bfaa339dd73..e7ca9f4d684 100644
--- a/include/system/cpus.h
+++ b/include/system/cpus.h
@@ -9,9 +9,7 @@ const AccelOpsClass *cpus_get_accel(void);
 
 /* accel/dummy-cpus.c */
 void dummy_thread_precreate(CPUState *cpu);
-
-/* Create a dummy vcpu for AccelOpsClass->create_vcpu_thread */
-void dummy_start_vcpu_thread(CPUState *);
+void *dummy_cpu_thread_routine(void *arg);
 
 /* interface available for cpus accelerator threads */
 
diff --git a/accel/dummy-cpus.c b/accel/dummy-cpus.c
index 7c34e6c0fc5..c6756252550 100644
--- a/accel/dummy-cpus.c
+++ b/accel/dummy-cpus.c
@@ -18,7 +18,7 @@
 #include "qemu/main-loop.h"
 #include "hw/core/cpu.h"
 
-static void *dummy_cpu_thread_fn(void *arg)
+void *dummy_cpu_thread_routine(void *arg)
 {
     CPUState *cpu = arg;
 
@@ -70,15 +70,3 @@ void dummy_thread_precreate(CPUState *cpu)
     qemu_sem_init(&cpu->sem, 0);
 #endif
 }
-
-void dummy_start_vcpu_thread(CPUState *cpu)
-{
-    char thread_name[VCPU_THREAD_NAME_SIZE];
-
-    dummy_thread_precreate(cpu);
-
-    snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/DUMMY",
-             cpu->cpu_index);
-    qemu_thread_create(cpu->thread, thread_name, dummy_cpu_thread_fn, cpu,
-                       QEMU_THREAD_JOINABLE);
-}
diff --git a/accel/qtest/qtest.c b/accel/qtest/qtest.c
index c5a53ecb6dd..6930cc4cf3e 100644
--- a/accel/qtest/qtest.c
+++ b/accel/qtest/qtest.c
@@ -63,7 +63,8 @@ static void qtest_accel_ops_class_init(ObjectClass *oc, const void *data)
 {
     AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);
 
-    ops->create_vcpu_thread = dummy_start_vcpu_thread;
+    ops->thread_precreate = dummy_thread_precreate;
+    ops->cpu_thread_routine = dummy_cpu_thread_routine;
     ops->get_virtual_clock = qtest_get_virtual_clock;
     ops->set_virtual_clock = qtest_set_virtual_clock;
 };
diff --git a/accel/xen/xen-all.c b/accel/xen/xen-all.c
index de52a8f882a..3bd91889925 100644
--- a/accel/xen/xen-all.c
+++ b/accel/xen/xen-all.c
@@ -151,7 +151,8 @@ static void xen_accel_ops_class_init(ObjectClass *oc, const void *data)
 {
     AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);
 
-    ops->create_vcpu_thread = dummy_start_vcpu_thread;
+    ops->thread_precreate = dummy_thread_precreate;
+    ops->cpu_thread_routine = dummy_cpu_thread_routine;
 }
 
 static const TypeInfo xen_accel_ops_type = {
-- 
2.49.0

[RFC PATCH 15/19] accel/tcg: Convert to AccelOpsClass::cpu_thread_routine

[Philippe Mathieu-Daudé]

By converting to AccelOpsClass::cpu_thread_routine we can
let the common accel_create_vcpu_thread() create the thread.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/tcg/tcg-accel-ops-mttcg.h |  3 +--
 accel/tcg/tcg-accel-ops-mttcg.c | 16 +---------------
 accel/tcg/tcg-accel-ops.c       |  3 ++-
 3 files changed, 4 insertions(+), 18 deletions(-)

diff --git a/accel/tcg/tcg-accel-ops-mttcg.h b/accel/tcg/tcg-accel-ops-mttcg.h
index 8ffa7a9a9fe..8bf2452c886 100644
--- a/accel/tcg/tcg-accel-ops-mttcg.h
+++ b/accel/tcg/tcg-accel-ops-mttcg.h
@@ -13,7 +13,6 @@
 /* kick MTTCG vCPU thread */
 void mttcg_kick_vcpu_thread(CPUState *cpu);
 
-/* start an mttcg vCPU thread */
-void mttcg_start_vcpu_thread(CPUState *cpu);
+void *mttcg_cpu_thread_routine(void *arg);
 
 #endif /* TCG_ACCEL_OPS_MTTCG_H */
diff --git a/accel/tcg/tcg-accel-ops-mttcg.c b/accel/tcg/tcg-accel-ops-mttcg.c
index 462be7596b9..96ce065eb59 100644
--- a/accel/tcg/tcg-accel-ops-mttcg.c
+++ b/accel/tcg/tcg-accel-ops-mttcg.c
@@ -61,7 +61,7 @@ static void mttcg_force_rcu(Notifier *notify, void *data)
  * current CPUState for a given thread.
  */
 
-static void *mttcg_cpu_thread_fn(void *arg)
+void *mttcg_cpu_thread_routine(void *arg)
 {
     MttcgForceRcuNotifier force_rcu;
     CPUState *cpu = arg;
@@ -128,17 +128,3 @@ void mttcg_kick_vcpu_thread(CPUState *cpu)
 {
     cpu_exit(cpu);
 }
-
-void mttcg_start_vcpu_thread(CPUState *cpu)
-{
-    char thread_name[VCPU_THREAD_NAME_SIZE];
-
-    tcg_vcpu_thread_precreate(cpu);
-
-    /* create a thread per vCPU with TCG (MTTCG) */
-    snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/TCG",
-             cpu->cpu_index);
-
-    qemu_thread_create(cpu->thread, thread_name, mttcg_cpu_thread_fn,
-                       cpu, QEMU_THREAD_JOINABLE);
-}
diff --git a/accel/tcg/tcg-accel-ops.c b/accel/tcg/tcg-accel-ops.c
index e58e1c3738f..954e85dc13c 100644
--- a/accel/tcg/tcg-accel-ops.c
+++ b/accel/tcg/tcg-accel-ops.c
@@ -210,7 +210,7 @@ static void tcg_accel_ops_init(AccelClass *ac)
     AccelOpsClass *ops = ac->ops;
 
     if (qemu_tcg_mttcg_enabled()) {
-        ops->create_vcpu_thread = mttcg_start_vcpu_thread;
+        ops->cpu_thread_routine = mttcg_cpu_thread_routine;
         ops->kick_vcpu_thread = mttcg_kick_vcpu_thread;
         ops->handle_interrupt = tcg_handle_interrupt;
     } else {
@@ -226,6 +226,7 @@ static void tcg_accel_ops_init(AccelClass *ac)
         }
     }
 
+    ops->thread_precreate = tcg_vcpu_thread_precreate;
     ops->cpu_reset_hold = tcg_cpu_reset_hold;
     ops->supports_guest_debug = tcg_supports_guest_debug;
     ops->insert_breakpoint = tcg_insert_breakpoint;
-- 
2.49.0

[RFC PATCH 16/19] accel/hvf: Convert to AccelOpsClass::cpu_thread_routine

[Philippe Mathieu-Daudé]

By converting to AccelOpsClass::cpu_thread_routine we can
let the common accel_create_vcpu_thread() create the thread.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/hvf/hvf-accel-ops.c | 18 +-----------------
 1 file changed, 1 insertion(+), 17 deletions(-)

diff --git a/accel/hvf/hvf-accel-ops.c b/accel/hvf/hvf-accel-ops.c
index f4a983d1328..d9b533820c5 100644
--- a/accel/hvf/hvf-accel-ops.c
+++ b/accel/hvf/hvf-accel-ops.c
@@ -458,22 +458,6 @@ static void *hvf_cpu_thread_fn(void *arg)
     return NULL;
 }
 
-static void hvf_start_vcpu_thread(CPUState *cpu)
-{
-    char thread_name[VCPU_THREAD_NAME_SIZE];
-
-    /*
-     * HVF currently does not support TCG, and only runs in
-     * unrestricted-guest mode.
-     */
-    assert(hvf_enabled());
-
-    snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/HVF",
-             cpu->cpu_index);
-    qemu_thread_create(cpu->thread, thread_name, hvf_cpu_thread_fn,
-                       cpu, QEMU_THREAD_JOINABLE);
-}
-
 static int hvf_insert_breakpoint(CPUState *cpu, int type, vaddr addr, vaddr len)
 {
     struct hvf_sw_breakpoint *bp;
@@ -580,7 +564,7 @@ static void hvf_accel_ops_class_init(ObjectClass *oc, const void *data)
 {
     AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);
 
-    ops->create_vcpu_thread = hvf_start_vcpu_thread;
+    ops->cpu_thread_routine = hvf_cpu_thread_fn,
     ops->kick_vcpu_thread = hvf_kick_vcpu_thread;
 
     ops->synchronize_post_reset = hvf_cpu_synchronize_post_reset;
-- 
2.49.0

[RFC PATCH 17/19] accel/kvm: Convert to AccelOpsClass::cpu_thread_routine

[Philippe Mathieu-Daudé]

By converting to AccelOpsClass::cpu_thread_routine we can
let the common accel_create_vcpu_thread() create the thread.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/kvm/kvm-accel-ops.c | 12 +-----------
 1 file changed, 1 insertion(+), 11 deletions(-)

diff --git a/accel/kvm/kvm-accel-ops.c b/accel/kvm/kvm-accel-ops.c
index e5c15449aa6..39334997fb1 100644
--- a/accel/kvm/kvm-accel-ops.c
+++ b/accel/kvm/kvm-accel-ops.c
@@ -63,16 +63,6 @@ static void *kvm_vcpu_thread_fn(void *arg)
     return NULL;
 }
 
-static void kvm_start_vcpu_thread(CPUState *cpu)
-{
-    char thread_name[VCPU_THREAD_NAME_SIZE];
-
-    snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/KVM",
-             cpu->cpu_index);
-    qemu_thread_create(cpu->thread, thread_name, kvm_vcpu_thread_fn,
-                       cpu, QEMU_THREAD_JOINABLE);
-}
-
 static bool kvm_vcpu_thread_is_idle(CPUState *cpu)
 {
     return !kvm_halt_in_kernel();
@@ -94,7 +84,7 @@ static void kvm_accel_ops_class_init(ObjectClass *oc, const void *data)
 {
     AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);
 
-    ops->create_vcpu_thread = kvm_start_vcpu_thread;
+    ops->cpu_thread_routine = kvm_vcpu_thread_fn;
     ops->cpu_thread_is_idle = kvm_vcpu_thread_is_idle;
     ops->cpus_are_resettable = kvm_cpus_are_resettable;
     ops->synchronize_post_reset = kvm_cpu_synchronize_post_reset;
-- 
2.49.0

[RFC PATCH 18/19] accel/nvmm: Convert to AccelOpsClass::cpu_thread_routine

[Philippe Mathieu-Daudé]

By converting to AccelOpsClass::cpu_thread_routine we can
let the common accel_create_vcpu_thread() create the thread.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 target/i386/nvmm/nvmm-accel-ops.c | 12 +-----------
 1 file changed, 1 insertion(+), 11 deletions(-)

diff --git a/target/i386/nvmm/nvmm-accel-ops.c b/target/i386/nvmm/nvmm-accel-ops.c
index 21443078b72..bef6f61b776 100644
--- a/target/i386/nvmm/nvmm-accel-ops.c
+++ b/target/i386/nvmm/nvmm-accel-ops.c
@@ -61,16 +61,6 @@ static void *qemu_nvmm_cpu_thread_fn(void *arg)
     return NULL;
 }
 
-static void nvmm_start_vcpu_thread(CPUState *cpu)
-{
-    char thread_name[VCPU_THREAD_NAME_SIZE];
-
-    snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/NVMM",
-             cpu->cpu_index);
-    qemu_thread_create(cpu->thread, thread_name, qemu_nvmm_cpu_thread_fn,
-                       cpu, QEMU_THREAD_JOINABLE);
-}
-
 /*
  * Abort the call to run the virtual processor by another thread, and to
  * return the control to that thread.
@@ -85,7 +75,7 @@ static void nvmm_accel_ops_class_init(ObjectClass *oc, const void *data)
 {
     AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);
 
-    ops->create_vcpu_thread = nvmm_start_vcpu_thread;
+    ops->cpu_thread_routine = qemu_nvmm_cpu_thread_fn;
     ops->kick_vcpu_thread = nvmm_kick_vcpu_thread;
 
     ops->synchronize_post_reset = nvmm_cpu_synchronize_post_reset;
-- 
2.49.0

[RFC PATCH 19/19] accel/whpx: Convert to AccelOpsClass::cpu_thread_routine

[Philippe Mathieu-Daudé]

By converting to AccelOpsClass::cpu_thread_routine we can
let the common accel_create_vcpu_thread() create the thread.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 target/i386/whpx/whpx-accel-ops.c | 12 +-----------
 1 file changed, 1 insertion(+), 11 deletions(-)

diff --git a/target/i386/whpx/whpx-accel-ops.c b/target/i386/whpx/whpx-accel-ops.c
index b8bebe403c9..c1b27d1b89d 100644
--- a/target/i386/whpx/whpx-accel-ops.c
+++ b/target/i386/whpx/whpx-accel-ops.c
@@ -61,16 +61,6 @@ static void *whpx_cpu_thread_fn(void *arg)
     return NULL;
 }
 
-static void whpx_start_vcpu_thread(CPUState *cpu)
-{
-    char thread_name[VCPU_THREAD_NAME_SIZE];
-
-    snprintf(thread_name, VCPU_THREAD_NAME_SIZE, "CPU %d/WHPX",
-             cpu->cpu_index);
-    qemu_thread_create(cpu->thread, thread_name, whpx_cpu_thread_fn,
-                       cpu, QEMU_THREAD_JOINABLE);
-}
-
 static void whpx_kick_vcpu_thread(CPUState *cpu)
 {
     if (!qemu_cpu_is_self(cpu)) {
@@ -87,7 +77,7 @@ static void whpx_accel_ops_class_init(ObjectClass *oc, const void *data)
 {
     AccelOpsClass *ops = ACCEL_OPS_CLASS(oc);
 
-    ops->create_vcpu_thread = whpx_start_vcpu_thread;
+    ops->cpu_thread_routine = whpx_cpu_thread_fn;
     ops->kick_vcpu_thread = whpx_kick_vcpu_thread;
     ops->cpu_thread_is_idle = whpx_vcpu_thread_is_idle;
 
-- 
2.49.0

Re: [RFC PATCH 01/19] hw/arm/virt: Only require TCG || QTest to use virtualization extension

[Richard Henderson]

On 6/6/25 17:44, Philippe Mathieu-Daudé wrote:
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   hw/arm/virt.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> index 9a6cd085a37..d55ce2c0f4e 100644
> --- a/hw/arm/virt.c
> +++ b/hw/arm/virt.c
> @@ -2205,7 +2205,7 @@ static void machvirt_init(MachineState *machine)
>           exit(1);
>       }
>   
> -    if (vms->virt && (kvm_enabled() || hvf_enabled())) {
> +    if (vms->virt && !tcg_enabled() && !qtest_enabled()) {
>           error_report("mach-virt: %s does not support providing "
>                        "Virtualization extensions to the guest CPU",
>                        current_accel_name());

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

Re: [RFC PATCH 02/19] system/cpus: Only kick running vCPUs

[Richard Henderson]

On 6/6/25 17:44, Philippe Mathieu-Daudé wrote:
> As an optimization, avoid kicking stopped vCPUs.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   system/cpus.c | 5 +++++
>   1 file changed, 5 insertions(+)
> 
> diff --git a/system/cpus.c b/system/cpus.c
> index d16b0dff989..4835e5ced48 100644
> --- a/system/cpus.c
> +++ b/system/cpus.c
> @@ -494,6 +494,11 @@ void cpus_kick_thread(CPUState *cpu)
>   void qemu_cpu_kick(CPUState *cpu)
>   {
>       qemu_cond_broadcast(cpu->halt_cond);
> +
> +    if (!cpu_can_run(cpu)) {
> +        return;
> +    }
> +

This would appear to be a race condition.  The evaluation of cpu_can_run should be done 
within the context of 'cpu', not here, and not *after* we've already woken 'cpu' via the 
broadcast.


r~

Re: [RFC PATCH 06/19] accel/hvf: Fix TYPE_HVF_ACCEL instance size

[Richard Henderson]

On 6/6/25 17:44, Philippe Mathieu-Daudé wrote:
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   include/system/hvf_int.h  | 1 +
>   accel/hvf/hvf-accel-ops.c | 1 +
>   2 files changed, 2 insertions(+)
> 
> diff --git a/include/system/hvf_int.h b/include/system/hvf_int.h
> index 8c8b84012d9..d774e58df91 100644
> --- a/include/system/hvf_int.h
> +++ b/include/system/hvf_int.h
> @@ -44,6 +44,7 @@ typedef struct hvf_vcpu_caps {
>   
>   struct HVFState {
>       AccelState parent;
> +
>       hvf_slot slots[32];
>       int num_slots;
>   
> diff --git a/accel/hvf/hvf-accel-ops.c b/accel/hvf/hvf-accel-ops.c
> index 3dd9de26dbb..808ecea3816 100644
> --- a/accel/hvf/hvf-accel-ops.c
> +++ b/accel/hvf/hvf-accel-ops.c
> @@ -366,6 +366,7 @@ static void hvf_accel_class_init(ObjectClass *oc, const void *data)
>   static const TypeInfo hvf_accel_type = {
>       .name = TYPE_HVF_ACCEL,
>       .parent = TYPE_ACCEL,
> +    .instance_size = sizeof(HVFState),
>       .class_init = hvf_accel_class_init,
>   };
>   

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

Re: [RFC PATCH 07/19] accel/hvf: Re-use QOM allocated state

[Richard Henderson]

On 6/6/25 17:44, Philippe Mathieu-Daudé wrote:
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   accel/hvf/hvf-accel-ops.c | 4 +---
>   1 file changed, 1 insertion(+), 3 deletions(-)
> 
> diff --git a/accel/hvf/hvf-accel-ops.c b/accel/hvf/hvf-accel-ops.c
> index 808ecea3816..f4a983d1328 100644
> --- a/accel/hvf/hvf-accel-ops.c
> +++ b/accel/hvf/hvf-accel-ops.c
> @@ -319,7 +319,7 @@ static int hvf_accel_init(MachineState *ms, AccelState *as)
>   {
>       int x;
>       hv_return_t ret;
> -    HVFState *s;
> +    HVFState *s = HVF_STATE(as);
>       int pa_range = 36;
>       MachineClass *mc = MACHINE_GET_CLASS(ms);
>   
> @@ -333,8 +333,6 @@ static int hvf_accel_init(MachineState *ms, AccelState *as)
>       ret = hvf_arch_vm_create(ms, (uint32_t)pa_range);
>       assert_hvf_ok(ret);
>   
> -    s = g_new0(HVFState, 1);
> -
>       s->num_slots = ARRAY_SIZE(s->slots);
>       for (x = 0; x < s->num_slots; ++x) {
>           s->slots[x].size = 0;

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

Re: [RFC PATCH 05/19] accel/kvm: Prefer local AccelState over global MachineState::accel

[Richard Henderson]

On 6/6/25 17:44, Philippe Mathieu-Daudé wrote:
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   accel/kvm/kvm-all.c | 4 +---
>   1 file changed, 1 insertion(+), 3 deletions(-)
> 
> diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
> index 5b28b15ccec..551a462926d 100644
> --- a/accel/kvm/kvm-all.c
> +++ b/accel/kvm/kvm-all.c
> @@ -2582,15 +2582,13 @@ static int kvm_init(MachineState *ms, AccelState *as)
>           { /* end of list */ }
>       }, *nc = num_cpus;
>       int soft_vcpus_limit, hard_vcpus_limit;
> -    KVMState *s;
> +    KVMState *s = KVM_STATE(as);
>       const KVMCapabilityInfo *missing_cap;
>       int ret;
>       int type;
>   
>       qemu_mutex_init(&kml_slots_lock);
>   
> -    s = KVM_STATE(ms->accelerator);
> -
>       /*
>        * On systems where the kernel can support different base page
>        * sizes, host page size may be different from TARGET_PAGE_SIZE,

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

Re: [RFC PATCH 04/19] accel: Propagate AccelState to AccelClass::init_machine()

[Richard Henderson]

On 6/6/25 17:44, Philippe Mathieu-Daudé wrote:
> In order to avoid init_machine() to call current_accel(),
> pass AccelState along.
> 
> Signed-off-by: Philippe Mathieu-Daudé<philmd@linaro.org>
> ---
>   include/qemu/accel.h        | 2 +-
>   accel/accel-system.c        | 2 +-
>   accel/hvf/hvf-accel-ops.c   | 2 +-
>   accel/kvm/kvm-all.c         | 2 +-
>   accel/qtest/qtest.c         | 2 +-
>   accel/tcg/tcg-all.c         | 2 +-
>   bsd-user/main.c             | 2 +-
>   linux-user/main.c           | 2 +-
>   target/i386/nvmm/nvmm-all.c | 2 +-
>   target/i386/whpx/whpx-all.c | 2 +-
>   10 files changed, 10 insertions(+), 10 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

Re: [RFC PATCH 03/19] accel: Keep reference to AccelOpsClass in AccelClass

[Richard Henderson]

On 6/6/25 17:44, Philippe Mathieu-Daudé wrote:
> Allow dereferencing AccelOpsClass outside of accel/accel-system.c.
> 
> Signed-off-by: Philippe Mathieu-Daudé<philmd@linaro.org>
> ---
>   include/qemu/accel.h       | 2 ++
>   include/system/accel-ops.h | 3 ++-
>   accel/accel-system.c       | 3 ++-
>   accel/tcg/tcg-accel-ops.c  | 4 +++-
>   4 files changed, 9 insertions(+), 3 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

Re: [RFC PATCH 08/19] accel/tcg: Prefer local AccelState over global current_accel()

[Richard Henderson]

On 6/6/25 17:44, Philippe Mathieu-Daudé wrote:
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   accel/tcg/tcg-all.c | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/accel/tcg/tcg-all.c b/accel/tcg/tcg-all.c
> index 6c5979861cf..4b1238ed345 100644
> --- a/accel/tcg/tcg-all.c
> +++ b/accel/tcg/tcg-all.c
> @@ -82,7 +82,7 @@ bool one_insn_per_tb;
>   
>   static int tcg_init_machine(MachineState *ms, AccelState *as)
>   {
> -    TCGState *s = TCG_STATE(current_accel());
> +    TCGState *s = TCG_STATE(as);
>       unsigned max_threads = 1;
>   
>   #ifndef CONFIG_USER_ONLY

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

Re: [RFC PATCH 01/19] hw/arm/virt: Only require TCG || QTest to use virtualization extension

[Alex Bennée]

Philippe Mathieu-Daudé <philmd@linaro.org> writes:

> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [RFC PATCH 03/19] accel: Keep reference to AccelOpsClass in AccelClass

[Alex Bennée]

Philippe Mathieu-Daudé <philmd@linaro.org> writes:

> Allow dereferencing AccelOpsClass outside of accel/accel-system.c.
>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>  include/qemu/accel.h       | 2 ++
>  include/system/accel-ops.h | 3 ++-
>  accel/accel-system.c       | 3 ++-
>  accel/tcg/tcg-accel-ops.c  | 4 +++-
>  4 files changed, 9 insertions(+), 3 deletions(-)
>
> diff --git a/include/qemu/accel.h b/include/qemu/accel.h
> index fbd3d897fef..23c62cbeb07 100644
> --- a/include/qemu/accel.h
> +++ b/include/qemu/accel.h
> @@ -37,6 +37,8 @@ typedef struct AccelClass {
>      /*< public >*/
>  
>      const char *name;

A comment wouldn't go a amiss here:

 "Cached by accel_init_ops_interfaces when created"?
 
> +    AccelOpsClass *ops;
> +
>      int (*init_machine)(MachineState *ms);
>      bool (*cpu_common_realize)(CPUState *cpu, Error **errp);
>      void (*cpu_common_unrealize)(CPUState *cpu);
> diff --git a/include/system/accel-ops.h b/include/system/accel-ops.h
> index 4c99d25aeff..44b37592d02 100644
> --- a/include/system/accel-ops.h
> +++ b/include/system/accel-ops.h
> @@ -10,6 +10,7 @@
>  #ifndef ACCEL_OPS_H
>  #define ACCEL_OPS_H
>  
> +#include "qemu/accel.h"
>  #include "exec/vaddr.h"
>  #include "qom/object.h"
>  
> @@ -31,7 +32,7 @@ struct AccelOpsClass {
>      /*< public >*/
>  
>      /* initialization function called when accel is chosen */
> -    void (*ops_init)(AccelOpsClass *ops);
> +    void (*ops_init)(AccelClass *ac);
>  
>      bool (*cpus_are_resettable)(void);
>      void (*cpu_reset_hold)(CPUState *cpu);
> diff --git a/accel/accel-system.c b/accel/accel-system.c
> index a0f562ae9ff..64bc991b1ce 100644
> --- a/accel/accel-system.c
> +++ b/accel/accel-system.c
> @@ -85,8 +85,9 @@ void accel_init_ops_interfaces(AccelClass *ac)
>       * non-NULL create_vcpu_thread operation.
>       */
>      ops = ACCEL_OPS_CLASS(oc);
> +    ac->ops = ops;
>      if (ops->ops_init) {
> -        ops->ops_init(ops);
> +        ops->ops_init(ac);
>      }
>      cpus_register_accel(ops);
>  }
> diff --git a/accel/tcg/tcg-accel-ops.c b/accel/tcg/tcg-accel-ops.c
> index b24d6a75625..da2e22a7dff 100644
> --- a/accel/tcg/tcg-accel-ops.c
> +++ b/accel/tcg/tcg-accel-ops.c
> @@ -198,8 +198,10 @@ static inline void tcg_remove_all_breakpoints(CPUState *cpu)
>      cpu_watchpoint_remove_all(cpu, BP_GDB);
>  }
>  
> -static void tcg_accel_ops_init(AccelOpsClass *ops)
> +static void tcg_accel_ops_init(AccelClass *ac)
>  {
> +    AccelOpsClass *ops = ac->ops;
> +
>      if (qemu_tcg_mttcg_enabled()) {
>          ops->create_vcpu_thread = mttcg_start_vcpu_thread;
>          ops->kick_vcpu_thread = mttcg_kick_vcpu_thread;

otherwise:

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [RFC PATCH 04/19] accel: Propagate AccelState to AccelClass::init_machine()

[Alex Bennée]

Philippe Mathieu-Daudé <philmd@linaro.org> writes:

> In order to avoid init_machine() to call current_accel(),
> pass AccelState along.
>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Reviewed-by: Alex Bennée <alex.bennee@linaro.org>

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [RFC PATCH 01/19] hw/arm/virt: Only require TCG || QTest to use virtualization extension

[Miguel Luis]

Hi Philippe,

> On 6 Jun 2025, at 16:44, Philippe Mathieu-Daudé <philmd@linaro.org> wrote:
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
> hw/arm/virt.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
> index 9a6cd085a37..d55ce2c0f4e 100644
> --- a/hw/arm/virt.c
> +++ b/hw/arm/virt.c
> @@ -2205,7 +2205,7 @@ static void machvirt_init(MachineState *machine)
>         exit(1);
>     }
> 
> -    if (vms->virt && (kvm_enabled() || hvf_enabled())) {
> +    if (vms->virt && !tcg_enabled() && !qtest_enabled()) {

Does this means TCG won’t run aarch64 NV?

Miguel

>         error_report("mach-virt: %s does not support providing "
>                      "Virtualization extensions to the guest CPU",
>                      current_accel_name());
> -- 
> 2.49.0
> 
> 

Re: [RFC PATCH 09/19] accel: Factor accel_cpu_realize() out

[Alex Bennée]

Philippe Mathieu-Daudé <philmd@linaro.org> writes:

> Factor accel_cpu_realize() out of accel_cpu_common_realize()
> for re-use.
>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>  accel/accel-internal.h |  2 ++
>  include/qemu/accel.h   |  2 ++
>  accel/accel-common.c   | 15 ++++++++++++---
>  3 files changed, 16 insertions(+), 3 deletions(-)
>
> diff --git a/accel/accel-internal.h b/accel/accel-internal.h
> index d3a4422cbf7..b541377c349 100644
> --- a/accel/accel-internal.h
> +++ b/accel/accel-internal.h
> @@ -14,4 +14,6 @@
>  
>  void accel_init_ops_interfaces(AccelClass *ac);
>  
> +bool accel_cpu_realize(AccelState *accel, CPUState *cpu, Error **errp);
> +
>  #endif /* ACCEL_SYSTEM_H */
> diff --git a/include/qemu/accel.h b/include/qemu/accel.h
> index c660c5f4b11..3c9aaf9523c 100644
> --- a/include/qemu/accel.h
> +++ b/include/qemu/accel.h
> @@ -91,6 +91,8 @@ void accel_setup_post(MachineState *ms);
>   */
>  void accel_cpu_instance_init(CPUState *cpu);
>  
> +bool accel_cpu_realize(AccelState *accel, CPUState *cpu, Error **errp);
> +

Duplicate forward declarations:

  FAILED: libuser.a.p/accel_accel-user.c.o 
  cc -m64 -Ilibuser.a.p -I. -I../.. -I../../common-user/host/x86_64 -I../../linux-user/include/host/x86_64 -I../../linux-user/include -Iqapi -Itrace -Iui -Iui/shader -I/usr/include/glib-2.0 -I/usr/lib/x86_64-linux-gnu/glib-2.0/include -I/usr/include/sysprof-6 -fdiagnostics-color=auto -Wall -Winvalid-pch -Werror -std=gnu11 -O2 -g -fstack-protector-strong -Wempty-body -Wendif-labels -Wexpansion-to-defined -Wformat-security -Wformat-y2k -Wignored-qualifiers -Wimplicit-fallthrough=2 -Winit-self -Wmissing-format-attribute -Wmissing-prototypes -Wnested-externs -Wold-style-declaration -Wold-style-definition -Wredundant-decls -Wshadow=local -Wstrict-prototypes -Wtype-limits -Wundef -Wvla -Wwrite-strings -Wno-missing-include-dirs -Wno-psabi -Wno-shift-negative-value -isystem /home/alex/lsrc/qemu.git/linux-headers -isystem linux-headers -iquote . -iquote /home/alex/lsrc/qemu.git -iquote /home/alex/lsrc/qemu.git/include -iquote /home/alex/lsrc/qemu.git/host/include/x86_64 -iquote /home/alex/lsrc/qemu.git/host/include/generic -iquote /home/alex/lsrc/qemu.git/tcg/i386 -pthread -mcx16 -msse2 -D_GNU_SOURCE -D_FILE_OFFSET_BITS=64 -D_LARGEFILE_SOURCE -fno-strict-aliasing -fno-common -fwrapv -ftrivial-auto-var-init=zero -fzero-call-used-regs=used-gpr -gsplit-dwarf -fPIE -DCONFIG_USER_ONLY -DCOMPILING_SYSTEM_VS_USER -MD -MQ libuser.a.p/accel_accel-user.c.o -MF libuser.a.p/accel_accel-user.c.o.d -o libuser.a.p/accel_accel-user.c.o -c ../../accel/accel-user.c
  In file included from ../../accel/accel-user.c:12:
  ../../accel/accel-internal.h:17:6: error: redundant redeclaration of ‘accel_cpu_realize’ [-Werror=redundant-decls]
     17 | bool accel_cpu_realize(AccelState *accel, CPUState *cpu, Error **errp);
        |      ^~~~~~~~~~~~~~~~~
  In file included from ../../accel/accel-user.c:11:
  /home/alex/lsrc/qemu.git/include/qemu/accel.h:94:6: note: previous declaration of ‘accel_cpu_realize’ with type ‘_Bool(AccelState *, CPUState *, Error **)’
     94 | bool accel_cpu_realize(AccelState *accel, CPUState *cpu, Error **errp);
        |      ^~~~~~~~~~~~~~~~~
  cc1: all warnings being treated as errors


-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [RFC PATCH 00/19] accel: Preparatory cleanups for split-accel

[Alex Bennée]

Philippe Mathieu-Daudé <philmd@linaro.org> writes:

> Few changes needed before being able to add the
> split acceleration. Mostly adapting few prototypes
> so callees have all necessary information.

make check-tcg failed:

  ➜  make check-tcg
  /home/alex/lsrc/qemu.git/builds/arm.all/pyvenv/bin/meson introspect --targets --tests --benchmarks | /home/alex/lsrc/qemu.git/builds/arm.all/pyvenv/bin/python3 -B scripts/mtest2make.py > Makefile.mtest
    BUILD   aarch64-softmmu guest-tests
  tests/tcg/aarch64-softmmu: -march=armv8.3-a detected
  tests/tcg/aarch64-softmmu: -march=armv8.5-a+memtag detected
    RUN     aarch64-softmmu guest-tests
    TEST    feat-xs on aarch64
  qemu-system-aarch64: ../../system/cpus.c:685: cpus_register_accel: Assertion `ops->create_vcpu_thread != NULL' failed.
  Aborted
  make[1]: *** [Makefile:195: run-feat-xs] Error 134
  make: *** [/home/alex/lsrc/qemu.git/tests/Makefile.include:56: run-tcg-tests-aarch64-softmmu] Error 2

<snip>

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [RFC PATCH 13/19] accel: Introduce AccelOpsClass::cpu_thread_routine handler

[Alex Bennée]

Philippe Mathieu-Daudé <philmd@linaro.org> writes:

> In order to have a generic function creating threads,
> introduce the thread_precreate() and cpu_thread_routine()
> handlers.
>
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>  include/system/accel-ops.h |  2 ++
>  accel/accel-common.c       | 16 +++++++++++++++-
>  2 files changed, 17 insertions(+), 1 deletion(-)
>
> diff --git a/include/system/accel-ops.h b/include/system/accel-ops.h
> index 44b37592d02..a6535a07343 100644
> --- a/include/system/accel-ops.h
> +++ b/include/system/accel-ops.h
> @@ -37,6 +37,8 @@ struct AccelOpsClass {
>      bool (*cpus_are_resettable)(void);
>      void (*cpu_reset_hold)(CPUState *cpu);
>  
> +    void *(*cpu_thread_routine)(void *);
> +    void (*thread_precreate)(CPUState *cpu);
>      void (*create_vcpu_thread)(CPUState *cpu); /* MANDATORY NON-NULL
> */

After this you start switching *create_vcpu_thread to
*cpu_thread_routine despite this being a mandatory non-null. You should
fix the comment and maybe the assert?

modified   system/cpus.c
@@ -682,7 +682,7 @@ void cpu_remove_sync(CPUState *cpu)
 void cpus_register_accel(const AccelOpsClass *ops)
 {
     assert(ops != NULL);
-    assert(ops->create_vcpu_thread != NULL); /* mandatory */
+    assert(ops->create_vcpu_thread || ops->cpu_thread_routine); /* mandatory */
     cpus_accel = ops;
 }

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [RFC PATCH 01/19] hw/arm/virt: Only require TCG || QTest to use virtualization extension

[Alex Bennée]

Miguel Luis <miguel.luis@oracle.com> writes:

> Hi Philippe,
>
>> On 6 Jun 2025, at 16:44, Philippe Mathieu-Daudé <philmd@linaro.org> wrote:
>> 
>> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
>> ---
>> hw/arm/virt.c | 2 +-
>> 1 file changed, 1 insertion(+), 1 deletion(-)
>> 
>> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
>> index 9a6cd085a37..d55ce2c0f4e 100644
>> --- a/hw/arm/virt.c
>> +++ b/hw/arm/virt.c
>> @@ -2205,7 +2205,7 @@ static void machvirt_init(MachineState *machine)
>>         exit(1);
>>     }
>> 
>> -    if (vms->virt && (kvm_enabled() || hvf_enabled())) {
>> +    if (vms->virt && !tcg_enabled() && !qtest_enabled()) {
>
> Does this means TCG won’t run aarch64 NV?

No - it just means we can't start in EL2 with anything but TCG (or
qtest). Currently we don't have NV support for KVM in QEMU.


BTW Philippe isn't the same test needed above for secure (which can't be
supported even with KVM NV support).

>
> Miguel
>
>>         error_report("mach-virt: %s does not support providing "
>>                      "Virtualization extensions to the guest CPU",
>>                      current_accel_name());
>> -- 
>> 2.49.0
>> 
>> 

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro

Re: [RFC PATCH 01/19] hw/arm/virt: Only require TCG || QTest to use virtualization extension

[Miguel Luis]

Hi Alex,

> On 11 Jun 2025, at 14:31, Alex Bennée <alex.bennee@linaro.org> wrote:
> 
> Miguel Luis <miguel.luis@oracle.com> writes:
> 
>> Hi Philippe,
>> 
>>> On 6 Jun 2025, at 16:44, Philippe Mathieu-Daudé <philmd@linaro.org> wrote:
>>> 
>>> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
>>> ---
>>> hw/arm/virt.c | 2 +-
>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>> 
>>> diff --git a/hw/arm/virt.c b/hw/arm/virt.c
>>> index 9a6cd085a37..d55ce2c0f4e 100644
>>> --- a/hw/arm/virt.c
>>> +++ b/hw/arm/virt.c
>>> @@ -2205,7 +2205,7 @@ static void machvirt_init(MachineState *machine)
>>>        exit(1);
>>>    }
>>> 
>>> -    if (vms->virt && (kvm_enabled() || hvf_enabled())) {
>>> +    if (vms->virt && !tcg_enabled() && !qtest_enabled()) {
>> 
>> Does this means TCG won’t run aarch64 NV?
> 
> No - it just means we can't start in EL2 with anything but TCG (or
> qtest).

Oh, of course.. now that I read it again.

Apologies for the noise and thanks for clarifying.

Regards,
Miguel

> Currently we don't have NV support for KVM in QEMU.
> 
> 
> BTW Philippe isn't the same test needed above for secure (which can't be
> supported even with KVM NV support).
> 
>> 
>> Miguel
>> 
>>>        error_report("mach-virt: %s does not support providing "
>>>                     "Virtualization extensions to the guest CPU",
>>>                     current_accel_name());
>>> -- 
>>> 2.49.0
>>> 
>>> 
> 
> -- 
> Alex Bennée
> Virtualisation Tech Lead @ Linaro

Re: [RFC PATCH 02/19] system/cpus: Only kick running vCPUs

[Philippe Mathieu-Daudé]

On 7/6/25 15:23, Richard Henderson wrote:
> On 6/6/25 17:44, Philippe Mathieu-Daudé wrote:
>> As an optimization, avoid kicking stopped vCPUs.
>>
>> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
>> ---
>>   system/cpus.c | 5 +++++
>>   1 file changed, 5 insertions(+)
>>
>> diff --git a/system/cpus.c b/system/cpus.c
>> index d16b0dff989..4835e5ced48 100644
>> --- a/system/cpus.c
>> +++ b/system/cpus.c
>> @@ -494,6 +494,11 @@ void cpus_kick_thread(CPUState *cpu)
>>   void qemu_cpu_kick(CPUState *cpu)
>>   {
>>       qemu_cond_broadcast(cpu->halt_cond);
>> +
>> +    if (!cpu_can_run(cpu)) {
>> +        return;
>> +    }
>> +
> 
> This would appear to be a race condition.  The evaluation of cpu_can_run 
> should be done within the context of 'cpu', not here, and not *after* 
> we've already woken 'cpu' via the broadcast.

OK.

Still I don't understand something, when putting this assertion:

-- >8 --
diff --git a/system/cpus.c b/system/cpus.c
index d16b0dff989..0631015f754 100644
--- a/system/cpus.c
+++ b/system/cpus.c
@@ -493,7 +493,10 @@ void cpus_kick_thread(CPUState *cpu)

  void qemu_cpu_kick(CPUState *cpu)
  {
+    assert(cpu_can_run(cpu));
+
      qemu_cond_broadcast(cpu->halt_cond);
      if (cpus_accel->kick_vcpu_thread) {
          cpus_accel->kick_vcpu_thread(cpu);
      } else { /* default */
---

I get:

(lldb) bt
* thread #1, queue = 'com.apple.main-thread', stop reason = hit program 
assert
     frame #0: 0x000000018a669388 libsystem_kernel.dylib`__pthread_kill + 8
     frame #1: 0x000000018a6a288c libsystem_pthread.dylib`pthread_kill + 296
     frame #2: 0x000000018a5abc60 libsystem_c.dylib`abort + 124
     frame #3: 0x000000018a5aaeec libsystem_c.dylib`__assert_rtn + 284
   * frame #4: 0x000000010057ddc4 qemu_cpu_kick(cpu=0x0000000130218000) 
at cpus.c:496:5
     frame #5: 0x00000001000106ec 
queue_work_on_cpu(cpu=0x0000000130218000, wi=0x000060000038c000) at 
cpu-common.c:140:5
     frame #6: 0x0000000100010780 
async_run_on_cpu(cpu=0x0000000130218000, func=(tcg_commit_cpu at 
physmem.c:2758), data=(host_int = 60885632, host_ulong = 
105553177152128, host_ptr = 0x0000600003a10a80, target_ptr = 
105553177152128)) at cpu-common.c:177:5
     frame #7: 0x000000010059ad34 
tcg_commit(listener=0x0000600003a10a98) at physmem.c:2789:9
     frame #8: 0x0000000100591240 
listener_add_address_space(listener=0x0000600003a10a98, 
as=0x0000600003611980) at memory.c:3082:9
     frame #9: 0x0000000100590f48 
memory_listener_register(listener=0x0000600003a10a98, 
as=0x0000600003611980) at memory.c:3170:5
     frame #10: 0x000000010059abe4 
cpu_address_space_init(cpu=0x0000000130218000, asidx=0, 
prefix="cpu-memory", mr=0x000000012b1faba0) at physmem.c:813:9
     frame #11: 0x0000000100750c40 
arm_cpu_realizefn(dev=0x0000000130218000, errp=0x000000016fdfe2c0) at 
cpu.c:2572:5
     frame #12: 0x0000000100b7ed9c 
device_set_realized(obj=0x0000000130218000, value=true, 
errp=0x000000016fdfe388) at qdev.c:494:13
     frame #13: 0x0000000100b8a880 
property_set_bool(obj=0x0000000130218000, v=0x0000600003f12d00, 
name="realized", opaque=0x000060000010c1d0, errp=0x000000016fdfe388) at 
object.c:2375:5
     frame #14: 0x0000000100b87acc 
object_property_set(obj=0x0000000130218000, name="realized", 
v=0x0000600003f12d00, errp=0x000000016fdfe388) at object.c:1450:5
     frame #15: 0x0000000100b8f14c 
object_property_set_qobject(obj=0x0000000130218000, name="realized", 
value=0x0000600000386920, errp=0x0000000101e39e28) at qom-qobject.c:28:10
     frame #16: 0x0000000100b882f8 
object_property_set_bool(obj=0x0000000130218000, name="realized", 
value=true, errp=0x0000000101e39e28) at object.c:1520:15
     frame #17: 0x0000000100b7d240 qdev_realize(dev=0x0000000130218000, 
bus=0x0000000000000000, errp=0x0000000101e39e28) at qdev.c:276:12
     frame #18: 0x000000010083a81c 
machvirt_init(machine=0x000000012b1fa710) at virt.c:2329:9
     frame #19: 0x0000000100136a40 
machine_run_board_init(machine=0x000000012b1fa710, 
mem_path=0x0000000000000000, errp=0x000000016fdfe6a8) at machine.c:1669:5
     frame #20: 0x0000000100571384 qemu_init_board at vl.c:2714:5
     frame #21: 0x0000000100571154 
qmp_x_exit_preconfig(errp=0x0000000101e39e28) at vl.c:2808:5
     frame #22: 0x0000000100573a14 qemu_init(argc=17, 
argv=0x000000016fdff138) at vl.c:3844:9
     frame #23: 0x0000000100d036e0 main(argc=17, 
argv=0x000000016fdff138) at main.c:71:5
     frame #24: 0x000000018a302b98 dyld`start + 6076
(lldb)

I expect a vCPU to be in a "stable" state and usable *after* it is
realized, as we are calling various hooks in many places. Here we are
processing the pending work queue while the vCPU isn't fully realized,
so some hooks might not have been called yet...

Git history of tcg_commit() points to commit 0d58c660689 ("softmmu: Use
async_run_on_cpu in tcg_commit").
This isn't the first time I ends there, see also:
https://lore.kernel.org/qemu-devel/20230907161415.6102-1-philmd@linaro.org/. 
Using the same reasoning of this patch, adding:

-- >8 --
diff --git a/system/physmem.c b/system/physmem.c
index a8a9ca309ea..479a7a88037 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -2773,6 +2774,14 @@ static void tcg_commit(MemoryListener *listener)
      cpuas = container_of(listener, CPUAddressSpace, tcg_as_listener);
      cpu = cpuas->cpu;

+    if (!qdev_is_realized(DEVICE(cpu))) {
+        /*
+         * The listener is also called during realize, before
+         * all of the tcg machinery for run-on is initialized.
+         */
+        return;
+    }
+
      /*
       * Defer changes to as->memory_dispatch until the cpu is quiescent.
       * Otherwise we race between (1) other cpu threads and (2) ongoing
---

makes my issues disappear; tcg_commit_cpu() calls are run on realized
vCPUs, and the order of pre-realize vcpu hooks doesn't alter anything.

I don't remember why I wrote this "The listener is also called during
realize, before all of the tcg machinery for run-on is initialized"
comment, it could be better to call memory_region_transaction_commit()
after CpuRealize, maybe in CpuReset.

Re: [RFC PATCH 02/19] system/cpus: Only kick running vCPUs

[Alex Bennée]

Philippe Mathieu-Daudé <philmd@linaro.org> writes:

> On 7/6/25 15:23, Richard Henderson wrote:
>> On 6/6/25 17:44, Philippe Mathieu-Daudé wrote:
>>> As an optimization, avoid kicking stopped vCPUs.

This also breaks gdbstub:

 pause_all_vcpus() -> cpu_pause(sets cpu->stop) -> qemu_cpu_kick(skips kicking)

>>>
>>> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
>>> ---
>>>   system/cpus.c | 5 +++++
>>>   1 file changed, 5 insertions(+)
>>>
>>> diff --git a/system/cpus.c b/system/cpus.c
>>> index d16b0dff989..4835e5ced48 100644
>>> --- a/system/cpus.c
>>> +++ b/system/cpus.c
>>> @@ -494,6 +494,11 @@ void cpus_kick_thread(CPUState *cpu)
>>>   void qemu_cpu_kick(CPUState *cpu)
>>>   {
>>>       qemu_cond_broadcast(cpu->halt_cond);
>>> +
>>> +    if (!cpu_can_run(cpu)) {
>>> +        return;
>>> +    }
>>> +
>> This would appear to be a race condition.  The evaluation of
>> cpu_can_run should be done within the context of 'cpu', not here,
>> and not *after* we've already woken 'cpu' via the broadcast.
>
> OK.
>
> Still I don't understand something, when putting this assertion:
>
> -- >8 --
> diff --git a/system/cpus.c b/system/cpus.c
> index d16b0dff989..0631015f754 100644
> --- a/system/cpus.c
> +++ b/system/cpus.c
> @@ -493,7 +493,10 @@ void cpus_kick_thread(CPUState *cpu)
>
>  void qemu_cpu_kick(CPUState *cpu)
>  {
> +    assert(cpu_can_run(cpu));
> +
>      qemu_cond_broadcast(cpu->halt_cond);
>      if (cpus_accel->kick_vcpu_thread) {
>          cpus_accel->kick_vcpu_thread(cpu);
>      } else { /* default */
> ---
>
> I get:
>
> (lldb) bt
> * thread #1, queue = 'com.apple.main-thread', stop reason = hit
>   program assert
>     frame #0: 0x000000018a669388 libsystem_kernel.dylib`__pthread_kill + 8
>     frame #1: 0x000000018a6a288c libsystem_pthread.dylib`pthread_kill + 296
>     frame #2: 0x000000018a5abc60 libsystem_c.dylib`abort + 124
>     frame #3: 0x000000018a5aaeec libsystem_c.dylib`__assert_rtn + 284
>   * frame #4: 0x000000010057ddc4 qemu_cpu_kick(cpu=0x0000000130218000)
>     at cpus.c:496:5
>     frame #5: 0x00000001000106ec
>     queue_work_on_cpu(cpu=0x0000000130218000, wi=0x000060000038c000)
>     at cpu-common.c:140:5
>     frame #6: 0x0000000100010780
>     async_run_on_cpu(cpu=0x0000000130218000, func=(tcg_commit_cpu at
>     physmem.c:2758), data=(host_int = 60885632, host_ulong =
>     105553177152128, host_ptr = 0x0000600003a10a80, target_ptr =
>     105553177152128)) at cpu-common.c:177:5
>     frame #7: 0x000000010059ad34
>     tcg_commit(listener=0x0000600003a10a98) at physmem.c:2789:9
>     frame #8: 0x0000000100591240
>     listener_add_address_space(listener=0x0000600003a10a98,
>     as=0x0000600003611980) at memory.c:3082:9
>     frame #9: 0x0000000100590f48
>     memory_listener_register(listener=0x0000600003a10a98,
>     as=0x0000600003611980) at memory.c:3170:5
>     frame #10: 0x000000010059abe4
>     cpu_address_space_init(cpu=0x0000000130218000, asidx=0,
>     prefix="cpu-memory", mr=0x000000012b1faba0) at physmem.c:813:9
>     frame #11: 0x0000000100750c40
>     arm_cpu_realizefn(dev=0x0000000130218000, errp=0x000000016fdfe2c0)
>     at cpu.c:2572:5
>     frame #12: 0x0000000100b7ed9c
>     device_set_realized(obj=0x0000000130218000, value=true,
>     errp=0x000000016fdfe388) at qdev.c:494:13
>     frame #13: 0x0000000100b8a880
>     property_set_bool(obj=0x0000000130218000, v=0x0000600003f12d00,
>     name="realized", opaque=0x000060000010c1d0,
>     errp=0x000000016fdfe388) at object.c:2375:5
>     frame #14: 0x0000000100b87acc
>     object_property_set(obj=0x0000000130218000, name="realized",
>     v=0x0000600003f12d00, errp=0x000000016fdfe388) at object.c:1450:5
>     frame #15: 0x0000000100b8f14c
>     object_property_set_qobject(obj=0x0000000130218000,
>     name="realized", value=0x0000600000386920,
>    errp=0x0000000101e39e28) at qom-qobject.c:28:10
>     frame #16: 0x0000000100b882f8
>     object_property_set_bool(obj=0x0000000130218000, name="realized",
>     value=true, errp=0x0000000101e39e28) at object.c:1520:15
>     frame #17: 0x0000000100b7d240 qdev_realize(dev=0x0000000130218000,
>     bus=0x0000000000000000, errp=0x0000000101e39e28) at qdev.c:276:12
>     frame #18: 0x000000010083a81c
>     machvirt_init(machine=0x000000012b1fa710) at virt.c:2329:9
>     frame #19: 0x0000000100136a40
>     machine_run_board_init(machine=0x000000012b1fa710,
>     mem_path=0x0000000000000000, errp=0x000000016fdfe6a8) at
>    machine.c:1669:5
>     frame #20: 0x0000000100571384 qemu_init_board at vl.c:2714:5
>     frame #21: 0x0000000100571154
>     qmp_x_exit_preconfig(errp=0x0000000101e39e28) at vl.c:2808:5
>     frame #22: 0x0000000100573a14 qemu_init(argc=17,
>     argv=0x000000016fdff138) at vl.c:3844:9
>     frame #23: 0x0000000100d036e0 main(argc=17,
>     argv=0x000000016fdff138) at main.c:71:5
>     frame #24: 0x000000018a302b98 dyld`start + 6076
> (lldb)
>
> I expect a vCPU to be in a "stable" state and usable *after* it is
> realized, as we are calling various hooks in many places. Here we are
> processing the pending work queue while the vCPU isn't fully realized,
> so some hooks might not have been called yet...
>
> Git history of tcg_commit() points to commit 0d58c660689 ("softmmu: Use
> async_run_on_cpu in tcg_commit").
> This isn't the first time I ends there, see also:
> https://lore.kernel.org/qemu-devel/20230907161415.6102-1-philmd@linaro.org/.
> Using the same reasoning of this patch, adding:
>
> -- >8 --
> diff --git a/system/physmem.c b/system/physmem.c
> index a8a9ca309ea..479a7a88037 100644
> --- a/system/physmem.c
> +++ b/system/physmem.c
> @@ -2773,6 +2774,14 @@ static void tcg_commit(MemoryListener *listener)
>      cpuas = container_of(listener, CPUAddressSpace, tcg_as_listener);
>      cpu = cpuas->cpu;
>
> +    if (!qdev_is_realized(DEVICE(cpu))) {
> +        /*
> +         * The listener is also called during realize, before
> +         * all of the tcg machinery for run-on is initialized.
> +         */
> +        return;
> +    }
> +
>      /*
>       * Defer changes to as->memory_dispatch until the cpu is quiescent.
>       * Otherwise we race between (1) other cpu threads and (2) ongoing
> ---
>
> makes my issues disappear; tcg_commit_cpu() calls are run on realized
> vCPUs, and the order of pre-realize vcpu hooks doesn't alter anything.
>
> I don't remember why I wrote this "The listener is also called during
> realize, before all of the tcg machinery for run-on is initialized"
> comment, it could be better to call memory_region_transaction_commit()
> after CpuRealize, maybe in CpuReset.

-- 
Alex Bennée
Virtualisation Tech Lead @ Linaro