[RFC PATCH 00/19] accel: Introduce AccelvCPUState opaque structure

[RFC PATCH 00/19] accel: Introduce AccelvCPUState opaque structure

[Philippe Mathieu-Daudé]

Hi,

This series introduces the 'AccelvCPUState' which is forward
declared opaque in "cpu.h", then each accelerator define it.

The opaque CPUState::accel_vcpu pointer is shared by all
accelerators (not a problem because there can be at most
one accelerator per vCPU).

Series is organized as:
- preliminary trivial cleanups
- introduce AccelvCPUState
- move WHPX fields (build-tested)
- move HAX fields (not tested)
- move KVM fields (build-tested)
- move HVF fields (not tested)

For now vcpu_dirty is still shared in CPUState.

Sending as RFC to see if it is worthwhile.

Regards,

Phil.

Philippe Mathieu-Daudé (19):
  target/i386/hvf: Use boolean value for vcpu_dirty
  target/s390x/kvm: Simplify debug code
  target/s390x/kvm: Reduce deref by declaring 'struct kvm_run' on stack
  cpu: Croup accelerator-specific fields altogether
  cpu: Introduce AccelvCPUState opaque structure
  accel/whpx: Add typedef for 'struct whpx_vcpu'
  accel/whpx: Rename struct whpx_vcpu -> AccelvCPUState
  accel/whpx: Use 'accel_vcpu' generic pointer
  accel/hax: Add typedef for 'struct hax_vcpu_state'
  accel/hax: Use 'accel_vcpu' generic pointer
  accel/kvm: Introduce kvm_vcpu_state() helper
  accel/kvm: Use kvm_vcpu_state() when possible
  accel/kvm: Declare and allocate AccelvCPUState struct
  accel/kvm: Move the 'kvm_fd' field to AccelvCPUState
  accel/kvm: Move the 'kvm_state' field to AccelvCPUState
  accel/kvm: Move the 'kvm_run' field to AccelvCPUState
  accel/hvf: Reduce deref by declaring 'hv_vcpuid_t hvf_fd' on stack
  accel/hvf: Declare and allocate AccelvCPUState struct
  accel/hvf: Move the 'hvf_fd' field to AccelvCPUState

 include/hw/core/cpu.h         |  17 +--
 include/sysemu/kvm.h          |   2 +
 include/sysemu/kvm_int.h      |   9 ++
 target/i386/hax/hax-i386.h    |  10 +-
 target/i386/hvf/hvf-i386.h    |   4 +
 target/i386/hvf/vmx.h         |  28 +++--
 accel/kvm/kvm-all.c           |  44 ++++---
 hw/s390x/pv.c                 |   3 +-
 target/arm/kvm.c              |   2 +-
 target/arm/kvm64.c            |  12 +-
 target/i386/cpu.c             |   4 +-
 target/i386/hax/hax-all.c     |  22 ++--
 target/i386/hax/hax-posix.c   |   4 +-
 target/i386/hax/hax-windows.c |   4 +-
 target/i386/hvf/hvf.c         | 118 +++++++++---------
 target/i386/hvf/x86.c         |  28 ++---
 target/i386/hvf/x86_descr.c   |  32 +++--
 target/i386/hvf/x86_emu.c     |  62 +++++-----
 target/i386/hvf/x86_mmu.c     |   4 +-
 target/i386/hvf/x86_task.c    |  14 ++-
 target/i386/hvf/x86hvf.c      | 227 +++++++++++++++++-----------------
 target/i386/kvm/kvm.c         |  36 +++---
 target/i386/whpx/whpx-all.c   |  34 ++---
 target/ppc/kvm.c              |  16 +--
 target/s390x/kvm.c            | 148 +++++++++++-----------
 25 files changed, 466 insertions(+), 418 deletions(-)

-- 
2.26.2

[PATCH 01/19] target/i386/hvf: Use boolean value for vcpu_dirty

[Philippe Mathieu-Daudé]

CPUState::vcpu_dirty is of type 'bool', not 'integer'.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 target/i386/hvf/hvf.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/target/i386/hvf/hvf.c b/target/i386/hvf/hvf.c
index 15f14ac69e7..3c5c9c8197e 100644
--- a/target/i386/hvf/hvf.c
+++ b/target/i386/hvf/hvf.c
@@ -533,7 +533,7 @@ int hvf_init_vcpu(CPUState *cpu)
     }
 
     r = hv_vcpu_create((hv_vcpuid_t *)&cpu->hvf_fd, HV_VCPU_DEFAULT);
-    cpu->vcpu_dirty = 1;
+    cpu->vcpu_dirty = true;
     assert_hvf_ok(r);
 
     if (hv_vmx_read_capability(HV_VMX_CAP_PINBASED,
-- 
2.26.2

[PATCH 02/19] target/s390x/kvm: Simplify debug code

[Philippe Mathieu-Daudé]

We already have the 'run' variable holding 'cs->kvm_run' value.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 target/s390x/kvm.c | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)

diff --git a/target/s390x/kvm.c b/target/s390x/kvm.c
index 7a892d663df..73f816a7222 100644
--- a/target/s390x/kvm.c
+++ b/target/s390x/kvm.c
@@ -1785,8 +1785,7 @@ static int handle_intercept(S390CPU *cpu)
     int icpt_code = run->s390_sieic.icptcode;
     int r = 0;
 
-    DPRINTF("intercept: 0x%x (at 0x%lx)\n", icpt_code,
-            (long)cs->kvm_run->psw_addr);
+    DPRINTF("intercept: 0x%x (at 0x%lx)\n", icpt_code, (long)run->psw_addr);
     switch (icpt_code) {
         case ICPT_INSTRUCTION:
         case ICPT_PV_INSTR:
-- 
2.26.2

Re: [PATCH 02/19] target/s390x/kvm: Simplify debug code

[Thomas Huth]

On 03/03/2021 19.22, Philippe Mathieu-Daudé wrote:
> We already have the 'run' variable holding 'cs->kvm_run' value.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>   target/s390x/kvm.c | 3 +--
>   1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/target/s390x/kvm.c b/target/s390x/kvm.c
> index 7a892d663df..73f816a7222 100644
> --- a/target/s390x/kvm.c
> +++ b/target/s390x/kvm.c
> @@ -1785,8 +1785,7 @@ static int handle_intercept(S390CPU *cpu)
>       int icpt_code = run->s390_sieic.icptcode;
>       int r = 0;
>   
> -    DPRINTF("intercept: 0x%x (at 0x%lx)\n", icpt_code,
> -            (long)cs->kvm_run->psw_addr);
> +    DPRINTF("intercept: 0x%x (at 0x%lx)\n", icpt_code, (long)run->psw_addr);
>       switch (icpt_code) {
>           case ICPT_INSTRUCTION:
>           case ICPT_PV_INSTR:
> 

Reviewed-by: Thomas Huth <thuth@redhat.com>

Re: [PATCH 02/19] target/s390x/kvm: Simplify debug code

[Cornelia Huck]

On Wed,  3 Mar 2021 19:22:02 +0100
Philippe Mathieu-Daudé <philmd@redhat.com> wrote:

> We already have the 'run' variable holding 'cs->kvm_run' value.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>  target/s390x/kvm.c | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
> 
> diff --git a/target/s390x/kvm.c b/target/s390x/kvm.c
> index 7a892d663df..73f816a7222 100644
> --- a/target/s390x/kvm.c
> +++ b/target/s390x/kvm.c
> @@ -1785,8 +1785,7 @@ static int handle_intercept(S390CPU *cpu)
>      int icpt_code = run->s390_sieic.icptcode;
>      int r = 0;
>  
> -    DPRINTF("intercept: 0x%x (at 0x%lx)\n", icpt_code,
> -            (long)cs->kvm_run->psw_addr);
> +    DPRINTF("intercept: 0x%x (at 0x%lx)\n", icpt_code, (long)run->psw_addr);
>      switch (icpt_code) {
>          case ICPT_INSTRUCTION:
>          case ICPT_PV_INSTR:

Thanks, queued this one to s390-next.

[PATCH 03/19] target/s390x/kvm: Reduce deref by declaring 'struct kvm_run' on stack

[Philippe Mathieu-Daudé]

In order to make one of the next commits easier to review,
declare 'struct kvm_run' on the stack when it is used in
various places in a function.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 target/s390x/kvm.c | 128 +++++++++++++++++++++++----------------------
 1 file changed, 65 insertions(+), 63 deletions(-)

diff --git a/target/s390x/kvm.c b/target/s390x/kvm.c
index 73f816a7222..d8ac12dfc11 100644
--- a/target/s390x/kvm.c
+++ b/target/s390x/kvm.c
@@ -467,6 +467,7 @@ int kvm_arch_put_registers(CPUState *cs, int level)
 {
     S390CPU *cpu = S390_CPU(cs);
     CPUS390XState *env = &cpu->env;
+    struct kvm_run *run = cs->kvm_run;
     struct kvm_sregs sregs;
     struct kvm_regs regs;
     struct kvm_fpu fpu = {};
@@ -474,13 +475,13 @@ int kvm_arch_put_registers(CPUState *cs, int level)
     int i;
 
     /* always save the PSW  and the GPRS*/
-    cs->kvm_run->psw_addr = env->psw.addr;
-    cs->kvm_run->psw_mask = env->psw.mask;
+    run->psw_addr = env->psw.addr;
+    run->psw_mask = env->psw.mask;
 
     if (can_sync_regs(cs, KVM_SYNC_GPRS)) {
         for (i = 0; i < 16; i++) {
-            cs->kvm_run->s.regs.gprs[i] = env->regs[i];
-            cs->kvm_run->kvm_dirty_regs |= KVM_SYNC_GPRS;
+            run->s.regs.gprs[i] = env->regs[i];
+            run->kvm_dirty_regs |= KVM_SYNC_GPRS;
         }
     } else {
         for (i = 0; i < 16; i++) {
@@ -494,17 +495,17 @@ int kvm_arch_put_registers(CPUState *cs, int level)
 
     if (can_sync_regs(cs, KVM_SYNC_VRS)) {
         for (i = 0; i < 32; i++) {
-            cs->kvm_run->s.regs.vrs[i][0] = env->vregs[i][0];
-            cs->kvm_run->s.regs.vrs[i][1] = env->vregs[i][1];
+            run->s.regs.vrs[i][0] = env->vregs[i][0];
+            run->s.regs.vrs[i][1] = env->vregs[i][1];
         }
-        cs->kvm_run->s.regs.fpc = env->fpc;
-        cs->kvm_run->kvm_dirty_regs |= KVM_SYNC_VRS;
+        run->s.regs.fpc = env->fpc;
+        run->kvm_dirty_regs |= KVM_SYNC_VRS;
     } else if (can_sync_regs(cs, KVM_SYNC_FPRS)) {
         for (i = 0; i < 16; i++) {
-            cs->kvm_run->s.regs.fprs[i] = *get_freg(env, i);
+            run->s.regs.fprs[i] = *get_freg(env, i);
         }
-        cs->kvm_run->s.regs.fpc = env->fpc;
-        cs->kvm_run->kvm_dirty_regs |= KVM_SYNC_FPRS;
+        run->s.regs.fpc = env->fpc;
+        run->kvm_dirty_regs |= KVM_SYNC_FPRS;
     } else {
         /* Floating point */
         for (i = 0; i < 16; i++) {
@@ -524,12 +525,12 @@ int kvm_arch_put_registers(CPUState *cs, int level)
     }
 
     if (can_sync_regs(cs, KVM_SYNC_ARCH0)) {
-        cs->kvm_run->s.regs.cputm = env->cputm;
-        cs->kvm_run->s.regs.ckc = env->ckc;
-        cs->kvm_run->s.regs.todpr = env->todpr;
-        cs->kvm_run->s.regs.gbea = env->gbea;
-        cs->kvm_run->s.regs.pp = env->pp;
-        cs->kvm_run->kvm_dirty_regs |= KVM_SYNC_ARCH0;
+        run->s.regs.cputm = env->cputm;
+        run->s.regs.ckc = env->ckc;
+        run->s.regs.todpr = env->todpr;
+        run->s.regs.gbea = env->gbea;
+        run->s.regs.pp = env->pp;
+        run->kvm_dirty_regs |= KVM_SYNC_ARCH0;
     } else {
         /*
          * These ONE_REGS are not protected by a capability. As they are only
@@ -544,16 +545,16 @@ int kvm_arch_put_registers(CPUState *cs, int level)
     }
 
     if (can_sync_regs(cs, KVM_SYNC_RICCB)) {
-        memcpy(cs->kvm_run->s.regs.riccb, env->riccb, 64);
-        cs->kvm_run->kvm_dirty_regs |= KVM_SYNC_RICCB;
+        memcpy(run->s.regs.riccb, env->riccb, 64);
+        run->kvm_dirty_regs |= KVM_SYNC_RICCB;
     }
 
     /* pfault parameters */
     if (can_sync_regs(cs, KVM_SYNC_PFAULT)) {
-        cs->kvm_run->s.regs.pft = env->pfault_token;
-        cs->kvm_run->s.regs.pfs = env->pfault_select;
-        cs->kvm_run->s.regs.pfc = env->pfault_compare;
-        cs->kvm_run->kvm_dirty_regs |= KVM_SYNC_PFAULT;
+        run->s.regs.pft = env->pfault_token;
+        run->s.regs.pfs = env->pfault_select;
+        run->s.regs.pfc = env->pfault_compare;
+        run->kvm_dirty_regs |= KVM_SYNC_PFAULT;
     } else if (cap_async_pf) {
         r = kvm_set_one_reg(cs, KVM_REG_S390_PFTOKEN, &env->pfault_token);
         if (r < 0) {
@@ -572,11 +573,11 @@ int kvm_arch_put_registers(CPUState *cs, int level)
     /* access registers and control registers*/
     if (can_sync_regs(cs, KVM_SYNC_ACRS | KVM_SYNC_CRS)) {
         for (i = 0; i < 16; i++) {
-            cs->kvm_run->s.regs.acrs[i] = env->aregs[i];
-            cs->kvm_run->s.regs.crs[i] = env->cregs[i];
+            run->s.regs.acrs[i] = env->aregs[i];
+            run->s.regs.crs[i] = env->cregs[i];
         }
-        cs->kvm_run->kvm_dirty_regs |= KVM_SYNC_ACRS;
-        cs->kvm_run->kvm_dirty_regs |= KVM_SYNC_CRS;
+        run->kvm_dirty_regs |= KVM_SYNC_ACRS;
+        run->kvm_dirty_regs |= KVM_SYNC_CRS;
     } else {
         for (i = 0; i < 16; i++) {
             sregs.acrs[i] = env->aregs[i];
@@ -589,30 +590,30 @@ int kvm_arch_put_registers(CPUState *cs, int level)
     }
 
     if (can_sync_regs(cs, KVM_SYNC_GSCB)) {
-        memcpy(cs->kvm_run->s.regs.gscb, env->gscb, 32);
-        cs->kvm_run->kvm_dirty_regs |= KVM_SYNC_GSCB;
+        memcpy(run->s.regs.gscb, env->gscb, 32);
+        run->kvm_dirty_regs |= KVM_SYNC_GSCB;
     }
 
     if (can_sync_regs(cs, KVM_SYNC_BPBC)) {
-        cs->kvm_run->s.regs.bpbc = env->bpbc;
-        cs->kvm_run->kvm_dirty_regs |= KVM_SYNC_BPBC;
+        run->s.regs.bpbc = env->bpbc;
+        run->kvm_dirty_regs |= KVM_SYNC_BPBC;
     }
 
     if (can_sync_regs(cs, KVM_SYNC_ETOKEN)) {
-        cs->kvm_run->s.regs.etoken = env->etoken;
-        cs->kvm_run->s.regs.etoken_extension  = env->etoken_extension;
-        cs->kvm_run->kvm_dirty_regs |= KVM_SYNC_ETOKEN;
+        run->s.regs.etoken = env->etoken;
+        run->s.regs.etoken_extension  = env->etoken_extension;
+        run->kvm_dirty_regs |= KVM_SYNC_ETOKEN;
     }
 
     if (can_sync_regs(cs, KVM_SYNC_DIAG318)) {
-        cs->kvm_run->s.regs.diag318 = env->diag318_info;
-        cs->kvm_run->kvm_dirty_regs |= KVM_SYNC_DIAG318;
+        run->s.regs.diag318 = env->diag318_info;
+        run->kvm_dirty_regs |= KVM_SYNC_DIAG318;
     }
 
     /* Finally the prefix */
     if (can_sync_regs(cs, KVM_SYNC_PREFIX)) {
-        cs->kvm_run->s.regs.prefix = env->psa;
-        cs->kvm_run->kvm_dirty_regs |= KVM_SYNC_PREFIX;
+        run->s.regs.prefix = env->psa;
+        run->kvm_dirty_regs |= KVM_SYNC_PREFIX;
     } else {
         /* prefix is only supported via sync regs */
     }
@@ -623,19 +624,20 @@ int kvm_arch_get_registers(CPUState *cs)
 {
     S390CPU *cpu = S390_CPU(cs);
     CPUS390XState *env = &cpu->env;
+    struct kvm_run *run = cs->kvm_run;
     struct kvm_sregs sregs;
     struct kvm_regs regs;
     struct kvm_fpu fpu;
     int i, r;
 
     /* get the PSW */
-    env->psw.addr = cs->kvm_run->psw_addr;
-    env->psw.mask = cs->kvm_run->psw_mask;
+    env->psw.addr = run->psw_addr;
+    env->psw.mask = run->psw_mask;
 
     /* the GPRS */
     if (can_sync_regs(cs, KVM_SYNC_GPRS)) {
         for (i = 0; i < 16; i++) {
-            env->regs[i] = cs->kvm_run->s.regs.gprs[i];
+            env->regs[i] = run->s.regs.gprs[i];
         }
     } else {
         r = kvm_vcpu_ioctl(cs, KVM_GET_REGS, &regs);
@@ -650,8 +652,8 @@ int kvm_arch_get_registers(CPUState *cs)
     /* The ACRS and CRS */
     if (can_sync_regs(cs, KVM_SYNC_ACRS | KVM_SYNC_CRS)) {
         for (i = 0; i < 16; i++) {
-            env->aregs[i] = cs->kvm_run->s.regs.acrs[i];
-            env->cregs[i] = cs->kvm_run->s.regs.crs[i];
+            env->aregs[i] = run->s.regs.acrs[i];
+            env->cregs[i] = run->s.regs.crs[i];
         }
     } else {
         r = kvm_vcpu_ioctl(cs, KVM_GET_SREGS, &sregs);
@@ -667,15 +669,15 @@ int kvm_arch_get_registers(CPUState *cs)
     /* Floating point and vector registers */
     if (can_sync_regs(cs, KVM_SYNC_VRS)) {
         for (i = 0; i < 32; i++) {
-            env->vregs[i][0] = cs->kvm_run->s.regs.vrs[i][0];
-            env->vregs[i][1] = cs->kvm_run->s.regs.vrs[i][1];
+            env->vregs[i][0] = run->s.regs.vrs[i][0];
+            env->vregs[i][1] = run->s.regs.vrs[i][1];
         }
-        env->fpc = cs->kvm_run->s.regs.fpc;
+        env->fpc = run->s.regs.fpc;
     } else if (can_sync_regs(cs, KVM_SYNC_FPRS)) {
         for (i = 0; i < 16; i++) {
-            *get_freg(env, i) = cs->kvm_run->s.regs.fprs[i];
+            *get_freg(env, i) = run->s.regs.fprs[i];
         }
-        env->fpc = cs->kvm_run->s.regs.fpc;
+        env->fpc = run->s.regs.fpc;
     } else {
         r = kvm_vcpu_ioctl(cs, KVM_GET_FPU, &fpu);
         if (r < 0) {
@@ -689,15 +691,15 @@ int kvm_arch_get_registers(CPUState *cs)
 
     /* The prefix */
     if (can_sync_regs(cs, KVM_SYNC_PREFIX)) {
-        env->psa = cs->kvm_run->s.regs.prefix;
+        env->psa = run->s.regs.prefix;
     }
 
     if (can_sync_regs(cs, KVM_SYNC_ARCH0)) {
-        env->cputm = cs->kvm_run->s.regs.cputm;
-        env->ckc = cs->kvm_run->s.regs.ckc;
-        env->todpr = cs->kvm_run->s.regs.todpr;
-        env->gbea = cs->kvm_run->s.regs.gbea;
-        env->pp = cs->kvm_run->s.regs.pp;
+        env->cputm = run->s.regs.cputm;
+        env->ckc = run->s.regs.ckc;
+        env->todpr = run->s.regs.todpr;
+        env->gbea = run->s.regs.gbea;
+        env->pp = run->s.regs.pp;
     } else {
         /*
          * These ONE_REGS are not protected by a capability. As they are only
@@ -712,27 +714,27 @@ int kvm_arch_get_registers(CPUState *cs)
     }
 
     if (can_sync_regs(cs, KVM_SYNC_RICCB)) {
-        memcpy(env->riccb, cs->kvm_run->s.regs.riccb, 64);
+        memcpy(env->riccb, run->s.regs.riccb, 64);
     }
 
     if (can_sync_regs(cs, KVM_SYNC_GSCB)) {
-        memcpy(env->gscb, cs->kvm_run->s.regs.gscb, 32);
+        memcpy(env->gscb, run->s.regs.gscb, 32);
     }
 
     if (can_sync_regs(cs, KVM_SYNC_BPBC)) {
-        env->bpbc = cs->kvm_run->s.regs.bpbc;
+        env->bpbc = run->s.regs.bpbc;
     }
 
     if (can_sync_regs(cs, KVM_SYNC_ETOKEN)) {
-        env->etoken = cs->kvm_run->s.regs.etoken;
-        env->etoken_extension = cs->kvm_run->s.regs.etoken_extension;
+        env->etoken = run->s.regs.etoken;
+        env->etoken_extension = run->s.regs.etoken_extension;
     }
 
     /* pfault parameters */
     if (can_sync_regs(cs, KVM_SYNC_PFAULT)) {
-        env->pfault_token = cs->kvm_run->s.regs.pft;
-        env->pfault_select = cs->kvm_run->s.regs.pfs;
-        env->pfault_compare = cs->kvm_run->s.regs.pfc;
+        env->pfault_token = run->s.regs.pft;
+        env->pfault_select = run->s.regs.pfs;
+        env->pfault_compare = run->s.regs.pfc;
     } else if (cap_async_pf) {
         r = kvm_get_one_reg(cs, KVM_REG_S390_PFTOKEN, &env->pfault_token);
         if (r < 0) {
@@ -749,7 +751,7 @@ int kvm_arch_get_registers(CPUState *cs)
     }
 
     if (can_sync_regs(cs, KVM_SYNC_DIAG318)) {
-        env->diag318_info = cs->kvm_run->s.regs.diag318;
+        env->diag318_info = run->s.regs.diag318;
     }
 
     return 0;
-- 
2.26.2

[PATCH 04/19] cpu: Croup accelerator-specific fields altogether

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 include/hw/core/cpu.h | 14 ++++++--------
 1 file changed, 6 insertions(+), 8 deletions(-)

diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index c005d3dc2d8..074199ce73c 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -393,10 +393,6 @@ struct CPUState {
      */
     uintptr_t mem_io_pc;
 
-    int kvm_fd;
-    struct KVMState *kvm_state;
-    struct kvm_run *kvm_run;
-
     /* Used for events with 'vcpu' and *without* the 'disabled' properties */
     DECLARE_BITMAP(trace_dstate_delayed, CPU_TRACE_DSTATE_MAX_EVENTS);
     DECLARE_BITMAP(trace_dstate, CPU_TRACE_DSTATE_MAX_EVENTS);
@@ -416,6 +412,12 @@ struct CPUState {
     uint32_t can_do_io;
     int32_t exception_index;
 
+    /* Accelerator-specific fields. */
+    int kvm_fd;
+    struct KVMState *kvm_state;
+    struct kvm_run *kvm_run;
+    struct hax_vcpu_state *hax_vcpu;
+    int hvf_fd;
     /* shared by kvm, hax and hvf */
     bool vcpu_dirty;
 
@@ -426,10 +428,6 @@ struct CPUState {
 
     bool ignore_memory_transaction_failures;
 
-    struct hax_vcpu_state *hax_vcpu;
-
-    int hvf_fd;
-
     /* track IOMMUs whose translations we've cached in the TCG TLB */
     GArray *iommu_notifiers;
 };
-- 
2.26.2

Re: [PATCH 04/19] cpu: Croup accelerator-specific fields altogether

[Philippe Mathieu-Daudé]

On 3/3/21 7:22 PM, Philippe Mathieu-Daudé wrote:
> Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
> ---
>  include/hw/core/cpu.h | 14 ++++++--------
>  1 file changed, 6 insertions(+), 8 deletions(-)

Typo in patch subject "Group" ;)

> 
> diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
> index c005d3dc2d8..074199ce73c 100644
> --- a/include/hw/core/cpu.h
> +++ b/include/hw/core/cpu.h
> @@ -393,10 +393,6 @@ struct CPUState {
>       */
>      uintptr_t mem_io_pc;
>  
> -    int kvm_fd;
> -    struct KVMState *kvm_state;
> -    struct kvm_run *kvm_run;
> -
>      /* Used for events with 'vcpu' and *without* the 'disabled' properties */
>      DECLARE_BITMAP(trace_dstate_delayed, CPU_TRACE_DSTATE_MAX_EVENTS);
>      DECLARE_BITMAP(trace_dstate, CPU_TRACE_DSTATE_MAX_EVENTS);
> @@ -416,6 +412,12 @@ struct CPUState {
>      uint32_t can_do_io;
>      int32_t exception_index;
>  
> +    /* Accelerator-specific fields. */
> +    int kvm_fd;
> +    struct KVMState *kvm_state;
> +    struct kvm_run *kvm_run;
> +    struct hax_vcpu_state *hax_vcpu;
> +    int hvf_fd;
>      /* shared by kvm, hax and hvf */
>      bool vcpu_dirty;
>  
> @@ -426,10 +428,6 @@ struct CPUState {
>  
>      bool ignore_memory_transaction_failures;
>  
> -    struct hax_vcpu_state *hax_vcpu;
> -
> -    int hvf_fd;
> -
>      /* track IOMMUs whose translations we've cached in the TCG TLB */
>      GArray *iommu_notifiers;
>  };
> 

[RFC PATCH 05/19] cpu: Introduce AccelvCPUState opaque structure

[Philippe Mathieu-Daudé]

Introduce the opaque 'AccelvCPUState' structure which will
be declared by each accelerator. Forward-declare it in "cpu.h".

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 include/hw/core/cpu.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index 074199ce73c..d807645af2b 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -274,6 +274,9 @@ struct qemu_work_item;
 #define CPU_UNSET_NUMA_NODE_ID -1
 #define CPU_TRACE_DSTATE_MAX_EVENTS 32
 
+/* This structure is defined by each accelerator. */
+struct AccelvCPUState;
+
 /**
  * CPUState:
  * @cpu_index: CPU index (informative).
@@ -312,6 +315,7 @@ struct qemu_work_item;
  * @next_cpu: Next CPU sharing TB cache.
  * @opaque: User data.
  * @mem_io_pc: Host Program Counter at which the memory was accessed.
+ * @accel_vcpu: Pointer to accelerator-specific AccelvCPUState field.
  * @kvm_fd: vCPU file descriptor for KVM.
  * @work_mutex: Lock to prevent multiple access to @work_list.
  * @work_list: List of pending asynchronous work.
@@ -413,6 +417,7 @@ struct CPUState {
     int32_t exception_index;
 
     /* Accelerator-specific fields. */
+    struct AccelvCPUState *accel_vcpu;
     int kvm_fd;
     struct KVMState *kvm_state;
     struct kvm_run *kvm_run;
-- 
2.26.2

[RFC PATCH 06/19] accel/whpx: Add typedef for 'struct whpx_vcpu'

[Philippe Mathieu-Daudé]

Use the 'whpx_vcpu' typedef instead of 'struct whpx_vcpu'.
This will make the next commits easier to review.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 target/i386/whpx/whpx-all.c | 28 +++++++++++++++-------------
 1 file changed, 15 insertions(+), 13 deletions(-)

diff --git a/target/i386/whpx/whpx-all.c b/target/i386/whpx/whpx-all.c
index f0a35df3bba..6469e388b6d 100644
--- a/target/i386/whpx/whpx-all.c
+++ b/target/i386/whpx/whpx-all.c
@@ -148,6 +148,8 @@ struct whpx_register_set {
     WHV_REGISTER_VALUE values[RTL_NUMBER_OF(whpx_register_names)];
 };
 
+typedef struct whpx_vcpu whpx_vcpu;
+
 struct whpx_vcpu {
     WHV_EMULATOR_HANDLE emulator;
     bool window_registered;
@@ -173,9 +175,9 @@ struct WHPDispatch whp_dispatch;
  * VP support
  */
 
-static struct whpx_vcpu *get_whpx_vcpu(CPUState *cpu)
+static whpx_vcpu *get_whpx_vcpu(CPUState *cpu)
 {
-    return (struct whpx_vcpu *)cpu->hax_vcpu;
+    return (whpx_vcpu *)cpu->hax_vcpu;
 }
 
 static WHV_X64_SEGMENT_REGISTER whpx_seg_q2h(const SegmentCache *qs, int v86,
@@ -259,7 +261,7 @@ static int whpx_set_tsc(CPUState *cpu)
 static void whpx_set_registers(CPUState *cpu, int level)
 {
     struct whpx_state *whpx = &whpx_global;
-    struct whpx_vcpu *vcpu = get_whpx_vcpu(cpu);
+    whpx_vcpu *vcpu = get_whpx_vcpu(cpu);
     struct CPUX86State *env = (CPUArchState *)(cpu->env_ptr);
     X86CPU *x86_cpu = X86_CPU(cpu);
     struct whpx_register_set vcxt;
@@ -448,7 +450,7 @@ static int whpx_get_tsc(CPUState *cpu)
 static void whpx_get_registers(CPUState *cpu)
 {
     struct whpx_state *whpx = &whpx_global;
-    struct whpx_vcpu *vcpu = get_whpx_vcpu(cpu);
+    whpx_vcpu *vcpu = get_whpx_vcpu(cpu);
     struct CPUX86State *env = (CPUArchState *)(cpu->env_ptr);
     X86CPU *x86_cpu = X86_CPU(cpu);
     struct whpx_register_set vcxt;
@@ -712,7 +714,7 @@ static const WHV_EMULATOR_CALLBACKS whpx_emu_callbacks = {
 static int whpx_handle_mmio(CPUState *cpu, WHV_MEMORY_ACCESS_CONTEXT *ctx)
 {
     HRESULT hr;
-    struct whpx_vcpu *vcpu = get_whpx_vcpu(cpu);
+    whpx_vcpu *vcpu = get_whpx_vcpu(cpu);
     WHV_EMULATOR_STATUS emu_status;
 
     hr = whp_dispatch.WHvEmulatorTryMmioEmulation(
@@ -737,7 +739,7 @@ static int whpx_handle_portio(CPUState *cpu,
                               WHV_X64_IO_PORT_ACCESS_CONTEXT *ctx)
 {
     HRESULT hr;
-    struct whpx_vcpu *vcpu = get_whpx_vcpu(cpu);
+    whpx_vcpu *vcpu = get_whpx_vcpu(cpu);
     WHV_EMULATOR_STATUS emu_status;
 
     hr = whp_dispatch.WHvEmulatorTryIoEmulation(
@@ -780,7 +782,7 @@ static void whpx_vcpu_pre_run(CPUState *cpu)
 {
     HRESULT hr;
     struct whpx_state *whpx = &whpx_global;
-    struct whpx_vcpu *vcpu = get_whpx_vcpu(cpu);
+    whpx_vcpu *vcpu = get_whpx_vcpu(cpu);
     struct CPUX86State *env = (CPUArchState *)(cpu->env_ptr);
     X86CPU *x86_cpu = X86_CPU(cpu);
     int irq;
@@ -902,7 +904,7 @@ static void whpx_vcpu_pre_run(CPUState *cpu)
 
 static void whpx_vcpu_post_run(CPUState *cpu)
 {
-    struct whpx_vcpu *vcpu = get_whpx_vcpu(cpu);
+    whpx_vcpu *vcpu = get_whpx_vcpu(cpu);
     struct CPUX86State *env = (CPUArchState *)(cpu->env_ptr);
     X86CPU *x86_cpu = X86_CPU(cpu);
 
@@ -929,7 +931,7 @@ static void whpx_vcpu_process_async_events(CPUState *cpu)
 {
     struct CPUX86State *env = (CPUArchState *)(cpu->env_ptr);
     X86CPU *x86_cpu = X86_CPU(cpu);
-    struct whpx_vcpu *vcpu = get_whpx_vcpu(cpu);
+    whpx_vcpu *vcpu = get_whpx_vcpu(cpu);
 
     if ((cpu->interrupt_request & CPU_INTERRUPT_INIT) &&
         !(env->hflags & HF_SMM_MASK)) {
@@ -968,7 +970,7 @@ static int whpx_vcpu_run(CPUState *cpu)
 {
     HRESULT hr;
     struct whpx_state *whpx = &whpx_global;
-    struct whpx_vcpu *vcpu = get_whpx_vcpu(cpu);
+    whpx_vcpu *vcpu = get_whpx_vcpu(cpu);
     int ret;
 
     whpx_vcpu_process_async_events(cpu);
@@ -1331,7 +1333,7 @@ int whpx_init_vcpu(CPUState *cpu)
 {
     HRESULT hr;
     struct whpx_state *whpx = &whpx_global;
-    struct whpx_vcpu *vcpu = NULL;
+    whpx_vcpu *vcpu = NULL;
     Error *local_error = NULL;
     struct CPUX86State *env = (CPUArchState *)(cpu->env_ptr);
     X86CPU *x86_cpu = X86_CPU(cpu);
@@ -1356,7 +1358,7 @@ int whpx_init_vcpu(CPUState *cpu)
         }
     }
 
-    vcpu = g_malloc0(sizeof(struct whpx_vcpu));
+    vcpu = g_malloc0(sizeof(whpx_vcpu));
 
     if (!vcpu) {
         error_report("WHPX: Failed to allocte VCPU context.");
@@ -1475,7 +1477,7 @@ int whpx_vcpu_exec(CPUState *cpu)
 void whpx_destroy_vcpu(CPUState *cpu)
 {
     struct whpx_state *whpx = &whpx_global;
-    struct whpx_vcpu *vcpu = get_whpx_vcpu(cpu);
+    whpx_vcpu *vcpu = get_whpx_vcpu(cpu);
 
     whp_dispatch.WHvDeleteVirtualProcessor(whpx->partition, cpu->cpu_index);
     whp_dispatch.WHvEmulatorDestroyEmulator(vcpu->emulator);
-- 
2.26.2

[RFC PATCH 07/19] accel/whpx: Rename struct whpx_vcpu -> AccelvCPUState

[Philippe Mathieu-Daudé]

The current 'struct whpx_vcpu' contains the vCPU fields
specific to the WHPX accelerator. Rename it as AccelvCPUState.
We keep the 'whpx_vcpu' typedef to reduce the amount of code
changed.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 target/i386/whpx/whpx-all.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/target/i386/whpx/whpx-all.c b/target/i386/whpx/whpx-all.c
index 6469e388b6d..f0b3266114d 100644
--- a/target/i386/whpx/whpx-all.c
+++ b/target/i386/whpx/whpx-all.c
@@ -148,9 +148,9 @@ struct whpx_register_set {
     WHV_REGISTER_VALUE values[RTL_NUMBER_OF(whpx_register_names)];
 };
 
-typedef struct whpx_vcpu whpx_vcpu;
+typedef struct AccelvCPUState whpx_vcpu;
 
-struct whpx_vcpu {
+struct AccelvCPUState {
     WHV_EMULATOR_HANDLE emulator;
     bool window_registered;
     bool interruptable;
-- 
2.26.2

[RFC PATCH 08/19] accel/whpx: Use 'accel_vcpu' generic pointer

[Philippe Mathieu-Daudé]

Instead of naming the HAX accelerator in WHPX, use the
'accel_vcpu' field which is meant for accelerators.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 target/i386/whpx/whpx-all.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/target/i386/whpx/whpx-all.c b/target/i386/whpx/whpx-all.c
index f0b3266114d..56ec82076cc 100644
--- a/target/i386/whpx/whpx-all.c
+++ b/target/i386/whpx/whpx-all.c
@@ -177,7 +177,7 @@ struct WHPDispatch whp_dispatch;
 
 static whpx_vcpu *get_whpx_vcpu(CPUState *cpu)
 {
-    return (whpx_vcpu *)cpu->hax_vcpu;
+    return cpu->accel_vcpu;
 }
 
 static WHV_X64_SEGMENT_REGISTER whpx_seg_q2h(const SegmentCache *qs, int v86,
@@ -1439,7 +1439,7 @@ int whpx_init_vcpu(CPUState *cpu)
 
     vcpu->interruptable = true;
     cpu->vcpu_dirty = true;
-    cpu->hax_vcpu = (struct hax_vcpu_state *)vcpu;
+    cpu->accel_vcpu = vcpu;
     max_vcpu_index = max(max_vcpu_index, cpu->cpu_index);
     qemu_add_vm_change_state_handler(whpx_cpu_update_state, cpu->env_ptr);
 
@@ -1481,7 +1481,7 @@ void whpx_destroy_vcpu(CPUState *cpu)
 
     whp_dispatch.WHvDeleteVirtualProcessor(whpx->partition, cpu->cpu_index);
     whp_dispatch.WHvEmulatorDestroyEmulator(vcpu->emulator);
-    g_free(cpu->hax_vcpu);
+    g_free(cpu->accel_vcpu);
     return;
 }
 
-- 
2.26.2

[RFC PATCH 09/19] accel/hax: Add typedef for 'struct hax_vcpu_state'

[Philippe Mathieu-Daudé]

Use the 'hax_vcpu_state' typedef instead of 'struct hax_vcpu_state'.
This will make the next commits easier to review.

Beside the typedef addition, patch created mechanically using:

  $ sed -i s/struct\ hax_vcpu_state/hax_vcpu_state/ \
      $(git grep -l 'struct hax_vcpu_state')

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 target/i386/hax/hax-i386.h    |  8 +++++---
 target/i386/hax/hax-all.c     | 18 +++++++++---------
 target/i386/hax/hax-posix.c   |  4 ++--
 target/i386/hax/hax-windows.c |  4 ++--
 4 files changed, 18 insertions(+), 16 deletions(-)

diff --git a/target/i386/hax/hax-i386.h b/target/i386/hax/hax-i386.h
index efbb3462389..ee77406a6a6 100644
--- a/target/i386/hax/hax-i386.h
+++ b/target/i386/hax/hax-i386.h
@@ -25,6 +25,8 @@ typedef HANDLE hax_fd;
 #endif
 
 extern struct hax_state hax_global;
+
+typedef struct hax_vcpu_state hax_vcpu_state;
 struct hax_vcpu_state {
     hax_fd fd;
     int vcpu_id;
@@ -46,7 +48,7 @@ struct hax_vm {
     hax_fd fd;
     int id;
     int numvcpus;
-    struct hax_vcpu_state **vcpus;
+    hax_vcpu_state **vcpus;
 };
 
 #ifdef NEED_CPU_H
@@ -58,7 +60,7 @@ int valid_hax_tunnel_size(uint16_t size);
 int hax_mod_version(struct hax_state *hax, struct hax_module_version *version);
 int hax_inject_interrupt(CPUArchState *env, int vector);
 struct hax_vm *hax_vm_create(struct hax_state *hax, int max_cpus);
-int hax_vcpu_run(struct hax_vcpu_state *vcpu);
+int hax_vcpu_run(hax_vcpu_state *vcpu);
 int hax_vcpu_create(int id);
 void hax_kick_vcpu_thread(CPUState *cpu);
 
@@ -78,7 +80,7 @@ int hax_host_create_vm(struct hax_state *hax, int *vm_id);
 hax_fd hax_host_open_vm(struct hax_state *hax, int vm_id);
 int hax_host_create_vcpu(hax_fd vm_fd, int vcpuid);
 hax_fd hax_host_open_vcpu(int vmid, int vcpuid);
-int hax_host_setup_vcpu_channel(struct hax_vcpu_state *vcpu);
+int hax_host_setup_vcpu_channel(hax_vcpu_state *vcpu);
 hax_fd hax_mod_open(void);
 void hax_memory_init(void);
 
diff --git a/target/i386/hax/hax-all.c b/target/i386/hax/hax-all.c
index bf65ed6fa92..08c2b60b437 100644
--- a/target/i386/hax/hax-all.c
+++ b/target/i386/hax/hax-all.c
@@ -68,7 +68,7 @@ int valid_hax_tunnel_size(uint16_t size)
 
 hax_fd hax_vcpu_get_fd(CPUArchState *env)
 {
-    struct hax_vcpu_state *vcpu = env_cpu(env)->hax_vcpu;
+    hax_vcpu_state *vcpu = env_cpu(env)->hax_vcpu;
     if (!vcpu) {
         return HAX_INVALID_FD;
     }
@@ -142,7 +142,7 @@ static int hax_version_support(struct hax_state *hax)
 
 int hax_vcpu_create(int id)
 {
-    struct hax_vcpu_state *vcpu = NULL;
+    hax_vcpu_state *vcpu = NULL;
     int ret;
 
     if (!hax_global.vm) {
@@ -155,7 +155,7 @@ int hax_vcpu_create(int id)
         return 0;
     }
 
-    vcpu = g_new0(struct hax_vcpu_state, 1);
+    vcpu = g_new0(hax_vcpu_state, 1);
 
     ret = hax_host_create_vcpu(hax_global.vm->fd, id);
     if (ret) {
@@ -194,7 +194,7 @@ int hax_vcpu_create(int id)
 
 int hax_vcpu_destroy(CPUState *cpu)
 {
-    struct hax_vcpu_state *vcpu = cpu->hax_vcpu;
+    hax_vcpu_state *vcpu = cpu->hax_vcpu;
 
     if (!hax_global.vm) {
         fprintf(stderr, "vcpu %x destroy failed, vm is null\n", vcpu->vcpu_id);
@@ -225,7 +225,7 @@ int hax_init_vcpu(CPUState *cpu)
         exit(-1);
     }
 
-    cpu->hax_vcpu = hax_global.vm->vcpus[cpu->cpu_index];
+    cpu->accel_vcpu = hax_global.vm->vcpus[cpu->cpu_index];
     cpu->vcpu_dirty = true;
     qemu_register_reset(hax_reset_vcpu_state, (CPUArchState *) (cpu->env_ptr));
 
@@ -265,7 +265,7 @@ struct hax_vm *hax_vm_create(struct hax_state *hax, int max_cpus)
     }
 
     vm->numvcpus = max_cpus;
-    vm->vcpus = g_new0(struct hax_vcpu_state *, vm->numvcpus);
+    vm->vcpus = g_new0(hax_vcpu_state *, vm->numvcpus);
     for (i = 0; i < vm->numvcpus; i++) {
         vm->vcpus[i] = NULL;
     }
@@ -414,7 +414,7 @@ static int hax_handle_io(CPUArchState *env, uint32_t df, uint16_t port,
 static int hax_vcpu_interrupt(CPUArchState *env)
 {
     CPUState *cpu = env_cpu(env);
-    struct hax_vcpu_state *vcpu = cpu->hax_vcpu;
+    hax_vcpu_state *vcpu = cpu->hax_vcpu;
     struct hax_tunnel *ht = vcpu->tunnel;
 
     /*
@@ -446,7 +446,7 @@ static int hax_vcpu_interrupt(CPUArchState *env)
 
 void hax_raise_event(CPUState *cpu)
 {
-    struct hax_vcpu_state *vcpu = cpu->hax_vcpu;
+    hax_vcpu_state *vcpu = cpu->hax_vcpu;
 
     if (!vcpu) {
         return;
@@ -467,7 +467,7 @@ static int hax_vcpu_hax_exec(CPUArchState *env)
     int ret = 0;
     CPUState *cpu = env_cpu(env);
     X86CPU *x86_cpu = X86_CPU(cpu);
-    struct hax_vcpu_state *vcpu = cpu->hax_vcpu;
+    hax_vcpu_state *vcpu = cpu->hax_vcpu;
     struct hax_tunnel *ht = vcpu->tunnel;
 
     if (!hax_enabled()) {
diff --git a/target/i386/hax/hax-posix.c b/target/i386/hax/hax-posix.c
index ac1a51096eb..8ee247845b7 100644
--- a/target/i386/hax/hax-posix.c
+++ b/target/i386/hax/hax-posix.c
@@ -205,7 +205,7 @@ hax_fd hax_host_open_vcpu(int vmid, int vcpuid)
     return fd;
 }
 
-int hax_host_setup_vcpu_channel(struct hax_vcpu_state *vcpu)
+int hax_host_setup_vcpu_channel(hax_vcpu_state *vcpu)
 {
     int ret;
     struct hax_tunnel_info info;
@@ -227,7 +227,7 @@ int hax_host_setup_vcpu_channel(struct hax_vcpu_state *vcpu)
     return 0;
 }
 
-int hax_vcpu_run(struct hax_vcpu_state *vcpu)
+int hax_vcpu_run(hax_vcpu_state *vcpu)
 {
     return ioctl(vcpu->fd, HAX_VCPU_IOCTL_RUN, NULL);
 }
diff --git a/target/i386/hax/hax-windows.c b/target/i386/hax/hax-windows.c
index 59afa213a6d..08ec93a256c 100644
--- a/target/i386/hax/hax-windows.c
+++ b/target/i386/hax/hax-windows.c
@@ -301,7 +301,7 @@ hax_fd hax_host_open_vcpu(int vmid, int vcpuid)
     return hDeviceVCPU;
 }
 
-int hax_host_setup_vcpu_channel(struct hax_vcpu_state *vcpu)
+int hax_host_setup_vcpu_channel(hax_vcpu_state *vcpu)
 {
     hax_fd hDeviceVCPU = vcpu->fd;
     int ret;
@@ -327,7 +327,7 @@ int hax_host_setup_vcpu_channel(struct hax_vcpu_state *vcpu)
     return 0;
 }
 
-int hax_vcpu_run(struct hax_vcpu_state *vcpu)
+int hax_vcpu_run(hax_vcpu_state *vcpu)
 {
     int ret;
     HANDLE hDeviceVCPU = vcpu->fd;
-- 
2.26.2

[RFC PATCH 10/19] accel/hax: Use 'accel_vcpu' generic pointer

[Philippe Mathieu-Daudé]

Use the 'accel_vcpu' field which is meant for accelerators.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 include/hw/core/cpu.h      |  3 ---
 target/i386/hax/hax-i386.h |  4 ++--
 target/i386/hax/hax-all.c  | 14 +++++++-------
 3 files changed, 9 insertions(+), 12 deletions(-)

diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index d807645af2b..65ff8d86dbc 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -244,8 +244,6 @@ typedef struct SavedIOTLB {
 struct KVMState;
 struct kvm_run;
 
-struct hax_vcpu_state;
-
 #define TB_JMP_CACHE_BITS 12
 #define TB_JMP_CACHE_SIZE (1 << TB_JMP_CACHE_BITS)
 
@@ -421,7 +419,6 @@ struct CPUState {
     int kvm_fd;
     struct KVMState *kvm_state;
     struct kvm_run *kvm_run;
-    struct hax_vcpu_state *hax_vcpu;
     int hvf_fd;
     /* shared by kvm, hax and hvf */
     bool vcpu_dirty;
diff --git a/target/i386/hax/hax-i386.h b/target/i386/hax/hax-i386.h
index ee77406a6a6..61ff0d84f2b 100644
--- a/target/i386/hax/hax-i386.h
+++ b/target/i386/hax/hax-i386.h
@@ -26,8 +26,8 @@ typedef HANDLE hax_fd;
 
 extern struct hax_state hax_global;
 
-typedef struct hax_vcpu_state hax_vcpu_state;
-struct hax_vcpu_state {
+typedef struct AccelvCPUState hax_vcpu_state;
+struct AccelvCPUState {
     hax_fd fd;
     int vcpu_id;
     struct hax_tunnel *tunnel;
diff --git a/target/i386/hax/hax-all.c b/target/i386/hax/hax-all.c
index 08c2b60b437..ce671760e64 100644
--- a/target/i386/hax/hax-all.c
+++ b/target/i386/hax/hax-all.c
@@ -68,7 +68,7 @@ int valid_hax_tunnel_size(uint16_t size)
 
 hax_fd hax_vcpu_get_fd(CPUArchState *env)
 {
-    hax_vcpu_state *vcpu = env_cpu(env)->hax_vcpu;
+    hax_vcpu_state *vcpu = env_cpu(env)->accel_vcpu;
     if (!vcpu) {
         return HAX_INVALID_FD;
     }
@@ -194,7 +194,7 @@ int hax_vcpu_create(int id)
 
 int hax_vcpu_destroy(CPUState *cpu)
 {
-    hax_vcpu_state *vcpu = cpu->hax_vcpu;
+    hax_vcpu_state *vcpu = cpu->accel_vcpu;
 
     if (!hax_global.vm) {
         fprintf(stderr, "vcpu %x destroy failed, vm is null\n", vcpu->vcpu_id);
@@ -414,7 +414,7 @@ static int hax_handle_io(CPUArchState *env, uint32_t df, uint16_t port,
 static int hax_vcpu_interrupt(CPUArchState *env)
 {
     CPUState *cpu = env_cpu(env);
-    hax_vcpu_state *vcpu = cpu->hax_vcpu;
+    hax_vcpu_state *vcpu = cpu->accel_vcpu;
     struct hax_tunnel *ht = vcpu->tunnel;
 
     /*
@@ -446,7 +446,7 @@ static int hax_vcpu_interrupt(CPUArchState *env)
 
 void hax_raise_event(CPUState *cpu)
 {
-    hax_vcpu_state *vcpu = cpu->hax_vcpu;
+    hax_vcpu_state *vcpu = cpu->accel_vcpu;
 
     if (!vcpu) {
         return;
@@ -467,7 +467,7 @@ static int hax_vcpu_hax_exec(CPUArchState *env)
     int ret = 0;
     CPUState *cpu = env_cpu(env);
     X86CPU *x86_cpu = X86_CPU(cpu);
-    hax_vcpu_state *vcpu = cpu->hax_vcpu;
+    hax_vcpu_state *vcpu = cpu->accel_vcpu;
     struct hax_tunnel *ht = vcpu->tunnel;
 
     if (!hax_enabled()) {
@@ -1113,8 +1113,8 @@ void hax_reset_vcpu_state(void *opaque)
 {
     CPUState *cpu;
     for (cpu = first_cpu; cpu != NULL; cpu = CPU_NEXT(cpu)) {
-        cpu->hax_vcpu->tunnel->user_event_pending = 0;
-        cpu->hax_vcpu->tunnel->ready_for_interrupt_injection = 0;
+        cpu->accel_vcpu->tunnel->user_event_pending = 0;
+        cpu->accel_vcpu->tunnel->ready_for_interrupt_injection = 0;
     }
 }
 
-- 
2.26.2

[RFC PATCH 11/19] accel/kvm: Introduce kvm_vcpu_state() helper

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 include/sysemu/kvm.h | 2 ++
 accel/kvm/kvm-all.c  | 5 +++++
 target/i386/cpu.c    | 4 ++--
 3 files changed, 9 insertions(+), 2 deletions(-)

diff --git a/include/sysemu/kvm.h b/include/sysemu/kvm.h
index 687c598be9b..f339be31d1b 100644
--- a/include/sysemu/kvm.h
+++ b/include/sysemu/kvm.h
@@ -361,6 +361,8 @@ int kvm_arch_init_vcpu(CPUState *cpu);
 int kvm_arch_destroy_vcpu(CPUState *cpu);
 
 bool kvm_vcpu_id_is_valid(int vcpu_id);
+/* Returns a pointer to the KVMState associated with this vCPU */
+KVMState *kvm_vcpu_state(CPUState *cpu);
 
 /* Returns VCPU ID to be used on KVM_CREATE_VCPU ioctl() */
 unsigned long kvm_arch_vcpu_id(CPUState *cpu);
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 84c943fcdb2..b787d590a9a 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -1978,6 +1978,11 @@ bool kvm_vcpu_id_is_valid(int vcpu_id)
     return vcpu_id >= 0 && vcpu_id < kvm_max_vcpu_id(s);
 }
 
+KVMState *kvm_vcpu_state(CPUState *cpu)
+{
+    return cpu->kvm_state;
+}
+
 static int kvm_init(MachineState *ms)
 {
     MachineClass *mc = MACHINE_GET_CLASS(ms);
diff --git a/target/i386/cpu.c b/target/i386/cpu.c
index 6a53446e6a5..0d6376322bb 100644
--- a/target/i386/cpu.c
+++ b/target/i386/cpu.c
@@ -5755,7 +5755,7 @@ void cpu_x86_cpuid(CPUX86State *env, uint32_t index, uint32_t count,
     case 0xA:
         /* Architectural Performance Monitoring Leaf */
         if (kvm_enabled() && cpu->enable_pmu) {
-            KVMState *s = cs->kvm_state;
+            KVMState *s = kvm_vcpu_state(cs);
 
             *eax = kvm_arch_get_supported_cpuid(s, 0xA, count, R_EAX);
             *ebx = kvm_arch_get_supported_cpuid(s, 0xA, count, R_EBX);
@@ -6620,7 +6620,7 @@ static void x86_cpu_filter_features(X86CPU *cpu, bool verbose)
 
     if ((env->features[FEAT_7_0_EBX] & CPUID_7_0_EBX_INTEL_PT) &&
         kvm_enabled()) {
-        KVMState *s = CPU(cpu)->kvm_state;
+        KVMState *s = kvm_vcpu_state(CPU(cpu));
         uint32_t eax_0 = kvm_arch_get_supported_cpuid(s, 0x14, 0, R_EAX);
         uint32_t ebx_0 = kvm_arch_get_supported_cpuid(s, 0x14, 0, R_EBX);
         uint32_t ecx_0 = kvm_arch_get_supported_cpuid(s, 0x14, 0, R_ECX);
-- 
2.26.2

[RFC PATCH 12/19] accel/kvm: Use kvm_vcpu_state() when possible

[Philippe Mathieu-Daudé]

In preparation to move the kvm_state field out of CPUState in few
commits, replace the CPUState->kvm_state dereference by a call to
kvm_vcpu_state().

Patch created mechanically using:

  $ sed -i 's/cpu->kvm_state/kvm_vcpu_state(cpu)/' \
        -i 's/cs->kvm_state/kvm_vcpu_state(cs)/' \
        -i 's/c->kvm_state/kvm_vcpu_state(c)/' $(git grep -l kvm_state)

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 accel/kvm/kvm-all.c   | 10 +++++-----
 target/arm/kvm.c      |  2 +-
 target/arm/kvm64.c    | 12 ++++++------
 target/i386/kvm/kvm.c | 34 +++++++++++++++++-----------------
 target/ppc/kvm.c      | 16 ++++++++--------
 5 files changed, 37 insertions(+), 37 deletions(-)

diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index b787d590a9a..8259e89bbaf 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -2760,7 +2760,7 @@ struct kvm_sw_breakpoint *kvm_find_sw_breakpoint(CPUState *cpu,
 {
     struct kvm_sw_breakpoint *bp;
 
-    QTAILQ_FOREACH(bp, &cpu->kvm_state->kvm_sw_breakpoints, entry) {
+    QTAILQ_FOREACH(bp, &kvm_vcpu_state(cpu)->kvm_sw_breakpoints, entry) {
         if (bp->pc == pc) {
             return bp;
         }
@@ -2770,7 +2770,7 @@ struct kvm_sw_breakpoint *kvm_find_sw_breakpoint(CPUState *cpu,
 
 int kvm_sw_breakpoints_active(CPUState *cpu)
 {
-    return !QTAILQ_EMPTY(&cpu->kvm_state->kvm_sw_breakpoints);
+    return !QTAILQ_EMPTY(&kvm_vcpu_state(cpu)->kvm_sw_breakpoints);
 }
 
 struct kvm_set_guest_debug_data {
@@ -2825,7 +2825,7 @@ int kvm_insert_breakpoint(CPUState *cpu, target_ulong addr,
             return err;
         }
 
-        QTAILQ_INSERT_HEAD(&cpu->kvm_state->kvm_sw_breakpoints, bp, entry);
+        QTAILQ_INSERT_HEAD(&kvm_vcpu_state(cpu)->kvm_sw_breakpoints, bp, entry);
     } else {
         err = kvm_arch_insert_hw_breakpoint(addr, len, type);
         if (err) {
@@ -2864,7 +2864,7 @@ int kvm_remove_breakpoint(CPUState *cpu, target_ulong addr,
             return err;
         }
 
-        QTAILQ_REMOVE(&cpu->kvm_state->kvm_sw_breakpoints, bp, entry);
+        QTAILQ_REMOVE(&kvm_vcpu_state(cpu)->kvm_sw_breakpoints, bp, entry);
         g_free(bp);
     } else {
         err = kvm_arch_remove_hw_breakpoint(addr, len, type);
@@ -2885,7 +2885,7 @@ int kvm_remove_breakpoint(CPUState *cpu, target_ulong addr,
 void kvm_remove_all_breakpoints(CPUState *cpu)
 {
     struct kvm_sw_breakpoint *bp, *next;
-    KVMState *s = cpu->kvm_state;
+    KVMState *s = kvm_vcpu_state(cpu);
     CPUState *tmpcpu;
 
     QTAILQ_FOREACH_SAFE(bp, &s->kvm_sw_breakpoints, entry, next) {
diff --git a/target/arm/kvm.c b/target/arm/kvm.c
index 00e124c8123..ed7c4e4815c 100644
--- a/target/arm/kvm.c
+++ b/target/arm/kvm.c
@@ -61,7 +61,7 @@ int kvm_arm_vcpu_finalize(CPUState *cs, int feature)
 
 void kvm_arm_init_serror_injection(CPUState *cs)
 {
-    cap_has_inject_serror_esr = kvm_check_extension(cs->kvm_state,
+    cap_has_inject_serror_esr = kvm_check_extension(kvm_vcpu_state(cs),
                                     KVM_CAP_ARM_INJECT_SERROR_ESR);
 }
 
diff --git a/target/arm/kvm64.c b/target/arm/kvm64.c
index dff85f6db94..c15df0cb1b7 100644
--- a/target/arm/kvm64.c
+++ b/target/arm/kvm64.c
@@ -85,14 +85,14 @@ GArray *hw_breakpoints, *hw_watchpoints;
  */
 static void kvm_arm_init_debug(CPUState *cs)
 {
-    have_guest_debug = kvm_check_extension(cs->kvm_state,
+    have_guest_debug = kvm_check_extension(kvm_vcpu_state(cs),
                                            KVM_CAP_SET_GUEST_DEBUG);
 
-    max_hw_wps = kvm_check_extension(cs->kvm_state, KVM_CAP_GUEST_DEBUG_HW_WPS);
+    max_hw_wps = kvm_check_extension(kvm_vcpu_state(cs), KVM_CAP_GUEST_DEBUG_HW_WPS);
     hw_watchpoints = g_array_sized_new(true, true,
                                        sizeof(HWWatchpoint), max_hw_wps);
 
-    max_hw_bps = kvm_check_extension(cs->kvm_state, KVM_CAP_GUEST_DEBUG_HW_BPS);
+    max_hw_bps = kvm_check_extension(kvm_vcpu_state(cs), KVM_CAP_GUEST_DEBUG_HW_BPS);
     hw_breakpoints = g_array_sized_new(true, true,
                                        sizeof(HWBreakpoint), max_hw_bps);
     return;
@@ -837,14 +837,14 @@ int kvm_arch_init_vcpu(CPUState *cs)
     if (cs->start_powered_off) {
         cpu->kvm_init_features[0] |= 1 << KVM_ARM_VCPU_POWER_OFF;
     }
-    if (kvm_check_extension(cs->kvm_state, KVM_CAP_ARM_PSCI_0_2)) {
+    if (kvm_check_extension(kvm_vcpu_state(cs), KVM_CAP_ARM_PSCI_0_2)) {
         cpu->psci_version = 2;
         cpu->kvm_init_features[0] |= 1 << KVM_ARM_VCPU_PSCI_0_2;
     }
     if (!arm_feature(&cpu->env, ARM_FEATURE_AARCH64)) {
         cpu->kvm_init_features[0] |= 1 << KVM_ARM_VCPU_EL1_32BIT;
     }
-    if (!kvm_check_extension(cs->kvm_state, KVM_CAP_ARM_PMU_V3)) {
+    if (!kvm_check_extension(kvm_vcpu_state(cs), KVM_CAP_ARM_PMU_V3)) {
         cpu->has_pmu = false;
     }
     if (cpu->has_pmu) {
@@ -1411,7 +1411,7 @@ void kvm_arch_on_sigbus_vcpu(CPUState *c, int code, void *addr)
             object_property_get_bool(obj, "ras", NULL)) {
         ram_addr = qemu_ram_addr_from_host(addr);
         if (ram_addr != RAM_ADDR_INVALID &&
-            kvm_physical_memory_addr_from_host(c->kvm_state, addr, &paddr)) {
+            kvm_physical_memory_addr_from_host(kvm_vcpu_state(c), addr, &paddr)) {
             kvm_hwpoison_page_add(ram_addr);
             /*
              * If this is a BUS_MCEERR_AR, we know we have been called
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index 0b5755e42b8..b2facf4f7c1 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -583,7 +583,7 @@ void kvm_arch_on_sigbus_vcpu(CPUState *c, int code, void *addr)
     if ((env->mcg_cap & MCG_SER_P) && addr) {
         ram_addr = qemu_ram_addr_from_host(addr);
         if (ram_addr != RAM_ADDR_INVALID &&
-            kvm_physical_memory_addr_from_host(c->kvm_state, addr, &paddr)) {
+            kvm_physical_memory_addr_from_host(kvm_vcpu_state(c), addr, &paddr)) {
             kvm_hwpoison_page_add(ram_addr);
             kvm_mce_inject(cpu, paddr, code);
 
@@ -715,7 +715,7 @@ unsigned long kvm_arch_vcpu_id(CPUState *cs)
 static bool hyperv_enabled(X86CPU *cpu)
 {
     CPUState *cs = CPU(cpu);
-    return kvm_check_extension(cs->kvm_state, KVM_CAP_HYPERV) > 0 &&
+    return kvm_check_extension(kvm_vcpu_state(cs), KVM_CAP_HYPERV) > 0 &&
         ((cpu->hyperv_spinlock_attempts != HYPERV_SPINLOCK_NEVER_NOTIFY) ||
          cpu->hyperv_features || cpu->hyperv_passthrough);
 }
@@ -747,13 +747,13 @@ static int kvm_arch_set_tsc_khz(CPUState *cs)
         return 0;
     }
 
-    cur_freq = kvm_check_extension(cs->kvm_state, KVM_CAP_GET_TSC_KHZ) ?
+    cur_freq = kvm_check_extension(kvm_vcpu_state(cs), KVM_CAP_GET_TSC_KHZ) ?
                kvm_vcpu_ioctl(cs, KVM_GET_TSC_KHZ) : -ENOTSUP;
 
     /*
      * If TSC scaling is supported, attempt to set TSC frequency.
      */
-    if (kvm_check_extension(cs->kvm_state, KVM_CAP_TSC_CONTROL)) {
+    if (kvm_check_extension(kvm_vcpu_state(cs), KVM_CAP_TSC_CONTROL)) {
         set_ioctl = true;
     }
 
@@ -773,9 +773,9 @@ static int kvm_arch_set_tsc_khz(CPUState *cs)
         /* When KVM_SET_TSC_KHZ fails, it's an error only if the current
          * TSC frequency doesn't match the one we want.
          */
-        cur_freq = kvm_check_extension(cs->kvm_state, KVM_CAP_GET_TSC_KHZ) ?
-                   kvm_vcpu_ioctl(cs, KVM_GET_TSC_KHZ) :
-                   -ENOTSUP;
+        cur_freq = kvm_check_extension(kvm_vcpu_state(cs), KVM_CAP_GET_TSC_KHZ)
+                   ? kvm_vcpu_ioctl(cs, KVM_GET_TSC_KHZ)
+                   : -ENOTSUP;
         if (cur_freq <= 0 || cur_freq != env->tsc_khz) {
             warn_report("TSC frequency mismatch between "
                         "VM (%" PRId64 " kHz) and host (%d kHz), "
@@ -994,7 +994,7 @@ static struct kvm_cpuid2 *get_supported_hv_cpuid_legacy(CPUState *cs)
     entry_recomm->function = HV_CPUID_ENLIGHTMENT_INFO;
     entry_recomm->ebx = cpu->hyperv_spinlock_attempts;
 
-    if (kvm_check_extension(cs->kvm_state, KVM_CAP_HYPERV) > 0) {
+    if (kvm_check_extension(kvm_vcpu_state(cs), KVM_CAP_HYPERV) > 0) {
         entry_feat->eax |= HV_HYPERCALL_AVAILABLE;
         entry_feat->eax |= HV_APIC_ACCESS_AVAILABLE;
         entry_feat->edx |= HV_CPU_DYNAMIC_PARTITIONING_AVAILABLE;
@@ -1002,7 +1002,7 @@ static struct kvm_cpuid2 *get_supported_hv_cpuid_legacy(CPUState *cs)
         entry_recomm->eax |= HV_APIC_ACCESS_RECOMMENDED;
     }
 
-    if (kvm_check_extension(cs->kvm_state, KVM_CAP_HYPERV_TIME) > 0) {
+    if (kvm_check_extension(kvm_vcpu_state(cs), KVM_CAP_HYPERV_TIME) > 0) {
         entry_feat->eax |= HV_TIME_REF_COUNT_AVAILABLE;
         entry_feat->eax |= HV_REFERENCE_TSC_AVAILABLE;
     }
@@ -1036,7 +1036,7 @@ static struct kvm_cpuid2 *get_supported_hv_cpuid_legacy(CPUState *cs)
         unsigned int cap = cpu->hyperv_synic_kvm_only ?
             KVM_CAP_HYPERV_SYNIC : KVM_CAP_HYPERV_SYNIC2;
 
-        if (kvm_check_extension(cs->kvm_state, cap) > 0) {
+        if (kvm_check_extension(kvm_vcpu_state(cs), cap) > 0) {
             entry_feat->eax |= HV_SYNIC_AVAILABLE;
         }
     }
@@ -1045,18 +1045,18 @@ static struct kvm_cpuid2 *get_supported_hv_cpuid_legacy(CPUState *cs)
         entry_feat->eax |= HV_SYNTIMERS_AVAILABLE;
     }
 
-    if (kvm_check_extension(cs->kvm_state,
+    if (kvm_check_extension(kvm_vcpu_state(cs),
                             KVM_CAP_HYPERV_TLBFLUSH) > 0) {
         entry_recomm->eax |= HV_REMOTE_TLB_FLUSH_RECOMMENDED;
         entry_recomm->eax |= HV_EX_PROCESSOR_MASKS_RECOMMENDED;
     }
 
-    if (kvm_check_extension(cs->kvm_state,
+    if (kvm_check_extension(kvm_vcpu_state(cs),
                             KVM_CAP_HYPERV_ENLIGHTENED_VMCS) > 0) {
         entry_recomm->eax |= HV_ENLIGHTENED_VMCS_RECOMMENDED;
     }
 
-    if (kvm_check_extension(cs->kvm_state,
+    if (kvm_check_extension(kvm_vcpu_state(cs),
                             KVM_CAP_HYPERV_SEND_IPI) > 0) {
         entry_recomm->eax |= HV_CLUSTER_IPI_RECOMMENDED;
         entry_recomm->eax |= HV_EX_PROCESSOR_MASKS_RECOMMENDED;
@@ -1200,7 +1200,7 @@ static int hyperv_handle_properties(CPUState *cs,
         }
     }
 
-    if (kvm_check_extension(cs->kvm_state, KVM_CAP_HYPERV_CPUID) > 0) {
+    if (kvm_check_extension(kvm_vcpu_state(cs), KVM_CAP_HYPERV_CPUID) > 0) {
         cpuid = get_supported_hv_cpuid(cs);
     } else {
         cpuid = get_supported_hv_cpuid_legacy(cs);
@@ -1504,7 +1504,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
      * so that vcpu's TSC frequency can be migrated later via this field.
      */
     if (!env->tsc_khz) {
-        r = kvm_check_extension(cs->kvm_state, KVM_CAP_GET_TSC_KHZ) ?
+        r = kvm_check_extension(kvm_vcpu_state(cs), KVM_CAP_GET_TSC_KHZ) ?
             kvm_vcpu_ioctl(cs, KVM_GET_TSC_KHZ) :
             -ENOTSUP;
         if (r > 0) {
@@ -1746,12 +1746,12 @@ int kvm_arch_init_vcpu(CPUState *cs)
     if (((env->cpuid_version >> 8)&0xF) >= 6
         && (env->features[FEAT_1_EDX] & (CPUID_MCE | CPUID_MCA)) ==
            (CPUID_MCE | CPUID_MCA)
-        && kvm_check_extension(cs->kvm_state, KVM_CAP_MCE) > 0) {
+        && kvm_check_extension(kvm_vcpu_state(cs), KVM_CAP_MCE) > 0) {
         uint64_t mcg_cap, unsupported_caps;
         int banks;
         int ret;
 
-        ret = kvm_get_mce_cap_supported(cs->kvm_state, &mcg_cap, &banks);
+        ret = kvm_get_mce_cap_supported(kvm_vcpu_state(cs), &mcg_cap, &banks);
         if (ret < 0) {
             fprintf(stderr, "kvm_get_mce_cap_supported: %s", strerror(-ret));
             return ret;
diff --git a/target/ppc/kvm.c b/target/ppc/kvm.c
index 298c1f882c6..d9a8f019a74 100644
--- a/target/ppc/kvm.c
+++ b/target/ppc/kvm.c
@@ -205,7 +205,7 @@ static int kvm_booke206_tlb_init(PowerPCCPU *cpu)
     int ret, i;
 
     if (!kvm_enabled() ||
-        !kvm_check_extension(cs->kvm_state, KVM_CAP_SW_TLB)) {
+        !kvm_check_extension(kvm_vcpu_state(cs), KVM_CAP_SW_TLB)) {
         return 0;
     }
 
@@ -303,7 +303,7 @@ target_ulong kvmppc_configure_v3_mmu(PowerPCCPU *cpu,
         flags |= KVM_PPC_MMUV3_GTSE;
     }
     cfg.flags = flags;
-    ret = kvm_vm_ioctl(cs->kvm_state, KVM_PPC_CONFIGURE_V3_MMU, &cfg);
+    ret = kvm_vm_ioctl(kvm_vcpu_state(cs), KVM_PPC_CONFIGURE_V3_MMU, &cfg);
     switch (ret) {
     case 0:
         return H_SUCCESS;
@@ -483,7 +483,7 @@ int kvm_arch_init_vcpu(CPUState *cs)
         ret = kvm_booke206_tlb_init(cpu);
         break;
     case POWERPC_MMU_2_07:
-        if (!cap_htm && !kvmppc_is_pr(cs->kvm_state)) {
+        if (!cap_htm && !kvmppc_is_pr(kvm_vcpu_state(cs))) {
             /*
              * KVM-HV has transactional memory on POWER8 also without
              * the KVM_CAP_PPC_HTM extension, so enable it here
@@ -1947,8 +1947,8 @@ static int kvmppc_get_pvinfo(CPUPPCState *env, struct kvm_ppc_pvinfo *pvinfo)
 {
     CPUState *cs = env_cpu(env);
 
-    if (kvm_vm_check_extension(cs->kvm_state, KVM_CAP_PPC_GET_PVINFO) &&
-        !kvm_vm_ioctl(cs->kvm_state, KVM_PPC_GET_PVINFO, pvinfo)) {
+    if (kvm_vm_check_extension(kvm_vcpu_state(cs), KVM_CAP_PPC_GET_PVINFO) &&
+        !kvm_vm_ioctl(kvm_vcpu_state(cs), KVM_PPC_GET_PVINFO, pvinfo)) {
         return 0;
     }
 
@@ -2864,7 +2864,7 @@ int kvmppc_resize_hpt_prepare(PowerPCCPU *cpu, target_ulong flags, int shift)
         return -ENOSYS;
     }
 
-    return kvm_vm_ioctl(cs->kvm_state, KVM_PPC_RESIZE_HPT_PREPARE, &rhpt);
+    return kvm_vm_ioctl(kvm_vcpu_state(cs), KVM_PPC_RESIZE_HPT_PREPARE, &rhpt);
 }
 
 int kvmppc_resize_hpt_commit(PowerPCCPU *cpu, target_ulong flags, int shift)
@@ -2879,7 +2879,7 @@ int kvmppc_resize_hpt_commit(PowerPCCPU *cpu, target_ulong flags, int shift)
         return -ENOSYS;
     }
 
-    return kvm_vm_ioctl(cs->kvm_state, KVM_PPC_RESIZE_HPT_COMMIT, &rhpt);
+    return kvm_vm_ioctl(kvm_vcpu_state(cs), KVM_PPC_RESIZE_HPT_COMMIT, &rhpt);
 }
 
 /*
@@ -2909,7 +2909,7 @@ bool kvmppc_pvr_workaround_required(PowerPCCPU *cpu)
         return false;
     }
 
-    return !kvmppc_is_pr(cs->kvm_state);
+    return !kvmppc_is_pr(kvm_vcpu_state(cs));
 }
 
 void kvmppc_set_reg_ppc_online(PowerPCCPU *cpu, unsigned int online)
-- 
2.26.2

[RFC PATCH 13/19] accel/kvm: Declare and allocate AccelvCPUState struct

[Philippe Mathieu-Daudé]

In preparation of moving KVM-specific fields from CPUState
to the accelerator-specific AccelvCPUState structure, first
declare it empty and allocate it. This will make the following
commits easier to review.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 include/sysemu/kvm_int.h | 3 +++
 accel/kvm/kvm-all.c      | 5 +++++
 target/s390x/kvm.c       | 2 +-
 3 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h
index ccb8869f01b..f57be10adde 100644
--- a/include/sysemu/kvm_int.h
+++ b/include/sysemu/kvm_int.h
@@ -13,6 +13,9 @@
 #include "qemu/accel.h"
 #include "sysemu/kvm.h"
 
+struct AccelvCPUState {
+};
+
 typedef struct KVMSlot
 {
     hwaddr start_addr;
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 8259e89bbaf..4ccd12ea56a 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -399,6 +399,7 @@ void kvm_destroy_vcpu(CPUState *cpu)
         error_report("kvm_destroy_vcpu failed");
         exit(EXIT_FAILURE);
     }
+    g_free(cpu->accel_vcpu);
 }
 
 static int kvm_get_vcpu(KVMState *s, unsigned long vcpu_id)
@@ -434,6 +435,7 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)
         goto err;
     }
 
+    cpu->accel_vcpu = g_new(struct AccelvCPUState, 1);
     cpu->kvm_fd = ret;
     cpu->kvm_state = s;
     cpu->vcpu_dirty = true;
@@ -468,6 +470,9 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)
                          kvm_arch_vcpu_id(cpu));
     }
 err:
+    if (ret < 0) {
+        g_free(cpu->accel_vcpu);
+    }
     return ret;
 }
 
diff --git a/target/s390x/kvm.c b/target/s390x/kvm.c
index d8ac12dfc11..cf6790b2678 100644
--- a/target/s390x/kvm.c
+++ b/target/s390x/kvm.c
@@ -2085,7 +2085,7 @@ int kvm_s390_set_cpu_state(S390CPU *cpu, uint8_t cpu_state)
     int ret;
 
     /* the kvm part might not have been initialized yet */
-    if (CPU(cpu)->kvm_state == NULL) {
+    if (CPU(cpu)->accel_vcpu == NULL) {
         return 0;
     }
 
-- 
2.26.2

[RFC PATCH 14/19] accel/kvm: Move the 'kvm_fd' field to AccelvCPUState

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 include/hw/core/cpu.h    | 2 --
 include/sysemu/kvm_int.h | 4 ++++
 accel/kvm/kvm-all.c      | 8 ++++----
 3 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index 65ff8d86dbc..ca2526e6a23 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -314,7 +314,6 @@ struct AccelvCPUState;
  * @opaque: User data.
  * @mem_io_pc: Host Program Counter at which the memory was accessed.
  * @accel_vcpu: Pointer to accelerator-specific AccelvCPUState field.
- * @kvm_fd: vCPU file descriptor for KVM.
  * @work_mutex: Lock to prevent multiple access to @work_list.
  * @work_list: List of pending asynchronous work.
  * @trace_dstate_delayed: Delayed changes to trace_dstate (includes all changes
@@ -416,7 +415,6 @@ struct CPUState {
 
     /* Accelerator-specific fields. */
     struct AccelvCPUState *accel_vcpu;
-    int kvm_fd;
     struct KVMState *kvm_state;
     struct kvm_run *kvm_run;
     int hvf_fd;
diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h
index f57be10adde..3bf75e62293 100644
--- a/include/sysemu/kvm_int.h
+++ b/include/sysemu/kvm_int.h
@@ -14,6 +14,10 @@
 #include "sysemu/kvm.h"
 
 struct AccelvCPUState {
+    /**
+     * @kvm_fd: vCPU file descriptor for KVM
+     */
+    int kvm_fd;
 };
 
 typedef struct KVMSlot
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 4ccd12ea56a..1c08ff3fbe0 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -387,7 +387,7 @@ static int do_kvm_destroy_vcpu(CPUState *cpu)
 
     vcpu = g_malloc0(sizeof(*vcpu));
     vcpu->vcpu_id = kvm_arch_vcpu_id(cpu);
-    vcpu->kvm_fd = cpu->kvm_fd;
+    vcpu->kvm_fd = cpu->accel_vcpu->kvm_fd;
     QLIST_INSERT_HEAD(&kvm_state->kvm_parked_vcpus, vcpu, node);
 err:
     return ret;
@@ -436,7 +436,7 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)
     }
 
     cpu->accel_vcpu = g_new(struct AccelvCPUState, 1);
-    cpu->kvm_fd = ret;
+    cpu->accel_vcpu->kvm_fd = ret;
     cpu->kvm_state = s;
     cpu->vcpu_dirty = true;
 
@@ -449,7 +449,7 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)
     }
 
     cpu->kvm_run = mmap(NULL, mmap_size, PROT_READ | PROT_WRITE, MAP_SHARED,
-                        cpu->kvm_fd, 0);
+                        cpu->accel_vcpu->kvm_fd, 0);
     if (cpu->kvm_run == MAP_FAILED) {
         ret = -errno;
         error_setg_errno(errp, ret,
@@ -2631,7 +2631,7 @@ int kvm_vcpu_ioctl(CPUState *cpu, int type, ...)
     va_end(ap);
 
     trace_kvm_vcpu_ioctl(cpu->cpu_index, type, arg);
-    ret = ioctl(cpu->kvm_fd, type, arg);
+    ret = ioctl(cpu->accel_vcpu->kvm_fd, type, arg);
     if (ret == -1) {
         ret = -errno;
     }
-- 
2.26.2

[RFC PATCH 15/19] accel/kvm: Move the 'kvm_state' field to AccelvCPUState

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 include/hw/core/cpu.h    | 1 -
 include/sysemu/kvm_int.h | 1 +
 accel/kvm/kvm-all.c      | 4 ++--
 3 files changed, 3 insertions(+), 3 deletions(-)

diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index ca2526e6a23..4f280509f9b 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -415,7 +415,6 @@ struct CPUState {
 
     /* Accelerator-specific fields. */
     struct AccelvCPUState *accel_vcpu;
-    struct KVMState *kvm_state;
     struct kvm_run *kvm_run;
     int hvf_fd;
     /* shared by kvm, hax and hvf */
diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h
index 3bf75e62293..dc45b3c3afa 100644
--- a/include/sysemu/kvm_int.h
+++ b/include/sysemu/kvm_int.h
@@ -18,6 +18,7 @@ struct AccelvCPUState {
      * @kvm_fd: vCPU file descriptor for KVM
      */
     int kvm_fd;
+    struct KVMState *kvm_state;
 };
 
 typedef struct KVMSlot
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 1c08ff3fbe0..737db3d3e0e 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -437,7 +437,7 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)
 
     cpu->accel_vcpu = g_new(struct AccelvCPUState, 1);
     cpu->accel_vcpu->kvm_fd = ret;
-    cpu->kvm_state = s;
+    cpu->accel_vcpu->kvm_state = s;
     cpu->vcpu_dirty = true;
 
     mmap_size = kvm_ioctl(s, KVM_GET_VCPU_MMAP_SIZE, 0);
@@ -1985,7 +1985,7 @@ bool kvm_vcpu_id_is_valid(int vcpu_id)
 
 KVMState *kvm_vcpu_state(CPUState *cpu)
 {
-    return cpu->kvm_state;
+    return cpu->accel_vcpu->kvm_state;
 }
 
 static int kvm_init(MachineState *ms)
-- 
2.26.2

[RFC PATCH 16/19] accel/kvm: Move the 'kvm_run' field to AccelvCPUState

[Philippe Mathieu-Daudé]

Patch created mechanically using:

  $ sed -i 's/->kvm_run/->accel_vcpu->kvm_run/' $(git grep -l kvm_run)

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 include/hw/core/cpu.h    |  1 -
 include/sysemu/kvm_int.h |  1 +
 accel/kvm/kvm-all.c      | 16 ++++++++--------
 hw/s390x/pv.c            |  3 ++-
 target/i386/kvm/kvm.c    |  2 +-
 target/s390x/kvm.c       | 19 ++++++++++---------
 6 files changed, 22 insertions(+), 20 deletions(-)

diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index 4f280509f9b..3268f1393f1 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -415,7 +415,6 @@ struct CPUState {
 
     /* Accelerator-specific fields. */
     struct AccelvCPUState *accel_vcpu;
-    struct kvm_run *kvm_run;
     int hvf_fd;
     /* shared by kvm, hax and hvf */
     bool vcpu_dirty;
diff --git a/include/sysemu/kvm_int.h b/include/sysemu/kvm_int.h
index dc45b3c3afa..b83264847aa 100644
--- a/include/sysemu/kvm_int.h
+++ b/include/sysemu/kvm_int.h
@@ -19,6 +19,7 @@ struct AccelvCPUState {
      */
     int kvm_fd;
     struct KVMState *kvm_state;
+    struct kvm_run *kvm_run;
 };
 
 typedef struct KVMSlot
diff --git a/accel/kvm/kvm-all.c b/accel/kvm/kvm-all.c
index 737db3d3e0e..69df35497d2 100644
--- a/accel/kvm/kvm-all.c
+++ b/accel/kvm/kvm-all.c
@@ -380,7 +380,7 @@ static int do_kvm_destroy_vcpu(CPUState *cpu)
         goto err;
     }
 
-    ret = munmap(cpu->kvm_run, mmap_size);
+    ret = munmap(cpu->accel_vcpu->kvm_run, mmap_size);
     if (ret < 0) {
         goto err;
     }
@@ -448,9 +448,9 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)
         goto err;
     }
 
-    cpu->kvm_run = mmap(NULL, mmap_size, PROT_READ | PROT_WRITE, MAP_SHARED,
-                        cpu->accel_vcpu->kvm_fd, 0);
-    if (cpu->kvm_run == MAP_FAILED) {
+    cpu->accel_vcpu->kvm_run = mmap(NULL, mmap_size, PROT_READ | PROT_WRITE,
+                                    MAP_SHARED, cpu->accel_vcpu->kvm_fd, 0);
+    if (cpu->accel_vcpu->kvm_run == MAP_FAILED) {
         ret = -errno;
         error_setg_errno(errp, ret,
                          "kvm_init_vcpu: mmap'ing vcpu state failed (%lu)",
@@ -460,7 +460,7 @@ int kvm_init_vcpu(CPUState *cpu, Error **errp)
 
     if (s->coalesced_mmio && !s->coalesced_mmio_ring) {
         s->coalesced_mmio_ring =
-            (void *)cpu->kvm_run + s->coalesced_mmio * PAGE_SIZE;
+            (void *)cpu->accel_vcpu->kvm_run + s->coalesced_mmio * PAGE_SIZE;
     }
 
     ret = kvm_arch_init_vcpu(cpu);
@@ -2382,7 +2382,7 @@ static __thread bool have_sigbus_pending;
 
 static void kvm_cpu_kick(CPUState *cpu)
 {
-    qatomic_set(&cpu->kvm_run->immediate_exit, 1);
+    qatomic_set(&cpu->accel_vcpu->kvm_run->immediate_exit, 1);
 }
 
 static void kvm_cpu_kick_self(void)
@@ -2403,7 +2403,7 @@ static void kvm_eat_signals(CPUState *cpu)
     int r;
 
     if (kvm_immediate_exit) {
-        qatomic_set(&cpu->kvm_run->immediate_exit, 0);
+        qatomic_set(&cpu->accel_vcpu->kvm_run->immediate_exit, 0);
         /* Write kvm_run->immediate_exit before the cpu->exit_request
          * write in kvm_cpu_exec.
          */
@@ -2431,7 +2431,7 @@ static void kvm_eat_signals(CPUState *cpu)
 
 int kvm_cpu_exec(CPUState *cpu)
 {
-    struct kvm_run *run = cpu->kvm_run;
+    struct kvm_run *run = cpu->accel_vcpu->kvm_run;
     int ret, run_ret;
 
     DPRINTF("kvm_cpu_exec()\n");
diff --git a/hw/s390x/pv.c b/hw/s390x/pv.c
index 93eccfc05d5..061c92fc4e3 100644
--- a/hw/s390x/pv.c
+++ b/hw/s390x/pv.c
@@ -17,6 +17,7 @@
 #include "qapi/error.h"
 #include "qemu/error-report.h"
 #include "sysemu/kvm.h"
+#include "sysemu/kvm_int.h"
 #include "qom/object_interfaces.h"
 #include "exec/confidential-guest-support.h"
 #include "hw/s390x/ipl.h"
@@ -108,7 +109,7 @@ void s390_pv_unshare(void)
 
 void s390_pv_inject_reset_error(CPUState *cs)
 {
-    int r1 = (cs->kvm_run->s390_sieic.ipa & 0x00f0) >> 4;
+    int r1 = (cs->accel_vcpu->kvm_run->s390_sieic.ipa & 0x00f0) >> 4;
     CPUS390XState *env = &S390_CPU(cs)->env;
 
     /* Report that we are unable to enter protected mode */
diff --git a/target/i386/kvm/kvm.c b/target/i386/kvm/kvm.c
index b2facf4f7c1..c0ccaf6b06a 100644
--- a/target/i386/kvm/kvm.c
+++ b/target/i386/kvm/kvm.c
@@ -4329,7 +4329,7 @@ static int kvm_handle_halt(X86CPU *cpu)
 static int kvm_handle_tpr_access(X86CPU *cpu)
 {
     CPUState *cs = CPU(cpu);
-    struct kvm_run *run = cs->kvm_run;
+    struct kvm_run *run = cs->accel_vcpu->kvm_run;
 
     apic_handle_tpr_access_report(cpu->apic_state, run->tpr_access.rip,
                                   run->tpr_access.is_write ? TPR_ACCESS_WRITE
diff --git a/target/s390x/kvm.c b/target/s390x/kvm.c
index cf6790b2678..7968b10fa52 100644
--- a/target/s390x/kvm.c
+++ b/target/s390x/kvm.c
@@ -460,14 +460,15 @@ void kvm_s390_reset_vcpu_normal(S390CPU *cpu)
 
 static int can_sync_regs(CPUState *cs, int regs)
 {
-    return cap_sync_regs && (cs->kvm_run->kvm_valid_regs & regs) == regs;
+    return cap_sync_regs
+           && (cs->accel_vcpu->kvm_run->kvm_valid_regs & regs) == regs;
 }
 
 int kvm_arch_put_registers(CPUState *cs, int level)
 {
     S390CPU *cpu = S390_CPU(cs);
     CPUS390XState *env = &cpu->env;
-    struct kvm_run *run = cs->kvm_run;
+    struct kvm_run *run = cs->accel_vcpu->kvm_run;
     struct kvm_sregs sregs;
     struct kvm_regs regs;
     struct kvm_fpu fpu = {};
@@ -624,7 +625,7 @@ int kvm_arch_get_registers(CPUState *cs)
 {
     S390CPU *cpu = S390_CPU(cs);
     CPUS390XState *env = &cpu->env;
-    struct kvm_run *run = cs->kvm_run;
+    struct kvm_run *run = cs->accel_vcpu->kvm_run;
     struct kvm_sregs sregs;
     struct kvm_regs regs;
     struct kvm_fpu fpu;
@@ -1621,8 +1622,8 @@ void kvm_s390_set_diag318(CPUState *cs, uint64_t diag318_info)
     /* Feat bit is set only if KVM supports sync for diag318 */
     if (s390_has_feat(S390_FEAT_DIAG_318)) {
         env->diag318_info = diag318_info;
-        cs->kvm_run->s.regs.diag318 = diag318_info;
-        cs->kvm_run->kvm_dirty_regs |= KVM_SYNC_DIAG318;
+        cs->accel_vcpu->kvm_run->s.regs.diag318 = diag318_info;
+        cs->accel_vcpu->kvm_run->kvm_dirty_regs |= KVM_SYNC_DIAG318;
     }
 }
 
@@ -1783,7 +1784,7 @@ static int handle_oper_loop(S390CPU *cpu, struct kvm_run *run)
 static int handle_intercept(S390CPU *cpu)
 {
     CPUState *cs = CPU(cpu);
-    struct kvm_run *run = cs->kvm_run;
+    struct kvm_run *run = cs->accel_vcpu->kvm_run;
     int icpt_code = run->s390_sieic.icptcode;
     int r = 0;
 
@@ -1844,7 +1845,7 @@ static int handle_intercept(S390CPU *cpu)
 static int handle_tsch(S390CPU *cpu)
 {
     CPUState *cs = CPU(cpu);
-    struct kvm_run *run = cs->kvm_run;
+    struct kvm_run *run = cs->accel_vcpu->kvm_run;
     int ret;
 
     ret = ioinst_handle_tsch(cpu, cpu->env.regs[1], run->s390_tsch.ipb,
@@ -1934,7 +1935,7 @@ static void insert_stsi_3_2_2(S390CPU *cpu, __u64 addr, uint8_t ar)
 static int handle_stsi(S390CPU *cpu)
 {
     CPUState *cs = CPU(cpu);
-    struct kvm_run *run = cs->kvm_run;
+    struct kvm_run *run = cs->accel_vcpu->kvm_run;
 
     switch (run->s390_stsi.fc) {
     case 3:
@@ -1952,7 +1953,7 @@ static int handle_stsi(S390CPU *cpu)
 static int kvm_arch_handle_debug_exit(S390CPU *cpu)
 {
     CPUState *cs = CPU(cpu);
-    struct kvm_run *run = cs->kvm_run;
+    struct kvm_run *run = cs->accel_vcpu->kvm_run;
 
     int ret = 0;
     struct kvm_debug_exit_arch *arch_info = &run->debug.arch;
-- 
2.26.2

[RFC PATCH 17/19] accel/hvf: Reduce deref by declaring 'hv_vcpuid_t hvf_fd' on stack

[Philippe Mathieu-Daudé]

In order to make the next commits easier to review,
declare 'hvf_fd' on the stack when it is used in various
places in a function.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 target/i386/hvf/hvf.c       |  95 ++++++++--------
 target/i386/hvf/x86_descr.c |  19 ++--
 target/i386/hvf/x86hvf.c    | 209 ++++++++++++++++++------------------
 3 files changed, 166 insertions(+), 157 deletions(-)

diff --git a/target/i386/hvf/hvf.c b/target/i386/hvf/hvf.c
index 3c5c9c8197e..effee39ee9b 100644
--- a/target/i386/hvf/hvf.c
+++ b/target/i386/hvf/hvf.c
@@ -504,6 +504,7 @@ int hvf_init_vcpu(CPUState *cpu)
 
     X86CPU *x86cpu = X86_CPU(cpu);
     CPUX86State *env = &x86cpu->env;
+    hv_vcpuid_t hvf_fd;
     int r;
 
     /* init cpu signals */
@@ -532,9 +533,10 @@ int hvf_init_vcpu(CPUState *cpu)
         }
     }
 
-    r = hv_vcpu_create((hv_vcpuid_t *)&cpu->hvf_fd, HV_VCPU_DEFAULT);
+    r = hv_vcpu_create(&hvf_fd, HV_VCPU_DEFAULT);
     cpu->vcpu_dirty = true;
     assert_hvf_ok(r);
+    cpu->hvf_fd = (int)hvf_fd
 
     if (hv_vmx_read_capability(HV_VMX_CAP_PINBASED,
         &hvf_state->hvf_caps->vmx_cap_pinbased)) {
@@ -554,43 +556,43 @@ int hvf_init_vcpu(CPUState *cpu)
     }
 
     /* set VMCS control fields */
-    wvmcs(cpu->hvf_fd, VMCS_PIN_BASED_CTLS,
+    wvmcs(hvf_fd, VMCS_PIN_BASED_CTLS,
           cap2ctrl(hvf_state->hvf_caps->vmx_cap_pinbased,
           VMCS_PIN_BASED_CTLS_EXTINT |
           VMCS_PIN_BASED_CTLS_NMI |
           VMCS_PIN_BASED_CTLS_VNMI));
-    wvmcs(cpu->hvf_fd, VMCS_PRI_PROC_BASED_CTLS,
+    wvmcs(hvf_fd, VMCS_PRI_PROC_BASED_CTLS,
           cap2ctrl(hvf_state->hvf_caps->vmx_cap_procbased,
           VMCS_PRI_PROC_BASED_CTLS_HLT |
           VMCS_PRI_PROC_BASED_CTLS_MWAIT |
           VMCS_PRI_PROC_BASED_CTLS_TSC_OFFSET |
           VMCS_PRI_PROC_BASED_CTLS_TPR_SHADOW) |
           VMCS_PRI_PROC_BASED_CTLS_SEC_CONTROL);
-    wvmcs(cpu->hvf_fd, VMCS_SEC_PROC_BASED_CTLS,
+    wvmcs(hvf_fd, VMCS_SEC_PROC_BASED_CTLS,
           cap2ctrl(hvf_state->hvf_caps->vmx_cap_procbased2,
                    VMCS_PRI_PROC_BASED2_CTLS_APIC_ACCESSES));
 
-    wvmcs(cpu->hvf_fd, VMCS_ENTRY_CTLS, cap2ctrl(hvf_state->hvf_caps->vmx_cap_entry,
+    wvmcs(hvf_fd, VMCS_ENTRY_CTLS, cap2ctrl(hvf_state->hvf_caps->vmx_cap_entry,
           0));
-    wvmcs(cpu->hvf_fd, VMCS_EXCEPTION_BITMAP, 0); /* Double fault */
+    wvmcs(hvf_fd, VMCS_EXCEPTION_BITMAP, 0); /* Double fault */
 
-    wvmcs(cpu->hvf_fd, VMCS_TPR_THRESHOLD, 0);
+    wvmcs(hvf_fd, VMCS_TPR_THRESHOLD, 0);
 
     x86cpu = X86_CPU(cpu);
     x86cpu->env.xsave_buf = qemu_memalign(4096, 4096);
 
-    hv_vcpu_enable_native_msr(cpu->hvf_fd, MSR_STAR, 1);
-    hv_vcpu_enable_native_msr(cpu->hvf_fd, MSR_LSTAR, 1);
-    hv_vcpu_enable_native_msr(cpu->hvf_fd, MSR_CSTAR, 1);
-    hv_vcpu_enable_native_msr(cpu->hvf_fd, MSR_FMASK, 1);
-    hv_vcpu_enable_native_msr(cpu->hvf_fd, MSR_FSBASE, 1);
-    hv_vcpu_enable_native_msr(cpu->hvf_fd, MSR_GSBASE, 1);
-    hv_vcpu_enable_native_msr(cpu->hvf_fd, MSR_KERNELGSBASE, 1);
-    hv_vcpu_enable_native_msr(cpu->hvf_fd, MSR_TSC_AUX, 1);
-    hv_vcpu_enable_native_msr(cpu->hvf_fd, MSR_IA32_TSC, 1);
-    hv_vcpu_enable_native_msr(cpu->hvf_fd, MSR_IA32_SYSENTER_CS, 1);
-    hv_vcpu_enable_native_msr(cpu->hvf_fd, MSR_IA32_SYSENTER_EIP, 1);
-    hv_vcpu_enable_native_msr(cpu->hvf_fd, MSR_IA32_SYSENTER_ESP, 1);
+    hv_vcpu_enable_native_msr(hvf_fd, MSR_STAR, 1);
+    hv_vcpu_enable_native_msr(hvf_fd, MSR_LSTAR, 1);
+    hv_vcpu_enable_native_msr(hvf_fd, MSR_CSTAR, 1);
+    hv_vcpu_enable_native_msr(hvf_fd, MSR_FMASK, 1);
+    hv_vcpu_enable_native_msr(hvf_fd, MSR_FSBASE, 1);
+    hv_vcpu_enable_native_msr(hvf_fd, MSR_GSBASE, 1);
+    hv_vcpu_enable_native_msr(hvf_fd, MSR_KERNELGSBASE, 1);
+    hv_vcpu_enable_native_msr(hvf_fd, MSR_TSC_AUX, 1);
+    hv_vcpu_enable_native_msr(hvf_fd, MSR_IA32_TSC, 1);
+    hv_vcpu_enable_native_msr(hvf_fd, MSR_IA32_SYSENTER_CS, 1);
+    hv_vcpu_enable_native_msr(hvf_fd, MSR_IA32_SYSENTER_EIP, 1);
+    hv_vcpu_enable_native_msr(hvf_fd, MSR_IA32_SYSENTER_ESP, 1);
 
     return 0;
 }
@@ -695,6 +697,7 @@ int hvf_vcpu_exec(CPUState *cpu)
 {
     X86CPU *x86_cpu = X86_CPU(cpu);
     CPUX86State *env = &x86_cpu->env;
+    hv_vcpuid_t hvf_fd = (hv_vcpuid_t)cpu_state->hvf_fd;
     int ret = 0;
     uint64_t rip = 0;
 
@@ -719,20 +722,20 @@ int hvf_vcpu_exec(CPUState *cpu)
             return EXCP_HLT;
         }
 
-        hv_return_t r  = hv_vcpu_run(cpu->hvf_fd);
+        hv_return_t r  = hv_vcpu_run(hvf_fd);
         assert_hvf_ok(r);
 
         /* handle VMEXIT */
-        uint64_t exit_reason = rvmcs(cpu->hvf_fd, VMCS_EXIT_REASON);
-        uint64_t exit_qual = rvmcs(cpu->hvf_fd, VMCS_EXIT_QUALIFICATION);
-        uint32_t ins_len = (uint32_t)rvmcs(cpu->hvf_fd,
+        uint64_t exit_reason = rvmcs(hvf_fd, VMCS_EXIT_REASON);
+        uint64_t exit_qual = rvmcs(hvf_fd, VMCS_EXIT_QUALIFICATION);
+        uint32_t ins_len = (uint32_t)rvmcs(hvf_fd,
                                            VMCS_EXIT_INSTRUCTION_LENGTH);
 
-        uint64_t idtvec_info = rvmcs(cpu->hvf_fd, VMCS_IDT_VECTORING_INFO);
+        uint64_t idtvec_info = rvmcs(hvf_fd, VMCS_IDT_VECTORING_INFO);
 
         hvf_store_events(cpu, ins_len, idtvec_info);
-        rip = rreg(cpu->hvf_fd, HV_X86_RIP);
-        env->eflags = rreg(cpu->hvf_fd, HV_X86_RFLAGS);
+        rip = rreg(hvf_fd, HV_X86_RIP);
+        env->eflags = rreg(hvf_fd, HV_X86_RFLAGS);
 
         qemu_mutex_lock_iothread();
 
@@ -762,7 +765,7 @@ int hvf_vcpu_exec(CPUState *cpu)
         case EXIT_REASON_EPT_FAULT:
         {
             hvf_slot *slot;
-            uint64_t gpa = rvmcs(cpu->hvf_fd, VMCS_GUEST_PHYSICAL_ADDRESS);
+            uint64_t gpa = rvmcs(hvf_fd, VMCS_GUEST_PHYSICAL_ADDRESS);
 
             if (((idtvec_info & VMCS_IDT_VEC_VALID) == 0) &&
                 ((exit_qual & EXIT_QUAL_NMIUDTI) != 0)) {
@@ -807,7 +810,7 @@ int hvf_vcpu_exec(CPUState *cpu)
                 store_regs(cpu);
                 break;
             } else if (!string && !in) {
-                RAX(env) = rreg(cpu->hvf_fd, HV_X86_RAX);
+                RAX(env) = rreg(hvf_fd, HV_X86_RAX);
                 hvf_handle_io(env, port, &RAX(env), 1, size, 1);
                 macvm_set_rip(cpu, rip + ins_len);
                 break;
@@ -823,21 +826,21 @@ int hvf_vcpu_exec(CPUState *cpu)
             break;
         }
         case EXIT_REASON_CPUID: {
-            uint32_t rax = (uint32_t)rreg(cpu->hvf_fd, HV_X86_RAX);
-            uint32_t rbx = (uint32_t)rreg(cpu->hvf_fd, HV_X86_RBX);
-            uint32_t rcx = (uint32_t)rreg(cpu->hvf_fd, HV_X86_RCX);
-            uint32_t rdx = (uint32_t)rreg(cpu->hvf_fd, HV_X86_RDX);
+            uint32_t rax = (uint32_t)rreg(hvf_fd, HV_X86_RAX);
+            uint32_t rbx = (uint32_t)rreg(hvf_fd, HV_X86_RBX);
+            uint32_t rcx = (uint32_t)rreg(hvf_fd, HV_X86_RCX);
+            uint32_t rdx = (uint32_t)rreg(hvf_fd, HV_X86_RDX);
 
             if (rax == 1) {
                 /* CPUID1.ecx.OSXSAVE needs to know CR4 */
-                env->cr[4] = rvmcs(cpu->hvf_fd, VMCS_GUEST_CR4);
+                env->cr[4] = rvmcs(hvf_fd, VMCS_GUEST_CR4);
             }
             hvf_cpu_x86_cpuid(env, rax, rcx, &rax, &rbx, &rcx, &rdx);
 
-            wreg(cpu->hvf_fd, HV_X86_RAX, rax);
-            wreg(cpu->hvf_fd, HV_X86_RBX, rbx);
-            wreg(cpu->hvf_fd, HV_X86_RCX, rcx);
-            wreg(cpu->hvf_fd, HV_X86_RDX, rdx);
+            wreg(hvf_fd, HV_X86_RAX, rax);
+            wreg(hvf_fd, HV_X86_RBX, rbx);
+            wreg(hvf_fd, HV_X86_RCX, rcx);
+            wreg(hvf_fd, HV_X86_RDX, rdx);
 
             macvm_set_rip(cpu, rip + ins_len);
             break;
@@ -845,16 +848,16 @@ int hvf_vcpu_exec(CPUState *cpu)
         case EXIT_REASON_XSETBV: {
             X86CPU *x86_cpu = X86_CPU(cpu);
             CPUX86State *env = &x86_cpu->env;
-            uint32_t eax = (uint32_t)rreg(cpu->hvf_fd, HV_X86_RAX);
-            uint32_t ecx = (uint32_t)rreg(cpu->hvf_fd, HV_X86_RCX);
-            uint32_t edx = (uint32_t)rreg(cpu->hvf_fd, HV_X86_RDX);
+            uint32_t eax = (uint32_t)rreg(hvf_fd, HV_X86_RAX);
+            uint32_t ecx = (uint32_t)rreg(hvf_fd, HV_X86_RCX);
+            uint32_t edx = (uint32_t)rreg(hvf_fd, HV_X86_RDX);
 
             if (ecx) {
                 macvm_set_rip(cpu, rip + ins_len);
                 break;
             }
             env->xcr0 = ((uint64_t)edx << 32) | eax;
-            wreg(cpu->hvf_fd, HV_X86_XCR0, env->xcr0 | 1);
+            wreg(hvf_fd, HV_X86_XCR0, env->xcr0 | 1);
             macvm_set_rip(cpu, rip + ins_len);
             break;
         }
@@ -893,11 +896,11 @@ int hvf_vcpu_exec(CPUState *cpu)
 
             switch (cr) {
             case 0x0: {
-                macvm_set_cr0(cpu->hvf_fd, RRX(env, reg));
+                macvm_set_cr0(hvf_fd, RRX(env, reg));
                 break;
             }
             case 4: {
-                macvm_set_cr4(cpu->hvf_fd, RRX(env, reg));
+                macvm_set_cr4(hvf_fd, RRX(env, reg));
                 break;
             }
             case 8: {
@@ -933,7 +936,7 @@ int hvf_vcpu_exec(CPUState *cpu)
             break;
         }
         case EXIT_REASON_TASK_SWITCH: {
-            uint64_t vinfo = rvmcs(cpu->hvf_fd, VMCS_IDT_VECTORING_INFO);
+            uint64_t vinfo = rvmcs(hvf_fd, VMCS_IDT_VECTORING_INFO);
             x68_segment_selector sel = {.sel = exit_qual & 0xffff};
             vmx_handle_task_switch(cpu, sel, (exit_qual >> 30) & 0x3,
              vinfo & VMCS_INTR_VALID, vinfo & VECTORING_INFO_VECTOR_MASK, vinfo
@@ -946,8 +949,8 @@ int hvf_vcpu_exec(CPUState *cpu)
             break;
         }
         case EXIT_REASON_RDPMC:
-            wreg(cpu->hvf_fd, HV_X86_RAX, 0);
-            wreg(cpu->hvf_fd, HV_X86_RDX, 0);
+            wreg(hvf_fd, HV_X86_RAX, 0);
+            wreg(hvf_fd, HV_X86_RDX, 0);
             macvm_set_rip(cpu, rip + ins_len);
             break;
         case VMX_REASON_VMCALL:
diff --git a/target/i386/hvf/x86_descr.c b/target/i386/hvf/x86_descr.c
index 9f539e73f6d..1c6220baa0d 100644
--- a/target/i386/hvf/x86_descr.c
+++ b/target/i386/hvf/x86_descr.c
@@ -75,20 +75,23 @@ void vmx_write_segment_selector(struct CPUState *cpu, x68_segment_selector selec
 
 void vmx_read_segment_descriptor(struct CPUState *cpu, struct vmx_segment *desc, X86Seg seg)
 {
-    desc->sel = rvmcs(cpu->hvf_fd, vmx_segment_fields[seg].selector);
-    desc->base = rvmcs(cpu->hvf_fd, vmx_segment_fields[seg].base);
-    desc->limit = rvmcs(cpu->hvf_fd, vmx_segment_fields[seg].limit);
-    desc->ar = rvmcs(cpu->hvf_fd, vmx_segment_fields[seg].ar_bytes);
+    hv_vcpuid_t hvf_fd = (hv_vcpuid_t)cpu_state->hvf_fd;
+
+    desc->sel = rvmcs(hvf_fd, vmx_segment_fields[seg].selector);
+    desc->base = rvmcs(hvf_fd, vmx_segment_fields[seg].base);
+    desc->limit = rvmcs(hvf_fd, vmx_segment_fields[seg].limit);
+    desc->ar = rvmcs(hvf_fd, vmx_segment_fields[seg].ar_bytes);
 }
 
 void vmx_write_segment_descriptor(CPUState *cpu, struct vmx_segment *desc, X86Seg seg)
 {
     const struct vmx_segment_field *sf = &vmx_segment_fields[seg];
+    hv_vcpuid_t hvf_fd = (hv_vcpuid_t)cpu_state->hvf_fd;
 
-    wvmcs(cpu->hvf_fd, sf->base, desc->base);
-    wvmcs(cpu->hvf_fd, sf->limit, desc->limit);
-    wvmcs(cpu->hvf_fd, sf->selector, desc->sel);
-    wvmcs(cpu->hvf_fd, sf->ar_bytes, desc->ar);
+    wvmcs(hvf_fd, sf->base, desc->base);
+    wvmcs(hvf_fd, sf->limit, desc->limit);
+    wvmcs(hvf_fd, sf->selector, desc->sel);
+    wvmcs(hvf_fd, sf->ar_bytes, desc->ar);
 }
 
 void x86_segment_descriptor_to_vmx(struct CPUState *cpu, x68_segment_selector selector, struct x86_segment_descriptor *desc, struct vmx_segment *vmx_desc)
diff --git a/target/i386/hvf/x86hvf.c b/target/i386/hvf/x86hvf.c
index 0d7533742eb..2f291f2ad53 100644
--- a/target/i386/hvf/x86hvf.c
+++ b/target/i386/hvf/x86hvf.c
@@ -89,21 +89,22 @@ void hvf_put_xsave(CPUState *cpu_state)
 void hvf_put_segments(CPUState *cpu_state)
 {
     CPUX86State *env = &X86_CPU(cpu_state)->env;
+    hv_vcpuid_t hvf_fd = (hv_vcpuid_t)cpu_state->hvf_fd;
     struct vmx_segment seg;
     
-    wvmcs(cpu_state->hvf_fd, VMCS_GUEST_IDTR_LIMIT, env->idt.limit);
-    wvmcs(cpu_state->hvf_fd, VMCS_GUEST_IDTR_BASE, env->idt.base);
+    wvmcs(hvf_fd, VMCS_GUEST_IDTR_LIMIT, env->idt.limit);
+    wvmcs(hvf_fd, VMCS_GUEST_IDTR_BASE, env->idt.base);
 
-    wvmcs(cpu_state->hvf_fd, VMCS_GUEST_GDTR_LIMIT, env->gdt.limit);
-    wvmcs(cpu_state->hvf_fd, VMCS_GUEST_GDTR_BASE, env->gdt.base);
+    wvmcs(hvf_fd, VMCS_GUEST_GDTR_LIMIT, env->gdt.limit);
+    wvmcs(hvf_fd, VMCS_GUEST_GDTR_BASE, env->gdt.base);
 
-    /* wvmcs(cpu_state->hvf_fd, VMCS_GUEST_CR2, env->cr[2]); */
-    wvmcs(cpu_state->hvf_fd, VMCS_GUEST_CR3, env->cr[3]);
+    /* wvmcs(hvf_fd, VMCS_GUEST_CR2, env->cr[2]); */
+    wvmcs(hvf_fd, VMCS_GUEST_CR3, env->cr[3]);
     vmx_update_tpr(cpu_state);
-    wvmcs(cpu_state->hvf_fd, VMCS_GUEST_IA32_EFER, env->efer);
+    wvmcs(hvf_fd, VMCS_GUEST_IA32_EFER, env->efer);
 
-    macvm_set_cr4(cpu_state->hvf_fd, env->cr[4]);
-    macvm_set_cr0(cpu_state->hvf_fd, env->cr[0]);
+    macvm_set_cr4(hvf_fd, env->cr[4]);
+    macvm_set_cr0(hvf_fd, env->cr[0]);
 
     hvf_set_segment(cpu_state, &seg, &env->segs[R_CS], false);
     vmx_write_segment_descriptor(cpu_state, &seg, R_CS);
@@ -129,31 +130,29 @@ void hvf_put_segments(CPUState *cpu_state)
     hvf_set_segment(cpu_state, &seg, &env->ldt, false);
     vmx_write_segment_descriptor(cpu_state, &seg, R_LDTR);
     
-    hv_vcpu_flush(cpu_state->hvf_fd);
+    hv_vcpu_flush(hvf_fd);
 }
     
 void hvf_put_msrs(CPUState *cpu_state)
 {
     CPUX86State *env = &X86_CPU(cpu_state)->env;
+    hv_vcpuid_t hvf_fd = (hv_vcpuid_t)cpu_state->hvf_fd;
 
-    hv_vcpu_write_msr(cpu_state->hvf_fd, MSR_IA32_SYSENTER_CS,
-                      env->sysenter_cs);
-    hv_vcpu_write_msr(cpu_state->hvf_fd, MSR_IA32_SYSENTER_ESP,
-                      env->sysenter_esp);
-    hv_vcpu_write_msr(cpu_state->hvf_fd, MSR_IA32_SYSENTER_EIP,
-                      env->sysenter_eip);
+    hv_vcpu_write_msr(hvf_fd, MSR_IA32_SYSENTER_CS, env->sysenter_cs);
+    hv_vcpu_write_msr(hvf_fd, MSR_IA32_SYSENTER_ESP, env->sysenter_esp);
+    hv_vcpu_write_msr(hvf_fd, MSR_IA32_SYSENTER_EIP, env->sysenter_eip);
 
-    hv_vcpu_write_msr(cpu_state->hvf_fd, MSR_STAR, env->star);
+    hv_vcpu_write_msr(hvf_fd, MSR_STAR, env->star);
 
 #ifdef TARGET_X86_64
-    hv_vcpu_write_msr(cpu_state->hvf_fd, MSR_CSTAR, env->cstar);
-    hv_vcpu_write_msr(cpu_state->hvf_fd, MSR_KERNELGSBASE, env->kernelgsbase);
-    hv_vcpu_write_msr(cpu_state->hvf_fd, MSR_FMASK, env->fmask);
-    hv_vcpu_write_msr(cpu_state->hvf_fd, MSR_LSTAR, env->lstar);
+    hv_vcpu_write_msr(hvf_fd, MSR_CSTAR, env->cstar);
+    hv_vcpu_write_msr(hvf_fd, MSR_KERNELGSBASE, env->kernelgsbase);
+    hv_vcpu_write_msr(hvf_fd, MSR_FMASK, env->fmask);
+    hv_vcpu_write_msr(hvf_fd, MSR_LSTAR, env->lstar);
 #endif
 
-    hv_vcpu_write_msr(cpu_state->hvf_fd, MSR_GSBASE, env->segs[R_GS].base);
-    hv_vcpu_write_msr(cpu_state->hvf_fd, MSR_FSBASE, env->segs[R_FS].base);
+    hv_vcpu_write_msr(hvf_fd, MSR_GSBASE, env->segs[R_GS].base);
+    hv_vcpu_write_msr(hvf_fd, MSR_FSBASE, env->segs[R_FS].base);
 }
 
 
@@ -173,7 +172,7 @@ void hvf_get_xsave(CPUState *cpu_state)
 void hvf_get_segments(CPUState *cpu_state)
 {
     CPUX86State *env = &X86_CPU(cpu_state)->env;
-
+    hv_vcpuid_t hvf_fd = (hv_vcpuid_t)cpu_state->hvf_fd;
     struct vmx_segment seg;
 
     env->interrupt_injected = -1;
@@ -202,72 +201,74 @@ void hvf_get_segments(CPUState *cpu_state)
     vmx_read_segment_descriptor(cpu_state, &seg, R_LDTR);
     hvf_get_segment(&env->ldt, &seg);
 
-    env->idt.limit = rvmcs(cpu_state->hvf_fd, VMCS_GUEST_IDTR_LIMIT);
-    env->idt.base = rvmcs(cpu_state->hvf_fd, VMCS_GUEST_IDTR_BASE);
-    env->gdt.limit = rvmcs(cpu_state->hvf_fd, VMCS_GUEST_GDTR_LIMIT);
-    env->gdt.base = rvmcs(cpu_state->hvf_fd, VMCS_GUEST_GDTR_BASE);
+    env->idt.limit = rvmcs(hvf_fd, VMCS_GUEST_IDTR_LIMIT);
+    env->idt.base = rvmcs(hvf_fd, VMCS_GUEST_IDTR_BASE);
+    env->gdt.limit = rvmcs(hvf_fd, VMCS_GUEST_GDTR_LIMIT);
+    env->gdt.base = rvmcs(hvf_fd, VMCS_GUEST_GDTR_BASE);
 
-    env->cr[0] = rvmcs(cpu_state->hvf_fd, VMCS_GUEST_CR0);
+    env->cr[0] = rvmcs(hvf_fd, VMCS_GUEST_CR0);
     env->cr[2] = 0;
-    env->cr[3] = rvmcs(cpu_state->hvf_fd, VMCS_GUEST_CR3);
-    env->cr[4] = rvmcs(cpu_state->hvf_fd, VMCS_GUEST_CR4);
+    env->cr[3] = rvmcs(hvf_fd, VMCS_GUEST_CR3);
+    env->cr[4] = rvmcs(hvf_fd, VMCS_GUEST_CR4);
     
-    env->efer = rvmcs(cpu_state->hvf_fd, VMCS_GUEST_IA32_EFER);
+    env->efer = rvmcs(hvf_fd, VMCS_GUEST_IA32_EFER);
 }
 
 void hvf_get_msrs(CPUState *cpu_state)
 {
     CPUX86State *env = &X86_CPU(cpu_state)->env;
+    hv_vcpuid_t hvf_fd = (hv_vcpuid_t)cpu_state->hvf_fd;
     uint64_t tmp;
     
-    hv_vcpu_read_msr(cpu_state->hvf_fd, MSR_IA32_SYSENTER_CS, &tmp);
+    hv_vcpu_read_msr(hvf_fd, MSR_IA32_SYSENTER_CS, &tmp);
     env->sysenter_cs = tmp;
     
-    hv_vcpu_read_msr(cpu_state->hvf_fd, MSR_IA32_SYSENTER_ESP, &tmp);
+    hv_vcpu_read_msr(hvf_fd, MSR_IA32_SYSENTER_ESP, &tmp);
     env->sysenter_esp = tmp;
 
-    hv_vcpu_read_msr(cpu_state->hvf_fd, MSR_IA32_SYSENTER_EIP, &tmp);
+    hv_vcpu_read_msr(hvf_fd, MSR_IA32_SYSENTER_EIP, &tmp);
     env->sysenter_eip = tmp;
 
-    hv_vcpu_read_msr(cpu_state->hvf_fd, MSR_STAR, &env->star);
+    hv_vcpu_read_msr(hvf_fd, MSR_STAR, &env->star);
 
 #ifdef TARGET_X86_64
-    hv_vcpu_read_msr(cpu_state->hvf_fd, MSR_CSTAR, &env->cstar);
-    hv_vcpu_read_msr(cpu_state->hvf_fd, MSR_KERNELGSBASE, &env->kernelgsbase);
-    hv_vcpu_read_msr(cpu_state->hvf_fd, MSR_FMASK, &env->fmask);
-    hv_vcpu_read_msr(cpu_state->hvf_fd, MSR_LSTAR, &env->lstar);
+    hv_vcpu_read_msr(hvf_fd, MSR_CSTAR, &env->cstar);
+    hv_vcpu_read_msr(hvf_fd, MSR_KERNELGSBASE, &env->kernelgsbase);
+    hv_vcpu_read_msr(hvf_fd, MSR_FMASK, &env->fmask);
+    hv_vcpu_read_msr(hvf_fd, MSR_LSTAR, &env->lstar);
 #endif
 
-    hv_vcpu_read_msr(cpu_state->hvf_fd, MSR_IA32_APICBASE, &tmp);
+    hv_vcpu_read_msr(hvf_fd, MSR_IA32_APICBASE, &tmp);
     
-    env->tsc = rdtscp() + rvmcs(cpu_state->hvf_fd, VMCS_TSC_OFFSET);
+    env->tsc = rdtscp() + rvmcs(hvf_fd, VMCS_TSC_OFFSET);
 }
 
 int hvf_put_registers(CPUState *cpu_state)
 {
     X86CPU *x86cpu = X86_CPU(cpu_state);
     CPUX86State *env = &x86cpu->env;
+    hv_vcpuid_t hvf_fd = (hv_vcpuid_t)cpu_state->hvf_fd;
 
-    wreg(cpu_state->hvf_fd, HV_X86_RAX, env->regs[R_EAX]);
-    wreg(cpu_state->hvf_fd, HV_X86_RBX, env->regs[R_EBX]);
-    wreg(cpu_state->hvf_fd, HV_X86_RCX, env->regs[R_ECX]);
-    wreg(cpu_state->hvf_fd, HV_X86_RDX, env->regs[R_EDX]);
-    wreg(cpu_state->hvf_fd, HV_X86_RBP, env->regs[R_EBP]);
-    wreg(cpu_state->hvf_fd, HV_X86_RSP, env->regs[R_ESP]);
-    wreg(cpu_state->hvf_fd, HV_X86_RSI, env->regs[R_ESI]);
-    wreg(cpu_state->hvf_fd, HV_X86_RDI, env->regs[R_EDI]);
-    wreg(cpu_state->hvf_fd, HV_X86_R8, env->regs[8]);
-    wreg(cpu_state->hvf_fd, HV_X86_R9, env->regs[9]);
-    wreg(cpu_state->hvf_fd, HV_X86_R10, env->regs[10]);
-    wreg(cpu_state->hvf_fd, HV_X86_R11, env->regs[11]);
-    wreg(cpu_state->hvf_fd, HV_X86_R12, env->regs[12]);
-    wreg(cpu_state->hvf_fd, HV_X86_R13, env->regs[13]);
-    wreg(cpu_state->hvf_fd, HV_X86_R14, env->regs[14]);
-    wreg(cpu_state->hvf_fd, HV_X86_R15, env->regs[15]);
-    wreg(cpu_state->hvf_fd, HV_X86_RFLAGS, env->eflags);
-    wreg(cpu_state->hvf_fd, HV_X86_RIP, env->eip);
+    wreg(hvf_fd, HV_X86_RAX, env->regs[R_EAX]);
+    wreg(hvf_fd, HV_X86_RBX, env->regs[R_EBX]);
+    wreg(hvf_fd, HV_X86_RCX, env->regs[R_ECX]);
+    wreg(hvf_fd, HV_X86_RDX, env->regs[R_EDX]);
+    wreg(hvf_fd, HV_X86_RBP, env->regs[R_EBP]);
+    wreg(hvf_fd, HV_X86_RSP, env->regs[R_ESP]);
+    wreg(hvf_fd, HV_X86_RSI, env->regs[R_ESI]);
+    wreg(hvf_fd, HV_X86_RDI, env->regs[R_EDI]);
+    wreg(hvf_fd, HV_X86_R8, env->regs[8]);
+    wreg(hvf_fd, HV_X86_R9, env->regs[9]);
+    wreg(hvf_fd, HV_X86_R10, env->regs[10]);
+    wreg(hvf_fd, HV_X86_R11, env->regs[11]);
+    wreg(hvf_fd, HV_X86_R12, env->regs[12]);
+    wreg(hvf_fd, HV_X86_R13, env->regs[13]);
+    wreg(hvf_fd, HV_X86_R14, env->regs[14]);
+    wreg(hvf_fd, HV_X86_R15, env->regs[15]);
+    wreg(hvf_fd, HV_X86_RFLAGS, env->eflags);
+    wreg(hvf_fd, HV_X86_RIP, env->eip);
    
-    wreg(cpu_state->hvf_fd, HV_X86_XCR0, env->xcr0);
+    wreg(hvf_fd, HV_X86_XCR0, env->xcr0);
     
     hvf_put_xsave(cpu_state);
     
@@ -275,14 +276,14 @@ int hvf_put_registers(CPUState *cpu_state)
     
     hvf_put_msrs(cpu_state);
     
-    wreg(cpu_state->hvf_fd, HV_X86_DR0, env->dr[0]);
-    wreg(cpu_state->hvf_fd, HV_X86_DR1, env->dr[1]);
-    wreg(cpu_state->hvf_fd, HV_X86_DR2, env->dr[2]);
-    wreg(cpu_state->hvf_fd, HV_X86_DR3, env->dr[3]);
-    wreg(cpu_state->hvf_fd, HV_X86_DR4, env->dr[4]);
-    wreg(cpu_state->hvf_fd, HV_X86_DR5, env->dr[5]);
-    wreg(cpu_state->hvf_fd, HV_X86_DR6, env->dr[6]);
-    wreg(cpu_state->hvf_fd, HV_X86_DR7, env->dr[7]);
+    wreg(hvf_fd, HV_X86_DR0, env->dr[0]);
+    wreg(hvf_fd, HV_X86_DR1, env->dr[1]);
+    wreg(hvf_fd, HV_X86_DR2, env->dr[2]);
+    wreg(hvf_fd, HV_X86_DR3, env->dr[3]);
+    wreg(hvf_fd, HV_X86_DR4, env->dr[4]);
+    wreg(hvf_fd, HV_X86_DR5, env->dr[5]);
+    wreg(hvf_fd, HV_X86_DR6, env->dr[6]);
+    wreg(hvf_fd, HV_X86_DR7, env->dr[7]);
     
     return 0;
 }
@@ -291,41 +292,42 @@ int hvf_get_registers(CPUState *cpu_state)
 {
     X86CPU *x86cpu = X86_CPU(cpu_state);
     CPUX86State *env = &x86cpu->env;
+    hv_vcpuid_t hvf_fd = (hv_vcpuid_t)cpu_state->hvf_fd;
 
-    env->regs[R_EAX] = rreg(cpu_state->hvf_fd, HV_X86_RAX);
-    env->regs[R_EBX] = rreg(cpu_state->hvf_fd, HV_X86_RBX);
-    env->regs[R_ECX] = rreg(cpu_state->hvf_fd, HV_X86_RCX);
-    env->regs[R_EDX] = rreg(cpu_state->hvf_fd, HV_X86_RDX);
-    env->regs[R_EBP] = rreg(cpu_state->hvf_fd, HV_X86_RBP);
-    env->regs[R_ESP] = rreg(cpu_state->hvf_fd, HV_X86_RSP);
-    env->regs[R_ESI] = rreg(cpu_state->hvf_fd, HV_X86_RSI);
-    env->regs[R_EDI] = rreg(cpu_state->hvf_fd, HV_X86_RDI);
-    env->regs[8] = rreg(cpu_state->hvf_fd, HV_X86_R8);
-    env->regs[9] = rreg(cpu_state->hvf_fd, HV_X86_R9);
-    env->regs[10] = rreg(cpu_state->hvf_fd, HV_X86_R10);
-    env->regs[11] = rreg(cpu_state->hvf_fd, HV_X86_R11);
-    env->regs[12] = rreg(cpu_state->hvf_fd, HV_X86_R12);
-    env->regs[13] = rreg(cpu_state->hvf_fd, HV_X86_R13);
-    env->regs[14] = rreg(cpu_state->hvf_fd, HV_X86_R14);
-    env->regs[15] = rreg(cpu_state->hvf_fd, HV_X86_R15);
+    env->regs[R_EAX] = rreg(hvf_fd, HV_X86_RAX);
+    env->regs[R_EBX] = rreg(hvf_fd, HV_X86_RBX);
+    env->regs[R_ECX] = rreg(hvf_fd, HV_X86_RCX);
+    env->regs[R_EDX] = rreg(hvf_fd, HV_X86_RDX);
+    env->regs[R_EBP] = rreg(hvf_fd, HV_X86_RBP);
+    env->regs[R_ESP] = rreg(hvf_fd, HV_X86_RSP);
+    env->regs[R_ESI] = rreg(hvf_fd, HV_X86_RSI);
+    env->regs[R_EDI] = rreg(hvf_fd, HV_X86_RDI);
+    env->regs[8] = rreg(hvf_fd, HV_X86_R8);
+    env->regs[9] = rreg(hvf_fd, HV_X86_R9);
+    env->regs[10] = rreg(hvf_fd, HV_X86_R10);
+    env->regs[11] = rreg(hvf_fd, HV_X86_R11);
+    env->regs[12] = rreg(hvf_fd, HV_X86_R12);
+    env->regs[13] = rreg(hvf_fd, HV_X86_R13);
+    env->regs[14] = rreg(hvf_fd, HV_X86_R14);
+    env->regs[15] = rreg(hvf_fd, HV_X86_R15);
     
-    env->eflags = rreg(cpu_state->hvf_fd, HV_X86_RFLAGS);
-    env->eip = rreg(cpu_state->hvf_fd, HV_X86_RIP);
+    env->eflags = rreg(hvf_fd, HV_X86_RFLAGS);
+    env->eip = rreg(hvf_fd, HV_X86_RIP);
    
     hvf_get_xsave(cpu_state);
-    env->xcr0 = rreg(cpu_state->hvf_fd, HV_X86_XCR0);
+    env->xcr0 = rreg(hvf_fd, HV_X86_XCR0);
     
     hvf_get_segments(cpu_state);
     hvf_get_msrs(cpu_state);
     
-    env->dr[0] = rreg(cpu_state->hvf_fd, HV_X86_DR0);
-    env->dr[1] = rreg(cpu_state->hvf_fd, HV_X86_DR1);
-    env->dr[2] = rreg(cpu_state->hvf_fd, HV_X86_DR2);
-    env->dr[3] = rreg(cpu_state->hvf_fd, HV_X86_DR3);
-    env->dr[4] = rreg(cpu_state->hvf_fd, HV_X86_DR4);
-    env->dr[5] = rreg(cpu_state->hvf_fd, HV_X86_DR5);
-    env->dr[6] = rreg(cpu_state->hvf_fd, HV_X86_DR6);
-    env->dr[7] = rreg(cpu_state->hvf_fd, HV_X86_DR7);
+    env->dr[0] = rreg(hvf_fd, HV_X86_DR0);
+    env->dr[1] = rreg(hvf_fd, HV_X86_DR1);
+    env->dr[2] = rreg(hvf_fd, HV_X86_DR2);
+    env->dr[3] = rreg(hvf_fd, HV_X86_DR3);
+    env->dr[4] = rreg(hvf_fd, HV_X86_DR4);
+    env->dr[5] = rreg(hvf_fd, HV_X86_DR5);
+    env->dr[6] = rreg(hvf_fd, HV_X86_DR6);
+    env->dr[7] = rreg(hvf_fd, HV_X86_DR7);
     
     x86_update_hflags(env);
     return 0;
@@ -351,6 +353,7 @@ bool hvf_inject_interrupts(CPUState *cpu_state)
 {
     X86CPU *x86cpu = X86_CPU(cpu_state);
     CPUX86State *env = &x86cpu->env;
+    hv_vcpuid_t hvf_fd = (hv_vcpuid_t)cpu_state->hvf_fd;
 
     uint8_t vector;
     uint64_t intr_type;
@@ -379,7 +382,7 @@ bool hvf_inject_interrupts(CPUState *cpu_state)
     uint64_t info = 0;
     if (have_event) {
         info = vector | intr_type | VMCS_INTR_VALID;
-        uint64_t reason = rvmcs(cpu_state->hvf_fd, VMCS_EXIT_REASON);
+        uint64_t reason = rvmcs(hvf_fd, VMCS_EXIT_REASON);
         if (env->nmi_injected && reason != EXIT_REASON_TASK_SWITCH) {
             vmx_clear_nmi_blocking(cpu_state);
         }
@@ -388,17 +391,17 @@ bool hvf_inject_interrupts(CPUState *cpu_state)
             info &= ~(1 << 12); /* clear undefined bit */
             if (intr_type == VMCS_INTR_T_SWINTR ||
                 intr_type == VMCS_INTR_T_SWEXCEPTION) {
-                wvmcs(cpu_state->hvf_fd, VMCS_ENTRY_INST_LENGTH, env->ins_len);
+                wvmcs(hvf_fd, VMCS_ENTRY_INST_LENGTH, env->ins_len);
             }
             
             if (env->has_error_code) {
-                wvmcs(cpu_state->hvf_fd, VMCS_ENTRY_EXCEPTION_ERROR,
+                wvmcs(hvf_fd, VMCS_ENTRY_EXCEPTION_ERROR,
                       env->error_code);
                 /* Indicate that VMCS_ENTRY_EXCEPTION_ERROR is valid */
                 info |= VMCS_INTR_DEL_ERRCODE;
             }
             /*printf("reinject  %lx err %d\n", info, err);*/
-            wvmcs(cpu_state->hvf_fd, VMCS_ENTRY_INTR_INFO, info);
+            wvmcs(hvf_fd, VMCS_ENTRY_INTR_INFO, info);
         };
     }
 
@@ -406,7 +409,7 @@ bool hvf_inject_interrupts(CPUState *cpu_state)
         if (!(env->hflags2 & HF2_NMI_MASK) && !(info & VMCS_INTR_VALID)) {
             cpu_state->interrupt_request &= ~CPU_INTERRUPT_NMI;
             info = VMCS_INTR_VALID | VMCS_INTR_T_NMI | EXCP02_NMI;
-            wvmcs(cpu_state->hvf_fd, VMCS_ENTRY_INTR_INFO, info);
+            wvmcs(hvf_fd, VMCS_ENTRY_INTR_INFO, info);
         } else {
             vmx_set_nmi_window_exiting(cpu_state);
         }
@@ -418,8 +421,8 @@ bool hvf_inject_interrupts(CPUState *cpu_state)
         int line = cpu_get_pic_interrupt(&x86cpu->env);
         cpu_state->interrupt_request &= ~CPU_INTERRUPT_HARD;
         if (line >= 0) {
-            wvmcs(cpu_state->hvf_fd, VMCS_ENTRY_INTR_INFO, line |
-                  VMCS_INTR_VALID | VMCS_INTR_T_HWINTR);
+            wvmcs(hvf_fd, VMCS_ENTRY_INTR_INFO,
+                  line | VMCS_INTR_VALID | VMCS_INTR_T_HWINTR);
         }
     }
     if (cpu_state->interrupt_request & CPU_INTERRUPT_HARD) {
-- 
2.26.2

[RFC PATCH 18/19] accel/hvf: Declare and allocate AccelvCPUState struct

[Philippe Mathieu-Daudé]

In preparation of moving HVF-specific fields from CPUState
to the accelerator-specific AccelvCPUState structure, first
declare it empty and allocate it. This will make the following
commits easier to review.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 target/i386/hvf/hvf-i386.h | 3 +++
 target/i386/hvf/hvf.c      | 4 +++-
 2 files changed, 6 insertions(+), 1 deletion(-)

diff --git a/target/i386/hvf/hvf-i386.h b/target/i386/hvf/hvf-i386.h
index 59cfca8875e..1f12eb647a0 100644
--- a/target/i386/hvf/hvf-i386.h
+++ b/target/i386/hvf/hvf-i386.h
@@ -51,6 +51,9 @@ struct HVFState {
 };
 extern HVFState *hvf_state;
 
+struct AccelvCPUState {
+};
+
 void hvf_set_phys_mem(MemoryRegionSection *, bool);
 void hvf_handle_io(CPUArchState *, uint16_t, void *, int, int, int);
 hvf_slot *hvf_find_overlap_slot(uint64_t, uint64_t);
diff --git a/target/i386/hvf/hvf.c b/target/i386/hvf/hvf.c
index effee39ee9b..342659f1e15 100644
--- a/target/i386/hvf/hvf.c
+++ b/target/i386/hvf/hvf.c
@@ -451,6 +451,7 @@ void hvf_vcpu_destroy(CPUState *cpu)
     hv_return_t ret = hv_vcpu_destroy((hv_vcpuid_t)cpu->hvf_fd);
     g_free(env->hvf_mmio_buf);
     assert_hvf_ok(ret);
+    g_free(cpu->accel_vcpu);
 }
 
 static void dummy_signal(int sig)
@@ -534,9 +535,10 @@ int hvf_init_vcpu(CPUState *cpu)
     }
 
     r = hv_vcpu_create(&hvf_fd, HV_VCPU_DEFAULT);
-    cpu->vcpu_dirty = true;
     assert_hvf_ok(r);
+    cpu->accel_vcpu = g_new(struct AccelvCPUState, 1);
     cpu->hvf_fd = (int)hvf_fd
+    cpu->vcpu_dirty = true;
 
     if (hv_vmx_read_capability(HV_VMX_CAP_PINBASED,
         &hvf_state->hvf_caps->vmx_cap_pinbased)) {
-- 
2.26.2

[RFC PATCH 19/19] accel/hvf: Move the 'hvf_fd' field to AccelvCPUState

[Philippe Mathieu-Daudé]

Move the 'hvf_fd' field from CPUState to AccelvCPUState,
and declare it with its correct type: hv_vcpuid_t.

Signed-off-by: Philippe Mathieu-Daudé <philmd@redhat.com>
---
 include/hw/core/cpu.h       |  1 -
 target/i386/hvf/hvf-i386.h  |  1 +
 target/i386/hvf/vmx.h       | 28 +++++++++--------
 target/i386/hvf/hvf.c       | 23 +++++++-------
 target/i386/hvf/x86.c       | 28 ++++++++---------
 target/i386/hvf/x86_descr.c | 17 +++++-----
 target/i386/hvf/x86_emu.c   | 62 ++++++++++++++++++-------------------
 target/i386/hvf/x86_mmu.c   |  4 +--
 target/i386/hvf/x86_task.c  | 14 +++++----
 target/i386/hvf/x86hvf.c    | 32 ++++++++++---------
 10 files changed, 110 insertions(+), 100 deletions(-)

diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index 3268f1393f1..69a456415c0 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -415,7 +415,6 @@ struct CPUState {
 
     /* Accelerator-specific fields. */
     struct AccelvCPUState *accel_vcpu;
-    int hvf_fd;
     /* shared by kvm, hax and hvf */
     bool vcpu_dirty;
 
diff --git a/target/i386/hvf/hvf-i386.h b/target/i386/hvf/hvf-i386.h
index 1f12eb647a0..e17f9f42c0e 100644
--- a/target/i386/hvf/hvf-i386.h
+++ b/target/i386/hvf/hvf-i386.h
@@ -52,6 +52,7 @@ struct HVFState {
 extern HVFState *hvf_state;
 
 struct AccelvCPUState {
+    hv_vcpuid_t hvf_fd;
 };
 
 void hvf_set_phys_mem(MemoryRegionSection *, bool);
diff --git a/target/i386/hvf/vmx.h b/target/i386/hvf/vmx.h
index 24c4cdf0be0..bed94856268 100644
--- a/target/i386/hvf/vmx.h
+++ b/target/i386/hvf/vmx.h
@@ -179,15 +179,15 @@ static inline void macvm_set_rip(CPUState *cpu, uint64_t rip)
     uint64_t val;
 
     /* BUG, should take considering overlap.. */
-    wreg(cpu->hvf_fd, HV_X86_RIP, rip);
+    wreg(cpu->accel_vcpu->hvf_fd, HV_X86_RIP, rip);
     env->eip = rip;
 
     /* after moving forward in rip, we need to clean INTERRUPTABILITY */
-   val = rvmcs(cpu->hvf_fd, VMCS_GUEST_INTERRUPTIBILITY);
+   val = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_INTERRUPTIBILITY);
    if (val & (VMCS_INTERRUPTIBILITY_STI_BLOCKING |
                VMCS_INTERRUPTIBILITY_MOVSS_BLOCKING)) {
         env->hflags &= ~HF_INHIBIT_IRQ_MASK;
-        wvmcs(cpu->hvf_fd, VMCS_GUEST_INTERRUPTIBILITY,
+        wvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_INTERRUPTIBILITY,
                val & ~(VMCS_INTERRUPTIBILITY_STI_BLOCKING |
                VMCS_INTERRUPTIBILITY_MOVSS_BLOCKING));
    }
@@ -199,9 +199,10 @@ static inline void vmx_clear_nmi_blocking(CPUState *cpu)
     CPUX86State *env = &x86_cpu->env;
 
     env->hflags2 &= ~HF2_NMI_MASK;
-    uint32_t gi = (uint32_t) rvmcs(cpu->hvf_fd, VMCS_GUEST_INTERRUPTIBILITY);
+    uint32_t gi = (uint32_t) rvmcs(cpu->accel_vcpu->hvf_fd,
+                                   VMCS_GUEST_INTERRUPTIBILITY);
     gi &= ~VMCS_INTERRUPTIBILITY_NMI_BLOCKING;
-    wvmcs(cpu->hvf_fd, VMCS_GUEST_INTERRUPTIBILITY, gi);
+    wvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_INTERRUPTIBILITY, gi);
 }
 
 static inline void vmx_set_nmi_blocking(CPUState *cpu)
@@ -210,17 +211,18 @@ static inline void vmx_set_nmi_blocking(CPUState *cpu)
     CPUX86State *env = &x86_cpu->env;
 
     env->hflags2 |= HF2_NMI_MASK;
-    uint32_t gi = (uint32_t)rvmcs(cpu->hvf_fd, VMCS_GUEST_INTERRUPTIBILITY);
+    uint32_t gi = (uint32_t)rvmcs(cpu->accel_vcpu->hvf_fd,
+                                  VMCS_GUEST_INTERRUPTIBILITY);
     gi |= VMCS_INTERRUPTIBILITY_NMI_BLOCKING;
-    wvmcs(cpu->hvf_fd, VMCS_GUEST_INTERRUPTIBILITY, gi);
+    wvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_INTERRUPTIBILITY, gi);
 }
 
 static inline void vmx_set_nmi_window_exiting(CPUState *cpu)
 {
     uint64_t val;
-    val = rvmcs(cpu->hvf_fd, VMCS_PRI_PROC_BASED_CTLS);
-    wvmcs(cpu->hvf_fd, VMCS_PRI_PROC_BASED_CTLS, val |
-          VMCS_PRI_PROC_BASED_CTLS_NMI_WINDOW_EXITING);
+    val = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_PRI_PROC_BASED_CTLS);
+    wvmcs(cpu->accel_vcpu->hvf_fd, VMCS_PRI_PROC_BASED_CTLS,
+          val | VMCS_PRI_PROC_BASED_CTLS_NMI_WINDOW_EXITING);
 
 }
 
@@ -228,9 +230,9 @@ static inline void vmx_clear_nmi_window_exiting(CPUState *cpu)
 {
 
     uint64_t val;
-    val = rvmcs(cpu->hvf_fd, VMCS_PRI_PROC_BASED_CTLS);
-    wvmcs(cpu->hvf_fd, VMCS_PRI_PROC_BASED_CTLS, val &
-          ~VMCS_PRI_PROC_BASED_CTLS_NMI_WINDOW_EXITING);
+    val = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_PRI_PROC_BASED_CTLS);
+    wvmcs(cpu->accel_vcpu->hvf_fd, VMCS_PRI_PROC_BASED_CTLS,
+          val & ~VMCS_PRI_PROC_BASED_CTLS_NMI_WINDOW_EXITING);
 }
 
 #endif
diff --git a/target/i386/hvf/hvf.c b/target/i386/hvf/hvf.c
index 342659f1e15..022975d093e 100644
--- a/target/i386/hvf/hvf.c
+++ b/target/i386/hvf/hvf.c
@@ -245,19 +245,19 @@ void vmx_update_tpr(CPUState *cpu)
     int tpr = cpu_get_apic_tpr(x86_cpu->apic_state) << 4;
     int irr = apic_get_highest_priority_irr(x86_cpu->apic_state);
 
-    wreg(cpu->hvf_fd, HV_X86_TPR, tpr);
+    wreg(cpu->accel_vcpu->hvf_fd, HV_X86_TPR, tpr);
     if (irr == -1) {
-        wvmcs(cpu->hvf_fd, VMCS_TPR_THRESHOLD, 0);
+        wvmcs(cpu->accel_vcpu->hvf_fd, VMCS_TPR_THRESHOLD, 0);
     } else {
-        wvmcs(cpu->hvf_fd, VMCS_TPR_THRESHOLD, (irr > tpr) ? tpr >> 4 :
-              irr >> 4);
+        wvmcs(cpu->accel_vcpu->hvf_fd, VMCS_TPR_THRESHOLD,
+              (irr > tpr) ? tpr >> 4 : irr >> 4);
     }
 }
 
 static void update_apic_tpr(CPUState *cpu)
 {
     X86CPU *x86_cpu = X86_CPU(cpu);
-    int tpr = rreg(cpu->hvf_fd, HV_X86_TPR) >> 4;
+    int tpr = rreg(cpu->accel_vcpu->hvf_fd, HV_X86_TPR) >> 4;
     cpu_set_apic_tpr(x86_cpu->apic_state, tpr);
 }
 
@@ -448,7 +448,7 @@ void hvf_vcpu_destroy(CPUState *cpu)
     X86CPU *x86_cpu = X86_CPU(cpu);
     CPUX86State *env = &x86_cpu->env;
 
-    hv_return_t ret = hv_vcpu_destroy((hv_vcpuid_t)cpu->hvf_fd);
+    hv_return_t ret = hv_vcpu_destroy(cpu->accel_vcpu->hvf_fd);
     g_free(env->hvf_mmio_buf);
     assert_hvf_ok(ret);
     g_free(cpu->accel_vcpu);
@@ -537,7 +537,7 @@ int hvf_init_vcpu(CPUState *cpu)
     r = hv_vcpu_create(&hvf_fd, HV_VCPU_DEFAULT);
     assert_hvf_ok(r);
     cpu->accel_vcpu = g_new(struct AccelvCPUState, 1);
-    cpu->hvf_fd = (int)hvf_fd
+    cpu->accel_vcpu->hvf_fd = hvf_fd
     cpu->vcpu_dirty = true;
 
     if (hv_vmx_read_capability(HV_VMX_CAP_PINBASED,
@@ -635,16 +635,17 @@ static void hvf_store_events(CPUState *cpu, uint32_t ins_len, uint64_t idtvec_in
         }
         if (idtvec_info & VMCS_IDT_VEC_ERRCODE_VALID) {
             env->has_error_code = true;
-            env->error_code = rvmcs(cpu->hvf_fd, VMCS_IDT_VECTORING_ERROR);
+            env->error_code = rvmcs(cpu->accel_vcpu->hvf_fd,
+                                    VMCS_IDT_VECTORING_ERROR);
         }
     }
-    if ((rvmcs(cpu->hvf_fd, VMCS_GUEST_INTERRUPTIBILITY) &
+    if ((rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_INTERRUPTIBILITY) &
         VMCS_INTERRUPTIBILITY_NMI_BLOCKING)) {
         env->hflags2 |= HF2_NMI_MASK;
     } else {
         env->hflags2 &= ~HF2_NMI_MASK;
     }
-    if (rvmcs(cpu->hvf_fd, VMCS_GUEST_INTERRUPTIBILITY) &
+    if (rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_INTERRUPTIBILITY) &
          (VMCS_INTERRUPTIBILITY_STI_BLOCKING |
          VMCS_INTERRUPTIBILITY_MOVSS_BLOCKING)) {
         env->hflags |= HF_INHIBIT_IRQ_MASK;
@@ -699,7 +700,7 @@ int hvf_vcpu_exec(CPUState *cpu)
 {
     X86CPU *x86_cpu = X86_CPU(cpu);
     CPUX86State *env = &x86_cpu->env;
-    hv_vcpuid_t hvf_fd = (hv_vcpuid_t)cpu_state->hvf_fd;
+    hv_vcpuid_t hvf_fd = cpu_state->accel_vcpu->hvf_fd;
     int ret = 0;
     uint64_t rip = 0;
 
diff --git a/target/i386/hvf/x86.c b/target/i386/hvf/x86.c
index cd045183a81..23fbdb91eb0 100644
--- a/target/i386/hvf/x86.c
+++ b/target/i386/hvf/x86.c
@@ -62,11 +62,11 @@ bool x86_read_segment_descriptor(struct CPUState *cpu,
     }
 
     if (GDT_SEL == sel.ti) {
-        base  = rvmcs(cpu->hvf_fd, VMCS_GUEST_GDTR_BASE);
-        limit = rvmcs(cpu->hvf_fd, VMCS_GUEST_GDTR_LIMIT);
+        base  = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_GDTR_BASE);
+        limit = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_GDTR_LIMIT);
     } else {
-        base  = rvmcs(cpu->hvf_fd, VMCS_GUEST_LDTR_BASE);
-        limit = rvmcs(cpu->hvf_fd, VMCS_GUEST_LDTR_LIMIT);
+        base  = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_LDTR_BASE);
+        limit = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_LDTR_LIMIT);
     }
 
     if (sel.index * 8 >= limit) {
@@ -85,11 +85,11 @@ bool x86_write_segment_descriptor(struct CPUState *cpu,
     uint32_t limit;
     
     if (GDT_SEL == sel.ti) {
-        base  = rvmcs(cpu->hvf_fd, VMCS_GUEST_GDTR_BASE);
-        limit = rvmcs(cpu->hvf_fd, VMCS_GUEST_GDTR_LIMIT);
+        base  = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_GDTR_BASE);
+        limit = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_GDTR_LIMIT);
     } else {
-        base  = rvmcs(cpu->hvf_fd, VMCS_GUEST_LDTR_BASE);
-        limit = rvmcs(cpu->hvf_fd, VMCS_GUEST_LDTR_LIMIT);
+        base  = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_LDTR_BASE);
+        limit = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_LDTR_LIMIT);
     }
     
     if (sel.index * 8 >= limit) {
@@ -103,8 +103,8 @@ bool x86_write_segment_descriptor(struct CPUState *cpu,
 bool x86_read_call_gate(struct CPUState *cpu, struct x86_call_gate *idt_desc,
                         int gate)
 {
-    target_ulong base  = rvmcs(cpu->hvf_fd, VMCS_GUEST_IDTR_BASE);
-    uint32_t limit = rvmcs(cpu->hvf_fd, VMCS_GUEST_IDTR_LIMIT);
+    target_ulong base  = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_IDTR_BASE);
+    uint32_t limit = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_IDTR_LIMIT);
 
     memset(idt_desc, 0, sizeof(*idt_desc));
     if (gate * 8 >= limit) {
@@ -118,7 +118,7 @@ bool x86_read_call_gate(struct CPUState *cpu, struct x86_call_gate *idt_desc,
 
 bool x86_is_protected(struct CPUState *cpu)
 {
-    uint64_t cr0 = rvmcs(cpu->hvf_fd, VMCS_GUEST_CR0);
+    uint64_t cr0 = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_CR0);
     return cr0 & CR0_PE;
 }
 
@@ -136,7 +136,7 @@ bool x86_is_v8086(struct CPUState *cpu)
 
 bool x86_is_long_mode(struct CPUState *cpu)
 {
-    return rvmcs(cpu->hvf_fd, VMCS_GUEST_IA32_EFER) & MSR_EFER_LMA;
+    return rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_IA32_EFER) & MSR_EFER_LMA;
 }
 
 bool x86_is_long64_mode(struct CPUState *cpu)
@@ -149,13 +149,13 @@ bool x86_is_long64_mode(struct CPUState *cpu)
 
 bool x86_is_paging_mode(struct CPUState *cpu)
 {
-    uint64_t cr0 = rvmcs(cpu->hvf_fd, VMCS_GUEST_CR0);
+    uint64_t cr0 = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_CR0);
     return cr0 & CR0_PG;
 }
 
 bool x86_is_pae_enabled(struct CPUState *cpu)
 {
-    uint64_t cr4 = rvmcs(cpu->hvf_fd, VMCS_GUEST_CR4);
+    uint64_t cr4 = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_CR4);
     return cr4 & CR4_PAE;
 }
 
diff --git a/target/i386/hvf/x86_descr.c b/target/i386/hvf/x86_descr.c
index 1c6220baa0d..4f716cc5942 100644
--- a/target/i386/hvf/x86_descr.c
+++ b/target/i386/hvf/x86_descr.c
@@ -48,34 +48,37 @@ static const struct vmx_segment_field {
 
 uint32_t vmx_read_segment_limit(CPUState *cpu, X86Seg seg)
 {
-    return (uint32_t)rvmcs(cpu->hvf_fd, vmx_segment_fields[seg].limit);
+    return (uint32_t)rvmcs(cpu->accel_vcpu->hvf_fd,
+                           vmx_segment_fields[seg].limit);
 }
 
 uint32_t vmx_read_segment_ar(CPUState *cpu, X86Seg seg)
 {
-    return (uint32_t)rvmcs(cpu->hvf_fd, vmx_segment_fields[seg].ar_bytes);
+    return (uint32_t)rvmcs(cpu->accel_vcpu->hvf_fd,
+                           vmx_segment_fields[seg].ar_bytes);
 }
 
 uint64_t vmx_read_segment_base(CPUState *cpu, X86Seg seg)
 {
-    return rvmcs(cpu->hvf_fd, vmx_segment_fields[seg].base);
+    return rvmcs(cpu->accel_vcpu->hvf_fd, vmx_segment_fields[seg].base);
 }
 
 x68_segment_selector vmx_read_segment_selector(CPUState *cpu, X86Seg seg)
 {
     x68_segment_selector sel;
-    sel.sel = rvmcs(cpu->hvf_fd, vmx_segment_fields[seg].selector);
+    sel.sel = rvmcs(cpu->accel_vcpu->hvf_fd, vmx_segment_fields[seg].selector);
     return sel;
 }
 
 void vmx_write_segment_selector(struct CPUState *cpu, x68_segment_selector selector, X86Seg seg)
 {
-    wvmcs(cpu->hvf_fd, vmx_segment_fields[seg].selector, selector.sel);
+    wvmcs(cpu->accel_vcpu->hvf_fd, vmx_segment_fields[seg].selector,
+          selector.sel);
 }
 
 void vmx_read_segment_descriptor(struct CPUState *cpu, struct vmx_segment *desc, X86Seg seg)
 {
-    hv_vcpuid_t hvf_fd = (hv_vcpuid_t)cpu_state->hvf_fd;
+    hv_vcpuid_t hvf_fd = cpu_state->accel_vcpu->hvf_fd;
 
     desc->sel = rvmcs(hvf_fd, vmx_segment_fields[seg].selector);
     desc->base = rvmcs(hvf_fd, vmx_segment_fields[seg].base);
@@ -86,7 +89,7 @@ void vmx_read_segment_descriptor(struct CPUState *cpu, struct vmx_segment *desc,
 void vmx_write_segment_descriptor(CPUState *cpu, struct vmx_segment *desc, X86Seg seg)
 {
     const struct vmx_segment_field *sf = &vmx_segment_fields[seg];
-    hv_vcpuid_t hvf_fd = (hv_vcpuid_t)cpu_state->hvf_fd;
+    hv_vcpuid_t hvf_fd = cpu_state->accel_vcpu->hvf_fd;
 
     wvmcs(hvf_fd, sf->base, desc->base);
     wvmcs(hvf_fd, sf->limit, desc->limit);
diff --git a/target/i386/hvf/x86_emu.c b/target/i386/hvf/x86_emu.c
index e52c39ddb1f..dd7dee6f880 100644
--- a/target/i386/hvf/x86_emu.c
+++ b/target/i386/hvf/x86_emu.c
@@ -674,7 +674,7 @@ void simulate_rdmsr(struct CPUState *cpu)
 
     switch (msr) {
     case MSR_IA32_TSC:
-        val = rdtscp() + rvmcs(cpu->hvf_fd, VMCS_TSC_OFFSET);
+        val = rdtscp() + rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_TSC_OFFSET);
         break;
     case MSR_IA32_APICBASE:
         val = cpu_get_apic_base(X86_CPU(cpu)->apic_state);
@@ -683,16 +683,16 @@ void simulate_rdmsr(struct CPUState *cpu)
         val = x86_cpu->ucode_rev;
         break;
     case MSR_EFER:
-        val = rvmcs(cpu->hvf_fd, VMCS_GUEST_IA32_EFER);
+        val = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_IA32_EFER);
         break;
     case MSR_FSBASE:
-        val = rvmcs(cpu->hvf_fd, VMCS_GUEST_FS_BASE);
+        val = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_FS_BASE);
         break;
     case MSR_GSBASE:
-        val = rvmcs(cpu->hvf_fd, VMCS_GUEST_GS_BASE);
+        val = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_GS_BASE);
         break;
     case MSR_KERNELGSBASE:
-        val = rvmcs(cpu->hvf_fd, VMCS_HOST_FS_BASE);
+        val = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_HOST_FS_BASE);
         break;
     case MSR_STAR:
         abort();
@@ -780,13 +780,13 @@ void simulate_wrmsr(struct CPUState *cpu)
         cpu_set_apic_base(X86_CPU(cpu)->apic_state, data);
         break;
     case MSR_FSBASE:
-        wvmcs(cpu->hvf_fd, VMCS_GUEST_FS_BASE, data);
+        wvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_FS_BASE, data);
         break;
     case MSR_GSBASE:
-        wvmcs(cpu->hvf_fd, VMCS_GUEST_GS_BASE, data);
+        wvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_GS_BASE, data);
         break;
     case MSR_KERNELGSBASE:
-        wvmcs(cpu->hvf_fd, VMCS_HOST_FS_BASE, data);
+        wvmcs(cpu->accel_vcpu->hvf_fd, VMCS_HOST_FS_BASE, data);
         break;
     case MSR_STAR:
         abort();
@@ -799,9 +799,9 @@ void simulate_wrmsr(struct CPUState *cpu)
         break;
     case MSR_EFER:
         /*printf("new efer %llx\n", EFER(cpu));*/
-        wvmcs(cpu->hvf_fd, VMCS_GUEST_IA32_EFER, data);
+        wvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_IA32_EFER, data);
         if (data & MSR_EFER_NXE) {
-            hv_vcpu_invalidate_tlb(cpu->hvf_fd);
+            hv_vcpu_invalidate_tlb(cpu->accel_vcpu->hvf_fd);
         }
         break;
     case MSR_MTRRphysBase(0):
@@ -1425,21 +1425,21 @@ void load_regs(struct CPUState *cpu)
     CPUX86State *env = &x86_cpu->env;
 
     int i = 0;
-    RRX(env, R_EAX) = rreg(cpu->hvf_fd, HV_X86_RAX);
-    RRX(env, R_EBX) = rreg(cpu->hvf_fd, HV_X86_RBX);
-    RRX(env, R_ECX) = rreg(cpu->hvf_fd, HV_X86_RCX);
-    RRX(env, R_EDX) = rreg(cpu->hvf_fd, HV_X86_RDX);
-    RRX(env, R_ESI) = rreg(cpu->hvf_fd, HV_X86_RSI);
-    RRX(env, R_EDI) = rreg(cpu->hvf_fd, HV_X86_RDI);
-    RRX(env, R_ESP) = rreg(cpu->hvf_fd, HV_X86_RSP);
-    RRX(env, R_EBP) = rreg(cpu->hvf_fd, HV_X86_RBP);
+    RRX(env, R_EAX) = rreg(cpu->accel_vcpu->hvf_fd, HV_X86_RAX);
+    RRX(env, R_EBX) = rreg(cpu->accel_vcpu->hvf_fd, HV_X86_RBX);
+    RRX(env, R_ECX) = rreg(cpu->accel_vcpu->hvf_fd, HV_X86_RCX);
+    RRX(env, R_EDX) = rreg(cpu->accel_vcpu->hvf_fd, HV_X86_RDX);
+    RRX(env, R_ESI) = rreg(cpu->accel_vcpu->hvf_fd, HV_X86_RSI);
+    RRX(env, R_EDI) = rreg(cpu->accel_vcpu->hvf_fd, HV_X86_RDI);
+    RRX(env, R_ESP) = rreg(cpu->accel_vcpu->hvf_fd, HV_X86_RSP);
+    RRX(env, R_EBP) = rreg(cpu->accel_vcpu->hvf_fd, HV_X86_RBP);
     for (i = 8; i < 16; i++) {
-        RRX(env, i) = rreg(cpu->hvf_fd, HV_X86_RAX + i);
+        RRX(env, i) = rreg(cpu->accel_vcpu->hvf_fd, HV_X86_RAX + i);
     }
 
-    env->eflags = rreg(cpu->hvf_fd, HV_X86_RFLAGS);
+    env->eflags = rreg(cpu->accel_vcpu->hvf_fd, HV_X86_RFLAGS);
     rflags_to_lflags(env);
-    env->eip = rreg(cpu->hvf_fd, HV_X86_RIP);
+    env->eip = rreg(cpu->accel_vcpu->hvf_fd, HV_X86_RIP);
 }
 
 void store_regs(struct CPUState *cpu)
@@ -1448,20 +1448,20 @@ void store_regs(struct CPUState *cpu)
     CPUX86State *env = &x86_cpu->env;
 
     int i = 0;
-    wreg(cpu->hvf_fd, HV_X86_RAX, RAX(env));
-    wreg(cpu->hvf_fd, HV_X86_RBX, RBX(env));
-    wreg(cpu->hvf_fd, HV_X86_RCX, RCX(env));
-    wreg(cpu->hvf_fd, HV_X86_RDX, RDX(env));
-    wreg(cpu->hvf_fd, HV_X86_RSI, RSI(env));
-    wreg(cpu->hvf_fd, HV_X86_RDI, RDI(env));
-    wreg(cpu->hvf_fd, HV_X86_RBP, RBP(env));
-    wreg(cpu->hvf_fd, HV_X86_RSP, RSP(env));
+    wreg(cpu->accel_vcpu->hvf_fd, HV_X86_RAX, RAX(env));
+    wreg(cpu->accel_vcpu->hvf_fd, HV_X86_RBX, RBX(env));
+    wreg(cpu->accel_vcpu->hvf_fd, HV_X86_RCX, RCX(env));
+    wreg(cpu->accel_vcpu->hvf_fd, HV_X86_RDX, RDX(env));
+    wreg(cpu->accel_vcpu->hvf_fd, HV_X86_RSI, RSI(env));
+    wreg(cpu->accel_vcpu->hvf_fd, HV_X86_RDI, RDI(env));
+    wreg(cpu->accel_vcpu->hvf_fd, HV_X86_RBP, RBP(env));
+    wreg(cpu->accel_vcpu->hvf_fd, HV_X86_RSP, RSP(env));
     for (i = 8; i < 16; i++) {
-        wreg(cpu->hvf_fd, HV_X86_RAX + i, RRX(env, i));
+        wreg(cpu->accel_vcpu->hvf_fd, HV_X86_RAX + i, RRX(env, i));
     }
 
     lflags_to_rflags(env);
-    wreg(cpu->hvf_fd, HV_X86_RFLAGS, env->eflags);
+    wreg(cpu->accel_vcpu->hvf_fd, HV_X86_RFLAGS, env->eflags);
     macvm_set_rip(cpu, env->eip);
 }
 
diff --git a/target/i386/hvf/x86_mmu.c b/target/i386/hvf/x86_mmu.c
index 882a6237eea..deb3608f2be 100644
--- a/target/i386/hvf/x86_mmu.c
+++ b/target/i386/hvf/x86_mmu.c
@@ -128,7 +128,7 @@ static bool test_pt_entry(struct CPUState *cpu, struct gpt_translation *pt,
         pt->err_code |= MMU_PAGE_PT;
     }
 
-    uint32_t cr0 = rvmcs(cpu->hvf_fd, VMCS_GUEST_CR0);
+    uint32_t cr0 = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_CR0);
     /* check protection */
     if (cr0 & CR0_WP) {
         if (pt->write_access && !pte_write_access(pte)) {
@@ -173,7 +173,7 @@ static bool walk_gpt(struct CPUState *cpu, target_ulong addr, int err_code,
 {
     int top_level, level;
     bool is_large = false;
-    target_ulong cr3 = rvmcs(cpu->hvf_fd, VMCS_GUEST_CR3);
+    target_ulong cr3 = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_CR3);
     uint64_t page_mask = pae ? PAE_PTE_PAGE_MASK : LEGACY_PTE_PAGE_MASK;
     
     memset(pt, 0, sizeof(*pt));
diff --git a/target/i386/hvf/x86_task.c b/target/i386/hvf/x86_task.c
index d66dfd76690..baa4c5ca87e 100644
--- a/target/i386/hvf/x86_task.c
+++ b/target/i386/hvf/x86_task.c
@@ -62,7 +62,7 @@ static void load_state_from_tss32(CPUState *cpu, struct x86_tss_segment32 *tss)
     X86CPU *x86_cpu = X86_CPU(cpu);
     CPUX86State *env = &x86_cpu->env;
 
-    wvmcs(cpu->hvf_fd, VMCS_GUEST_CR3, tss->cr3);
+    wvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_CR3, tss->cr3);
 
     env->eip = tss->eip;
     env->eflags = tss->eflags | 2;
@@ -111,11 +111,12 @@ static int task_switch_32(CPUState *cpu, x68_segment_selector tss_sel, x68_segme
 
 void vmx_handle_task_switch(CPUState *cpu, x68_segment_selector tss_sel, int reason, bool gate_valid, uint8_t gate, uint64_t gate_type)
 {
-    uint64_t rip = rreg(cpu->hvf_fd, HV_X86_RIP);
+    uint64_t rip = rreg(cpu->accel_vcpu->hvf_fd, HV_X86_RIP);
     if (!gate_valid || (gate_type != VMCS_INTR_T_HWEXCEPTION &&
                         gate_type != VMCS_INTR_T_HWINTR &&
                         gate_type != VMCS_INTR_T_NMI)) {
-        int ins_len = rvmcs(cpu->hvf_fd, VMCS_EXIT_INSTRUCTION_LENGTH);
+        int ins_len = rvmcs(cpu->accel_vcpu->hvf_fd,
+                            VMCS_EXIT_INSTRUCTION_LENGTH);
         macvm_set_rip(cpu, rip + ins_len);
         return;
     }
@@ -174,12 +175,13 @@ void vmx_handle_task_switch(CPUState *cpu, x68_segment_selector tss_sel, int rea
         //ret = task_switch_16(cpu, tss_sel, old_tss_sel, old_tss_base, &next_tss_desc);
         VM_PANIC("task_switch_16");
 
-    macvm_set_cr0(cpu->hvf_fd, rvmcs(cpu->hvf_fd, VMCS_GUEST_CR0) | CR0_TS);
+    macvm_set_cr0(cpu->accel_vcpu->hvf_fd,
+                  rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_GUEST_CR0) | CR0_TS);
     x86_segment_descriptor_to_vmx(cpu, tss_sel, &next_tss_desc, &vmx_seg);
     vmx_write_segment_descriptor(cpu, &vmx_seg, R_TR);
 
     store_regs(cpu);
 
-    hv_vcpu_invalidate_tlb(cpu->hvf_fd);
-    hv_vcpu_flush(cpu->hvf_fd);
+    hv_vcpu_invalidate_tlb(cpu->accel_vcpu->hvf_fd);
+    hv_vcpu_flush(cpu->accel_vcpu->hvf_fd);
 }
diff --git a/target/i386/hvf/x86hvf.c b/target/i386/hvf/x86hvf.c
index 2f291f2ad53..c68400b9729 100644
--- a/target/i386/hvf/x86hvf.c
+++ b/target/i386/hvf/x86hvf.c
@@ -81,7 +81,8 @@ void hvf_put_xsave(CPUState *cpu_state)
 
     x86_cpu_xsave_all_areas(X86_CPU(cpu_state), xsave);
 
-    if (hv_vcpu_write_fpstate(cpu_state->hvf_fd, (void*)xsave, 4096)) {
+    if (hv_vcpu_write_fpstate(cpu_state->accel_vcpu->hvf_fd,
+                              (void *)xsave, 4096)) {
         abort();
     }
 }
@@ -89,7 +90,7 @@ void hvf_put_xsave(CPUState *cpu_state)
 void hvf_put_segments(CPUState *cpu_state)
 {
     CPUX86State *env = &X86_CPU(cpu_state)->env;
-    hv_vcpuid_t hvf_fd = (hv_vcpuid_t)cpu_state->hvf_fd;
+    hv_vcpuid_t hvf_fd = cpu_state->accel_vcpu->hvf_fd;
     struct vmx_segment seg;
     
     wvmcs(hvf_fd, VMCS_GUEST_IDTR_LIMIT, env->idt.limit);
@@ -136,7 +137,7 @@ void hvf_put_segments(CPUState *cpu_state)
 void hvf_put_msrs(CPUState *cpu_state)
 {
     CPUX86State *env = &X86_CPU(cpu_state)->env;
-    hv_vcpuid_t hvf_fd = (hv_vcpuid_t)cpu_state->hvf_fd;
+    hv_vcpuid_t hvf_fd = cpu_state->accel_vcpu->hvf_fd;
 
     hv_vcpu_write_msr(hvf_fd, MSR_IA32_SYSENTER_CS, env->sysenter_cs);
     hv_vcpu_write_msr(hvf_fd, MSR_IA32_SYSENTER_ESP, env->sysenter_esp);
@@ -162,7 +163,8 @@ void hvf_get_xsave(CPUState *cpu_state)
 
     xsave = X86_CPU(cpu_state)->env.xsave_buf;
 
-    if (hv_vcpu_read_fpstate(cpu_state->hvf_fd, (void*)xsave, 4096)) {
+    if (hv_vcpu_read_fpstate(cpu_state->accel_vcpu->hvf_fd,
+                             (void *)xsave, 4096)) {
         abort();
     }
 
@@ -172,7 +174,7 @@ void hvf_get_xsave(CPUState *cpu_state)
 void hvf_get_segments(CPUState *cpu_state)
 {
     CPUX86State *env = &X86_CPU(cpu_state)->env;
-    hv_vcpuid_t hvf_fd = (hv_vcpuid_t)cpu_state->hvf_fd;
+    hv_vcpuid_t hvf_fd = cpu_state->accel_vcpu->hvf_fd;
     struct vmx_segment seg;
 
     env->interrupt_injected = -1;
@@ -217,7 +219,7 @@ void hvf_get_segments(CPUState *cpu_state)
 void hvf_get_msrs(CPUState *cpu_state)
 {
     CPUX86State *env = &X86_CPU(cpu_state)->env;
-    hv_vcpuid_t hvf_fd = (hv_vcpuid_t)cpu_state->hvf_fd;
+    hv_vcpuid_t hvf_fd = cpu_state->accel_vcpu->hvf_fd;
     uint64_t tmp;
     
     hv_vcpu_read_msr(hvf_fd, MSR_IA32_SYSENTER_CS, &tmp);
@@ -247,7 +249,7 @@ int hvf_put_registers(CPUState *cpu_state)
 {
     X86CPU *x86cpu = X86_CPU(cpu_state);
     CPUX86State *env = &x86cpu->env;
-    hv_vcpuid_t hvf_fd = (hv_vcpuid_t)cpu_state->hvf_fd;
+    hv_vcpuid_t hvf_fd = cpu_state->accel_vcpu->hvf_fd;
 
     wreg(hvf_fd, HV_X86_RAX, env->regs[R_EAX]);
     wreg(hvf_fd, HV_X86_RBX, env->regs[R_EBX]);
@@ -292,7 +294,7 @@ int hvf_get_registers(CPUState *cpu_state)
 {
     X86CPU *x86cpu = X86_CPU(cpu_state);
     CPUX86State *env = &x86cpu->env;
-    hv_vcpuid_t hvf_fd = (hv_vcpuid_t)cpu_state->hvf_fd;
+    hv_vcpuid_t hvf_fd = cpu_state->accel_vcpu->hvf_fd;
 
     env->regs[R_EAX] = rreg(hvf_fd, HV_X86_RAX);
     env->regs[R_EBX] = rreg(hvf_fd, HV_X86_RBX);
@@ -336,24 +338,24 @@ int hvf_get_registers(CPUState *cpu_state)
 static void vmx_set_int_window_exiting(CPUState *cpu)
 {
      uint64_t val;
-     val = rvmcs(cpu->hvf_fd, VMCS_PRI_PROC_BASED_CTLS);
-     wvmcs(cpu->hvf_fd, VMCS_PRI_PROC_BASED_CTLS, val |
+     val = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_PRI_PROC_BASED_CTLS);
+     wvmcs(cpu->accel_vcpu->hvf_fd, VMCS_PRI_PROC_BASED_CTLS, val |
              VMCS_PRI_PROC_BASED_CTLS_INT_WINDOW_EXITING);
 }
 
 void vmx_clear_int_window_exiting(CPUState *cpu)
 {
      uint64_t val;
-     val = rvmcs(cpu->hvf_fd, VMCS_PRI_PROC_BASED_CTLS);
-     wvmcs(cpu->hvf_fd, VMCS_PRI_PROC_BASED_CTLS, val &
-             ~VMCS_PRI_PROC_BASED_CTLS_INT_WINDOW_EXITING);
+     val = rvmcs(cpu->accel_vcpu->hvf_fd, VMCS_PRI_PROC_BASED_CTLS);
+     wvmcs(cpu->accel_vcpu->hvf_fd, VMCS_PRI_PROC_BASED_CTLS,\
+           val & ~VMCS_PRI_PROC_BASED_CTLS_INT_WINDOW_EXITING);
 }
 
 bool hvf_inject_interrupts(CPUState *cpu_state)
 {
     X86CPU *x86cpu = X86_CPU(cpu_state);
     CPUX86State *env = &x86cpu->env;
-    hv_vcpuid_t hvf_fd = (hv_vcpuid_t)cpu_state->hvf_fd;
+    hv_vcpuid_t hvf_fd = cpu_state->accel_vcpu->hvf_fd;
 
     uint8_t vector;
     uint64_t intr_type;
@@ -437,7 +439,7 @@ int hvf_process_events(CPUState *cpu_state)
     X86CPU *cpu = X86_CPU(cpu_state);
     CPUX86State *env = &cpu->env;
 
-    env->eflags = rreg(cpu_state->hvf_fd, HV_X86_RFLAGS);
+    env->eflags = rreg(cpu_state->accel_vcpu->hvf_fd, HV_X86_RFLAGS);
 
     if (cpu_state->interrupt_request & CPU_INTERRUPT_INIT) {
         hvf_cpu_synchronize_state(cpu_state);
-- 
2.26.2

Re: [RFC PATCH 00/19] accel: Introduce AccelvCPUState opaque structure

[Paolo Bonzini]

On 03/03/21 19:22, Philippe Mathieu-Daudé wrote:
> Series is organized as:
> - preliminary trivial cleanups
> - introduce AccelvCPUState
> - move WHPX fields (build-tested)
> - move HAX fields (not tested)
> - move KVM fields (build-tested)
> - move HVF fields (not tested)

This approach prevents adding a TCG state.  Have you thought of using a 
union instead, or even a void pointer?

Paolo

Re: [RFC PATCH 00/19] accel: Introduce AccelvCPUState opaque structure

[Philippe Mathieu-Daudé]

On 3/4/21 2:56 PM, Paolo Bonzini wrote:
> On 03/03/21 19:22, Philippe Mathieu-Daudé wrote:
>> Series is organized as:
>> - preliminary trivial cleanups
>> - introduce AccelvCPUState
>> - move WHPX fields (build-tested)
>> - move HAX fields (not tested)
>> - move KVM fields (build-tested)
>> - move HVF fields (not tested)
> 
> This approach prevents adding a TCG state.  Have you thought of using a
> union instead, or even a void pointer?

Why does it prevent it? We can only have one accelerator per vCPU.

TCG state has to be declared as another AccelvCPUState implementation.

Am I missing something?

Preventing building different accelerator-specific code in the same
unit file is on purpose.

Regards,

Phil.

Re: [RFC PATCH 00/19] accel: Introduce AccelvCPUState opaque structure

[Paolo Bonzini]

On 04/03/21 15:54, Philippe Mathieu-Daudé wrote:
> On 3/4/21 2:56 PM, Paolo Bonzini wrote:
>> On 03/03/21 19:22, Philippe Mathieu-Daudé wrote:
>>> Series is organized as:
>>> - preliminary trivial cleanups
>>> - introduce AccelvCPUState
>>> - move WHPX fields (build-tested)
>>> - move HAX fields (not tested)
>>> - move KVM fields (build-tested)
>>> - move HVF fields (not tested)
>>
>> This approach prevents adding a TCG state.  Have you thought of using a
>> union instead, or even a void pointer?
> 
> Why does it prevent it? We can only have one accelerator per vCPU.

You're right, my misguided assumption was that there can only be one of 
WHPX/HAX/KVM/HVF.  This is true for WHPX/KVM/HVF but HAX can live with 
any of the others.

However this means that AccelvCPUState would have multiple definitions. 
  Did you check that gdb copes well with it?  It's also forbidden by 
C++[1], so another thing to check would be LTO when using the C++ 
compiler for linking.

Paolo

[1] https://en.wikipedia.org/wiki/One_Definition_Rule

> TCG state has to be declared as another AccelvCPUState implementation.
> 
> Am I missing something?
> 
> Preventing building different accelerator-specific code in the same
> unit file is on purpose.
> 
> Regards,
> 
> Phil.
> 

Re: [RFC PATCH 00/19] accel: Introduce AccelvCPUState opaque structure

[Philippe Mathieu-Daudé]

On 3/4/21 4:40 PM, Paolo Bonzini wrote:
> On 04/03/21 15:54, Philippe Mathieu-Daudé wrote:
>> On 3/4/21 2:56 PM, Paolo Bonzini wrote:
>>> On 03/03/21 19:22, Philippe Mathieu-Daudé wrote:
>>>> Series is organized as:
>>>> - preliminary trivial cleanups
>>>> - introduce AccelvCPUState
>>>> - move WHPX fields (build-tested)
>>>> - move HAX fields (not tested)
>>>> - move KVM fields (build-tested)
>>>> - move HVF fields (not tested)
>>>
>>> This approach prevents adding a TCG state.  Have you thought of using a
>>> union instead, or even a void pointer?
>>
>> Why does it prevent it? We can only have one accelerator per vCPU.
> 
> You're right, my misguided assumption was that there can only be one of
> WHPX/HAX/KVM/HVF.  This is true for WHPX/KVM/HVF but HAX can live with
> any of the others.

I suppose you aren't talking about build-time but runtime. There should
be no distinction related to accelerator at runtime. We should be able
to have multiple accelerators at runtime, and eventually be able to
migrate vCPU from one accelerator to another, if it is proven useful.

How accelerators are orchestrated is obviously out of the scope of this
series.

> However this means that AccelvCPUState would have multiple definitions.

Yes.

>  Did you check that gdb copes well with it?

No, I haven't, because we already use opaque pointers elsewhere.

> It's also forbidden by
> C++[1], so another thing to check would be LTO when using the C++
> compiler for linking.

OK, I have no clue about C++ (and tries to keep QEMU way from it)
or about LTO. So I'd need to investigate that.

> 
> Paolo
> 
> [1] https://en.wikipedia.org/wiki/One_Definition_Rule
> 
>> TCG state has to be declared as another AccelvCPUState implementation.
>>
>> Am I missing something?
>>
>> Preventing building different accelerator-specific code in the same
>> unit file is on purpose.
>>
>> Regards,
>>
>> Phil.
>>
>