[PATCH 00/24] exec: Rework around CPUState user fields (part 2)

[PATCH 00/24] exec: Rework around CPUState user fields (part 2)

[Philippe Mathieu-Daudé]

Finish extracting TCG fields from CPUState:
- Extract tcg_cpu_exit() from cpu_exit()
- Introduce AccelOpsClass::exit_vcpu_thread()
- cpu_exit() calls exit_vcpu_thread=tcg_cpu_exit for TCG
- Forward declare TaskState and more uses of get_task_state()
- Introduce TCG AccelCPUState
- Move TCG specific fields from CPUState to AccelCPUState
- Restrict "exec/tlb-common.h" to TCG
- Restrict iommu_notifiers, icount to system emulation

Based-on: <20240428214915.10339-1-philmd@linaro.org>

Philippe Mathieu-Daudé (24):
  exec/user: Move 'thunk.h' from 'exec/user' to 'user'
  coverity: Update user emulation regexp
  accel/tcg: Move user definition of cpu_interrupt() to user-exec.c
  accel/tcg: Duplicate cpu_exit() for user / system
  accel/tcg: Extract tcg_cpu_exit() from cpu_exit()
  accel: Introduce AccelOpsClass::exit_vcpu_thread() handler
  accel/tcg: Implement AccelOpsClass::exit_vcpu_thread() handler
  user: Forward declare TaskState type definition
  user: Declare get_task_state() once in 'accel/tcg/vcpu-state.h'
  user: Use get_task_state() helper
  accel/tcg: Allocate per-vCPU accel state in create_vcpu_thread()
  accel/tcg: Move TaskState from CPUState to TCG AccelCPUState
  accel/tcg: Update CPUNegativeOffsetState::can_do_io field
    documentation
  accel/tcg: Move plugin fields to CPUNegativeOffsetState
  accel/tcg: Restrict IcountDecr and CPUTLB to TCG
  accel/tcg: Move @jmp_env from CPUState to TCG AccelCPUState
  accel/tcg: Move @mem_io_pc from CPUState to TCG AccelCPUState
  accel/tcg: Move @cflags_next_tb from CPUState to TCG AccelCPUState
  accel/tcg: Move @iommu_notifiers from CPUState to TCG AccelCPUState
  accel/tcg: Move @tb_jmp_cache from CPUState to TCG AccelCPUState
  accel/tcg: Remove NULL check in tcg_flush_jmp_cache()
  accel/tcg: Move @tcg_cflags from CPUState to TCG AccelCPUState
  accel/tcg: Restrict icount to system emulation
  accel/tcg: Move icount fields from CPUState to TCG AccelCPUState

 accel/tcg/internal-common.h         |  3 ++
 accel/tcg/tb-jmp-cache.h            |  4 +-
 accel/tcg/tcg-accel-ops.h           |  1 +
 accel/tcg/vcpu-state.h              | 45 +++++++++++++++++++++++
 bsd-user/qemu.h                     | 12 ++----
 include/exec/tlb-common.h           |  4 ++
 include/hw/core/cpu.h               | 57 +++++++++++------------------
 include/qemu/plugin.h               |  2 +-
 include/qemu/typedefs.h             |  2 +-
 include/sysemu/accel-ops.h          |  1 +
 include/{exec => }/user/thunk.h     |  8 +++-
 linux-user/qemu.h                   | 10 ++---
 linux-user/user-internals.h         |  2 +-
 accel/tcg/cpu-exec-common.c         |  2 +-
 accel/tcg/cpu-exec.c                | 51 ++++++++++++++------------
 accel/tcg/cputlb.c                  |  4 +-
 accel/tcg/icount-common.c           |  7 ++--
 accel/tcg/plugin-gen.c              |  8 ++--
 accel/tcg/tb-maint.c                |  6 +--
 accel/tcg/tcg-accel-ops-icount.c    | 14 +++----
 accel/tcg/tcg-accel-ops-mttcg.c     |  2 +
 accel/tcg/tcg-accel-ops-rr.c        |  2 +
 accel/tcg/tcg-accel-ops.c           |  5 +++
 accel/tcg/translate-all.c           | 25 +++++--------
 accel/tcg/user-exec.c               | 14 +++++++
 accel/tcg/watchpoint.c              |  5 ++-
 bsd-user/main.c                     |  2 +-
 gdbstub/gdbstub.c                   |  3 +-
 gdbstub/user-target.c               |  4 +-
 hw/core/cpu-common.c                | 13 +------
 hw/misc/mips_itu.c                  |  3 +-
 linux-user/main.c                   |  4 +-
 linux-user/syscall.c                |  6 +--
 linux-user/thunk.c                  |  2 +-
 plugins/core.c                      |  8 ++--
 system/cpus.c                       |  8 ++++
 system/physmem.c                    | 37 ++++++++++++++-----
 target/i386/helper.c                |  3 +-
 scripts/coverity-scan/COMPONENTS.md |  2 +-
 39 files changed, 235 insertions(+), 156 deletions(-)
 create mode 100644 accel/tcg/vcpu-state.h
 rename include/{exec => }/user/thunk.h (97%)

-- 
2.41.0

[PATCH 01/24] exec/user: Move 'thunk.h' from 'exec/user' to 'user'

[Philippe Mathieu-Daudé]

Keep all user emulation headers under the same user/ directory.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 bsd-user/qemu.h                 | 2 +-
 include/{exec => }/user/thunk.h | 8 ++++++--
 linux-user/user-internals.h     | 2 +-
 linux-user/thunk.c              | 2 +-
 4 files changed, 9 insertions(+), 5 deletions(-)
 rename include/{exec => }/user/thunk.h (97%)

diff --git a/bsd-user/qemu.h b/bsd-user/qemu.h
index 63ee07d534..e562adde65 100644
--- a/bsd-user/qemu.h
+++ b/bsd-user/qemu.h
@@ -26,7 +26,7 @@
 
 extern char **environ;
 
-#include "exec/user/thunk.h"
+#include "user/thunk.h"
 #include "target_arch.h"
 #include "syscall_defs.h"
 #include "target_syscall.h"
diff --git a/include/exec/user/thunk.h b/include/user/thunk.h
similarity index 97%
rename from include/exec/user/thunk.h
rename to include/user/thunk.h
index 2ebfecf58e..b97def8d53 100644
--- a/include/exec/user/thunk.h
+++ b/include/user/thunk.h
@@ -17,8 +17,12 @@
  * License along with this library; if not, see <http://www.gnu.org/licenses/>.
  */
 
-#ifndef THUNK_H
-#define THUNK_H
+#ifndef USER_THUNK_H
+#define USER_THUNK_H
+
+#ifndef CONFIG_USER_ONLY
+#error Cannot include this header from system emulation
+#endif
 
 #include "cpu.h"
 #include "exec/user/abitypes.h"
diff --git a/linux-user/user-internals.h b/linux-user/user-internals.h
index ce11d9e21c..5c7f173ceb 100644
--- a/linux-user/user-internals.h
+++ b/linux-user/user-internals.h
@@ -18,7 +18,7 @@
 #ifndef LINUX_USER_USER_INTERNALS_H
 #define LINUX_USER_USER_INTERNALS_H
 
-#include "exec/user/thunk.h"
+#include "user/thunk.h"
 #include "exec/exec-all.h"
 #include "exec/tb-flush.h"
 #include "qemu/log.h"
diff --git a/linux-user/thunk.c b/linux-user/thunk.c
index 071aad4b5f..3cd19e79c6 100644
--- a/linux-user/thunk.c
+++ b/linux-user/thunk.c
@@ -20,7 +20,7 @@
 #include "qemu/log.h"
 
 #include "qemu.h"
-#include "exec/user/thunk.h"
+#include "user/thunk.h"
 
 //#define DEBUG
 
-- 
2.41.0

[PATCH 02/24] coverity: Update user emulation regexp

[Philippe Mathieu-Daudé]

All user emulation headers are now under include/user/.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 scripts/coverity-scan/COMPONENTS.md | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/scripts/coverity-scan/COMPONENTS.md b/scripts/coverity-scan/COMPONENTS.md
index 91be8d1c36..1537e49cd5 100644
--- a/scripts/coverity-scan/COMPONENTS.md
+++ b/scripts/coverity-scan/COMPONENTS.md
@@ -121,7 +121,7 @@ usb
   ~ (/qemu)?(/hw/usb/.*|/include/hw/usb/.*)
 
 user
-  ~ (/qemu)?(/linux-user/.*|/bsd-user/.*|/user-exec\.c|/thunk\.c|/include/exec/user/.*)
+  ~ (/qemu)?(/linux-user/.*|/bsd-user/.*|/user-exec\.c|/thunk\.c|/include/user/.*)
 
 util
   ~ (/qemu)?(/util/.*|/include/qemu/.*)
-- 
2.41.0

[PATCH 03/24] accel/tcg: Move user definition of cpu_interrupt() to user-exec.c

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/tcg/translate-all.c | 9 ---------
 accel/tcg/user-exec.c     | 8 ++++++++
 2 files changed, 8 insertions(+), 9 deletions(-)

diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index 83cc14fbde..fdf6d8ac19 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -644,15 +644,6 @@ void cpu_io_recompile(CPUState *cpu, uintptr_t retaddr)
     cpu_loop_exit_noexc(cpu);
 }
 
-#else /* CONFIG_USER_ONLY */
-
-void cpu_interrupt(CPUState *cpu, int mask)
-{
-    g_assert(bql_locked());
-    cpu->interrupt_request |= mask;
-    qatomic_set(&cpu->neg.icount_decr.u16.high, -1);
-}
-
 #endif /* CONFIG_USER_ONLY */
 
 /*
diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c
index ca27746fe4..f49435df9d 100644
--- a/accel/tcg/user-exec.c
+++ b/accel/tcg/user-exec.c
@@ -24,6 +24,7 @@
 #include "qemu/bitops.h"
 #include "qemu/rcu.h"
 #include "exec/cpu_ldst.h"
+#include "qemu/main-loop.h"
 #include "exec/translate-all.h"
 #include "exec/page-protection.h"
 #include "exec/helper-proto.h"
@@ -38,6 +39,13 @@ __thread uintptr_t helper_retaddr;
 
 //#define DEBUG_SIGNAL
 
+void cpu_interrupt(CPUState *cpu, int mask)
+{
+    g_assert(bql_locked());
+    cpu->interrupt_request |= mask;
+    qatomic_set(&cpu->neg.icount_decr.u16.high, -1);
+}
+
 /*
  * Adjust the pc to pass to cpu_restore_state; return the memop type.
  */
-- 
2.41.0

[PATCH 04/24] accel/tcg: Duplicate cpu_exit() for user / system

[Philippe Mathieu-Daudé]

We will introduce a per-accelerator vCPU exit() handler
for system emulation. Duplicate cpu_exit() because the
handler won't be accessible from user emulation.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/tcg/user-exec.c | 8 ++++++++
 hw/core/cpu-common.c  | 8 --------
 system/cpus.c         | 8 ++++++++
 3 files changed, 16 insertions(+), 8 deletions(-)

diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c
index f49435df9d..404d8a145c 100644
--- a/accel/tcg/user-exec.c
+++ b/accel/tcg/user-exec.c
@@ -46,6 +46,14 @@ void cpu_interrupt(CPUState *cpu, int mask)
     qatomic_set(&cpu->neg.icount_decr.u16.high, -1);
 }
 
+void cpu_exit(CPUState *cpu)
+{
+    qatomic_set(&cpu->exit_request, 1);
+    /* Ensure cpu_exec will see the exit request after TCG has exited.  */
+    smp_wmb();
+    qatomic_set(&cpu->neg.icount_decr.u16.high, -1);
+}
+
 /*
  * Adjust the pc to pass to cpu_restore_state; return the memop type.
  */
diff --git a/hw/core/cpu-common.c b/hw/core/cpu-common.c
index 9b3efba82f..cbafc79033 100644
--- a/hw/core/cpu-common.c
+++ b/hw/core/cpu-common.c
@@ -78,14 +78,6 @@ void cpu_reset_interrupt(CPUState *cpu, int mask)
     }
 }
 
-void cpu_exit(CPUState *cpu)
-{
-    qatomic_set(&cpu->exit_request, 1);
-    /* Ensure cpu_exec will see the exit request after TCG has exited.  */
-    smp_wmb();
-    qatomic_set(&cpu->neg.icount_decr.u16.high, -1);
-}
-
 static int cpu_common_gdb_read_register(CPUState *cpu, GByteArray *buf, int reg)
 {
     return 0;
diff --git a/system/cpus.c b/system/cpus.c
index 68d161d96b..e0530ad603 100644
--- a/system/cpus.c
+++ b/system/cpus.c
@@ -419,6 +419,14 @@ void run_on_cpu(CPUState *cpu, run_on_cpu_func func, run_on_cpu_data data)
     do_run_on_cpu(cpu, func, data, &bql);
 }
 
+void cpu_exit(CPUState *cpu)
+{
+    qatomic_set(&cpu->exit_request, 1);
+    /* Ensure cpu_exec will see the exit request after TCG has exited.  */
+    smp_wmb();
+    qatomic_set(&cpu->neg.icount_decr.u16.high, -1);
+}
+
 static void qemu_cpu_stop(CPUState *cpu, bool exit)
 {
     g_assert(qemu_cpu_is_self(cpu));
-- 
2.41.0

[PATCH 05/24] accel/tcg: Extract tcg_cpu_exit() from cpu_exit()

[Philippe Mathieu-Daudé]

Extract tcg_cpu_exit() from the user cpu_exit() implementation.
In few commits we will re-use it in the system one (via a
per-accel handler).

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/tcg/internal-common.h | 2 ++
 accel/tcg/translate-all.c   | 7 +++++++
 accel/tcg/user-exec.c       | 4 +---
 3 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/accel/tcg/internal-common.h b/accel/tcg/internal-common.h
index ead53cb8a5..df317e7496 100644
--- a/accel/tcg/internal-common.h
+++ b/accel/tcg/internal-common.h
@@ -24,4 +24,6 @@ static inline bool cpu_in_serial_context(CPUState *cs)
     return !tcg_cflags_has(cs, CF_PARALLEL) || cpu_in_exclusive_context(cs);
 }
 
+void tcg_cpu_exit(CPUState *cpu);
+
 #endif
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index fdf6d8ac19..b67adce20e 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -663,3 +663,10 @@ void tcg_flush_jmp_cache(CPUState *cpu)
         qatomic_set(&jc->array[i].tb, NULL);
     }
 }
+
+void tcg_cpu_exit(CPUState *cpu)
+{
+    /* Ensure cpu_exec will see the exit request after TCG has exited.  */
+    smp_wmb();
+    qatomic_set(&cpu->neg.icount_decr.u16.high, -1);
+}
diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c
index 404d8a145c..d7e5f00c97 100644
--- a/accel/tcg/user-exec.c
+++ b/accel/tcg/user-exec.c
@@ -49,9 +49,7 @@ void cpu_interrupt(CPUState *cpu, int mask)
 void cpu_exit(CPUState *cpu)
 {
     qatomic_set(&cpu->exit_request, 1);
-    /* Ensure cpu_exec will see the exit request after TCG has exited.  */
-    smp_wmb();
-    qatomic_set(&cpu->neg.icount_decr.u16.high, -1);
+    tcg_cpu_exit(cpu);
 }
 
 /*
-- 
2.41.0

[PATCH 06/24] accel: Introduce AccelOpsClass::exit_vcpu_thread() handler

[Philippe Mathieu-Daudé]

Introduce a per-accelerator handler that can be call
when a vCPU exits.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/sysemu/accel-ops.h | 1 +
 system/cpus.c              | 3 +++
 2 files changed, 4 insertions(+)

diff --git a/include/sysemu/accel-ops.h b/include/sysemu/accel-ops.h
index ef91fc28bb..35dfb7e266 100644
--- a/include/sysemu/accel-ops.h
+++ b/include/sysemu/accel-ops.h
@@ -35,6 +35,7 @@ struct AccelOpsClass {
     void (*create_vcpu_thread)(CPUState *cpu); /* MANDATORY NON-NULL */
     void (*kick_vcpu_thread)(CPUState *cpu);
     bool (*cpu_thread_is_idle)(CPUState *cpu);
+    void (*exit_vcpu_thread)(CPUState *cpu);
 
     void (*synchronize_post_reset)(CPUState *cpu);
     void (*synchronize_post_init)(CPUState *cpu);
diff --git a/system/cpus.c b/system/cpus.c
index e0530ad603..d0d585e8b9 100644
--- a/system/cpus.c
+++ b/system/cpus.c
@@ -425,6 +425,9 @@ void cpu_exit(CPUState *cpu)
     /* Ensure cpu_exec will see the exit request after TCG has exited.  */
     smp_wmb();
     qatomic_set(&cpu->neg.icount_decr.u16.high, -1);
+    if (cpus_accel->exit_vcpu_thread) {
+        cpus_accel->exit_vcpu_thread(cpu);
+    }
 }
 
 static void qemu_cpu_stop(CPUState *cpu, bool exit)
-- 
2.41.0

[PATCH 07/24] accel/tcg: Implement AccelOpsClass::exit_vcpu_thread() handler

[Philippe Mathieu-Daudé]

The TCG-specific code from the system generic cpu_exit()
is equivalent of tcg_cpu_exit(). Define the AccelOpsClass
exit_vcpu_thread() handler to it, making cpu_exit() generic.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/tcg/tcg-accel-ops.c | 2 ++
 system/cpus.c             | 3 ---
 2 files changed, 2 insertions(+), 3 deletions(-)

diff --git a/accel/tcg/tcg-accel-ops.c b/accel/tcg/tcg-accel-ops.c
index 7ac5f0c974..56bbad9fcd 100644
--- a/accel/tcg/tcg-accel-ops.c
+++ b/accel/tcg/tcg-accel-ops.c
@@ -43,6 +43,7 @@
 #include "tcg-accel-ops-mttcg.h"
 #include "tcg-accel-ops-rr.h"
 #include "tcg-accel-ops-icount.h"
+#include "internal-common.h"
 
 /* common functionality among all TCG variants */
 
@@ -218,6 +219,7 @@ static void tcg_accel_ops_init(AccelOpsClass *ops)
         }
     }
 
+    ops->exit_vcpu_thread = tcg_cpu_exit;
     ops->cpu_reset_hold = tcg_cpu_reset_hold;
     ops->supports_guest_debug = tcg_supports_guest_debug;
     ops->insert_breakpoint = tcg_insert_breakpoint;
diff --git a/system/cpus.c b/system/cpus.c
index d0d585e8b9..3e86b488d3 100644
--- a/system/cpus.c
+++ b/system/cpus.c
@@ -422,9 +422,6 @@ void run_on_cpu(CPUState *cpu, run_on_cpu_func func, run_on_cpu_data data)
 void cpu_exit(CPUState *cpu)
 {
     qatomic_set(&cpu->exit_request, 1);
-    /* Ensure cpu_exec will see the exit request after TCG has exited.  */
-    smp_wmb();
-    qatomic_set(&cpu->neg.icount_decr.u16.high, -1);
     if (cpus_accel->exit_vcpu_thread) {
         cpus_accel->exit_vcpu_thread(cpu);
     }
-- 
2.41.0

[PATCH 08/24] user: Forward declare TaskState type definition

[Philippe Mathieu-Daudé]

Forward declare TaskState in "qemu/typedefs.h" so we can
use it in generic headers like "hw/cpu/core.h".

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 bsd-user/qemu.h         | 4 ++--
 include/qemu/typedefs.h | 1 +
 linux-user/qemu.h       | 4 ++--
 3 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/bsd-user/qemu.h b/bsd-user/qemu.h
index e562adde65..be57374b41 100644
--- a/bsd-user/qemu.h
+++ b/bsd-user/qemu.h
@@ -76,7 +76,7 @@ struct emulated_sigtable {
 /*
  * NOTE: we force a big alignment so that the stack stored after is aligned too
  */
-typedef struct TaskState {
+struct TaskState {
     pid_t ts_tid;     /* tid (or pid) of this task */
 
     struct TaskState *next;
@@ -114,7 +114,7 @@ typedef struct TaskState {
 
     /* This thread's sigaltstack, if it has one */
     struct target_sigaltstack sigaltstack_used;
-} __attribute__((aligned(16))) TaskState;
+} __attribute__((aligned(16)));
 
 static inline TaskState *get_task_state(CPUState *cs)
 {
diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
index 50c277cf0b..36f2825725 100644
--- a/include/qemu/typedefs.h
+++ b/include/qemu/typedefs.h
@@ -134,6 +134,7 @@ typedef struct SHPCDevice SHPCDevice;
 typedef struct SSIBus SSIBus;
 typedef struct TCGCPUOps TCGCPUOps;
 typedef struct TCGHelperInfo TCGHelperInfo;
+typedef struct TaskState TaskState;
 typedef struct TranslationBlock TranslationBlock;
 typedef struct VirtIODevice VirtIODevice;
 typedef struct Visitor Visitor;
diff --git a/linux-user/qemu.h b/linux-user/qemu.h
index 4777856b52..16d9f6ae8c 100644
--- a/linux-user/qemu.h
+++ b/linux-user/qemu.h
@@ -95,7 +95,7 @@ struct emulated_sigtable {
     target_siginfo_t info;
 };
 
-typedef struct TaskState {
+struct TaskState {
     pid_t ts_tid;     /* tid (or pid) of this task */
 #ifdef TARGET_ARM
 # ifdef TARGET_ABI32
@@ -158,7 +158,7 @@ typedef struct TaskState {
 
     /* Start time of task after system boot in clock ticks */
     uint64_t start_boottime;
-} TaskState;
+};
 
 static inline TaskState *get_task_state(CPUState *cs)
 {
-- 
2.41.0

[PATCH 09/24] user: Declare get_task_state() once in 'accel/tcg/vcpu-state.h'

[Philippe Mathieu-Daudé]

While each user emulation implentation defines its own
TaskState structure, both use the same get_task_state()
declaration, in particular in common code (such gdbstub).
Declare the method once in "accel/tcg/vcpu-state.h".

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/tcg/vcpu-state.h | 18 ++++++++++++++++++
 bsd-user/qemu.h        |  6 +-----
 linux-user/qemu.h      |  6 +-----
 3 files changed, 20 insertions(+), 10 deletions(-)
 create mode 100644 accel/tcg/vcpu-state.h

diff --git a/accel/tcg/vcpu-state.h b/accel/tcg/vcpu-state.h
new file mode 100644
index 0000000000..e407d914df
--- /dev/null
+++ b/accel/tcg/vcpu-state.h
@@ -0,0 +1,18 @@
+/*
+ * SPDX-FileContributor: Philippe Mathieu-Daudé <philmd@linaro.org>
+ * SPDX-FileCopyrightText: 2023 Linaro Ltd.
+ * SPDX-License-Identifier: GPL-2.0-or-later
+ */
+#ifndef ACCEL_TCG_VCPU_STATE_H
+#define ACCEL_TCG_VCPU_STATE_H
+
+#include "hw/core/cpu.h"
+
+#ifdef CONFIG_USER_ONLY
+static inline TaskState *get_task_state(const CPUState *cs)
+{
+    return cs->opaque;
+}
+#endif
+
+#endif
diff --git a/bsd-user/qemu.h b/bsd-user/qemu.h
index be57374b41..65fe95fed1 100644
--- a/bsd-user/qemu.h
+++ b/bsd-user/qemu.h
@@ -36,6 +36,7 @@ extern char **environ;
 #include "exec/gdbstub.h"
 #include "exec/page-protection.h"
 #include "qemu/clang-tsa.h"
+#include "accel/tcg/vcpu-state.h"
 
 #include "qemu-os.h"
 /*
@@ -116,11 +117,6 @@ struct TaskState {
     struct target_sigaltstack sigaltstack_used;
 } __attribute__((aligned(16)));
 
-static inline TaskState *get_task_state(CPUState *cs)
-{
-    return cs->opaque;
-}
-
 void stop_all_tasks(void);
 extern const char *interp_prefix;
 extern const char *qemu_uname_release;
diff --git a/linux-user/qemu.h b/linux-user/qemu.h
index 16d9f6ae8c..515af82d8b 100644
--- a/linux-user/qemu.h
+++ b/linux-user/qemu.h
@@ -8,6 +8,7 @@
 
 #include "syscall_defs.h"
 #include "target_syscall.h"
+#include "accel/tcg/vcpu-state.h"
 
 /*
  * This is the size of the host kernel's sigset_t, needed where we make
@@ -160,11 +161,6 @@ struct TaskState {
     uint64_t start_boottime;
 };
 
-static inline TaskState *get_task_state(CPUState *cs)
-{
-    return cs->opaque;
-}
-
 abi_long do_brk(abi_ulong new_brk);
 int do_guest_openat(CPUArchState *cpu_env, int dirfd, const char *pathname,
                     int flags, mode_t mode, bool safe);
-- 
2.41.0

[PATCH 10/24] user: Use get_task_state() helper

[Philippe Mathieu-Daudé]

Get the TaskState pointer calling get_task_state().

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 gdbstub/gdbstub.c     | 3 ++-
 gdbstub/user-target.c | 4 ++--
 linux-user/syscall.c  | 4 ++--
 3 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/gdbstub/gdbstub.c b/gdbstub/gdbstub.c
index 9c2b8b5d0a..b3574997ea 100644
--- a/gdbstub/gdbstub.c
+++ b/gdbstub/gdbstub.c
@@ -32,6 +32,7 @@
 #include "exec/gdbstub.h"
 #include "gdbstub/syscalls.h"
 #ifdef CONFIG_USER_ONLY
+#include "accel/tcg/vcpu-state.h"
 #include "gdbstub/user.h"
 #else
 #include "hw/cpu/cluster.h"
@@ -1661,7 +1662,7 @@ static void handle_query_supported(GArray *params, void *user_ctx)
 
 #if defined(CONFIG_USER_ONLY)
 #if defined(CONFIG_LINUX)
-    if (gdbserver_state.c_cpu->opaque) {
+    if (get_task_state(gdbserver_state.c_cpu)) {
         g_string_append(gdbserver_state.str_buf, ";qXfer:auxv:read+");
     }
     g_string_append(gdbserver_state.str_buf, ";QCatchSyscalls+");
diff --git a/gdbstub/user-target.c b/gdbstub/user-target.c
index 6646684a4c..a9c6c64512 100644
--- a/gdbstub/user-target.c
+++ b/gdbstub/user-target.c
@@ -216,7 +216,7 @@ void gdb_handle_query_offsets(GArray *params, void *user_ctx)
 {
     TaskState *ts;
 
-    ts = gdbserver_state.c_cpu->opaque;
+    ts = get_task_state(gdbserver_state.c_cpu);
     g_string_printf(gdbserver_state.str_buf,
                     "Text=" TARGET_ABI_FMT_lx
                     ";Data=" TARGET_ABI_FMT_lx
@@ -252,7 +252,7 @@ void gdb_handle_query_xfer_auxv(GArray *params, void *user_ctx)
 
     offset = get_param(params, 0)->val_ul;
     len = get_param(params, 1)->val_ul;
-    ts = gdbserver_state.c_cpu->opaque;
+    ts = get_task_state(gdbserver_state.c_cpu);
     saved_auxv = ts->info->saved_auxv;
     auxv_len = ts->info->auxv_len;
 
diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index 1b42e80f9a..b9b5a387b3 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -6463,7 +6463,7 @@ static abi_long do_prctl(CPUArchState *env, abi_long option, abi_long arg2,
 
     case PR_GET_TID_ADDRESS:
         {
-            TaskState *ts = env_cpu(env)->opaque;
+            TaskState *ts = get_task_state(env_cpu(env));
             return put_user_ual(ts->child_tidptr, arg2);
         }
 
@@ -8124,7 +8124,7 @@ static int open_self_maps_2(void *opaque, target_ulong guest_start,
 static int open_self_maps_1(CPUArchState *env, int fd, bool smaps)
 {
     struct open_self_maps_data d = {
-        .ts = env_cpu(env)->opaque,
+        .ts = get_task_state(env_cpu(env)),
         .host_maps = read_self_maps(),
         .fd = fd,
         .smaps = smaps
-- 
2.41.0

[PATCH 11/24] accel/tcg: Allocate per-vCPU accel state in create_vcpu_thread()

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/tcg/vcpu-state.h          | 7 +++++++
 accel/tcg/tcg-accel-ops-mttcg.c | 2 ++
 accel/tcg/tcg-accel-ops-rr.c    | 2 ++
 3 files changed, 11 insertions(+)

diff --git a/accel/tcg/vcpu-state.h b/accel/tcg/vcpu-state.h
index e407d914df..d0dd1bbff8 100644
--- a/accel/tcg/vcpu-state.h
+++ b/accel/tcg/vcpu-state.h
@@ -8,6 +8,13 @@
 
 #include "hw/core/cpu.h"
 
+/**
+ * AccelCPUState:
+ */
+struct AccelCPUState {
+    /* Empty */
+};
+
 #ifdef CONFIG_USER_ONLY
 static inline TaskState *get_task_state(const CPUState *cs)
 {
diff --git a/accel/tcg/tcg-accel-ops-mttcg.c b/accel/tcg/tcg-accel-ops-mttcg.c
index c552b45b8e..767b321a0d 100644
--- a/accel/tcg/tcg-accel-ops-mttcg.c
+++ b/accel/tcg/tcg-accel-ops-mttcg.c
@@ -35,6 +35,7 @@
 #include "tcg/startup.h"
 #include "tcg-accel-ops.h"
 #include "tcg-accel-ops-mttcg.h"
+#include "accel/tcg/vcpu-state.h"
 
 typedef struct MttcgForceRcuNotifier {
     Notifier notifier;
@@ -135,6 +136,7 @@ void mttcg_start_vcpu_thread(CPUState *cpu)
     char thread_name[VCPU_THREAD_NAME_SIZE];
 
     g_assert(tcg_enabled());
+    cpu->accel = g_new0(AccelCPUState, 1);
     tcg_cpu_init_cflags(cpu, current_machine->smp.max_cpus > 1);
 
     cpu->thread = g_new0(QemuThread, 1);
diff --git a/accel/tcg/tcg-accel-ops-rr.c b/accel/tcg/tcg-accel-ops-rr.c
index 894e73e52c..419b94f786 100644
--- a/accel/tcg/tcg-accel-ops-rr.c
+++ b/accel/tcg/tcg-accel-ops-rr.c
@@ -36,6 +36,7 @@
 #include "tcg-accel-ops.h"
 #include "tcg-accel-ops-rr.h"
 #include "tcg-accel-ops-icount.h"
+#include "accel/tcg/vcpu-state.h"
 
 /* Kick all RR vCPUs */
 void rr_kick_vcpu_thread(CPUState *unused)
@@ -314,6 +315,7 @@ void rr_start_vcpu_thread(CPUState *cpu)
     static QemuThread *single_tcg_cpu_thread;
 
     g_assert(tcg_enabled());
+    cpu->accel = g_new0(AccelCPUState, 1);
     tcg_cpu_init_cflags(cpu, false);
 
     if (!single_tcg_cpu_thread) {
-- 
2.41.0

[PATCH 12/24] accel/tcg: Move TaskState from CPUState to TCG AccelCPUState

[Philippe Mathieu-Daudé]

TaskState is specific to TCG user emulation, move it
to AccelCPUState.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/tcg/vcpu-state.h | 6 ++++--
 include/hw/core/cpu.h  | 2 --
 bsd-user/main.c        | 2 +-
 linux-user/main.c      | 2 +-
 linux-user/syscall.c   | 2 +-
 5 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/accel/tcg/vcpu-state.h b/accel/tcg/vcpu-state.h
index d0dd1bbff8..cf8e3acef9 100644
--- a/accel/tcg/vcpu-state.h
+++ b/accel/tcg/vcpu-state.h
@@ -12,13 +12,15 @@
  * AccelCPUState:
  */
 struct AccelCPUState {
-    /* Empty */
+#ifdef CONFIG_USER_ONLY
+    TaskState *ts;
+#endif
 };
 
 #ifdef CONFIG_USER_ONLY
 static inline TaskState *get_task_state(const CPUState *cs)
 {
-    return cs->opaque;
+    return cs->accel->ts;
 }
 #endif
 
diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index 46b99a7ea5..9b99d8e8fe 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -490,8 +490,6 @@ struct CPUState {
     QTAILQ_HEAD(, CPUWatchpoint) watchpoints;
     CPUWatchpoint *watchpoint_hit;
 
-    void *opaque;
-
     /* In order to avoid passing too many arguments to the MMIO helpers,
      * we store some rarely used information in the CPU context.
      */
diff --git a/bsd-user/main.c b/bsd-user/main.c
index 29a629d877..1ce4b0b6e4 100644
--- a/bsd-user/main.c
+++ b/bsd-user/main.c
@@ -590,7 +590,7 @@ int main(int argc, char **argv)
     init_task_state(ts);
     ts->info = info;
     ts->bprm = &bprm;
-    cpu->opaque = ts;
+    cpu->accel->ts = ts;
 
     target_set_brk(info->brk);
     syscall_init();
diff --git a/linux-user/main.c b/linux-user/main.c
index 94e4c47f05..5f7f03f4b0 100644
--- a/linux-user/main.c
+++ b/linux-user/main.c
@@ -949,7 +949,7 @@ int main(int argc, char **argv, char **envp)
     /* build Task State */
     ts->info = info;
     ts->bprm = &bprm;
-    cpu->opaque = ts;
+    cpu->accel->ts = ts;
     task_settid(ts);
 
     fd_trans_init();
diff --git a/linux-user/syscall.c b/linux-user/syscall.c
index b9b5a387b3..49db3052aa 100644
--- a/linux-user/syscall.c
+++ b/linux-user/syscall.c
@@ -6594,7 +6594,7 @@ static int do_fork(CPUArchState *env, unsigned int flags, abi_ulong newsp,
         cpu_clone_regs_child(new_env, newsp, flags);
         cpu_clone_regs_parent(env, flags);
         new_cpu = env_cpu(new_env);
-        new_cpu->opaque = ts;
+        new_cpu->accel->ts = ts;
         ts->bprm = parent_ts->bprm;
         ts->info = parent_ts->info;
         ts->signal_mask = parent_ts->signal_mask;
-- 
2.41.0

[PATCH 13/24] accel/tcg: Update CPUNegativeOffsetState::can_do_io field documentation

[Philippe Mathieu-Daudé]

The @can_do_io field got moved from CPUState to
CPUNegativeOffsetState in commit 464dacf609 ("accel/tcg:
Move can_do_io to CPUNegativeOffsetState").

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/core/cpu.h | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index 9b99d8e8fe..beb37342e9 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -338,9 +338,10 @@ typedef union IcountDecr {
     } u16;
 } IcountDecr;
 
-/*
- * Elements of CPUState most efficiently accessed from CPUArchState,
- * via small negative offsets.
+/**
+ * CPUNegativeOffsetState: Elements of CPUState most efficiently accessed
+ *                         from CPUArchState, via small negative offsets.
+ * @can_do_io: True if memory-mapped IO is allowed.
  */
 typedef struct CPUNegativeOffsetState {
     CPUTLB tlb;
@@ -400,7 +401,6 @@ struct qemu_work_item;
  * @crash_occurred: Indicates the OS reported a crash (panic) for this CPU
  * @singlestep_enabled: Flags for single-stepping.
  * @icount_extra: Instructions until next timer event.
- * @neg.can_do_io: True if memory-mapped IO is allowed.
  * @cpu_ases: Pointer to array of CPUAddressSpaces (which define the
  *            AddressSpaces this CPU has)
  * @num_ases: number of CPUAddressSpaces in @cpu_ases
-- 
2.41.0

[PATCH 14/24] accel/tcg: Move plugin fields to CPUNegativeOffsetState

[Philippe Mathieu-Daudé]

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/hw/core/cpu.h  | 22 ++++++++++------------
 include/qemu/plugin.h  |  2 +-
 accel/tcg/plugin-gen.c |  8 +++++---
 hw/core/cpu-common.c   |  2 +-
 plugins/core.c         |  8 ++++----
 5 files changed, 21 insertions(+), 21 deletions(-)

diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index beb37342e9..ef8b85b6fe 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -342,9 +342,18 @@ typedef union IcountDecr {
  * CPUNegativeOffsetState: Elements of CPUState most efficiently accessed
  *                         from CPUArchState, via small negative offsets.
  * @can_do_io: True if memory-mapped IO is allowed.
+ * @plugin_mem_cbs: active plugin memory callbacks
+ * @plugin_state: per-CPU plugin state
  */
 typedef struct CPUNegativeOffsetState {
     CPUTLB tlb;
+#ifdef CONFIG_PLUGIN
+    /*
+     * The callback pointer are accessed via TCG (see gen_empty_mem_helper).
+     */
+    GArray *plugin_mem_cbs;
+    CPUPluginState *plugin_state;
+#endif
     IcountDecr icount_decr;
     bool can_do_io;
 } CPUNegativeOffsetState;
@@ -416,8 +425,6 @@ struct qemu_work_item;
  * @kvm_fd: vCPU file descriptor for KVM.
  * @work_mutex: Lock to prevent multiple access to @work_list.
  * @work_list: List of pending asynchronous work.
- * @plugin_mem_cbs: active plugin memory callbacks
- * @plugin_state: per-CPU plugin state
  * @ignore_memory_transaction_failures: Cached copy of the MachineState
  *    flag of the same name: allows the board to suppress calling of the
  *    CPU do_transaction_failed hook function.
@@ -508,15 +515,6 @@ struct CPUState {
     /* Use by accel-block: CPU is executing an ioctl() */
     QemuLockCnt in_ioctl_lock;
 
-#ifdef CONFIG_PLUGIN
-    /*
-     * The callback pointer stays in the main CPUState as it is
-     * accessed via TCG (see gen_empty_mem_helper).
-     */
-    GArray *plugin_mem_cbs;
-    CPUPluginState *plugin_state;
-#endif
-
     /* TODO Move common fields from CPUArchState here. */
     int cpu_index;
     int cluster_index;
@@ -1120,7 +1118,7 @@ void cpu_watchpoint_remove_all(CPUState *cpu, int mask);
 static inline bool cpu_plugin_mem_cbs_enabled(const CPUState *cpu)
 {
 #ifdef CONFIG_PLUGIN
-    return !!cpu->plugin_mem_cbs;
+    return !!cpu->neg.plugin_mem_cbs;
 #else
     return false;
 #endif
diff --git a/include/qemu/plugin.h b/include/qemu/plugin.h
index 41db748eda..99a32446e9 100644
--- a/include/qemu/plugin.h
+++ b/include/qemu/plugin.h
@@ -229,7 +229,7 @@ void qemu_plugin_add_dyn_cb_arr(GArray *arr);
 
 static inline void qemu_plugin_disable_mem_helpers(CPUState *cpu)
 {
-    cpu->plugin_mem_cbs = NULL;
+    cpu->neg.plugin_mem_cbs = NULL;
 }
 
 /**
diff --git a/accel/tcg/plugin-gen.c b/accel/tcg/plugin-gen.c
index cd78ef94a1..3766870108 100644
--- a/accel/tcg/plugin-gen.c
+++ b/accel/tcg/plugin-gen.c
@@ -178,7 +178,7 @@ static void gen_empty_mem_helper(void)
     TCGv_ptr ptr = tcg_temp_ebb_new_ptr();
 
     tcg_gen_movi_ptr(ptr, 0);
-    tcg_gen_st_ptr(ptr, tcg_env, offsetof(CPUState, plugin_mem_cbs) -
+    tcg_gen_st_ptr(ptr, tcg_env, offsetof(CPUState, neg.plugin_mem_cbs) -
                                  offsetof(ArchCPU, env));
     tcg_temp_free_ptr(ptr);
 }
@@ -634,7 +634,8 @@ void plugin_gen_disable_mem_helpers(void)
         return;
     }
     tcg_gen_st_ptr(tcg_constant_ptr(NULL), tcg_env,
-                   offsetof(CPUState, plugin_mem_cbs) - offsetof(ArchCPU, env));
+                   offsetof(CPUState, neg.plugin_mem_cbs) -
+                   offsetof(ArchCPU, env));
 }
 
 static void plugin_gen_tb_udata(const struct qemu_plugin_tb *ptb,
@@ -871,7 +872,8 @@ bool plugin_gen_tb_start(CPUState *cpu, const DisasContextBase *db,
 {
     bool ret = false;
 
-    if (test_bit(QEMU_PLUGIN_EV_VCPU_TB_TRANS, cpu->plugin_state->event_mask)) {
+    if (test_bit(QEMU_PLUGIN_EV_VCPU_TB_TRANS,
+                 cpu->neg.plugin_state->event_mask)) {
         struct qemu_plugin_tb *ptb = tcg_ctx->plugin_tb;
         int i;
 
diff --git a/hw/core/cpu-common.c b/hw/core/cpu-common.c
index cbafc79033..3e00ea94be 100644
--- a/hw/core/cpu-common.c
+++ b/hw/core/cpu-common.c
@@ -213,7 +213,7 @@ static void cpu_common_realizefn(DeviceState *dev, Error **errp)
     /* Plugin initialization must wait until the cpu start executing code */
 #ifdef CONFIG_PLUGIN
     if (tcg_enabled()) {
-        cpu->plugin_state = qemu_plugin_create_vcpu_state();
+        cpu->neg.plugin_state = qemu_plugin_create_vcpu_state();
         async_run_on_cpu(cpu, qemu_plugin_vcpu_init__async, RUN_ON_CPU_NULL);
     }
 #endif
diff --git a/plugins/core.c b/plugins/core.c
index 09c98382f5..1286a18f09 100644
--- a/plugins/core.c
+++ b/plugins/core.c
@@ -55,7 +55,7 @@ struct qemu_plugin_ctx *plugin_id_to_ctx_locked(qemu_plugin_id_t id)
 
 static void plugin_cpu_update__async(CPUState *cpu, run_on_cpu_data data)
 {
-    bitmap_copy(cpu->plugin_state->event_mask,
+    bitmap_copy(cpu->neg.plugin_state->event_mask,
                 &data.host_ulong, QEMU_PLUGIN_EV_MAX);
     tcg_flush_jmp_cache(cpu);
 }
@@ -396,7 +396,7 @@ qemu_plugin_vcpu_syscall(CPUState *cpu, int64_t num, uint64_t a1, uint64_t a2,
     struct qemu_plugin_cb *cb, *next;
     enum qemu_plugin_event ev = QEMU_PLUGIN_EV_VCPU_SYSCALL;
 
-    if (!test_bit(ev, cpu->plugin_state->event_mask)) {
+    if (!test_bit(ev, cpu->neg.plugin_state->event_mask)) {
         return;
     }
 
@@ -418,7 +418,7 @@ void qemu_plugin_vcpu_syscall_ret(CPUState *cpu, int64_t num, int64_t ret)
     struct qemu_plugin_cb *cb, *next;
     enum qemu_plugin_event ev = QEMU_PLUGIN_EV_VCPU_SYSCALL_RET;
 
-    if (!test_bit(ev, cpu->plugin_state->event_mask)) {
+    if (!test_bit(ev, cpu->neg.plugin_state->event_mask)) {
         return;
     }
 
@@ -496,7 +496,7 @@ void exec_inline_op(struct qemu_plugin_dyn_cb *cb, int cpu_index)
 void qemu_plugin_vcpu_mem_cb(CPUState *cpu, uint64_t vaddr,
                              MemOpIdx oi, enum qemu_plugin_mem_rw rw)
 {
-    GArray *arr = cpu->plugin_mem_cbs;
+    GArray *arr = cpu->neg.plugin_mem_cbs;
     size_t i;
 
     if (arr == NULL) {
-- 
2.41.0

[PATCH 15/24] accel/tcg: Restrict IcountDecr and CPUTLB to TCG

[Philippe Mathieu-Daudé]

IcountDecr union, the CPUTLB* structures and the
"exec/tlb-common.h" header are only required for
TCG.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 include/exec/tlb-common.h | 4 ++++
 include/hw/core/cpu.h     | 9 ++++++---
 2 files changed, 10 insertions(+), 3 deletions(-)

diff --git a/include/exec/tlb-common.h b/include/exec/tlb-common.h
index dc5a5faa0b..a529c9f056 100644
--- a/include/exec/tlb-common.h
+++ b/include/exec/tlb-common.h
@@ -19,6 +19,10 @@
 #ifndef EXEC_TLB_COMMON_H
 #define EXEC_TLB_COMMON_H 1
 
+#ifndef CONFIG_TCG
+#error Can only include this header with TCG
+#endif
+
 #define CPU_TLB_ENTRY_BITS 5
 
 /* Minimalized TLB entry for use by TCG fast path. */
diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index ef8b85b6fe..dc28920bcc 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -27,7 +27,6 @@
 #include "exec/vaddr.h"
 #include "exec/memattrs.h"
 #include "exec/mmu-access-type.h"
-#include "exec/tlb-common.h"
 #include "qapi/qapi-types-run-state.h"
 #include "qemu/bitmap.h"
 #include "qemu/rcu_queue.h"
@@ -256,6 +255,9 @@ typedef struct CPUTLBEntryFull {
     } extra;
 } CPUTLBEntryFull;
 
+#ifdef CONFIG_TCG
+#include "exec/tlb-common.h"
+
 /*
  * Data elements that are per MMU mode, minus the bits accessed by
  * the TCG fast path.
@@ -311,11 +313,9 @@ typedef struct CPUTLBCommon {
  * negative offsets are at the end of the struct.
  */
 typedef struct CPUTLB {
-#ifdef CONFIG_TCG
     CPUTLBCommon c;
     CPUTLBDesc d[NB_MMU_MODES];
     CPUTLBDescFast f[NB_MMU_MODES];
-#endif
 } CPUTLB;
 
 /*
@@ -337,6 +337,7 @@ typedef union IcountDecr {
 #endif
     } u16;
 } IcountDecr;
+#endif
 
 /**
  * CPUNegativeOffsetState: Elements of CPUState most efficiently accessed
@@ -346,6 +347,7 @@ typedef union IcountDecr {
  * @plugin_state: per-CPU plugin state
  */
 typedef struct CPUNegativeOffsetState {
+#ifdef CONFIG_TCG
     CPUTLB tlb;
 #ifdef CONFIG_PLUGIN
     /*
@@ -356,6 +358,7 @@ typedef struct CPUNegativeOffsetState {
 #endif
     IcountDecr icount_decr;
     bool can_do_io;
+#endif
 } CPUNegativeOffsetState;
 
 struct KVMState;
-- 
2.41.0

[PATCH 16/24] accel/tcg: Move @jmp_env from CPUState to TCG AccelCPUState

[Philippe Mathieu-Daudé]

@jmp_env is specific to TCG accelerator, move it to its AccelCPUState.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/tcg/internal-common.h | 1 +
 accel/tcg/tcg-accel-ops.h   | 1 +
 accel/tcg/vcpu-state.h      | 2 ++
 include/hw/core/cpu.h       | 1 -
 accel/tcg/cpu-exec-common.c | 2 +-
 accel/tcg/cpu-exec.c        | 6 +++---
 6 files changed, 8 insertions(+), 5 deletions(-)

diff --git a/accel/tcg/internal-common.h b/accel/tcg/internal-common.h
index df317e7496..e2c80bc7ff 100644
--- a/accel/tcg/internal-common.h
+++ b/accel/tcg/internal-common.h
@@ -11,6 +11,7 @@
 
 #include "exec/cpu-common.h"
 #include "exec/translation-block.h"
+#include "accel/tcg/vcpu-state.h"
 
 extern int64_t max_delay;
 extern int64_t max_advance;
diff --git a/accel/tcg/tcg-accel-ops.h b/accel/tcg/tcg-accel-ops.h
index 44c4079972..ed41a087a3 100644
--- a/accel/tcg/tcg-accel-ops.h
+++ b/accel/tcg/tcg-accel-ops.h
@@ -13,6 +13,7 @@
 #define TCG_ACCEL_OPS_H
 
 #include "sysemu/cpus.h"
+#include "accel/tcg/vcpu-state.h"
 
 void tcg_cpu_destroy(CPUState *cpu);
 int tcg_cpu_exec(CPUState *cpu);
diff --git a/accel/tcg/vcpu-state.h b/accel/tcg/vcpu-state.h
index cf8e3acef9..cb06f0412b 100644
--- a/accel/tcg/vcpu-state.h
+++ b/accel/tcg/vcpu-state.h
@@ -12,6 +12,8 @@
  * AccelCPUState:
  */
 struct AccelCPUState {
+    sigjmp_buf jmp_env;
+
 #ifdef CONFIG_USER_ONLY
     TaskState *ts;
 #endif
diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index dc28920bcc..6d3716f619 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -477,7 +477,6 @@ struct CPUState {
     int64_t icount_budget;
     int64_t icount_extra;
     uint64_t random_seed;
-    sigjmp_buf jmp_env;
 
     QemuMutex work_mutex;
     QSIMPLEQ_HEAD(, qemu_work_item) work_list;
diff --git a/accel/tcg/cpu-exec-common.c b/accel/tcg/cpu-exec-common.c
index bc9b1a260e..ec45482305 100644
--- a/accel/tcg/cpu-exec-common.c
+++ b/accel/tcg/cpu-exec-common.c
@@ -38,7 +38,7 @@ void cpu_loop_exit(CPUState *cpu)
     cpu->neg.can_do_io = true;
     /* Undo any setting in generated code.  */
     qemu_plugin_disable_mem_helpers(cpu);
-    siglongjmp(cpu->jmp_env, 1);
+    siglongjmp(cpu->accel->jmp_env, 1);
 }
 
 void cpu_loop_exit_restore(CPUState *cpu, uintptr_t pc)
diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index 9af66bc191..46ad16f911 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -553,7 +553,7 @@ static void cpu_exec_longjmp_cleanup(CPUState *cpu)
      * support such a thing.  We'd have to properly register unwind info
      * for the JIT for EH, rather that just for GDB.
      *
-     * Alternative 2: Set and restore cpu->jmp_env in tb_gen_code to
+     * Alternative 2: Set and restore cpu->accel->jmp_env in tb_gen_code to
      * capture the cpu_loop_exit longjmp, perform the cleanup, and
      * jump again to arrive here.
      */
@@ -577,7 +577,7 @@ void cpu_exec_step_atomic(CPUState *cpu)
     uint32_t flags, cflags;
     int tb_exit;
 
-    if (sigsetjmp(cpu->jmp_env, 0) == 0) {
+    if (sigsetjmp(cpu->accel->jmp_env, 0) == 0) {
         start_exclusive();
         g_assert(cpu == current_cpu);
         g_assert(!cpu->running);
@@ -1038,7 +1038,7 @@ cpu_exec_loop(CPUState *cpu, SyncClocks *sc)
 static int cpu_exec_setjmp(CPUState *cpu, SyncClocks *sc)
 {
     /* Prepare setjmp context for exception handling. */
-    if (unlikely(sigsetjmp(cpu->jmp_env, 0) != 0)) {
+    if (unlikely(sigsetjmp(cpu->accel->jmp_env, 0) != 0)) {
         cpu_exec_longjmp_cleanup(cpu);
     }
 
-- 
2.41.0

[PATCH 17/24] accel/tcg: Move @mem_io_pc from CPUState to TCG AccelCPUState

[Philippe Mathieu-Daudé]

@mem_io_pc is specific to TCG system emulation, move it to
AccelCPUState.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/tcg/vcpu-state.h    | 3 +++
 include/hw/core/cpu.h     | 2 --
 accel/tcg/cputlb.c        | 2 +-
 accel/tcg/tcg-accel-ops.c | 1 +
 hw/core/cpu-common.c      | 1 -
 hw/misc/mips_itu.c        | 3 ++-
 target/i386/helper.c      | 3 ++-
 7 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/accel/tcg/vcpu-state.h b/accel/tcg/vcpu-state.h
index cb06f0412b..d1f989c625 100644
--- a/accel/tcg/vcpu-state.h
+++ b/accel/tcg/vcpu-state.h
@@ -10,12 +10,15 @@
 
 /**
  * AccelCPUState:
+ * @mem_io_pc: Host Program Counter at which the memory was accessed.
  */
 struct AccelCPUState {
     sigjmp_buf jmp_env;
 
 #ifdef CONFIG_USER_ONLY
     TaskState *ts;
+#else
+    uintptr_t mem_io_pc;
 #endif
 };
 
diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index 6d3716f619..4df9bfeba9 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -423,7 +423,6 @@ struct qemu_work_item;
  * @gdb_num_g_regs: Number of registers in GDB 'g' packets.
  * @node: QTAILQ of CPUs sharing TB cache.
  * @opaque: User data.
- * @mem_io_pc: Host Program Counter at which the memory was accessed.
  * @accel: Pointer to accelerator specific state.
  * @kvm_fd: vCPU file descriptor for KVM.
  * @work_mutex: Lock to prevent multiple access to @work_list.
@@ -502,7 +501,6 @@ struct CPUState {
     /* In order to avoid passing too many arguments to the MMIO helpers,
      * we store some rarely used information in the CPU context.
      */
-    uintptr_t mem_io_pc;
 
     /* Only used in KVM */
     int kvm_fd;
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index cdb3e12dfb..e9d6faf78f 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -1383,7 +1383,7 @@ io_prepare(hwaddr *out_offset, CPUState *cpu, hwaddr xlat,
 
     section = iotlb_to_section(cpu, xlat, attrs);
     mr_offset = (xlat & TARGET_PAGE_MASK) + addr;
-    cpu->mem_io_pc = retaddr;
+    cpu->accel->mem_io_pc = retaddr;
     if (!cpu->neg.can_do_io) {
         cpu_io_recompile(cpu, retaddr);
     }
diff --git a/accel/tcg/tcg-accel-ops.c b/accel/tcg/tcg-accel-ops.c
index 56bbad9fcd..dfa0357558 100644
--- a/accel/tcg/tcg-accel-ops.c
+++ b/accel/tcg/tcg-accel-ops.c
@@ -89,6 +89,7 @@ static void tcg_cpu_reset_hold(CPUState *cpu)
 
     qatomic_set(&cpu->neg.icount_decr.u32, 0);
     cpu->neg.can_do_io = true;
+    cpu->accel->mem_io_pc = 0;
 }
 
 /* mask must never be zero, except for A20 change call */
diff --git a/hw/core/cpu-common.c b/hw/core/cpu-common.c
index 3e00ea94be..21151f5634 100644
--- a/hw/core/cpu-common.c
+++ b/hw/core/cpu-common.c
@@ -117,7 +117,6 @@ static void cpu_common_reset_hold(Object *obj, ResetType type)
 
     cpu->interrupt_request = 0;
     cpu->halted = cpu->start_powered_off;
-    cpu->mem_io_pc = 0;
     cpu->icount_extra = 0;
     cpu->exception_index = -1;
     cpu->crash_occurred = false;
diff --git a/hw/misc/mips_itu.c b/hw/misc/mips_itu.c
index f8acfb3ee2..aa11548e71 100644
--- a/hw/misc/mips_itu.c
+++ b/hw/misc/mips_itu.c
@@ -26,6 +26,7 @@
 #include "hw/misc/mips_itu.h"
 #include "hw/qdev-properties.h"
 #include "target/mips/cpu.h"
+#include "accel/tcg/vcpu-state.h"
 
 #define ITC_TAG_ADDRSPACE_SZ (ITC_ADDRESSMAP_NUM * 8)
 /* Initialize as 4kB area to fit all 32 cells with default 128B grain.
@@ -185,7 +186,7 @@ void block_thread_and_exit(ITCStorageCell *c)
     c->blocked_threads |= 1ULL << current_cpu->cpu_index;
     current_cpu->halted = 1;
     current_cpu->exception_index = EXCP_HLT;
-    cpu_loop_exit_restore(current_cpu, current_cpu->mem_io_pc);
+    cpu_loop_exit_restore(current_cpu, current_cpu->accel->mem_io_pc);
 }
 
 /* ITC Bypass View */
diff --git a/target/i386/helper.c b/target/i386/helper.c
index 48d1513a35..6c188ea94c 100644
--- a/target/i386/helper.c
+++ b/target/i386/helper.c
@@ -30,6 +30,7 @@
 #include "qemu/log.h"
 #ifdef CONFIG_TCG
 #include "tcg/insn-start-words.h"
+#include "accel/tcg/vcpu-state.h" // ???
 #endif
 
 void cpu_sync_avx_hflag(CPUX86State *env)
@@ -518,7 +519,7 @@ static inline target_ulong get_memio_eip(CPUX86State *env)
     uint64_t data[TARGET_INSN_START_WORDS];
     CPUState *cs = env_cpu(env);
 
-    if (!cpu_unwind_state_data(cs, cs->mem_io_pc, data)) {
+    if (!cpu_unwind_state_data(cs, cs->accel->mem_io_pc, data)) {
         return env->eip;
     }
 
-- 
2.41.0

[PATCH 18/24] accel/tcg: Move @cflags_next_tb from CPUState to TCG AccelCPUState

[Philippe Mathieu-Daudé]

@cflags_next_tb is specific to TCG accelerator, move it to
its AccelCPUState.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/tcg/vcpu-state.h    |  2 ++
 include/hw/core/cpu.h     |  1 -
 accel/tcg/cpu-exec.c      | 12 ++++++------
 accel/tcg/tb-maint.c      |  4 ++--
 accel/tcg/tcg-accel-ops.c |  1 +
 accel/tcg/translate-all.c |  2 +-
 accel/tcg/watchpoint.c    |  5 +++--
 hw/core/cpu-common.c      |  1 -
 8 files changed, 15 insertions(+), 13 deletions(-)

diff --git a/accel/tcg/vcpu-state.h b/accel/tcg/vcpu-state.h
index d1f989c625..79e1490631 100644
--- a/accel/tcg/vcpu-state.h
+++ b/accel/tcg/vcpu-state.h
@@ -13,6 +13,8 @@
  * @mem_io_pc: Host Program Counter at which the memory was accessed.
  */
 struct AccelCPUState {
+    uint32_t cflags_next_tb;
+
     sigjmp_buf jmp_env;
 
 #ifdef CONFIG_USER_ONLY
diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index 4df9bfeba9..a8b4ae25f1 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -469,7 +469,6 @@ struct CPUState {
     bool crash_occurred;
     bool exit_request;
     int exclusive_context_count;
-    uint32_t cflags_next_tb;
     /* updates protected by BQL */
     uint32_t interrupt_request;
     int singlestep_enabled;
diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index 46ad16f911..55235d3e5e 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -720,7 +720,7 @@ static inline bool cpu_handle_exception(CPUState *cpu, int *ret)
         if (replay_has_exception()
             && cpu->neg.icount_decr.u16.low + cpu->icount_extra == 0) {
             /* Execute just one insn to trigger exception pending in the log */
-            cpu->cflags_next_tb = (curr_cflags(cpu) & ~CF_USE_ICOUNT)
+            cpu->accel->cflags_next_tb = (curr_cflags(cpu) & ~CF_USE_ICOUNT)
                 | CF_NOIRQ | 1;
         }
 #endif
@@ -783,7 +783,7 @@ static inline bool icount_exit_request(CPUState *cpu)
     if (!icount_enabled()) {
         return false;
     }
-    if (cpu->cflags_next_tb != -1 && !(cpu->cflags_next_tb & CF_USE_ICOUNT)) {
+    if (!(cpu->accel->cflags_next_tb == -1 || cpu->accel->cflags_next_tb & CF_USE_ICOUNT)) {
         return false;
     }
     return cpu->neg.icount_decr.u16.low + cpu->icount_extra == 0;
@@ -797,7 +797,7 @@ static inline bool cpu_handle_interrupt(CPUState *cpu,
      * skip checking here. Any pending interrupts will get picked up
      * by the next TB we execute under normal cflags.
      */
-    if (cpu->cflags_next_tb != -1 && cpu->cflags_next_tb & CF_NOIRQ) {
+    if (cpu->accel->cflags_next_tb != -1 && cpu->accel->cflags_next_tb & CF_NOIRQ) {
         return false;
     }
 
@@ -947,7 +947,7 @@ static inline void cpu_loop_exec_tb(CPUState *cpu, TranslationBlock *tb,
     if (insns_left > 0 && insns_left < tb->icount)  {
         assert(insns_left <= CF_COUNT_MASK);
         assert(cpu->icount_extra == 0);
-        cpu->cflags_next_tb = (tb->cflags & ~CF_COUNT_MASK) | insns_left;
+        cpu->accel->cflags_next_tb = (tb->cflags & ~CF_COUNT_MASK) | insns_left;
     }
 #endif
 }
@@ -979,11 +979,11 @@ cpu_exec_loop(CPUState *cpu, SyncClocks *sc)
              * have CF_INVALID set, -1 is a convenient invalid value that
              * does not require tcg headers for cpu_common_reset.
              */
-            cflags = cpu->cflags_next_tb;
+            cflags = cpu->accel->cflags_next_tb;
             if (cflags == -1) {
                 cflags = curr_cflags(cpu);
             } else {
-                cpu->cflags_next_tb = -1;
+                cpu->accel->cflags_next_tb = -1;
             }
 
             if (check_for_breakpoints(cpu, pc, &cflags)) {
diff --git a/accel/tcg/tb-maint.c b/accel/tcg/tb-maint.c
index 19ae6793f3..2d5faca9fd 100644
--- a/accel/tcg/tb-maint.c
+++ b/accel/tcg/tb-maint.c
@@ -1084,7 +1084,7 @@ bool tb_invalidate_phys_page_unwind(tb_page_addr_t addr, uintptr_t pc)
     if (current_tb_modified) {
         /* Force execution of one insn next time.  */
         CPUState *cpu = current_cpu;
-        cpu->cflags_next_tb = 1 | CF_NOIRQ | curr_cflags(current_cpu);
+        cpu->accel->cflags_next_tb = 1 | CF_NOIRQ | curr_cflags(current_cpu);
         return true;
     }
     return false;
@@ -1154,7 +1154,7 @@ tb_invalidate_phys_page_range__locked(struct page_collection *pages,
     if (current_tb_modified) {
         page_collection_unlock(pages);
         /* Force execution of one insn next time.  */
-        current_cpu->cflags_next_tb = 1 | CF_NOIRQ | curr_cflags(current_cpu);
+        current_cpu->accel->cflags_next_tb = 1 | CF_NOIRQ | curr_cflags(current_cpu);
         mmap_unlock();
         cpu_loop_exit_noexc(current_cpu);
     }
diff --git a/accel/tcg/tcg-accel-ops.c b/accel/tcg/tcg-accel-ops.c
index dfa0357558..5429e2d219 100644
--- a/accel/tcg/tcg-accel-ops.c
+++ b/accel/tcg/tcg-accel-ops.c
@@ -90,6 +90,7 @@ static void tcg_cpu_reset_hold(CPUState *cpu)
     qatomic_set(&cpu->neg.icount_decr.u32, 0);
     cpu->neg.can_do_io = true;
     cpu->accel->mem_io_pc = 0;
+    cpu->accel->cflags_next_tb = -1;
 }
 
 /* mask must never be zero, except for A20 change call */
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index b67adce20e..3a8199a761 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -631,7 +631,7 @@ void cpu_io_recompile(CPUState *cpu, uintptr_t retaddr)
      * operations only (which execute after completion) so we don't
      * double instrument the instruction.
      */
-    cpu->cflags_next_tb = curr_cflags(cpu) | CF_MEMI_ONLY | n;
+    cpu->accel->cflags_next_tb = curr_cflags(cpu) | CF_MEMI_ONLY | n;
 
     if (qemu_loglevel_mask(CPU_LOG_EXEC)) {
         vaddr pc = cpu->cc->get_pc(cpu);
diff --git a/accel/tcg/watchpoint.c b/accel/tcg/watchpoint.c
index d3aab11458..0a40bfdc85 100644
--- a/accel/tcg/watchpoint.c
+++ b/accel/tcg/watchpoint.c
@@ -26,6 +26,7 @@
 #include "sysemu/replay.h"
 #include "hw/core/tcg-cpu-ops.h"
 #include "hw/core/cpu.h"
+#include "accel/tcg/vcpu-state.h"
 
 /*
  * Return true if this watchpoint address matches the specified
@@ -100,7 +101,7 @@ void cpu_check_watchpoint(CPUState *cpu, vaddr addr, vaddr len,
                  */
                 if (!cpu->neg.can_do_io) {
                     /* Force execution of one insn next time.  */
-                    cpu->cflags_next_tb = 1 | CF_NOIRQ | curr_cflags(cpu);
+                    cpu->accel->cflags_next_tb = 1 | CF_NOIRQ | curr_cflags(cpu);
                     cpu_loop_exit_restore(cpu, ra);
                 }
                 /*
@@ -132,7 +133,7 @@ void cpu_check_watchpoint(CPUState *cpu, vaddr addr, vaddr len,
                 cpu_loop_exit(cpu);
             } else {
                 /* Force execution of one insn next time.  */
-                cpu->cflags_next_tb = 1 | CF_NOIRQ | curr_cflags(cpu);
+                cpu->accel->cflags_next_tb = 1 | CF_NOIRQ | curr_cflags(cpu);
                 mmap_unlock();
                 cpu_loop_exit_noexc(cpu);
             }
diff --git a/hw/core/cpu-common.c b/hw/core/cpu-common.c
index 21151f5634..684422991c 100644
--- a/hw/core/cpu-common.c
+++ b/hw/core/cpu-common.c
@@ -120,7 +120,6 @@ static void cpu_common_reset_hold(Object *obj, ResetType type)
     cpu->icount_extra = 0;
     cpu->exception_index = -1;
     cpu->crash_occurred = false;
-    cpu->cflags_next_tb = -1;
 
     cpu_exec_reset_hold(cpu);
 }
-- 
2.41.0

[PATCH 19/24] accel/tcg: Move @iommu_notifiers from CPUState to TCG AccelCPUState

[Philippe Mathieu-Daudé]

@iommu_notifiers is specific to TCG system emulation, move it to
AccelCPUState.

Restrict TCG specific code in system/physmem.c, adding an empty
stub for tcg_register_iommu_notifier().

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/tcg/vcpu-state.h |  3 +++
 include/hw/core/cpu.h  |  3 ---
 system/physmem.c       | 37 ++++++++++++++++++++++++++++---------
 3 files changed, 31 insertions(+), 12 deletions(-)

diff --git a/accel/tcg/vcpu-state.h b/accel/tcg/vcpu-state.h
index 79e1490631..2d09dc3857 100644
--- a/accel/tcg/vcpu-state.h
+++ b/accel/tcg/vcpu-state.h
@@ -21,6 +21,9 @@ struct AccelCPUState {
     TaskState *ts;
 #else
     uintptr_t mem_io_pc;
+
+    /* track IOMMUs whose translations we've cached in the TCG TLB */
+    GArray *iommu_notifiers;
 #endif
 };
 
diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index a8b4ae25f1..9e192f739a 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -539,9 +539,6 @@ struct CPUState {
     /* Used for user-only emulation of prctl(PR_SET_UNALIGN). */
     bool prctl_unalign_sigbus;
 
-    /* track IOMMUs whose translations we've cached in the TCG TLB */
-    GArray *iommu_notifiers;
-
     /*
      * MUST BE LAST in order to minimize the displacement to CPUArchState.
      */
diff --git a/system/physmem.c b/system/physmem.c
index 44e477a1a5..1e003e42bb 100644
--- a/system/physmem.c
+++ b/system/physmem.c
@@ -27,6 +27,8 @@
 #include "qemu/madvise.h"
 
 #ifdef CONFIG_TCG
+#include "exec/translate-all.h"
+#include "accel/tcg/vcpu-state.h"
 #include "hw/core/tcg-cpu-ops.h"
 #endif /* CONFIG_TCG */
 
@@ -578,6 +580,8 @@ MemoryRegion *flatview_translate(FlatView *fv, hwaddr addr, hwaddr *xlat,
     return mr;
 }
 
+#ifdef CONFIG_TCG
+
 typedef struct TCGIOMMUNotifier {
     IOMMUNotifier n;
     MemoryRegion *mr;
@@ -614,17 +618,20 @@ static void tcg_register_iommu_notifier(CPUState *cpu,
     TCGIOMMUNotifier *notifier = NULL;
     int i;
 
-    for (i = 0; i < cpu->iommu_notifiers->len; i++) {
-        notifier = g_array_index(cpu->iommu_notifiers, TCGIOMMUNotifier *, i);
+    for (i = 0; i < cpu->accel->iommu_notifiers->len; i++) {
+        notifier = g_array_index(cpu->accel->iommu_notifiers,
+                                 TCGIOMMUNotifier *, i);
         if (notifier->mr == mr && notifier->iommu_idx == iommu_idx) {
             break;
         }
     }
-    if (i == cpu->iommu_notifiers->len) {
+    if (i == cpu->accel->iommu_notifiers->len) {
         /* Not found, add a new entry at the end of the array */
-        cpu->iommu_notifiers = g_array_set_size(cpu->iommu_notifiers, i + 1);
+        cpu->accel->iommu_notifiers = g_array_set_size(cpu->accel->iommu_notifiers,
+                                                       i + 1);
         notifier = g_new0(TCGIOMMUNotifier, 1);
-        g_array_index(cpu->iommu_notifiers, TCGIOMMUNotifier *, i) = notifier;
+        g_array_index(cpu->accel->iommu_notifiers,
+                      TCGIOMMUNotifier *, i) = notifier;
 
         notifier->mr = mr;
         notifier->iommu_idx = iommu_idx;
@@ -656,19 +663,31 @@ void tcg_iommu_free_notifier_list(CPUState *cpu)
     int i;
     TCGIOMMUNotifier *notifier;
 
-    for (i = 0; i < cpu->iommu_notifiers->len; i++) {
-        notifier = g_array_index(cpu->iommu_notifiers, TCGIOMMUNotifier *, i);
+    for (i = 0; i < cpu->accel->iommu_notifiers->len; i++) {
+        notifier = g_array_index(cpu->accel->iommu_notifiers,
+                                 TCGIOMMUNotifier *, i);
         memory_region_unregister_iommu_notifier(notifier->mr, &notifier->n);
         g_free(notifier);
     }
-    g_array_free(cpu->iommu_notifiers, true);
+    g_array_free(cpu->accel->iommu_notifiers, true);
 }
 
 void tcg_iommu_init_notifier_list(CPUState *cpu)
 {
-    cpu->iommu_notifiers = g_array_new(false, true, sizeof(TCGIOMMUNotifier *));
+    cpu->accel->iommu_notifiers = g_array_new(false, true,
+                                              sizeof(TCGIOMMUNotifier *));
 }
 
+#else
+
+static void tcg_register_iommu_notifier(CPUState *cpu,
+                                        IOMMUMemoryRegion *iommu_mr,
+                                        int iommu_idx)
+{
+}
+
+#endif
+
 /* Called from RCU critical section */
 MemoryRegionSection *
 address_space_translate_for_iotlb(CPUState *cpu, int asidx, hwaddr orig_addr,
-- 
2.41.0

[PATCH 20/24] accel/tcg: Move @tb_jmp_cache from CPUState to TCG AccelCPUState

[Philippe Mathieu-Daudé]

@tb_jmp_cache is specific to TCG accelerator, move it to
its AccelCPUState.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/tcg/tb-jmp-cache.h  | 4 ++--
 accel/tcg/vcpu-state.h    | 2 ++
 include/hw/core/cpu.h     | 2 --
 include/qemu/typedefs.h   | 1 -
 accel/tcg/cpu-exec.c      | 7 +++----
 accel/tcg/cputlb.c        | 2 +-
 accel/tcg/tb-maint.c      | 2 +-
 accel/tcg/translate-all.c | 5 +++--
 8 files changed, 12 insertions(+), 13 deletions(-)

diff --git a/accel/tcg/tb-jmp-cache.h b/accel/tcg/tb-jmp-cache.h
index 184bb3e3e2..c3a505e394 100644
--- a/accel/tcg/tb-jmp-cache.h
+++ b/accel/tcg/tb-jmp-cache.h
@@ -22,12 +22,12 @@
  * non-NULL value of 'tb'.  Strictly speaking pc is only needed for
  * CF_PCREL, but it's used always for simplicity.
  */
-struct CPUJumpCache {
+typedef struct CPUJumpCache {
     struct rcu_head rcu;
     struct {
         TranslationBlock *tb;
         vaddr pc;
     } array[TB_JMP_CACHE_SIZE];
-};
+} CPUJumpCache;
 
 #endif /* ACCEL_TCG_TB_JMP_CACHE_H */
diff --git a/accel/tcg/vcpu-state.h b/accel/tcg/vcpu-state.h
index 2d09dc3857..5b31c801d8 100644
--- a/accel/tcg/vcpu-state.h
+++ b/accel/tcg/vcpu-state.h
@@ -7,6 +7,7 @@
 #define ACCEL_TCG_VCPU_STATE_H
 
 #include "hw/core/cpu.h"
+#include "tb-jmp-cache.h"
 
 /**
  * AccelCPUState:
@@ -16,6 +17,7 @@ struct AccelCPUState {
     uint32_t cflags_next_tb;
 
     sigjmp_buf jmp_env;
+    CPUJumpCache tb_jmp_cache;
 
 #ifdef CONFIG_USER_ONLY
     TaskState *ts;
diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index 9e192f739a..99709270fa 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -484,8 +484,6 @@ struct CPUState {
     AddressSpace *as;
     MemoryRegion *memory;
 
-    CPUJumpCache *tb_jmp_cache;
-
     GArray *gdb_regs;
     int gdb_num_regs;
     int gdb_num_g_regs;
diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
index 36f2825725..daf9009332 100644
--- a/include/qemu/typedefs.h
+++ b/include/qemu/typedefs.h
@@ -44,7 +44,6 @@ typedef struct CPUAddressSpace CPUAddressSpace;
 typedef struct CPUArchState CPUArchState;
 typedef struct CPUPluginState CPUPluginState;
 typedef struct CpuInfoFast CpuInfoFast;
-typedef struct CPUJumpCache CPUJumpCache;
 typedef struct CPUState CPUState;
 typedef struct CPUTLBEntryFull CPUTLBEntryFull;
 typedef struct DeviceListener DeviceListener;
diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index 55235d3e5e..8f8e1fa948 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -261,7 +261,7 @@ static inline TranslationBlock *tb_lookup(CPUState *cpu, vaddr pc,
     tcg_debug_assert(!(cflags & CF_INVALID));
 
     hash = tb_jmp_cache_hash_func(pc);
-    jc = cpu->tb_jmp_cache;
+    jc = &cpu->accel->tb_jmp_cache;
 
     tb = qatomic_read(&jc->array[hash].tb);
     if (likely(tb &&
@@ -1004,7 +1004,7 @@ cpu_exec_loop(CPUState *cpu, SyncClocks *sc)
                  * for the fast lookup
                  */
                 h = tb_jmp_cache_hash_func(pc);
-                jc = cpu->tb_jmp_cache;
+                jc = &cpu->accel->tb_jmp_cache;
                 jc->array[h].pc = pc;
                 qatomic_set(&jc->array[h].tb, tb);
             }
@@ -1083,7 +1083,6 @@ bool tcg_exec_realizefn(CPUState *cpu, Error **errp)
         tcg_target_initialized = true;
     }
 
-    cpu->tb_jmp_cache = g_new0(CPUJumpCache, 1);
     tlb_init(cpu);
 #ifndef CONFIG_USER_ONLY
     tcg_iommu_init_notifier_list(cpu);
@@ -1101,5 +1100,5 @@ void tcg_exec_unrealizefn(CPUState *cpu)
 #endif /* !CONFIG_USER_ONLY */
 
     tlb_destroy(cpu);
-    g_free_rcu(cpu->tb_jmp_cache, rcu);
+    g_free_rcu(&cpu->accel->tb_jmp_cache, rcu);
 }
diff --git a/accel/tcg/cputlb.c b/accel/tcg/cputlb.c
index e9d6faf78f..964dacaf44 100644
--- a/accel/tcg/cputlb.c
+++ b/accel/tcg/cputlb.c
@@ -156,7 +156,7 @@ static void tlb_window_reset(CPUTLBDesc *desc, int64_t ns,
 
 static void tb_jmp_cache_clear_page(CPUState *cpu, vaddr page_addr)
 {
-    CPUJumpCache *jc = cpu->tb_jmp_cache;
+    CPUJumpCache *jc = &cpu->accel->tb_jmp_cache;
     int i, i0;
 
     if (unlikely(!jc)) {
diff --git a/accel/tcg/tb-maint.c b/accel/tcg/tb-maint.c
index 2d5faca9fd..83758648f2 100644
--- a/accel/tcg/tb-maint.c
+++ b/accel/tcg/tb-maint.c
@@ -888,7 +888,7 @@ static void tb_jmp_cache_inval_tb(TranslationBlock *tb)
         uint32_t h = tb_jmp_cache_hash_func(tb->pc);
 
         CPU_FOREACH(cpu) {
-            CPUJumpCache *jc = cpu->tb_jmp_cache;
+            CPUJumpCache *jc = &cpu->accel->tb_jmp_cache;
 
             if (qatomic_read(&jc->array[h].tb) == tb) {
                 qatomic_set(&jc->array[h].tb, NULL);
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index 3a8199a761..ca1e193633 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -652,13 +652,14 @@ void cpu_io_recompile(CPUState *cpu, uintptr_t retaddr)
  */
 void tcg_flush_jmp_cache(CPUState *cpu)
 {
-    CPUJumpCache *jc = cpu->tb_jmp_cache;
+    CPUJumpCache *jc;
 
     /* During early initialization, the cache may not yet be allocated. */
-    if (unlikely(jc == NULL)) {
+    if (unlikely(cpu->accel == NULL)) {
         return;
     }
 
+    jc = &cpu->accel->tb_jmp_cache;
     for (int i = 0; i < TB_JMP_CACHE_SIZE; i++) {
         qatomic_set(&jc->array[i].tb, NULL);
     }
-- 
2.41.0

[PATCH 21/24] accel/tcg: Remove NULL check in tcg_flush_jmp_cache()

[Philippe Mathieu-Daudé]

I /think/ this check added in commit 4e4fa6c12d ("accel/tcg:
Complete cpu initialization before registration") is now
unnecessary, but I don't have the WASM reproducer mentioned
in:
https://lore.kernel.org/qemu-devel/20221027141856.w5umjgklawgu7pqv@heavy/
to confirm. Ilya, do you mind testing? If so, we could
squash this with the previous patch.

Cc: Ilya Leoshkevich <iii@linux.ibm.com>
Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/tcg/translate-all.c | 8 +-------
 1 file changed, 1 insertion(+), 7 deletions(-)

diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index ca1e193633..9b02f21b23 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -652,14 +652,8 @@ void cpu_io_recompile(CPUState *cpu, uintptr_t retaddr)
  */
 void tcg_flush_jmp_cache(CPUState *cpu)
 {
-    CPUJumpCache *jc;
+    CPUJumpCache *jc = &cpu->accel->tb_jmp_cache;
 
-    /* During early initialization, the cache may not yet be allocated. */
-    if (unlikely(cpu->accel == NULL)) {
-        return;
-    }
-
-    jc = &cpu->accel->tb_jmp_cache;
     for (int i = 0; i < TB_JMP_CACHE_SIZE; i++) {
         qatomic_set(&jc->array[i].tb, NULL);
     }
-- 
2.41.0

[PATCH 22/24] accel/tcg: Move @tcg_cflags from CPUState to TCG AccelCPUState

[Philippe Mathieu-Daudé]

@tcg_cflags is specific to TCG accelerator, move it to
its AccelCPUState.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/tcg/vcpu-state.h | 2 ++
 include/hw/core/cpu.h  | 4 +---
 accel/tcg/cpu-exec.c   | 6 +++---
 linux-user/main.c      | 2 +-
 4 files changed, 7 insertions(+), 7 deletions(-)

diff --git a/accel/tcg/vcpu-state.h b/accel/tcg/vcpu-state.h
index 5b31c801d8..1cdca0e0d4 100644
--- a/accel/tcg/vcpu-state.h
+++ b/accel/tcg/vcpu-state.h
@@ -11,9 +11,11 @@
 
 /**
  * AccelCPUState:
+ * @cflags: Pre-computed cflags for this cpu.
  * @mem_io_pc: Host Program Counter at which the memory was accessed.
  */
 struct AccelCPUState {
+    uint32_t cflags;
     uint32_t cflags_next_tb;
 
     sigjmp_buf jmp_env;
diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index 99709270fa..f6bafa090a 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -396,9 +396,8 @@ struct qemu_work_item;
  *   to a cluster this will be UNASSIGNED_CLUSTER_INDEX; otherwise it will
  *   be the same as the cluster-id property of the CPU object's TYPE_CPU_CLUSTER
  *   QOM parent.
- *   Under TCG this value is propagated to @tcg_cflags.
+ *   Under TCG this value is propagated to @accel->cflags.
  *   See TranslationBlock::TCG CF_CLUSTER_MASK.
- * @tcg_cflags: Pre-computed cflags for this cpu.
  * @nr_cores: Number of cores within this CPU package.
  * @nr_threads: Number of threads within this CPU core.
  * @running: #true if CPU is currently running (lockless).
@@ -515,7 +514,6 @@ struct CPUState {
     /* TODO Move common fields from CPUArchState here. */
     int cpu_index;
     int cluster_index;
-    uint32_t tcg_cflags;
     uint32_t halted;
     int32_t exception_index;
 
diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index 8f8e1fa948..84fd041aec 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -149,17 +149,17 @@ static void init_delay_params(SyncClocks *sc, const CPUState *cpu)
 
 bool tcg_cflags_has(CPUState *cpu, uint32_t flags)
 {
-    return cpu->tcg_cflags & flags;
+    return cpu->accel->cflags & flags;
 }
 
 void tcg_cflags_set(CPUState *cpu, uint32_t flags)
 {
-    cpu->tcg_cflags |= flags;
+    cpu->accel->cflags |= flags;
 }
 
 uint32_t curr_cflags(CPUState *cpu)
 {
-    uint32_t cflags = cpu->tcg_cflags;
+    uint32_t cflags = cpu->accel->cflags;
 
     /*
      * Record gdb single-step.  We should be exiting the TB by raising
diff --git a/linux-user/main.c b/linux-user/main.c
index 5f7f03f4b0..8be06627da 100644
--- a/linux-user/main.c
+++ b/linux-user/main.c
@@ -241,7 +241,7 @@ CPUArchState *cpu_copy(CPUArchState *env)
     /* Reset non arch specific state */
     cpu_reset(new_cpu);
 
-    new_cpu->tcg_cflags = cpu->tcg_cflags;
+    new_cpu->accel->cflags = cpu->accel->cflags;
     memcpy(new_env, env, sizeof(CPUArchState));
 #if defined(TARGET_I386) || defined(TARGET_X86_64)
     new_env->gdt.base = target_mmap(0, sizeof(uint64_t) * TARGET_GDT_ENTRIES,
-- 
2.41.0

[PATCH 23/24] accel/tcg: Restrict icount to system emulation

[Philippe Mathieu-Daudé]

So far we don't support icount on user emulation.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/tcg/cpu-exec.c | 6 ++++++
 1 file changed, 6 insertions(+)

diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index 84fd041aec..1f618f6c2e 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -780,6 +780,9 @@ static inline bool cpu_handle_exception(CPUState *cpu, int *ret)
 
 static inline bool icount_exit_request(CPUState *cpu)
 {
+#if defined(CONFIG_USER_ONLY)
+    return false;
+#else
     if (!icount_enabled()) {
         return false;
     }
@@ -787,6 +790,7 @@ static inline bool icount_exit_request(CPUState *cpu)
         return false;
     }
     return cpu->neg.icount_decr.u16.low + cpu->icount_extra == 0;
+#endif
 }
 
 static inline bool cpu_handle_interrupt(CPUState *cpu,
@@ -801,12 +805,14 @@ static inline bool cpu_handle_interrupt(CPUState *cpu,
         return false;
     }
 
+#if !defined(CONFIG_USER_ONLY)
     /* Clear the interrupt flag now since we're processing
      * cpu->interrupt_request and cpu->exit_request.
      * Ensure zeroing happens before reading cpu->exit_request or
      * cpu->interrupt_request (see also smp_wmb in cpu_exit())
      */
     qatomic_set_mb(&cpu->neg.icount_decr.u16.high, 0);
+#endif /* !CONFIG_USER_ONLY */
 
     if (unlikely(qatomic_read(&cpu->interrupt_request))) {
         int interrupt_request;
-- 
2.41.0

[PATCH 24/24] accel/tcg: Move icount fields from CPUState to TCG AccelCPUState

[Philippe Mathieu-Daudé]

Both @icount_budget and @icount_extra fields are specific
to TCG accelerator, move them to its AccelCPUState.

Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
---
 accel/tcg/vcpu-state.h           |  4 ++++
 include/hw/core/cpu.h            |  3 ---
 accel/tcg/cpu-exec.c             | 14 +++++++-------
 accel/tcg/icount-common.c        |  7 ++++---
 accel/tcg/tcg-accel-ops-icount.c | 14 +++++++-------
 accel/tcg/tcg-accel-ops.c        |  1 +
 hw/core/cpu-common.c             |  1 -
 7 files changed, 23 insertions(+), 21 deletions(-)

diff --git a/accel/tcg/vcpu-state.h b/accel/tcg/vcpu-state.h
index 1cdca0e0d4..9bb8afac57 100644
--- a/accel/tcg/vcpu-state.h
+++ b/accel/tcg/vcpu-state.h
@@ -12,6 +12,7 @@
 /**
  * AccelCPUState:
  * @cflags: Pre-computed cflags for this cpu.
+ * @icount_extra: Instructions until next timer event.
  * @mem_io_pc: Host Program Counter at which the memory was accessed.
  */
 struct AccelCPUState {
@@ -24,6 +25,9 @@ struct AccelCPUState {
 #ifdef CONFIG_USER_ONLY
     TaskState *ts;
 #else
+    int64_t icount_budget;
+    int64_t icount_extra;
+
     uintptr_t mem_io_pc;
 
     /* track IOMMUs whose translations we've cached in the TCG TLB */
diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index f6bafa090a..bdcb09b464 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -411,7 +411,6 @@ struct qemu_work_item;
  * @unplug: Indicates a pending CPU unplug request.
  * @crash_occurred: Indicates the OS reported a crash (panic) for this CPU
  * @singlestep_enabled: Flags for single-stepping.
- * @icount_extra: Instructions until next timer event.
  * @cpu_ases: Pointer to array of CPUAddressSpaces (which define the
  *            AddressSpaces this CPU has)
  * @num_ases: number of CPUAddressSpaces in @cpu_ases
@@ -471,8 +470,6 @@ struct CPUState {
     /* updates protected by BQL */
     uint32_t interrupt_request;
     int singlestep_enabled;
-    int64_t icount_budget;
-    int64_t icount_extra;
     uint64_t random_seed;
 
     QemuMutex work_mutex;
diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index 1f618f6c2e..7c21542e52 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -74,7 +74,7 @@ static void align_clocks(SyncClocks *sc, CPUState *cpu)
         return;
     }
 
-    cpu_icount = cpu->icount_extra + cpu->neg.icount_decr.u16.low;
+    cpu_icount = cpu->accel->icount_extra + cpu->neg.icount_decr.u16.low;
     sc->diff_clk += icount_to_ns(sc->last_cpu_icount - cpu_icount);
     sc->last_cpu_icount = cpu_icount;
 
@@ -125,7 +125,7 @@ static void init_delay_params(SyncClocks *sc, CPUState *cpu)
     sc->realtime_clock = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL_RT);
     sc->diff_clk = qemu_clock_get_ns(QEMU_CLOCK_VIRTUAL) - sc->realtime_clock;
     sc->last_cpu_icount
-        = cpu->icount_extra + cpu->neg.icount_decr.u16.low;
+        = cpu->accel->icount_extra + cpu->neg.icount_decr.u16.low;
     if (sc->diff_clk < max_delay) {
         max_delay = sc->diff_clk;
     }
@@ -718,7 +718,7 @@ static inline bool cpu_handle_exception(CPUState *cpu, int *ret)
     if (cpu->exception_index < 0) {
 #ifndef CONFIG_USER_ONLY
         if (replay_has_exception()
-            && cpu->neg.icount_decr.u16.low + cpu->icount_extra == 0) {
+            && cpu->neg.icount_decr.u16.low + cpu->accel->icount_extra == 0) {
             /* Execute just one insn to trigger exception pending in the log */
             cpu->accel->cflags_next_tb = (curr_cflags(cpu) & ~CF_USE_ICOUNT)
                 | CF_NOIRQ | 1;
@@ -789,7 +789,7 @@ static inline bool icount_exit_request(CPUState *cpu)
     if (!(cpu->accel->cflags_next_tb == -1 || cpu->accel->cflags_next_tb & CF_USE_ICOUNT)) {
         return false;
     }
-    return cpu->neg.icount_decr.u16.low + cpu->icount_extra == 0;
+    return cpu->neg.icount_decr.u16.low + cpu->accel->icount_extra == 0;
 #endif
 }
 
@@ -941,9 +941,9 @@ static inline void cpu_loop_exec_tb(CPUState *cpu, TranslationBlock *tb,
     /* Ensure global icount has gone forward */
     icount_update(cpu);
     /* Refill decrementer and continue execution.  */
-    int32_t insns_left = MIN(0xffff, cpu->icount_budget);
+    int32_t insns_left = MIN(0xffff, cpu->accel->icount_budget);
     cpu->neg.icount_decr.u16.low = insns_left;
-    cpu->icount_extra = cpu->icount_budget - insns_left;
+    cpu->accel->icount_extra = cpu->accel->icount_budget - insns_left;
 
     /*
      * If the next tb has more instructions than we have left to
@@ -952,7 +952,7 @@ static inline void cpu_loop_exec_tb(CPUState *cpu, TranslationBlock *tb,
      */
     if (insns_left > 0 && insns_left < tb->icount)  {
         assert(insns_left <= CF_COUNT_MASK);
-        assert(cpu->icount_extra == 0);
+        assert(cpu->accel->icount_extra == 0);
         cpu->accel->cflags_next_tb = (tb->cflags & ~CF_COUNT_MASK) | insns_left;
     }
 #endif
diff --git a/accel/tcg/icount-common.c b/accel/tcg/icount-common.c
index 8d3d3a7e9d..ff503f8e96 100644
--- a/accel/tcg/icount-common.c
+++ b/accel/tcg/icount-common.c
@@ -38,6 +38,7 @@
 #include "sysemu/cpu-timers.h"
 #include "sysemu/cpu-throttle.h"
 #include "sysemu/cpu-timers-internal.h"
+#include "accel/tcg/vcpu-state.h"
 
 /*
  * ICOUNT: Instruction Counter
@@ -71,8 +72,8 @@ static void icount_enable_adaptive(void)
  */
 static int64_t icount_get_executed(CPUState *cpu)
 {
-    return (cpu->icount_budget -
-            (cpu->neg.icount_decr.u16.low + cpu->icount_extra));
+    return (cpu->accel->icount_budget -
+            (cpu->neg.icount_decr.u16.low + cpu->accel->icount_extra));
 }
 
 /*
@@ -83,7 +84,7 @@ static int64_t icount_get_executed(CPUState *cpu)
 static void icount_update_locked(CPUState *cpu)
 {
     int64_t executed = icount_get_executed(cpu);
-    cpu->icount_budget -= executed;
+    cpu->accel->icount_budget -= executed;
 
     qatomic_set_i64(&timers_state.qemu_icount,
                     timers_state.qemu_icount + executed);
diff --git a/accel/tcg/tcg-accel-ops-icount.c b/accel/tcg/tcg-accel-ops-icount.c
index 9e1ae66f65..75073ec23f 100644
--- a/accel/tcg/tcg-accel-ops-icount.c
+++ b/accel/tcg/tcg-accel-ops-icount.c
@@ -112,16 +112,16 @@ void icount_prepare_for_run(CPUState *cpu, int64_t cpu_budget)
      * asynchronously by cpu_exit/cpu_interrupt/tcg_handle_interrupt
      */
     g_assert(cpu->neg.icount_decr.u16.low == 0);
-    g_assert(cpu->icount_extra == 0);
+    g_assert(cpu->accel->icount_extra == 0);
 
     replay_mutex_lock();
 
-    cpu->icount_budget = MIN(icount_get_limit(), cpu_budget);
-    insns_left = MIN(0xffff, cpu->icount_budget);
+    cpu->accel->icount_budget = MIN(icount_get_limit(), cpu_budget);
+    insns_left = MIN(0xffff, cpu->accel->icount_budget);
     cpu->neg.icount_decr.u16.low = insns_left;
-    cpu->icount_extra = cpu->icount_budget - insns_left;
+    cpu->accel->icount_extra = cpu->accel->icount_budget - insns_left;
 
-    if (cpu->icount_budget == 0) {
+    if (cpu->accel->icount_budget == 0) {
         /*
          * We're called without the BQL, so must take it while
          * we're calling timer handlers.
@@ -139,8 +139,8 @@ void icount_process_data(CPUState *cpu)
 
     /* Reset the counters */
     cpu->neg.icount_decr.u16.low = 0;
-    cpu->icount_extra = 0;
-    cpu->icount_budget = 0;
+    cpu->accel->icount_extra = 0;
+    cpu->accel->icount_budget = 0;
 
     replay_account_executed_instructions();
 
diff --git a/accel/tcg/tcg-accel-ops.c b/accel/tcg/tcg-accel-ops.c
index 5429e2d219..86bc82b891 100644
--- a/accel/tcg/tcg-accel-ops.c
+++ b/accel/tcg/tcg-accel-ops.c
@@ -89,6 +89,7 @@ static void tcg_cpu_reset_hold(CPUState *cpu)
 
     qatomic_set(&cpu->neg.icount_decr.u32, 0);
     cpu->neg.can_do_io = true;
+    cpu->accel->icount_extra = 0;
     cpu->accel->mem_io_pc = 0;
     cpu->accel->cflags_next_tb = -1;
 }
diff --git a/hw/core/cpu-common.c b/hw/core/cpu-common.c
index 684422991c..6dc94b1eaf 100644
--- a/hw/core/cpu-common.c
+++ b/hw/core/cpu-common.c
@@ -117,7 +117,6 @@ static void cpu_common_reset_hold(Object *obj, ResetType type)
 
     cpu->interrupt_request = 0;
     cpu->halted = cpu->start_powered_off;
-    cpu->icount_extra = 0;
     cpu->exception_index = -1;
     cpu->crash_occurred = false;
 
-- 
2.41.0

Re: [PATCH 00/24] exec: Rework around CPUState user fields (part 2)

[Philippe Mathieu-Daudé]

On 29/4/24 00:14, Philippe Mathieu-Daudé wrote:
> Finish extracting TCG fields from CPUState:
> - Extract tcg_cpu_exit() from cpu_exit()
> - Introduce AccelOpsClass::exit_vcpu_thread()
> - cpu_exit() calls exit_vcpu_thread=tcg_cpu_exit for TCG
> - Forward declare TaskState and more uses of get_task_state()
> - Introduce TCG AccelCPUState
> - Move TCG specific fields from CPUState to AccelCPUState
> - Restrict "exec/tlb-common.h" to TCG
> - Restrict iommu_notifiers, icount to system emulation
> 
> Based-on: <20240428214915.10339-1-philmd@linaro.org>

The CPUState changes (part 1 & 2) can be resumed as:

$ git diff master.. -- include/hw/core/cpu.h accel/tcg/vcpu-state.h
-- >8 --
diff --git a/accel/tcg/vcpu-state.h b/accel/tcg/vcpu-state.h
new file mode 100644
index 0000000000..9bb8afac57
--- /dev/null
+++ b/accel/tcg/vcpu-state.h
@@ -0,0 +1,45 @@
+/**
+ * AccelCPUState:
+ * @cflags: Pre-computed cflags for this cpu.
+ * @icount_extra: Instructions until next timer event.
+ * @mem_io_pc: Host Program Counter at which the memory was accessed.
+ */
+struct AccelCPUState {
+    uint32_t cflags;
+    uint32_t cflags_next_tb;
+
+    sigjmp_buf jmp_env;
+    CPUJumpCache tb_jmp_cache;
+
+#ifdef CONFIG_USER_ONLY
+    TaskState *ts;
+#else
+    int64_t icount_budget;
+    int64_t icount_extra;
+
+    uintptr_t mem_io_pc;
+
+    /* track IOMMUs whose translations we've cached in the TCG TLB */
+    GArray *iommu_notifiers;
+#endif
+};
+#endif
diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index 46b99a7ea5..bdcb09b464 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -337,15 +337,28 @@ typedef union IcountDecr {
-/*
- * Elements of CPUState most efficiently accessed from CPUArchState,
- * via small negative offsets.
+/**
+ * CPUNegativeOffsetState: Elements of CPUState most efficiently accessed
+ *                         from CPUArchState, via small negative offsets.
+ * @can_do_io: True if memory-mapped IO is allowed.
+ * @plugin_mem_cbs: active plugin memory callbacks
+ * @plugin_state: per-CPU plugin state
   */
  typedef struct CPUNegativeOffsetState {
+#ifdef CONFIG_TCG
      CPUTLB tlb;
+#ifdef CONFIG_PLUGIN
+    /*
+     * The callback pointer are accessed via TCG (see 
gen_empty_mem_helper).
+     */
+    GArray *plugin_mem_cbs;
+    CPUPluginState *plugin_state;
+#endif
      IcountDecr icount_decr;
      bool can_do_io;
+#endif
  } CPUNegativeOffsetState;

  struct KVMState;
@@ -383,9 +396,8 @@ struct qemu_work_item;
   *   to a cluster this will be UNASSIGNED_CLUSTER_INDEX; otherwise it will
   *   be the same as the cluster-id property of the CPU object's 
TYPE_CPU_CLUSTER
   *   QOM parent.
- *   Under TCG this value is propagated to @tcg_cflags.
+ *   Under TCG this value is propagated to @accel->cflags.
   *   See TranslationBlock::TCG CF_CLUSTER_MASK.
- * @tcg_cflags: Pre-computed cflags for this cpu.
   * @nr_cores: Number of cores within this CPU package.
   * @nr_threads: Number of threads within this CPU core.
   * @running: #true if CPU is currently running (lockless).
@@ -399,8 +411,6 @@ struct qemu_work_item;
   * @unplug: Indicates a pending CPU unplug request.
   * @crash_occurred: Indicates the OS reported a crash (panic) for this CPU
   * @singlestep_enabled: Flags for single-stepping.
- * @icount_extra: Instructions until next timer event.
- * @neg.can_do_io: True if memory-mapped IO is allowed.
   * @cpu_ases: Pointer to array of CPUAddressSpaces (which define the
   *            AddressSpaces this CPU has)
   * @num_ases: number of CPUAddressSpaces in @cpu_ases
@@ -411,13 +421,10 @@ struct qemu_work_item;
   * @gdb_num_g_regs: Number of registers in GDB 'g' packets.
   * @node: QTAILQ of CPUs sharing TB cache.
   * @opaque: User data.
- * @mem_io_pc: Host Program Counter at which the memory was accessed.
   * @accel: Pointer to accelerator specific state.
   * @kvm_fd: vCPU file descriptor for KVM.
   * @work_mutex: Lock to prevent multiple access to @work_list.
   * @work_list: List of pending asynchronous work.
- * @plugin_mem_cbs: active plugin memory callbacks
- * @plugin_state: per-CPU plugin state
   * @ignore_memory_transaction_failures: Cached copy of the MachineState
   *    flag of the same name: allows the board to suppress calling of the
   *    CPU do_transaction_failed hook function.
@@ -460,14 +467,10 @@ struct CPUState {
      bool crash_occurred;
      bool exit_request;
      int exclusive_context_count;
-    uint32_t cflags_next_tb;
      /* updates protected by BQL */
      uint32_t interrupt_request;
      int singlestep_enabled;
-    int64_t icount_budget;
-    int64_t icount_extra;
      uint64_t random_seed;
-    sigjmp_buf jmp_env;

      QemuMutex work_mutex;
      QSIMPLEQ_HEAD(, qemu_work_item) work_list;
@@ -477,8 +480,6 @@ struct CPUState {
      AddressSpace *as;
      MemoryRegion *memory;

-    CPUJumpCache *tb_jmp_cache;
-
      GArray *gdb_regs;
      int gdb_num_regs;
      int gdb_num_g_regs;
@@ -490,12 +491,9 @@ struct CPUState {
      QTAILQ_HEAD(, CPUWatchpoint) watchpoints;
      CPUWatchpoint *watchpoint_hit;

-    void *opaque;
-
      /* In order to avoid passing too many arguments to the MMIO helpers,
       * we store some rarely used information in the CPU context.
       */
-    uintptr_t mem_io_pc;

      /* Only used in KVM */
      int kvm_fd;
@@ -510,19 +508,9 @@ struct CPUState {
      /* Use by accel-block: CPU is executing an ioctl() */
      QemuLockCnt in_ioctl_lock;

-#ifdef CONFIG_PLUGIN
-    /*
-     * The callback pointer stays in the main CPUState as it is
-     * accessed via TCG (see gen_empty_mem_helper).
-     */
-    GArray *plugin_mem_cbs;
-    CPUPluginState *plugin_state;
-#endif
-
      /* TODO Move common fields from CPUArchState here. */
      int cpu_index;
      int cluster_index;
-    uint32_t tcg_cflags;
      uint32_t halted;
      int32_t exception_index;

@@ -544,9 +532,6 @@ struct CPUState {
      /* Used for user-only emulation of prctl(PR_SET_UNALIGN). */
      bool prctl_unalign_sigbus;

-    /* track IOMMUs whose translations we've cached in the TCG TLB */
-    GArray *iommu_notifiers;
-
      /*
       * MUST BE LAST in order to minimize the displacement to CPUArchState.
       */
---

Re: [PATCH 01/24] exec/user: Move 'thunk.h' from 'exec/user' to 'user'

[Richard Henderson]

On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
> Keep all user emulation headers under the same user/ directory.
> 
> Signed-off-by: Philippe Mathieu-Daudé<philmd@linaro.org>
> ---
>   bsd-user/qemu.h                 | 2 +-
>   include/{exec => }/user/thunk.h | 8 ++++++--
>   linux-user/user-internals.h     | 2 +-
>   linux-user/thunk.c              | 2 +-
>   4 files changed, 9 insertions(+), 5 deletions(-)
>   rename include/{exec => }/user/thunk.h (97%)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

Re: [PATCH 02/24] coverity: Update user emulation regexp

[Richard Henderson]

On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
> All user emulation headers are now under include/user/.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   scripts/coverity-scan/COMPONENTS.md | 2 +-
>   1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/scripts/coverity-scan/COMPONENTS.md b/scripts/coverity-scan/COMPONENTS.md
> index 91be8d1c36..1537e49cd5 100644
> --- a/scripts/coverity-scan/COMPONENTS.md
> +++ b/scripts/coverity-scan/COMPONENTS.md
> @@ -121,7 +121,7 @@ usb
>     ~ (/qemu)?(/hw/usb/.*|/include/hw/usb/.*)
>   
>   user
> -  ~ (/qemu)?(/linux-user/.*|/bsd-user/.*|/user-exec\.c|/thunk\.c|/include/exec/user/.*)
> +  ~ (/qemu)?(/linux-user/.*|/bsd-user/.*|/user-exec\.c|/thunk\.c|/include/user/.*)
>   
>   util
>     ~ (/qemu)?(/util/.*|/include/qemu/.*)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

Re: [PATCH 03/24] accel/tcg: Move user definition of cpu_interrupt() to user-exec.c

[Richard Henderson]

On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   accel/tcg/translate-all.c | 9 ---------
>   accel/tcg/user-exec.c     | 8 ++++++++
>   2 files changed, 8 insertions(+), 9 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

> 
> diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
> index 83cc14fbde..fdf6d8ac19 100644
> --- a/accel/tcg/translate-all.c
> +++ b/accel/tcg/translate-all.c
> @@ -644,15 +644,6 @@ void cpu_io_recompile(CPUState *cpu, uintptr_t retaddr)
>       cpu_loop_exit_noexc(cpu);
>   }
>   
> -#else /* CONFIG_USER_ONLY */
> -
> -void cpu_interrupt(CPUState *cpu, int mask)
> -{
> -    g_assert(bql_locked());
> -    cpu->interrupt_request |= mask;
> -    qatomic_set(&cpu->neg.icount_decr.u16.high, -1);
> -}
> -
>   #endif /* CONFIG_USER_ONLY */
>   
>   /*
> diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c
> index ca27746fe4..f49435df9d 100644
> --- a/accel/tcg/user-exec.c
> +++ b/accel/tcg/user-exec.c
> @@ -24,6 +24,7 @@
>   #include "qemu/bitops.h"
>   #include "qemu/rcu.h"
>   #include "exec/cpu_ldst.h"
> +#include "qemu/main-loop.h"
>   #include "exec/translate-all.h"
>   #include "exec/page-protection.h"
>   #include "exec/helper-proto.h"
> @@ -38,6 +39,13 @@ __thread uintptr_t helper_retaddr;
>   
>   //#define DEBUG_SIGNAL
>   
> +void cpu_interrupt(CPUState *cpu, int mask)
> +{
> +    g_assert(bql_locked());
> +    cpu->interrupt_request |= mask;
> +    qatomic_set(&cpu->neg.icount_decr.u16.high, -1);
> +}
> +
>   /*
>    * Adjust the pc to pass to cpu_restore_state; return the memop type.
>    */

Re: [PATCH 04/24] accel/tcg: Duplicate cpu_exit() for user / system

[Richard Henderson]

On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
> We will introduce a per-accelerator vCPU exit() handler
> for system emulation. Duplicate cpu_exit() because the
> handler won't be accessible from user emulation.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   accel/tcg/user-exec.c | 8 ++++++++
>   hw/core/cpu-common.c  | 8 --------
>   system/cpus.c         | 8 ++++++++
>   3 files changed, 16 insertions(+), 8 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>


r~

> 
> diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c
> index f49435df9d..404d8a145c 100644
> --- a/accel/tcg/user-exec.c
> +++ b/accel/tcg/user-exec.c
> @@ -46,6 +46,14 @@ void cpu_interrupt(CPUState *cpu, int mask)
>       qatomic_set(&cpu->neg.icount_decr.u16.high, -1);
>   }
>   
> +void cpu_exit(CPUState *cpu)
> +{
> +    qatomic_set(&cpu->exit_request, 1);
> +    /* Ensure cpu_exec will see the exit request after TCG has exited.  */
> +    smp_wmb();
> +    qatomic_set(&cpu->neg.icount_decr.u16.high, -1);
> +}
> +
>   /*
>    * Adjust the pc to pass to cpu_restore_state; return the memop type.
>    */
> diff --git a/hw/core/cpu-common.c b/hw/core/cpu-common.c
> index 9b3efba82f..cbafc79033 100644
> --- a/hw/core/cpu-common.c
> +++ b/hw/core/cpu-common.c
> @@ -78,14 +78,6 @@ void cpu_reset_interrupt(CPUState *cpu, int mask)
>       }
>   }
>   
> -void cpu_exit(CPUState *cpu)
> -{
> -    qatomic_set(&cpu->exit_request, 1);
> -    /* Ensure cpu_exec will see the exit request after TCG has exited.  */
> -    smp_wmb();
> -    qatomic_set(&cpu->neg.icount_decr.u16.high, -1);
> -}
> -
>   static int cpu_common_gdb_read_register(CPUState *cpu, GByteArray *buf, int reg)
>   {
>       return 0;
> diff --git a/system/cpus.c b/system/cpus.c
> index 68d161d96b..e0530ad603 100644
> --- a/system/cpus.c
> +++ b/system/cpus.c
> @@ -419,6 +419,14 @@ void run_on_cpu(CPUState *cpu, run_on_cpu_func func, run_on_cpu_data data)
>       do_run_on_cpu(cpu, func, data, &bql);
>   }
>   
> +void cpu_exit(CPUState *cpu)
> +{
> +    qatomic_set(&cpu->exit_request, 1);
> +    /* Ensure cpu_exec will see the exit request after TCG has exited.  */
> +    smp_wmb();
> +    qatomic_set(&cpu->neg.icount_decr.u16.high, -1);
> +}
> +
>   static void qemu_cpu_stop(CPUState *cpu, bool exit)
>   {
>       g_assert(qemu_cpu_is_self(cpu));

Re: [PATCH 05/24] accel/tcg: Extract tcg_cpu_exit() from cpu_exit()

[Richard Henderson]

On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
> Extract tcg_cpu_exit() from the user cpu_exit() implementation.
> In few commits we will re-use it in the system one (via a
> per-accel handler).
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   accel/tcg/internal-common.h | 2 ++
>   accel/tcg/translate-all.c   | 7 +++++++
>   accel/tcg/user-exec.c       | 4 +---
>   3 files changed, 10 insertions(+), 3 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

> 
> diff --git a/accel/tcg/internal-common.h b/accel/tcg/internal-common.h
> index ead53cb8a5..df317e7496 100644
> --- a/accel/tcg/internal-common.h
> +++ b/accel/tcg/internal-common.h
> @@ -24,4 +24,6 @@ static inline bool cpu_in_serial_context(CPUState *cs)
>       return !tcg_cflags_has(cs, CF_PARALLEL) || cpu_in_exclusive_context(cs);
>   }
>   
> +void tcg_cpu_exit(CPUState *cpu);
> +
>   #endif
> diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
> index fdf6d8ac19..b67adce20e 100644
> --- a/accel/tcg/translate-all.c
> +++ b/accel/tcg/translate-all.c
> @@ -663,3 +663,10 @@ void tcg_flush_jmp_cache(CPUState *cpu)
>           qatomic_set(&jc->array[i].tb, NULL);
>       }
>   }
> +
> +void tcg_cpu_exit(CPUState *cpu)
> +{
> +    /* Ensure cpu_exec will see the exit request after TCG has exited.  */
> +    smp_wmb();
> +    qatomic_set(&cpu->neg.icount_decr.u16.high, -1);
> +}
> diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c
> index 404d8a145c..d7e5f00c97 100644
> --- a/accel/tcg/user-exec.c
> +++ b/accel/tcg/user-exec.c
> @@ -49,9 +49,7 @@ void cpu_interrupt(CPUState *cpu, int mask)
>   void cpu_exit(CPUState *cpu)
>   {
>       qatomic_set(&cpu->exit_request, 1);
> -    /* Ensure cpu_exec will see the exit request after TCG has exited.  */
> -    smp_wmb();
> -    qatomic_set(&cpu->neg.icount_decr.u16.high, -1);
> +    tcg_cpu_exit(cpu);
>   }
>   
>   /*

Re: [PATCH 06/24] accel: Introduce AccelOpsClass::exit_vcpu_thread() handler

[Richard Henderson]

On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
> Introduce a per-accelerator handler that can be call
> when a vCPU exits.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   include/sysemu/accel-ops.h | 1 +
>   system/cpus.c              | 3 +++
>   2 files changed, 4 insertions(+)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

> 
> diff --git a/include/sysemu/accel-ops.h b/include/sysemu/accel-ops.h
> index ef91fc28bb..35dfb7e266 100644
> --- a/include/sysemu/accel-ops.h
> +++ b/include/sysemu/accel-ops.h
> @@ -35,6 +35,7 @@ struct AccelOpsClass {
>       void (*create_vcpu_thread)(CPUState *cpu); /* MANDATORY NON-NULL */
>       void (*kick_vcpu_thread)(CPUState *cpu);
>       bool (*cpu_thread_is_idle)(CPUState *cpu);
> +    void (*exit_vcpu_thread)(CPUState *cpu);
>   
>       void (*synchronize_post_reset)(CPUState *cpu);
>       void (*synchronize_post_init)(CPUState *cpu);
> diff --git a/system/cpus.c b/system/cpus.c
> index e0530ad603..d0d585e8b9 100644
> --- a/system/cpus.c
> +++ b/system/cpus.c
> @@ -425,6 +425,9 @@ void cpu_exit(CPUState *cpu)
>       /* Ensure cpu_exec will see the exit request after TCG has exited.  */
>       smp_wmb();
>       qatomic_set(&cpu->neg.icount_decr.u16.high, -1);
> +    if (cpus_accel->exit_vcpu_thread) {
> +        cpus_accel->exit_vcpu_thread(cpu);
> +    }
>   }
>   
>   static void qemu_cpu_stop(CPUState *cpu, bool exit)

Re: [PATCH 07/24] accel/tcg: Implement AccelOpsClass::exit_vcpu_thread() handler

[Richard Henderson]

On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
> The TCG-specific code from the system generic cpu_exit()
> is equivalent of tcg_cpu_exit(). Define the AccelOpsClass
> exit_vcpu_thread() handler to it, making cpu_exit() generic.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   accel/tcg/tcg-accel-ops.c | 2 ++
>   system/cpus.c             | 3 ---
>   2 files changed, 2 insertions(+), 3 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>


r~

> 
> diff --git a/accel/tcg/tcg-accel-ops.c b/accel/tcg/tcg-accel-ops.c
> index 7ac5f0c974..56bbad9fcd 100644
> --- a/accel/tcg/tcg-accel-ops.c
> +++ b/accel/tcg/tcg-accel-ops.c
> @@ -43,6 +43,7 @@
>   #include "tcg-accel-ops-mttcg.h"
>   #include "tcg-accel-ops-rr.h"
>   #include "tcg-accel-ops-icount.h"
> +#include "internal-common.h"
>   
>   /* common functionality among all TCG variants */
>   
> @@ -218,6 +219,7 @@ static void tcg_accel_ops_init(AccelOpsClass *ops)
>           }
>       }
>   
> +    ops->exit_vcpu_thread = tcg_cpu_exit;
>       ops->cpu_reset_hold = tcg_cpu_reset_hold;
>       ops->supports_guest_debug = tcg_supports_guest_debug;
>       ops->insert_breakpoint = tcg_insert_breakpoint;
> diff --git a/system/cpus.c b/system/cpus.c
> index d0d585e8b9..3e86b488d3 100644
> --- a/system/cpus.c
> +++ b/system/cpus.c
> @@ -422,9 +422,6 @@ void run_on_cpu(CPUState *cpu, run_on_cpu_func func, run_on_cpu_data data)
>   void cpu_exit(CPUState *cpu)
>   {
>       qatomic_set(&cpu->exit_request, 1);
> -    /* Ensure cpu_exec will see the exit request after TCG has exited.  */
> -    smp_wmb();
> -    qatomic_set(&cpu->neg.icount_decr.u16.high, -1);
>       if (cpus_accel->exit_vcpu_thread) {
>           cpus_accel->exit_vcpu_thread(cpu);
>       }

Re: [PATCH 08/24] user: Forward declare TaskState type definition

[Richard Henderson]

On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
> Forward declare TaskState in "qemu/typedefs.h" so we can
> use it in generic headers like "hw/cpu/core.h".
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   bsd-user/qemu.h         | 4 ++--
>   include/qemu/typedefs.h | 1 +
>   linux-user/qemu.h       | 4 ++--
>   3 files changed, 5 insertions(+), 4 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>


r~

> 
> diff --git a/bsd-user/qemu.h b/bsd-user/qemu.h
> index e562adde65..be57374b41 100644
> --- a/bsd-user/qemu.h
> +++ b/bsd-user/qemu.h
> @@ -76,7 +76,7 @@ struct emulated_sigtable {
>   /*
>    * NOTE: we force a big alignment so that the stack stored after is aligned too
>    */
> -typedef struct TaskState {
> +struct TaskState {
>       pid_t ts_tid;     /* tid (or pid) of this task */
>   
>       struct TaskState *next;
> @@ -114,7 +114,7 @@ typedef struct TaskState {
>   
>       /* This thread's sigaltstack, if it has one */
>       struct target_sigaltstack sigaltstack_used;
> -} __attribute__((aligned(16))) TaskState;
> +} __attribute__((aligned(16)));
>   
>   static inline TaskState *get_task_state(CPUState *cs)
>   {
> diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
> index 50c277cf0b..36f2825725 100644
> --- a/include/qemu/typedefs.h
> +++ b/include/qemu/typedefs.h
> @@ -134,6 +134,7 @@ typedef struct SHPCDevice SHPCDevice;
>   typedef struct SSIBus SSIBus;
>   typedef struct TCGCPUOps TCGCPUOps;
>   typedef struct TCGHelperInfo TCGHelperInfo;
> +typedef struct TaskState TaskState;
>   typedef struct TranslationBlock TranslationBlock;
>   typedef struct VirtIODevice VirtIODevice;
>   typedef struct Visitor Visitor;
> diff --git a/linux-user/qemu.h b/linux-user/qemu.h
> index 4777856b52..16d9f6ae8c 100644
> --- a/linux-user/qemu.h
> +++ b/linux-user/qemu.h
> @@ -95,7 +95,7 @@ struct emulated_sigtable {
>       target_siginfo_t info;
>   };
>   
> -typedef struct TaskState {
> +struct TaskState {
>       pid_t ts_tid;     /* tid (or pid) of this task */
>   #ifdef TARGET_ARM
>   # ifdef TARGET_ABI32
> @@ -158,7 +158,7 @@ typedef struct TaskState {
>   
>       /* Start time of task after system boot in clock ticks */
>       uint64_t start_boottime;
> -} TaskState;
> +};
>   
>   static inline TaskState *get_task_state(CPUState *cs)
>   {

Re: [PATCH 09/24] user: Declare get_task_state() once in 'accel/tcg/vcpu-state.h'

[Richard Henderson]

On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
> While each user emulation implentation defines its own
> TaskState structure, both use the same get_task_state()
> declaration, in particular in common code (such gdbstub).
> Declare the method once in "accel/tcg/vcpu-state.h".
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   accel/tcg/vcpu-state.h | 18 ++++++++++++++++++
>   bsd-user/qemu.h        |  6 +-----
>   linux-user/qemu.h      |  6 +-----
>   3 files changed, 20 insertions(+), 10 deletions(-)
>   create mode 100644 accel/tcg/vcpu-state.h

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

> 
> diff --git a/accel/tcg/vcpu-state.h b/accel/tcg/vcpu-state.h
> new file mode 100644
> index 0000000000..e407d914df
> --- /dev/null
> +++ b/accel/tcg/vcpu-state.h
> @@ -0,0 +1,18 @@
> +/*
> + * SPDX-FileContributor: Philippe Mathieu-Daudé <philmd@linaro.org>
> + * SPDX-FileCopyrightText: 2023 Linaro Ltd.
> + * SPDX-License-Identifier: GPL-2.0-or-later
> + */
> +#ifndef ACCEL_TCG_VCPU_STATE_H
> +#define ACCEL_TCG_VCPU_STATE_H
> +
> +#include "hw/core/cpu.h"
> +
> +#ifdef CONFIG_USER_ONLY
> +static inline TaskState *get_task_state(const CPUState *cs)
> +{
> +    return cs->opaque;
> +}
> +#endif
> +
> +#endif
> diff --git a/bsd-user/qemu.h b/bsd-user/qemu.h
> index be57374b41..65fe95fed1 100644
> --- a/bsd-user/qemu.h
> +++ b/bsd-user/qemu.h
> @@ -36,6 +36,7 @@ extern char **environ;
>   #include "exec/gdbstub.h"
>   #include "exec/page-protection.h"
>   #include "qemu/clang-tsa.h"
> +#include "accel/tcg/vcpu-state.h"
>   
>   #include "qemu-os.h"
>   /*
> @@ -116,11 +117,6 @@ struct TaskState {
>       struct target_sigaltstack sigaltstack_used;
>   } __attribute__((aligned(16)));
>   
> -static inline TaskState *get_task_state(CPUState *cs)
> -{
> -    return cs->opaque;
> -}
> -
>   void stop_all_tasks(void);
>   extern const char *interp_prefix;
>   extern const char *qemu_uname_release;
> diff --git a/linux-user/qemu.h b/linux-user/qemu.h
> index 16d9f6ae8c..515af82d8b 100644
> --- a/linux-user/qemu.h
> +++ b/linux-user/qemu.h
> @@ -8,6 +8,7 @@
>   
>   #include "syscall_defs.h"
>   #include "target_syscall.h"
> +#include "accel/tcg/vcpu-state.h"
>   
>   /*
>    * This is the size of the host kernel's sigset_t, needed where we make
> @@ -160,11 +161,6 @@ struct TaskState {
>       uint64_t start_boottime;
>   };
>   
> -static inline TaskState *get_task_state(CPUState *cs)
> -{
> -    return cs->opaque;
> -}
> -
>   abi_long do_brk(abi_ulong new_brk);
>   int do_guest_openat(CPUArchState *cpu_env, int dirfd, const char *pathname,
>                       int flags, mode_t mode, bool safe);

Re: [PATCH 10/24] user: Use get_task_state() helper

[Richard Henderson]

On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
> Get the TaskState pointer calling get_task_state().
> 
> Signed-off-by: Philippe Mathieu-Daudé<philmd@linaro.org>
> ---
>   gdbstub/gdbstub.c     | 3 ++-
>   gdbstub/user-target.c | 4 ++--
>   linux-user/syscall.c  | 4 ++--
>   3 files changed, 6 insertions(+), 5 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

Re: [PATCH 15/24] accel/tcg: Restrict IcountDecr and CPUTLB to TCG

[Philippe Mathieu-Daudé]

On 29/4/24 00:14, Philippe Mathieu-Daudé wrote:
> IcountDecr union, the CPUTLB* structures and the
> "exec/tlb-common.h" header are only required for
> TCG.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   include/exec/tlb-common.h | 4 ++++
>   include/hw/core/cpu.h     | 9 ++++++---
>   2 files changed, 10 insertions(+), 3 deletions(-)
> 
> diff --git a/include/exec/tlb-common.h b/include/exec/tlb-common.h
> index dc5a5faa0b..a529c9f056 100644
> --- a/include/exec/tlb-common.h
> +++ b/include/exec/tlb-common.h
> @@ -19,6 +19,10 @@
>   #ifndef EXEC_TLB_COMMON_H
>   #define EXEC_TLB_COMMON_H 1
>   
> +#ifndef CONFIG_TCG
> +#error Can only include this header with TCG
> +#endif
> +
>   #define CPU_TLB_ENTRY_BITS 5
>   
>   /* Minimalized TLB entry for use by TCG fast path. */
> diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
> index ef8b85b6fe..dc28920bcc 100644
> --- a/include/hw/core/cpu.h
> +++ b/include/hw/core/cpu.h
> @@ -27,7 +27,6 @@
>   #include "exec/vaddr.h"
>   #include "exec/memattrs.h"
>   #include "exec/mmu-access-type.h"
> -#include "exec/tlb-common.h"
>   #include "qapi/qapi-types-run-state.h"
>   #include "qemu/bitmap.h"
>   #include "qemu/rcu_queue.h"
> @@ -256,6 +255,9 @@ typedef struct CPUTLBEntryFull {
>       } extra;
>   } CPUTLBEntryFull;
>   
> +#ifdef CONFIG_TCG
> +#include "exec/tlb-common.h"
> +
>   /*
>    * Data elements that are per MMU mode, minus the bits accessed by
>    * the TCG fast path.
> @@ -311,11 +313,9 @@ typedef struct CPUTLBCommon {
>    * negative offsets are at the end of the struct.
>    */
>   typedef struct CPUTLB {
> -#ifdef CONFIG_TCG
>       CPUTLBCommon c;
>       CPUTLBDesc d[NB_MMU_MODES];
>       CPUTLBDescFast f[NB_MMU_MODES];
> -#endif
>   } CPUTLB;
>   
>   /*
> @@ -337,6 +337,7 @@ typedef union IcountDecr {
>   #endif
>       } u16;
>   } IcountDecr;
> +#endif
>   
>   /**
>    * CPUNegativeOffsetState: Elements of CPUState most efficiently accessed
> @@ -346,6 +347,7 @@ typedef union IcountDecr {
>    * @plugin_state: per-CPU plugin state
>    */
>   typedef struct CPUNegativeOffsetState {
> +#ifdef CONFIG_TCG
>       CPUTLB tlb;
>   #ifdef CONFIG_PLUGIN
>       /*
> @@ -356,6 +358,7 @@ typedef struct CPUNegativeOffsetState {
>   #endif
>       IcountDecr icount_decr;
>       bool can_do_io;
> +#endif
>   } CPUNegativeOffsetState;

We also need:

-- >8 --
@@ -1110,6 +1110,7 @@

+#ifdef CONFIG_TCG
  /**
   * cpu_plugin_mem_cbs_enabled() - are plugin memory callbacks enabled?
   * @cs: CPUState pointer
@@ -1126,6 +1127,7 @@ static inline bool 
cpu_plugin_mem_cbs_enabled(const CPUState *cpu)
      return false;
  #endif
  }
+#endif

---

Otherwise this inlined function fails to build when TCG is disabled...

But since it is only used from accel/tcg/, I'll move it to
accel/tcg/internal-common.h in a preliminary patch instead:

-- >8 --
Author: Philippe Mathieu-Daudé <philmd@linaro.org>
Date:   Mon Apr 29 16:01:18 2024 +0200

     accel/tcg: Restrict cpu_plugin_mem_cbs_enabled() to TCG

     So far cpu_plugin_mem_cbs_enabled() is only called from
     TCG, so reduce it to accel/tcg/.

     Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>

diff --git a/accel/tcg/internal-common.h b/accel/tcg/internal-common.h
index df317e7496..867426500f 100644
--- a/accel/tcg/internal-common.h
+++ b/accel/tcg/internal-common.h
@@ -26,0 +27,17 @@ static inline bool cpu_in_serial_context(CPUState *cs)
+/**
+ * cpu_plugin_mem_cbs_enabled() - are plugin memory callbacks enabled?
+ * @cs: CPUState pointer
+ *
+ * The memory callbacks are installed if a plugin has instrumented an
+ * instruction for memory. This can be useful to know if you want to
+ * force a slow path for a series of memory accesses.
+ */
+static inline bool cpu_plugin_mem_cbs_enabled(const CPUState *cpu)
+{
+#ifdef CONFIG_PLUGIN
+    return !!cpu->neg.plugin_mem_cbs;
+#else
+    return false;
+#endif
+}
+
diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
index ef8b85b6fe..24ad52af7d 100644
--- a/include/hw/core/cpu.h
+++ b/include/hw/core/cpu.h
@@ -1110,17 +1109,0 @@ void cpu_watchpoint_remove_all(CPUState *cpu, int 
mask);
-/**
- * cpu_plugin_mem_cbs_enabled() - are plugin memory callbacks enabled?
- * @cs: CPUState pointer
- *
- * The memory callbacks are installed if a plugin has instrumented an
- * instruction for memory. This can be useful to know if you want to
- * force a slow path for a series of memory accesses.
- */
-static inline bool cpu_plugin_mem_cbs_enabled(const CPUState *cpu)
-{
-#ifdef CONFIG_PLUGIN
-    return !!cpu->neg.plugin_mem_cbs;
-#else
-    return false;
-#endif
-}
-
---

Re: [PATCH 11/24] accel/tcg: Allocate per-vCPU accel state in create_vcpu_thread()

[Richard Henderson]

On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   accel/tcg/vcpu-state.h          | 7 +++++++
>   accel/tcg/tcg-accel-ops-mttcg.c | 2 ++
>   accel/tcg/tcg-accel-ops-rr.c    | 2 ++
>   3 files changed, 11 insertions(+)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

> 
> diff --git a/accel/tcg/vcpu-state.h b/accel/tcg/vcpu-state.h
> index e407d914df..d0dd1bbff8 100644
> --- a/accel/tcg/vcpu-state.h
> +++ b/accel/tcg/vcpu-state.h
> @@ -8,6 +8,13 @@
>   
>   #include "hw/core/cpu.h"
>   
> +/**
> + * AccelCPUState:
> + */
> +struct AccelCPUState {
> +    /* Empty */
> +};
> +
>   #ifdef CONFIG_USER_ONLY
>   static inline TaskState *get_task_state(const CPUState *cs)
>   {
> diff --git a/accel/tcg/tcg-accel-ops-mttcg.c b/accel/tcg/tcg-accel-ops-mttcg.c
> index c552b45b8e..767b321a0d 100644
> --- a/accel/tcg/tcg-accel-ops-mttcg.c
> +++ b/accel/tcg/tcg-accel-ops-mttcg.c
> @@ -35,6 +35,7 @@
>   #include "tcg/startup.h"
>   #include "tcg-accel-ops.h"
>   #include "tcg-accel-ops-mttcg.h"
> +#include "accel/tcg/vcpu-state.h"
>   
>   typedef struct MttcgForceRcuNotifier {
>       Notifier notifier;
> @@ -135,6 +136,7 @@ void mttcg_start_vcpu_thread(CPUState *cpu)
>       char thread_name[VCPU_THREAD_NAME_SIZE];
>   
>       g_assert(tcg_enabled());
> +    cpu->accel = g_new0(AccelCPUState, 1);
>       tcg_cpu_init_cflags(cpu, current_machine->smp.max_cpus > 1);
>   
>       cpu->thread = g_new0(QemuThread, 1);
> diff --git a/accel/tcg/tcg-accel-ops-rr.c b/accel/tcg/tcg-accel-ops-rr.c
> index 894e73e52c..419b94f786 100644
> --- a/accel/tcg/tcg-accel-ops-rr.c
> +++ b/accel/tcg/tcg-accel-ops-rr.c
> @@ -36,6 +36,7 @@
>   #include "tcg-accel-ops.h"
>   #include "tcg-accel-ops-rr.h"
>   #include "tcg-accel-ops-icount.h"
> +#include "accel/tcg/vcpu-state.h"
>   
>   /* Kick all RR vCPUs */
>   void rr_kick_vcpu_thread(CPUState *unused)
> @@ -314,6 +315,7 @@ void rr_start_vcpu_thread(CPUState *cpu)
>       static QemuThread *single_tcg_cpu_thread;
>   
>       g_assert(tcg_enabled());
> +    cpu->accel = g_new0(AccelCPUState, 1);
>       tcg_cpu_init_cflags(cpu, false);
>   
>       if (!single_tcg_cpu_thread) {

Re: [PATCH 12/24] accel/tcg: Move TaskState from CPUState to TCG AccelCPUState

[Richard Henderson]

On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
> TaskState is specific to TCG user emulation, move it
> to AccelCPUState.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   accel/tcg/vcpu-state.h | 6 ++++--
>   include/hw/core/cpu.h  | 2 --
>   bsd-user/main.c        | 2 +-
>   linux-user/main.c      | 2 +-
>   linux-user/syscall.c   | 2 +-
>   5 files changed, 7 insertions(+), 7 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

Re: [PATCH 13/24] accel/tcg: Update CPUNegativeOffsetState::can_do_io field documentation

[Richard Henderson]

On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
> The @can_do_io field got moved from CPUState to
> CPUNegativeOffsetState in commit 464dacf609 ("accel/tcg:
> Move can_do_io to CPUNegativeOffsetState").
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   include/hw/core/cpu.h | 8 ++++----
>   1 file changed, 4 insertions(+), 4 deletions(-)

Oops,

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

Re: [PATCH 14/24] accel/tcg: Move plugin fields to CPUNegativeOffsetState

[Richard Henderson]

On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   include/hw/core/cpu.h  | 22 ++++++++++------------
>   include/qemu/plugin.h  |  2 +-
>   accel/tcg/plugin-gen.c |  8 +++++---
>   hw/core/cpu-common.c   |  2 +-
>   plugins/core.c         |  8 ++++----
>   5 files changed, 21 insertions(+), 21 deletions(-)
> 
> diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
> index beb37342e9..ef8b85b6fe 100644
> --- a/include/hw/core/cpu.h
> +++ b/include/hw/core/cpu.h
> @@ -342,9 +342,18 @@ typedef union IcountDecr {
>    * CPUNegativeOffsetState: Elements of CPUState most efficiently accessed
>    *                         from CPUArchState, via small negative offsets.
>    * @can_do_io: True if memory-mapped IO is allowed.
> + * @plugin_mem_cbs: active plugin memory callbacks
> + * @plugin_state: per-CPU plugin state
>    */
>   typedef struct CPUNegativeOffsetState {
>       CPUTLB tlb;
> +#ifdef CONFIG_PLUGIN
> +    /*
> +     * The callback pointer are accessed via TCG (see gen_empty_mem_helper).
> +     */
> +    GArray *plugin_mem_cbs;
> +    CPUPluginState *plugin_state;

While plugin_mem_cbs is accessed by tcg generated code, plugin_state is not.
I would move plugin_state to cpu->accel.


r~

Re: [PATCH 15/24] accel/tcg: Restrict IcountDecr and CPUTLB to TCG

[Richard Henderson]

On 4/29/24 07:03, Philippe Mathieu-Daudé wrote:
> Author: Philippe Mathieu-Daudé <philmd@linaro.org>
> Date:   Mon Apr 29 16:01:18 2024 +0200
> 
>      accel/tcg: Restrict cpu_plugin_mem_cbs_enabled() to TCG
> 
>      So far cpu_plugin_mem_cbs_enabled() is only called from
>      TCG, so reduce it to accel/tcg/.
> 
>      Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

Re: [PATCH 15/24] accel/tcg: Restrict IcountDecr and CPUTLB to TCG

[Richard Henderson]

On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
> IcountDecr union, the CPUTLB* structures and the
> "exec/tlb-common.h" header are only required for
> TCG.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   include/exec/tlb-common.h | 4 ++++
>   include/hw/core/cpu.h     | 9 ++++++---
>   2 files changed, 10 insertions(+), 3 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

Though you might mention can_do_io also being restricted:

> @@ -346,6 +347,7 @@ typedef union IcountDecr {
>    * @plugin_state: per-CPU plugin state
>    */
>   typedef struct CPUNegativeOffsetState {
> +#ifdef CONFIG_TCG
>       CPUTLB tlb;
>   #ifdef CONFIG_PLUGIN
>       /*
> @@ -356,6 +358,7 @@ typedef struct CPUNegativeOffsetState {
>   #endif
>       IcountDecr icount_decr;
>       bool can_do_io;
> +#endif
>   } CPUNegativeOffsetState;
>   
>   struct KVMState;


r~

Re: [PATCH 16/24] accel/tcg: Move @jmp_env from CPUState to TCG AccelCPUState

[Richard Henderson]

On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
> @jmp_env is specific to TCG accelerator, move it to its AccelCPUState.
> 
> Signed-off-by: Philippe Mathieu-Daudé<philmd@linaro.org>
> ---
>   accel/tcg/internal-common.h | 1 +
>   accel/tcg/tcg-accel-ops.h   | 1 +
>   accel/tcg/vcpu-state.h      | 2 ++
>   include/hw/core/cpu.h       | 1 -
>   accel/tcg/cpu-exec-common.c | 2 +-
>   accel/tcg/cpu-exec.c        | 6 +++---
>   6 files changed, 8 insertions(+), 5 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

Re: [PATCH 17/24] accel/tcg: Move @mem_io_pc from CPUState to TCG AccelCPUState

[Richard Henderson]

On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
> diff --git a/target/i386/helper.c b/target/i386/helper.c
> index 48d1513a35..6c188ea94c 100644
> --- a/target/i386/helper.c
> +++ b/target/i386/helper.c
> @@ -30,6 +30,7 @@
>   #include "qemu/log.h"
>   #ifdef CONFIG_TCG
>   #include "tcg/insn-start-words.h"
> +#include "accel/tcg/vcpu-state.h" // ???
>   #endif
>   
>   void cpu_sync_avx_hflag(CPUX86State *env)
> @@ -518,7 +519,7 @@ static inline target_ulong get_memio_eip(CPUX86State *env)
>       uint64_t data[TARGET_INSN_START_WORDS];
>       CPUState *cs = env_cpu(env);
>   
> -    if (!cpu_unwind_state_data(cs, cs->mem_io_pc, data)) {
> +    if (!cpu_unwind_state_data(cs, cs->accel->mem_io_pc, data)) {
>           return env->eip;
>       }
>   

Not good.

Although considering

https://lore.kernel.org/qemu-devel/20240416040609.1313605-3-richard.henderson@linaro.org/

we should probably merge mem_io_pc and plugin_ra.  So the value to be accessed by i386 
would not need to be in a tcg private data structure, but in CPUNegativeOffsetState.


r~

Re: [PATCH 18/24] accel/tcg: Move @cflags_next_tb from CPUState to TCG AccelCPUState

[Richard Henderson]

On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
> @cflags_next_tb is specific to TCG accelerator, move it to
> its AccelCPUState.
> 
> Signed-off-by: Philippe Mathieu-Daudé<philmd@linaro.org>
> ---
>   accel/tcg/vcpu-state.h    |  2 ++
>   include/hw/core/cpu.h     |  1 -
>   accel/tcg/cpu-exec.c      | 12 ++++++------
>   accel/tcg/tb-maint.c      |  4 ++--
>   accel/tcg/tcg-accel-ops.c |  1 +
>   accel/tcg/translate-all.c |  2 +-
>   accel/tcg/watchpoint.c    |  5 +++--
>   hw/core/cpu-common.c      |  1 -
>   8 files changed, 15 insertions(+), 13 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

Re: [PATCH 19/24] accel/tcg: Move @iommu_notifiers from CPUState to TCG AccelCPUState

[Richard Henderson]

On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
> @iommu_notifiers is specific to TCG system emulation, move it to
> AccelCPUState.
> 
> Restrict TCG specific code in system/physmem.c, adding an empty
> stub for tcg_register_iommu_notifier().
> 
> Signed-off-by: Philippe Mathieu-Daudé<philmd@linaro.org>
> ---
>   accel/tcg/vcpu-state.h |  3 +++
>   include/hw/core/cpu.h  |  3 ---
>   system/physmem.c       | 37 ++++++++++++++++++++++++++++---------
>   3 files changed, 31 insertions(+), 12 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

Re: [PATCH 20/24] accel/tcg: Move @tb_jmp_cache from CPUState to TCG AccelCPUState

[Richard Henderson]

On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
> @tb_jmp_cache is specific to TCG accelerator, move it to
> its AccelCPUState.
> 
> Signed-off-by: Philippe Mathieu-Daudé<philmd@linaro.org>
> ---
>   accel/tcg/tb-jmp-cache.h  | 4 ++--
>   accel/tcg/vcpu-state.h    | 2 ++
>   include/hw/core/cpu.h     | 2 --
>   include/qemu/typedefs.h   | 1 -
>   accel/tcg/cpu-exec.c      | 7 +++----
>   accel/tcg/cputlb.c        | 2 +-
>   accel/tcg/tb-maint.c      | 2 +-
>   accel/tcg/translate-all.c | 5 +++--
>   8 files changed, 12 insertions(+), 13 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>


r~

Re: [PATCH 22/24] accel/tcg: Move @tcg_cflags from CPUState to TCG AccelCPUState

[Richard Henderson]

On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
> @tcg_cflags is specific to TCG accelerator, move it to
> its AccelCPUState.
> 
> Signed-off-by: Philippe Mathieu-Daudé<philmd@linaro.org>
> ---
>   accel/tcg/vcpu-state.h | 2 ++
>   include/hw/core/cpu.h  | 4 +---
>   accel/tcg/cpu-exec.c   | 6 +++---
>   linux-user/main.c      | 2 +-
>   4 files changed, 7 insertions(+), 7 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~

Re: [PATCH 14/24] accel/tcg: Move plugin fields to CPUNegativeOffsetState

[Philippe Mathieu-Daudé]

On 29/4/24 16:42, Richard Henderson wrote:
> On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
>> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
>> ---
>>   include/hw/core/cpu.h  | 22 ++++++++++------------
>>   include/qemu/plugin.h  |  2 +-
>>   accel/tcg/plugin-gen.c |  8 +++++---
>>   hw/core/cpu-common.c   |  2 +-
>>   plugins/core.c         |  8 ++++----
>>   5 files changed, 21 insertions(+), 21 deletions(-)
>>
>> diff --git a/include/hw/core/cpu.h b/include/hw/core/cpu.h
>> index beb37342e9..ef8b85b6fe 100644
>> --- a/include/hw/core/cpu.h
>> +++ b/include/hw/core/cpu.h
>> @@ -342,9 +342,18 @@ typedef union IcountDecr {
>>    * CPUNegativeOffsetState: Elements of CPUState most efficiently 
>> accessed
>>    *                         from CPUArchState, via small negative 
>> offsets.
>>    * @can_do_io: True if memory-mapped IO is allowed.
>> + * @plugin_mem_cbs: active plugin memory callbacks
>> + * @plugin_state: per-CPU plugin state
>>    */
>>   typedef struct CPUNegativeOffsetState {
>>       CPUTLB tlb;
>> +#ifdef CONFIG_PLUGIN
>> +    /*
>> +     * The callback pointer are accessed via TCG (see 
>> gen_empty_mem_helper).
>> +     */
>> +    GArray *plugin_mem_cbs;
>> +    CPUPluginState *plugin_state;
> 
> While plugin_mem_cbs is accessed by tcg generated code, plugin_state is 
> not.
> I would move plugin_state to cpu->accel.

Good idea.

Re: [PATCH 00/24] exec: Rework around CPUState user fields (part 2)

[Philippe Mathieu-Daudé]

On 29/4/24 00:14, Philippe Mathieu-Daudé wrote:

> Philippe Mathieu-Daudé (24):
>    exec/user: Move 'thunk.h' from 'exec/user' to 'user'
>    coverity: Update user emulation regexp
>    accel/tcg: Move user definition of cpu_interrupt() to user-exec.c
>    accel/tcg: Duplicate cpu_exit() for user / system
>    accel/tcg: Extract tcg_cpu_exit() from cpu_exit()
>    accel: Introduce AccelOpsClass::exit_vcpu_thread() handler
>    accel/tcg: Implement AccelOpsClass::exit_vcpu_thread() handler
>    user: Forward declare TaskState type definition
>    user: Declare get_task_state() once in 'accel/tcg/vcpu-state.h'
>    user: Use get_task_state() helper
>    accel/tcg: Allocate per-vCPU accel state in create_vcpu_thread()
>    accel/tcg: Move TaskState from CPUState to TCG AccelCPUState
>    accel/tcg: Update CPUNegativeOffsetState::can_do_io field
>      documentation

Queuing patches 1-13, thanks!

Re: [PATCH 23/24] accel/tcg: Restrict icount to system emulation

[Richard Henderson]

On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
> So far we don't support icount on user emulation.
> 
> Signed-off-by: Philippe Mathieu-Daudé <philmd@linaro.org>
> ---
>   accel/tcg/cpu-exec.c | 6 ++++++
>   1 file changed, 6 insertions(+)


Reviewed-by: Richard Henderson <richard.henderson@linaro.org>


r~

Re: [PATCH 24/24] accel/tcg: Move icount fields from CPUState to TCG AccelCPUState

[Richard Henderson]

On 4/28/24 15:14, Philippe Mathieu-Daudé wrote:
> Both @icount_budget and @icount_extra fields are specific
> to TCG accelerator, move them to its AccelCPUState.
> 
> Signed-off-by: Philippe Mathieu-Daudé<philmd@linaro.org>
> ---
>   accel/tcg/vcpu-state.h           |  4 ++++
>   include/hw/core/cpu.h            |  3 ---
>   accel/tcg/cpu-exec.c             | 14 +++++++-------
>   accel/tcg/icount-common.c        |  7 ++++---
>   accel/tcg/tcg-accel-ops-icount.c | 14 +++++++-------
>   accel/tcg/tcg-accel-ops.c        |  1 +
>   hw/core/cpu-common.c             |  1 -
>   7 files changed, 23 insertions(+), 21 deletions(-)

Reviewed-by: Richard Henderson <richard.henderson@linaro.org>

r~