* [PULL 00/47] tcg patch queue
@ 2022-10-26 2:10 Richard Henderson
2022-10-26 2:10 ` [PULL 01/47] Revert "accel/tcg: Init TCG cflags in vCPU thread handler" Richard Henderson
` (47 more replies)
0 siblings, 48 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha
The following changes since commit e750a7ace492f0b450653d4ad368a77d6f660fb8:
Merge tag 'pull-9p-20221024' of https://github.com/cschoenebeck/qemu into staging (2022-10-24 14:27:12 -0400)
are available in the Git repository at:
https://gitlab.com/rth7680/qemu.git tags/pull-tcg-20221026
for you to fetch changes up to 04f105758b0089f73ee47260671580cde35f96cc:
accel/tcg: Remove restore_state_to_opc function (2022-10-26 11:11:28 +1000)
----------------------------------------------------------------
Revert incorrect cflags initialization.
Add direct jumps for tcg/loongarch64.
Speed up breakpoint check.
Improve assertions for atomic.h.
Move restore_state_to_opc to TCGCPUOps.
Cleanups to TranslationBlock maintenance.
----------------------------------------------------------------
Leandro Lupori (1):
accel/tcg: Add a quicker check for breakpoints
Peter Maydell (1):
Revert "accel/tcg: Init TCG cflags in vCPU thread handler"
Qi Hu (2):
tcg/loongarch64: Add direct jump support
tcg/aarch64: Remove unused code in tcg_out_op
Richard Henderson (43):
include/qemu/osdep: Add qemu_build_assert
include/qemu/atomic: Use qemu_build_assert
include/qemu/thread: Use qatomic_* functions
accel/tcg: Make page_alloc_target_data allocation constant
accel/tcg: Remove disabled debug in translate-all.c
accel/tcg: Split out PageDesc to internal.h
accel/tcg: Split out tb-maint.c
accel/tcg: Move assert_no_pages_locked to internal.h
accel/tcg: Drop cpu_get_tb_cpu_state from TARGET_HAS_PRECISE_SMC
accel/tcg: Remove duplicate store to tb->page_addr[]
accel/tcg: Introduce tb_{set_}page_addr{0,1}
accel/tcg: Rename tb_invalidate_phys_page
accel/tcg: Rename tb_invalidate_phys_page_range and drop end parameter
accel/tcg: Unify declarations of tb_invalidate_phys_range
accel/tcg: Use tb_invalidate_phys_page in page_set_flags
accel/tcg: Call tb_invalidate_phys_page for PAGE_RESET
accel/tcg: Use page_reset_target_data in page_set_flags
accel/tcg: Use tb_invalidate_phys_range in page_set_flags
accel/tcg: Move TARGET_PAGE_DATA_SIZE impl to user-exec.c
accel/tcg: Simplify page_get/alloc_target_data
accel/tcg: Add restore_state_to_opc to TCGCPUOps
target/alpha: Convert to tcg_ops restore_state_to_opc
target/arm: Convert to tcg_ops restore_state_to_opc
target/avr: Convert to tcg_ops restore_state_to_opc
target/cris: Convert to tcg_ops restore_state_to_opc
target/hexagon: Convert to tcg_ops restore_state_to_opc
target/hppa: Convert to tcg_ops restore_state_to_opc
target/i386: Convert to tcg_ops restore_state_to_opc
target/loongarch: Convert to tcg_ops restore_state_to_opc
target/m68k: Convert to tcg_ops restore_state_to_opc
target/microblaze: Convert to tcg_ops restore_state_to_opc
target/mips: Convert to tcg_ops restore_state_to_opc
target/nios2: Convert to tcg_ops restore_state_to_opc
target/openrisc: Convert to tcg_ops restore_state_to_opc
target/ppc: Convert to tcg_ops restore_state_to_opc
target/riscv: Convert to tcg_ops restore_state_to_opc
target/rx: Convert to tcg_ops restore_state_to_opc
target/s390x: Convert to tcg_ops restore_state_to_opc
target/sh4: Convert to tcg_ops restore_state_to_opc
target/sparc: Convert to tcg_ops restore_state_to_opc
target/tricore: Convert to tcg_ops restore_state_to_opc
target/xtensa: Convert to tcg_ops restore_state_to_opc
accel/tcg: Remove restore_state_to_opc function
accel/tcg/internal.h | 91 ++++
include/exec/cpu-all.h | 22 +-
include/exec/exec-all.h | 35 +-
include/exec/ram_addr.h | 2 -
include/exec/translate-all.h | 2 +-
include/hw/core/tcg-cpu-ops.h | 11 +
include/qemu/atomic.h | 16 +-
include/qemu/osdep.h | 8 +
include/qemu/thread.h | 8 +-
target/arm/cpu.h | 8 +
target/arm/internals.h | 4 -
target/mips/tcg/tcg-internal.h | 3 +
target/s390x/s390x-internal.h | 4 +-
target/sparc/cpu.h | 3 +
tcg/loongarch64/tcg-target.h | 9 +-
accel/tcg/cpu-exec.c | 24 +-
accel/tcg/tb-maint.c | 704 ++++++++++++++++++++++++++
accel/tcg/tcg-accel-ops-mttcg.c | 5 +-
accel/tcg/tcg-accel-ops-rr.c | 7 +-
accel/tcg/translate-all.c | 1017 ++------------------------------------
accel/tcg/translator.c | 9 +-
accel/tcg/user-exec.c | 42 ++
bsd-user/mmap.c | 2 -
cpu.c | 4 +-
linux-user/mmap.c | 4 -
target/alpha/cpu.c | 9 +
target/alpha/translate.c | 6 -
target/arm/cpu.c | 26 +
target/arm/mte_helper.c | 5 -
target/arm/translate.c | 22 -
target/avr/cpu.c | 11 +
target/avr/translate.c | 6 -
target/cris/cpu.c | 11 +
target/cris/translate.c | 6 -
target/hexagon/cpu.c | 9 +-
target/hppa/cpu.c | 19 +
target/hppa/translate.c | 13 -
target/i386/tcg/tcg-cpu.c | 19 +
target/i386/tcg/translate.c | 15 -
target/loongarch/cpu.c | 11 +
target/loongarch/translate.c | 6 -
target/m68k/cpu.c | 14 +
target/m68k/translate.c | 10 -
target/microblaze/cpu.c | 11 +
target/microblaze/translate.c | 7 -
target/mips/cpu.c | 1 +
target/mips/tcg/translate.c | 8 +-
target/nios2/cpu.c | 11 +
target/nios2/translate.c | 6 -
target/openrisc/cpu.c | 13 +
target/openrisc/translate.c | 10 -
target/ppc/cpu_init.c | 10 +
target/ppc/translate.c | 6 -
target/riscv/cpu.c | 9 +-
target/rx/cpu.c | 10 +
target/rx/translate.c | 6 -
target/s390x/cpu.c | 1 +
target/s390x/tcg/translate.c | 7 +-
target/sh4/cpu.c | 16 +
target/sh4/translate.c | 10 -
target/sparc/cpu.c | 1 +
target/sparc/translate.c | 7 +-
target/tricore/cpu.c | 11 +
target/tricore/translate.c | 6 -
target/xtensa/cpu.c | 10 +
target/xtensa/translate.c | 6 -
tcg/aarch64/tcg-target.c.inc | 31 +-
tcg/loongarch64/tcg-target.c.inc | 48 +-
accel/tcg/meson.build | 1 +
69 files changed, 1304 insertions(+), 1221 deletions(-)
create mode 100644 accel/tcg/tb-maint.c
^ permalink raw reply [flat|nested] 56+ messages in thread
* [PULL 01/47] Revert "accel/tcg: Init TCG cflags in vCPU thread handler"
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 02/47] tcg/loongarch64: Add direct jump support Richard Henderson
` (46 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel
Cc: stefanha, Peter Maydell, qemu-stable, Philippe Mathieu-Daudé
From: Peter Maydell <peter.maydell@linaro.org>
Commit a82fd5a4ec24d was intended to be a code cleanup, but
unfortunately it has a bug. It moves the initialization of the
TCG cflags from the "start a new vcpu" function to the
thread handler; this is fine when each vcpu has its own thread,
but when we are doing round-robin of vcpus on a single thread
we end up only initializing the cflags for CPU 0, not for any
of the others.
The most obvious effect of this bug is that running in icount
mode with more than one CPU is broken; typically the guest
hangs shortly after it brings up the secondary CPUs.
This reverts commit a82fd5a4ec24d923ff1e6da128c0fd4a74079d99.
Cc: qemu-stable@nongnu.org
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Signed-off-by: Peter Maydell <peter.maydell@linaro.org>
Message-Id: <20221021163409.3674911-1-peter.maydell@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
accel/tcg/tcg-accel-ops-mttcg.c | 5 +++--
accel/tcg/tcg-accel-ops-rr.c | 7 ++++---
2 files changed, 7 insertions(+), 5 deletions(-)
diff --git a/accel/tcg/tcg-accel-ops-mttcg.c b/accel/tcg/tcg-accel-ops-mttcg.c
index ba997f6cfe..d50239e0e2 100644
--- a/accel/tcg/tcg-accel-ops-mttcg.c
+++ b/accel/tcg/tcg-accel-ops-mttcg.c
@@ -70,8 +70,6 @@ static void *mttcg_cpu_thread_fn(void *arg)
assert(tcg_enabled());
g_assert(!icount_enabled());
- tcg_cpu_init_cflags(cpu, current_machine->smp.max_cpus > 1);
-
rcu_register_thread();
force_rcu.notifier.notify = mttcg_force_rcu;
force_rcu.cpu = cpu;
@@ -141,6 +139,9 @@ void mttcg_start_vcpu_thread(CPUState *cpu)
{
char thread_name[VCPU_THREAD_NAME_SIZE];
+ g_assert(tcg_enabled());
+ tcg_cpu_init_cflags(cpu, current_machine->smp.max_cpus > 1);
+
cpu->thread = g_new0(QemuThread, 1);
cpu->halt_cond = g_malloc0(sizeof(QemuCond));
qemu_cond_init(cpu->halt_cond);
diff --git a/accel/tcg/tcg-accel-ops-rr.c b/accel/tcg/tcg-accel-ops-rr.c
index cc8adc2380..1a72149f0e 100644
--- a/accel/tcg/tcg-accel-ops-rr.c
+++ b/accel/tcg/tcg-accel-ops-rr.c
@@ -152,9 +152,7 @@ static void *rr_cpu_thread_fn(void *arg)
Notifier force_rcu;
CPUState *cpu = arg;
- g_assert(tcg_enabled());
- tcg_cpu_init_cflags(cpu, false);
-
+ assert(tcg_enabled());
rcu_register_thread();
force_rcu.notify = rr_force_rcu;
rcu_add_force_rcu_notifier(&force_rcu);
@@ -277,6 +275,9 @@ void rr_start_vcpu_thread(CPUState *cpu)
static QemuCond *single_tcg_halt_cond;
static QemuThread *single_tcg_cpu_thread;
+ g_assert(tcg_enabled());
+ tcg_cpu_init_cflags(cpu, false);
+
if (!single_tcg_cpu_thread) {
cpu->thread = g_new0(QemuThread, 1);
cpu->halt_cond = g_new0(QemuCond, 1);
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 02/47] tcg/loongarch64: Add direct jump support
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
2022-10-26 2:10 ` [PULL 01/47] Revert "accel/tcg: Init TCG cflags in vCPU thread handler" Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 03/47] tcg/aarch64: Remove unused code in tcg_out_op Richard Henderson
` (45 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Qi Hu, WANG Xuerui
From: Qi Hu <huqi@loongson.cn>
Similar to the ARM64, LoongArch has PC-relative instructions such as
PCADDU18I. These instructions can be used to support direct jump for
LoongArch. Additionally, if instruction "B offset" can cover the target
address(target is within ±128MB range), a single "B offset" plus a nop
will be used by "tb_target_set_jump_target".
Signed-off-by: Qi Hu <huqi@loongson.cn>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Reviewed-by: WANG Xuerui <git@xen0n.name>
Message-Id: <20221015092754.91971-1-huqi@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
tcg/loongarch64/tcg-target.h | 9 ++++--
tcg/loongarch64/tcg-target.c.inc | 48 +++++++++++++++++++++++++++++---
2 files changed, 50 insertions(+), 7 deletions(-)
diff --git a/tcg/loongarch64/tcg-target.h b/tcg/loongarch64/tcg-target.h
index d58a6162f2..a659c8d6fd 100644
--- a/tcg/loongarch64/tcg-target.h
+++ b/tcg/loongarch64/tcg-target.h
@@ -42,7 +42,11 @@
#define TCG_TARGET_INSN_UNIT_SIZE 4
#define TCG_TARGET_NB_REGS 32
-#define MAX_CODE_GEN_BUFFER_SIZE SIZE_MAX
+/*
+ * PCADDU18I + JIRL sequence can give 20 + 16 + 2 = 38 bits
+ * signed offset, which is +/- 128 GiB.
+ */
+#define MAX_CODE_GEN_BUFFER_SIZE (128 * GiB)
typedef enum {
TCG_REG_ZERO,
@@ -123,7 +127,7 @@ typedef enum {
#define TCG_TARGET_HAS_clz_i32 1
#define TCG_TARGET_HAS_ctz_i32 1
#define TCG_TARGET_HAS_ctpop_i32 0
-#define TCG_TARGET_HAS_direct_jump 0
+#define TCG_TARGET_HAS_direct_jump 1
#define TCG_TARGET_HAS_brcond2 0
#define TCG_TARGET_HAS_setcond2 0
#define TCG_TARGET_HAS_qemu_st8_i32 0
@@ -166,7 +170,6 @@ typedef enum {
#define TCG_TARGET_HAS_muluh_i64 1
#define TCG_TARGET_HAS_mulsh_i64 1
-/* not defined -- call should be eliminated at compile time */
void tb_target_set_jmp_target(uintptr_t, uintptr_t, uintptr_t, uintptr_t);
#define TCG_TARGET_DEFAULT_MO (0)
diff --git a/tcg/loongarch64/tcg-target.c.inc b/tcg/loongarch64/tcg-target.c.inc
index a3debf6da7..d326e28740 100644
--- a/tcg/loongarch64/tcg-target.c.inc
+++ b/tcg/loongarch64/tcg-target.c.inc
@@ -1031,6 +1031,36 @@ static void tcg_out_qemu_st(TCGContext *s, const TCGArg *args)
#endif
}
+/* LoongArch uses `andi zero, zero, 0` as NOP. */
+#define NOP OPC_ANDI
+static void tcg_out_nop(TCGContext *s)
+{
+ tcg_out32(s, NOP);
+}
+
+void tb_target_set_jmp_target(uintptr_t tc_ptr, uintptr_t jmp_rx,
+ uintptr_t jmp_rw, uintptr_t addr)
+{
+ tcg_insn_unit i1, i2;
+ ptrdiff_t upper, lower;
+ ptrdiff_t offset = (ptrdiff_t)(addr - jmp_rx) >> 2;
+
+ if (offset == sextreg(offset, 0, 26)) {
+ i1 = encode_sd10k16_insn(OPC_B, offset);
+ i2 = NOP;
+ } else {
+ tcg_debug_assert(offset == sextreg(offset, 0, 36));
+ lower = (int16_t)offset;
+ upper = (offset - lower) >> 16;
+
+ i1 = encode_dsj20_insn(OPC_PCADDU18I, TCG_REG_TMP0, upper);
+ i2 = encode_djsk16_insn(OPC_JIRL, TCG_REG_ZERO, TCG_REG_TMP0, lower);
+ }
+ uint64_t pair = ((uint64_t)i2 << 32) | i1;
+ qatomic_set((uint64_t *)jmp_rw, pair);
+ flush_idcache_range(jmp_rx, jmp_rw, 8);
+}
+
/*
* Entry-points
*/
@@ -1058,10 +1088,20 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc,
break;
case INDEX_op_goto_tb:
- assert(s->tb_jmp_insn_offset == 0);
- /* indirect jump method */
- tcg_out_ld(s, TCG_TYPE_PTR, TCG_REG_TMP0, TCG_REG_ZERO,
- (uintptr_t)(s->tb_jmp_target_addr + a0));
+ tcg_debug_assert(s->tb_jmp_insn_offset != NULL);
+ /*
+ * Ensure that patch area is 8-byte aligned so that an
+ * atomic write can be used to patch the target address.
+ */
+ if ((uintptr_t)s->code_ptr & 7) {
+ tcg_out_nop(s);
+ }
+ s->tb_jmp_insn_offset[a0] = tcg_current_code_size(s);
+ /*
+ * actual branch destination will be patched by
+ * tb_target_set_jmp_target later
+ */
+ tcg_out_opc_pcaddu18i(s, TCG_REG_TMP0, 0);
tcg_out_opc_jirl(s, TCG_REG_ZERO, TCG_REG_TMP0, 0);
set_jmp_reset_offset(s, a0);
break;
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 03/47] tcg/aarch64: Remove unused code in tcg_out_op
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
2022-10-26 2:10 ` [PULL 01/47] Revert "accel/tcg: Init TCG cflags in vCPU thread handler" Richard Henderson
2022-10-26 2:10 ` [PULL 02/47] tcg/loongarch64: Add direct jump support Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 04/47] accel/tcg: Add a quicker check for breakpoints Richard Henderson
` (44 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Qi Hu, WANG Xuerui, Philippe Mathieu-Daudé
From: Qi Hu <huqi@loongson.cn>
AArch64 defines the TCG_TARGET_HAS_direct_jump. So the "else" block is
useless in the case of "INDEX_op_goto_tb" in function "tcg_out_op". Add
an assertion and delete these codes for clarity.
Suggested-by: WANG Xuerui <git@xen0n.name>
Signed-off-by: Qi Hu <huqi@loongson.cn>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20221017020826.990729-1-huqi@loongson.cn>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
tcg/aarch64/tcg-target.c.inc | 31 ++++++++++++++-----------------
1 file changed, 14 insertions(+), 17 deletions(-)
diff --git a/tcg/aarch64/tcg-target.c.inc b/tcg/aarch64/tcg-target.c.inc
index d997f7922a..344b63e20f 100644
--- a/tcg/aarch64/tcg-target.c.inc
+++ b/tcg/aarch64/tcg-target.c.inc
@@ -1916,24 +1916,21 @@ static void tcg_out_op(TCGContext *s, TCGOpcode opc,
break;
case INDEX_op_goto_tb:
- if (s->tb_jmp_insn_offset != NULL) {
- /* TCG_TARGET_HAS_direct_jump */
- /* Ensure that ADRP+ADD are 8-byte aligned so that an atomic
- write can be used to patch the target address. */
- if ((uintptr_t)s->code_ptr & 7) {
- tcg_out32(s, NOP);
- }
- s->tb_jmp_insn_offset[a0] = tcg_current_code_size(s);
- /* actual branch destination will be patched by
- tb_target_set_jmp_target later. */
- tcg_out_insn(s, 3406, ADRP, TCG_REG_TMP, 0);
- tcg_out_insn(s, 3401, ADDI, TCG_TYPE_I64, TCG_REG_TMP, TCG_REG_TMP, 0);
- } else {
- /* !TCG_TARGET_HAS_direct_jump */
- tcg_debug_assert(s->tb_jmp_target_addr != NULL);
- intptr_t offset = tcg_pcrel_diff(s, (s->tb_jmp_target_addr + a0)) >> 2;
- tcg_out_insn(s, 3305, LDR, offset, TCG_REG_TMP);
+ tcg_debug_assert(s->tb_jmp_insn_offset != NULL);
+ /*
+ * Ensure that ADRP+ADD are 8-byte aligned so that an atomic
+ * write can be used to patch the target address.
+ */
+ if ((uintptr_t)s->code_ptr & 7) {
+ tcg_out32(s, NOP);
}
+ s->tb_jmp_insn_offset[a0] = tcg_current_code_size(s);
+ /*
+ * actual branch destination will be patched by
+ * tb_target_set_jmp_target later
+ */
+ tcg_out_insn(s, 3406, ADRP, TCG_REG_TMP, 0);
+ tcg_out_insn(s, 3401, ADDI, TCG_TYPE_I64, TCG_REG_TMP, TCG_REG_TMP, 0);
tcg_out_insn(s, 3207, BR, TCG_REG_TMP);
set_jmp_reset_offset(s, a0);
break;
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 04/47] accel/tcg: Add a quicker check for breakpoints
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (2 preceding siblings ...)
2022-10-26 2:10 ` [PULL 03/47] tcg/aarch64: Remove unused code in tcg_out_op Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 05/47] include/qemu/osdep: Add qemu_build_assert Richard Henderson
` (43 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Leandro Lupori
From: Leandro Lupori <leandro.lupori@eldorado.org.br>
Profiling QEMU during Fedora 35 for PPC64 boot revealed that a
considerable amount of time was being spent in
check_for_breakpoints() (0.61% of total time on PPC64 and 2.19% on
amd64), even though it was just checking that its queue was empty
and returning, when no breakpoints were set. It turns out this
function is not inlined by the compiler and it's always called by
helper_lookup_tb_ptr(), one of the most called functions.
By leaving only the check for empty queue in
check_for_breakpoints() and moving the remaining code to
check_for_breakpoints_slow(), called only when the queue is not
empty, it's possible to avoid the call overhead. An improvement of
about 3% in total time was measured on POWER9.
Signed-off-by: Leandro Lupori <leandro.lupori@eldorado.org.br>
Reviewed-by: Richard Henderson <richard.henderson@linaro.org>
Message-Id: <20221025202424.195984-2-leandro.lupori@eldorado.org.br>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
accel/tcg/cpu-exec.c | 15 +++++++++------
1 file changed, 9 insertions(+), 6 deletions(-)
diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index f9e5cc9ba0..bb4b9e92ce 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -304,16 +304,12 @@ static void log_cpu_exec(target_ulong pc, CPUState *cpu,
}
}
-static bool check_for_breakpoints(CPUState *cpu, target_ulong pc,
- uint32_t *cflags)
+static bool check_for_breakpoints_slow(CPUState *cpu, target_ulong pc,
+ uint32_t *cflags)
{
CPUBreakpoint *bp;
bool match_page = false;
- if (likely(QTAILQ_EMPTY(&cpu->breakpoints))) {
- return false;
- }
-
/*
* Singlestep overrides breakpoints.
* This requirement is visible in the record-replay tests, where
@@ -374,6 +370,13 @@ static bool check_for_breakpoints(CPUState *cpu, target_ulong pc,
return false;
}
+static inline bool check_for_breakpoints(CPUState *cpu, target_ulong pc,
+ uint32_t *cflags)
+{
+ return unlikely(!QTAILQ_EMPTY(&cpu->breakpoints)) &&
+ check_for_breakpoints_slow(cpu, pc, cflags);
+}
+
/**
* helper_lookup_tb_ptr: quick check for next tb
* @env: current cpu state
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 05/47] include/qemu/osdep: Add qemu_build_assert
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (3 preceding siblings ...)
2022-10-26 2:10 ` [PULL 04/47] accel/tcg: Add a quicker check for breakpoints Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 06/47] include/qemu/atomic: Use qemu_build_assert Richard Henderson
` (42 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Paolo Bonzini, Philippe Mathieu-Daudé
This differs from assert, in that with optimization enabled it
triggers at build-time. It differs from QEMU_BUILD_BUG_ON,
aka _Static_assert, in that it is sensitive to control flow
and is subject to dead-code elimination.
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
include/qemu/osdep.h | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/include/qemu/osdep.h b/include/qemu/osdep.h
index b1c161c035..2276094729 100644
--- a/include/qemu/osdep.h
+++ b/include/qemu/osdep.h
@@ -186,6 +186,14 @@ void QEMU_ERROR("code path is reachable")
#define qemu_build_not_reached() g_assert_not_reached()
#endif
+/**
+ * qemu_build_assert()
+ *
+ * The compiler, during optimization, is expected to prove that the
+ * assertion is true.
+ */
+#define qemu_build_assert(test) while (!(test)) qemu_build_not_reached()
+
/*
* According to waitpid man page:
* WCOREDUMP
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 06/47] include/qemu/atomic: Use qemu_build_assert
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (4 preceding siblings ...)
2022-10-26 2:10 ` [PULL 05/47] include/qemu/osdep: Add qemu_build_assert Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 07/47] include/qemu/thread: Use qatomic_* functions Richard Henderson
` (41 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Paolo Bonzini, Philippe Mathieu-Daudé
Change from QEMU_BUILD_BUG_ON, which requires ifdefs to avoid
problematic code, to qemu_build_assert, which can use C ifs.
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
include/qemu/atomic.h | 16 ++++++++--------
1 file changed, 8 insertions(+), 8 deletions(-)
diff --git a/include/qemu/atomic.h b/include/qemu/atomic.h
index 7e8fc8e7cd..874134fd19 100644
--- a/include/qemu/atomic.h
+++ b/include/qemu/atomic.h
@@ -133,7 +133,7 @@
#define qatomic_read(ptr) \
({ \
- QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
+ qemu_build_assert(sizeof(*ptr) <= ATOMIC_REG_SIZE); \
qatomic_read__nocheck(ptr); \
})
@@ -141,7 +141,7 @@
__atomic_store_n(ptr, i, __ATOMIC_RELAXED)
#define qatomic_set(ptr, i) do { \
- QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
+ qemu_build_assert(sizeof(*ptr) <= ATOMIC_REG_SIZE); \
qatomic_set__nocheck(ptr, i); \
} while(0)
@@ -159,27 +159,27 @@
#define qatomic_rcu_read(ptr) \
({ \
- QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
+ qemu_build_assert(sizeof(*ptr) <= ATOMIC_REG_SIZE); \
typeof_strip_qual(*ptr) _val; \
qatomic_rcu_read__nocheck(ptr, &_val); \
_val; \
})
#define qatomic_rcu_set(ptr, i) do { \
- QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
+ qemu_build_assert(sizeof(*ptr) <= ATOMIC_REG_SIZE); \
__atomic_store_n(ptr, i, __ATOMIC_RELEASE); \
} while(0)
#define qatomic_load_acquire(ptr) \
({ \
- QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
+ qemu_build_assert(sizeof(*ptr) <= ATOMIC_REG_SIZE); \
typeof_strip_qual(*ptr) _val; \
__atomic_load(ptr, &_val, __ATOMIC_ACQUIRE); \
_val; \
})
#define qatomic_store_release(ptr, i) do { \
- QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
+ qemu_build_assert(sizeof(*ptr) <= ATOMIC_REG_SIZE); \
__atomic_store_n(ptr, i, __ATOMIC_RELEASE); \
} while(0)
@@ -191,7 +191,7 @@
})
#define qatomic_xchg(ptr, i) ({ \
- QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
+ qemu_build_assert(sizeof(*ptr) <= ATOMIC_REG_SIZE); \
qatomic_xchg__nocheck(ptr, i); \
})
@@ -204,7 +204,7 @@
})
#define qatomic_cmpxchg(ptr, old, new) ({ \
- QEMU_BUILD_BUG_ON(sizeof(*ptr) > ATOMIC_REG_SIZE); \
+ qemu_build_assert(sizeof(*ptr) <= ATOMIC_REG_SIZE); \
qatomic_cmpxchg__nocheck(ptr, old, new); \
})
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 07/47] include/qemu/thread: Use qatomic_* functions
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (5 preceding siblings ...)
2022-10-26 2:10 ` [PULL 06/47] include/qemu/atomic: Use qemu_build_assert Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 08/47] accel/tcg: Make page_alloc_target_data allocation constant Richard Henderson
` (40 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Paolo Bonzini
Use qatomic_*, which expands to __atomic_* in preference
to the "legacy" __sync_* functions.
Acked-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
include/qemu/thread.h | 8 ++++----
1 file changed, 4 insertions(+), 4 deletions(-)
diff --git a/include/qemu/thread.h b/include/qemu/thread.h
index af19f2b3fc..20641e5844 100644
--- a/include/qemu/thread.h
+++ b/include/qemu/thread.h
@@ -227,7 +227,7 @@ struct QemuSpin {
static inline void qemu_spin_init(QemuSpin *spin)
{
- __sync_lock_release(&spin->value);
+ qatomic_set(&spin->value, 0);
#ifdef CONFIG_TSAN
__tsan_mutex_create(spin, __tsan_mutex_not_static);
#endif
@@ -246,7 +246,7 @@ static inline void qemu_spin_lock(QemuSpin *spin)
#ifdef CONFIG_TSAN
__tsan_mutex_pre_lock(spin, 0);
#endif
- while (unlikely(__sync_lock_test_and_set(&spin->value, true))) {
+ while (unlikely(qatomic_xchg(&spin->value, 1))) {
while (qatomic_read(&spin->value)) {
cpu_relax();
}
@@ -261,7 +261,7 @@ static inline bool qemu_spin_trylock(QemuSpin *spin)
#ifdef CONFIG_TSAN
__tsan_mutex_pre_lock(spin, __tsan_mutex_try_lock);
#endif
- bool busy = __sync_lock_test_and_set(&spin->value, true);
+ bool busy = qatomic_xchg(&spin->value, true);
#ifdef CONFIG_TSAN
unsigned flags = __tsan_mutex_try_lock;
flags |= busy ? __tsan_mutex_try_lock_failed : 0;
@@ -280,7 +280,7 @@ static inline void qemu_spin_unlock(QemuSpin *spin)
#ifdef CONFIG_TSAN
__tsan_mutex_pre_unlock(spin, 0);
#endif
- __sync_lock_release(&spin->value);
+ qatomic_store_release(&spin->value, 0);
#ifdef CONFIG_TSAN
__tsan_mutex_post_unlock(spin, 0);
#endif
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 08/47] accel/tcg: Make page_alloc_target_data allocation constant
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (6 preceding siblings ...)
2022-10-26 2:10 ` [PULL 07/47] include/qemu/thread: Use qatomic_* functions Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 09/47] accel/tcg: Remove disabled debug in translate-all.c Richard Henderson
` (39 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Alex Bennée
Use a constant target data allocation size for all pages.
This will be necessary to reduce overhead of page tracking.
Since TARGET_PAGE_DATA_SIZE is now required, we can use this
to omit data tracking for targets that don't require it.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
include/exec/cpu-all.h | 9 ++++-----
target/arm/cpu.h | 8 ++++++++
target/arm/internals.h | 4 ----
accel/tcg/translate-all.c | 8 ++++++--
target/arm/mte_helper.c | 3 +--
5 files changed, 19 insertions(+), 13 deletions(-)
diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
index 16b7df41bf..854adc4ac2 100644
--- a/include/exec/cpu-all.h
+++ b/include/exec/cpu-all.h
@@ -281,19 +281,18 @@ void page_reset_target_data(target_ulong start, target_ulong end);
int page_check_range(target_ulong start, target_ulong len, int flags);
/**
- * page_alloc_target_data(address, size)
+ * page_alloc_target_data(address)
* @address: guest virtual address
- * @size: size of data to allocate
*
- * Allocate @size bytes of out-of-band data to associate with the
- * guest page at @address. If the page is not mapped, NULL will
+ * Allocate TARGET_PAGE_DATA_SIZE bytes of out-of-band data to associate
+ * with the guest page at @address. If the page is not mapped, NULL will
* be returned. If there is existing data associated with @address,
* no new memory will be allocated.
*
* The memory will be freed when the guest page is deallocated,
* e.g. with the munmap system call.
*/
-void *page_alloc_target_data(target_ulong address, size_t size);
+void *page_alloc_target_data(target_ulong address);
/**
* page_get_target_data(address)
diff --git a/target/arm/cpu.h b/target/arm/cpu.h
index 64fc03214c..db9ec6a038 100644
--- a/target/arm/cpu.h
+++ b/target/arm/cpu.h
@@ -3410,6 +3410,14 @@ extern const uint64_t pred_esz_masks[5];
#define PAGE_MTE PAGE_TARGET_2
#define PAGE_TARGET_STICKY PAGE_MTE
+/* We associate one allocation tag per 16 bytes, the minimum. */
+#define LOG2_TAG_GRANULE 4
+#define TAG_GRANULE (1 << LOG2_TAG_GRANULE)
+
+#ifdef CONFIG_USER_ONLY
+#define TARGET_PAGE_DATA_SIZE (TARGET_PAGE_SIZE >> (LOG2_TAG_GRANULE + 1))
+#endif
+
#ifdef TARGET_TAGGED_ADDRESSES
/**
* cpu_untagged_addr:
diff --git a/target/arm/internals.h b/target/arm/internals.h
index c3c3920ded..b26c9ca17b 100644
--- a/target/arm/internals.h
+++ b/target/arm/internals.h
@@ -1164,10 +1164,6 @@ void arm_log_exception(CPUState *cs);
*/
#define GMID_EL1_BS 6
-/* We associate one allocation tag per 16 bytes, the minimum. */
-#define LOG2_TAG_GRANULE 4
-#define TAG_GRANULE (1 << LOG2_TAG_GRANULE)
-
/*
* SVE predicates are 1/8 the size of SVE vectors, and cannot use
* the same simd_desc() encoding due to restrictions on size.
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index 4ed75a13e1..64a2601f9f 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -2271,6 +2271,7 @@ void page_set_flags(target_ulong start, target_ulong end, int flags)
void page_reset_target_data(target_ulong start, target_ulong end)
{
+#ifdef TARGET_PAGE_DATA_SIZE
target_ulong addr, len;
/*
@@ -2293,15 +2294,17 @@ void page_reset_target_data(target_ulong start, target_ulong end)
g_free(p->target_data);
p->target_data = NULL;
}
+#endif
}
+#ifdef TARGET_PAGE_DATA_SIZE
void *page_get_target_data(target_ulong address)
{
PageDesc *p = page_find(address >> TARGET_PAGE_BITS);
return p ? p->target_data : NULL;
}
-void *page_alloc_target_data(target_ulong address, size_t size)
+void *page_alloc_target_data(target_ulong address)
{
PageDesc *p = page_find(address >> TARGET_PAGE_BITS);
void *ret = NULL;
@@ -2309,11 +2312,12 @@ void *page_alloc_target_data(target_ulong address, size_t size)
if (p->flags & PAGE_VALID) {
ret = p->target_data;
if (!ret) {
- p->target_data = ret = g_malloc0(size);
+ p->target_data = ret = g_malloc0(TARGET_PAGE_DATA_SIZE);
}
}
return ret;
}
+#endif /* TARGET_PAGE_DATA_SIZE */
int page_check_range(target_ulong start, target_ulong len, int flags)
{
diff --git a/target/arm/mte_helper.c b/target/arm/mte_helper.c
index e85208339e..a9c5fc2cb2 100644
--- a/target/arm/mte_helper.c
+++ b/target/arm/mte_helper.c
@@ -96,8 +96,7 @@ static uint8_t *allocation_tag_mem(CPUARMState *env, int ptr_mmu_idx,
tags = page_get_target_data(clean_ptr);
if (tags == NULL) {
- size_t alloc_size = TARGET_PAGE_SIZE >> (LOG2_TAG_GRANULE + 1);
- tags = page_alloc_target_data(clean_ptr, alloc_size);
+ tags = page_alloc_target_data(clean_ptr);
assert(tags != NULL);
}
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 09/47] accel/tcg: Remove disabled debug in translate-all.c
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (7 preceding siblings ...)
2022-10-26 2:10 ` [PULL 08/47] accel/tcg: Make page_alloc_target_data allocation constant Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 10/47] accel/tcg: Split out PageDesc to internal.h Richard Henderson
` (38 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Alex Bennée
These items printf, and could be replaced with proper
tracepoints if we really cared.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
accel/tcg/translate-all.c | 109 --------------------------------------
1 file changed, 109 deletions(-)
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index 64a2601f9f..42385fa032 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -63,33 +63,7 @@
#include "tb-context.h"
#include "internal.h"
-/* #define DEBUG_TB_INVALIDATE */
-/* #define DEBUG_TB_FLUSH */
/* make various TB consistency checks */
-/* #define DEBUG_TB_CHECK */
-
-#ifdef DEBUG_TB_INVALIDATE
-#define DEBUG_TB_INVALIDATE_GATE 1
-#else
-#define DEBUG_TB_INVALIDATE_GATE 0
-#endif
-
-#ifdef DEBUG_TB_FLUSH
-#define DEBUG_TB_FLUSH_GATE 1
-#else
-#define DEBUG_TB_FLUSH_GATE 0
-#endif
-
-#if !defined(CONFIG_USER_ONLY)
-/* TB consistency checks only implemented for usermode emulation. */
-#undef DEBUG_TB_CHECK
-#endif
-
-#ifdef DEBUG_TB_CHECK
-#define DEBUG_TB_CHECK_GATE 1
-#else
-#define DEBUG_TB_CHECK_GATE 0
-#endif
/* Access to the various translations structures need to be serialised via locks
* for consistency.
@@ -940,15 +914,6 @@ static void page_flush_tb(void)
}
}
-static gboolean tb_host_size_iter(gpointer key, gpointer value, gpointer data)
-{
- const TranslationBlock *tb = value;
- size_t *size = data;
-
- *size += tb->tc.size;
- return false;
-}
-
/* flush all the translation blocks */
static void do_tb_flush(CPUState *cpu, run_on_cpu_data tb_flush_count)
{
@@ -963,15 +928,6 @@ static void do_tb_flush(CPUState *cpu, run_on_cpu_data tb_flush_count)
}
did_flush = true;
- if (DEBUG_TB_FLUSH_GATE) {
- size_t nb_tbs = tcg_nb_tbs();
- size_t host_size = 0;
-
- tcg_tb_foreach(tb_host_size_iter, &host_size);
- printf("qemu: flush code_size=%zu nb_tbs=%zu avg_tb_size=%zu\n",
- tcg_code_size(), nb_tbs, nb_tbs > 0 ? host_size / nb_tbs : 0);
- }
-
CPU_FOREACH(cpu) {
tcg_flush_jmp_cache(cpu);
}
@@ -1005,57 +961,6 @@ void tb_flush(CPUState *cpu)
}
}
-/*
- * Formerly ifdef DEBUG_TB_CHECK. These debug functions are user-mode-only,
- * so in order to prevent bit rot we compile them unconditionally in user-mode,
- * and let the optimizer get rid of them by wrapping their user-only callers
- * with if (DEBUG_TB_CHECK_GATE).
- */
-#ifdef CONFIG_USER_ONLY
-
-static void do_tb_invalidate_check(void *p, uint32_t hash, void *userp)
-{
- TranslationBlock *tb = p;
- target_ulong addr = *(target_ulong *)userp;
-
- if (!(addr + TARGET_PAGE_SIZE <= tb_pc(tb) ||
- addr >= tb_pc(tb) + tb->size)) {
- printf("ERROR invalidate: address=" TARGET_FMT_lx
- " PC=%08lx size=%04x\n", addr, (long)tb_pc(tb), tb->size);
- }
-}
-
-/* verify that all the pages have correct rights for code
- *
- * Called with mmap_lock held.
- */
-static void tb_invalidate_check(target_ulong address)
-{
- address &= TARGET_PAGE_MASK;
- qht_iter(&tb_ctx.htable, do_tb_invalidate_check, &address);
-}
-
-static void do_tb_page_check(void *p, uint32_t hash, void *userp)
-{
- TranslationBlock *tb = p;
- int flags1, flags2;
-
- flags1 = page_get_flags(tb_pc(tb));
- flags2 = page_get_flags(tb_pc(tb) + tb->size - 1);
- if ((flags1 & PAGE_WRITE) || (flags2 & PAGE_WRITE)) {
- printf("ERROR page flags: PC=%08lx size=%04x f1=%x f2=%x\n",
- (long)tb_pc(tb), tb->size, flags1, flags2);
- }
-}
-
-/* verify that all the pages have correct rights for code */
-static void tb_page_check(void)
-{
- qht_iter(&tb_ctx.htable, do_tb_page_check, NULL);
-}
-
-#endif /* CONFIG_USER_ONLY */
-
/*
* user-mode: call with mmap_lock held
* !user-mode: call with @pd->lock held
@@ -1339,12 +1244,6 @@ tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc,
page_unlock(p2);
}
page_unlock(p);
-
-#ifdef CONFIG_USER_ONLY
- if (DEBUG_TB_CHECK_GATE) {
- tb_page_check();
- }
-#endif
return tb;
}
@@ -2400,9 +2299,6 @@ void page_protect(tb_page_addr_t page_addr)
}
mprotect(g2h_untagged(page_addr), qemu_host_page_size,
(prot & PAGE_BITS) & ~PAGE_WRITE);
- if (DEBUG_TB_INVALIDATE_GATE) {
- printf("protecting code page: 0x" TB_PAGE_ADDR_FMT "\n", page_addr);
- }
}
}
@@ -2458,11 +2354,6 @@ int page_unprotect(target_ulong address, uintptr_t pc)
/* and since the content will be modified, we must invalidate
the corresponding translated code. */
current_tb_invalidated |= tb_invalidate_phys_page(addr, pc);
-#ifdef CONFIG_USER_ONLY
- if (DEBUG_TB_CHECK_GATE) {
- tb_invalidate_check(addr);
- }
-#endif
}
mprotect((void *)g2h_untagged(host_start), qemu_host_page_size,
prot & PAGE_BITS);
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 10/47] accel/tcg: Split out PageDesc to internal.h
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (8 preceding siblings ...)
2022-10-26 2:10 ` [PULL 09/47] accel/tcg: Remove disabled debug in translate-all.c Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 11/47] accel/tcg: Split out tb-maint.c Richard Henderson
` (37 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Alex Bennée
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
accel/tcg/internal.h | 31 +++++++++++++++++++++++++++++++
accel/tcg/translate-all.c | 31 +------------------------------
2 files changed, 32 insertions(+), 30 deletions(-)
diff --git a/accel/tcg/internal.h b/accel/tcg/internal.h
index dc800fd485..62da49ed52 100644
--- a/accel/tcg/internal.h
+++ b/accel/tcg/internal.h
@@ -11,6 +11,37 @@
#include "exec/exec-all.h"
+/*
+ * Access to the various translations structures need to be serialised
+ * via locks for consistency. In user-mode emulation access to the
+ * memory related structures are protected with mmap_lock.
+ * In !user-mode we use per-page locks.
+ */
+#ifdef CONFIG_SOFTMMU
+#define assert_memory_lock()
+#else
+#define assert_memory_lock() tcg_debug_assert(have_mmap_lock())
+#endif
+
+typedef struct PageDesc {
+ /* list of TBs intersecting this ram page */
+ uintptr_t first_tb;
+#ifdef CONFIG_USER_ONLY
+ unsigned long flags;
+ void *target_data;
+#endif
+#ifdef CONFIG_SOFTMMU
+ QemuSpin lock;
+#endif
+} PageDesc;
+
+PageDesc *page_find_alloc(tb_page_addr_t index, bool alloc);
+
+static inline PageDesc *page_find(tb_page_addr_t index)
+{
+ return page_find_alloc(index, false);
+}
+
TranslationBlock *tb_gen_code(CPUState *cpu, target_ulong pc,
target_ulong cs_base, uint32_t flags,
int cflags);
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index 42385fa032..86848c6743 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -65,30 +65,6 @@
/* make various TB consistency checks */
-/* Access to the various translations structures need to be serialised via locks
- * for consistency.
- * In user-mode emulation access to the memory related structures are protected
- * with mmap_lock.
- * In !user-mode we use per-page locks.
- */
-#ifdef CONFIG_SOFTMMU
-#define assert_memory_lock()
-#else
-#define assert_memory_lock() tcg_debug_assert(have_mmap_lock())
-#endif
-
-typedef struct PageDesc {
- /* list of TBs intersecting this ram page */
- uintptr_t first_tb;
-#ifdef CONFIG_USER_ONLY
- unsigned long flags;
- void *target_data;
-#endif
-#ifdef CONFIG_SOFTMMU
- QemuSpin lock;
-#endif
-} PageDesc;
-
/**
* struct page_entry - page descriptor entry
* @pd: pointer to the &struct PageDesc of the page this entry represents
@@ -445,7 +421,7 @@ void page_init(void)
#endif
}
-static PageDesc *page_find_alloc(tb_page_addr_t index, bool alloc)
+PageDesc *page_find_alloc(tb_page_addr_t index, bool alloc)
{
PageDesc *pd;
void **lp;
@@ -511,11 +487,6 @@ static PageDesc *page_find_alloc(tb_page_addr_t index, bool alloc)
return pd + (index & (V_L2_SIZE - 1));
}
-static inline PageDesc *page_find(tb_page_addr_t index)
-{
- return page_find_alloc(index, false);
-}
-
static void page_lock_pair(PageDesc **ret_p1, tb_page_addr_t phys1,
PageDesc **ret_p2, tb_page_addr_t phys2, bool alloc);
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 11/47] accel/tcg: Split out tb-maint.c
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (9 preceding siblings ...)
2022-10-26 2:10 ` [PULL 10/47] accel/tcg: Split out PageDesc to internal.h Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 12/47] accel/tcg: Move assert_no_pages_locked to internal.h Richard Henderson
` (36 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Alex Bennée
Move all of the TranslationBlock flushing and page linking
code from translate-all.c to tb-maint.c.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
accel/tcg/internal.h | 55 +++
accel/tcg/tb-maint.c | 735 ++++++++++++++++++++++++++++++++++++
accel/tcg/translate-all.c | 766 +-------------------------------------
accel/tcg/meson.build | 1 +
4 files changed, 802 insertions(+), 755 deletions(-)
create mode 100644 accel/tcg/tb-maint.c
diff --git a/accel/tcg/internal.h b/accel/tcg/internal.h
index 62da49ed52..a77b110b78 100644
--- a/accel/tcg/internal.h
+++ b/accel/tcg/internal.h
@@ -35,6 +35,27 @@ typedef struct PageDesc {
#endif
} PageDesc;
+/* Size of the L2 (and L3, etc) page tables. */
+#define V_L2_BITS 10
+#define V_L2_SIZE (1 << V_L2_BITS)
+
+/*
+ * L1 Mapping properties
+ */
+extern int v_l1_size;
+extern int v_l1_shift;
+extern int v_l2_levels;
+
+/*
+ * The bottom level has pointers to PageDesc, and is indexed by
+ * anything from 4 to (V_L2_BITS + 3) bits, depending on target page size.
+ */
+#define V_L1_MIN_BITS 4
+#define V_L1_MAX_BITS (V_L2_BITS + 3)
+#define V_L1_MAX_SIZE (1 << V_L1_MAX_BITS)
+
+extern void *l1_map[V_L1_MAX_SIZE];
+
PageDesc *page_find_alloc(tb_page_addr_t index, bool alloc);
static inline PageDesc *page_find(tb_page_addr_t index)
@@ -42,12 +63,46 @@ static inline PageDesc *page_find(tb_page_addr_t index)
return page_find_alloc(index, false);
}
+/* list iterators for lists of tagged pointers in TranslationBlock */
+#define TB_FOR_EACH_TAGGED(head, tb, n, field) \
+ for (n = (head) & 1, tb = (TranslationBlock *)((head) & ~1); \
+ tb; tb = (TranslationBlock *)tb->field[n], n = (uintptr_t)tb & 1, \
+ tb = (TranslationBlock *)((uintptr_t)tb & ~1))
+
+#define PAGE_FOR_EACH_TB(pagedesc, tb, n) \
+ TB_FOR_EACH_TAGGED((pagedesc)->first_tb, tb, n, page_next)
+
+#define TB_FOR_EACH_JMP(head_tb, tb, n) \
+ TB_FOR_EACH_TAGGED((head_tb)->jmp_list_head, tb, n, jmp_list_next)
+
+/* In user-mode page locks aren't used; mmap_lock is enough */
+#ifdef CONFIG_USER_ONLY
+#define assert_page_locked(pd) tcg_debug_assert(have_mmap_lock())
+static inline void page_lock(PageDesc *pd) { }
+static inline void page_unlock(PageDesc *pd) { }
+#else
+#ifdef CONFIG_DEBUG_TCG
+void do_assert_page_locked(const PageDesc *pd, const char *file, int line);
+#define assert_page_locked(pd) do_assert_page_locked(pd, __FILE__, __LINE__)
+#else
+#define assert_page_locked(pd)
+#endif
+void page_lock(PageDesc *pd);
+void page_unlock(PageDesc *pd);
+#endif
+
TranslationBlock *tb_gen_code(CPUState *cpu, target_ulong pc,
target_ulong cs_base, uint32_t flags,
int cflags);
G_NORETURN void cpu_io_recompile(CPUState *cpu, uintptr_t retaddr);
void page_init(void);
void tb_htable_init(void);
+void tb_reset_jump(TranslationBlock *tb, int n);
+TranslationBlock *tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc,
+ tb_page_addr_t phys_page2);
+bool tb_invalidate_phys_page(tb_page_addr_t addr, uintptr_t pc);
+int cpu_restore_state_from_tb(CPUState *cpu, TranslationBlock *tb,
+ uintptr_t searched_pc, bool reset_icount);
/* Return the current PC from CPU, which may be cached in TB. */
static inline target_ulong log_pc(CPUState *cpu, const TranslationBlock *tb)
diff --git a/accel/tcg/tb-maint.c b/accel/tcg/tb-maint.c
new file mode 100644
index 0000000000..66c1900ae6
--- /dev/null
+++ b/accel/tcg/tb-maint.c
@@ -0,0 +1,735 @@
+/*
+ * Translation Block Maintaince
+ *
+ * Copyright (c) 2003 Fabrice Bellard
+ *
+ * This library is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU Lesser General Public
+ * License as published by the Free Software Foundation; either
+ * version 2.1 of the License, or (at your option) any later version.
+ *
+ * This library is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * Lesser General Public License for more details.
+ *
+ * You should have received a copy of the GNU Lesser General Public
+ * License along with this library; if not, see <http://www.gnu.org/licenses/>.
+ */
+
+#include "qemu/osdep.h"
+#include "exec/cputlb.h"
+#include "exec/log.h"
+#include "exec/translate-all.h"
+#include "sysemu/tcg.h"
+#include "tcg/tcg.h"
+#include "tb-hash.h"
+#include "tb-context.h"
+#include "internal.h"
+
+/* FIXME: tb_invalidate_phys_range is declared in different places. */
+#ifdef CONFIG_USER_ONLY
+#include "exec/exec-all.h"
+#else
+#include "exec/ram_addr.h"
+#endif
+
+static bool tb_cmp(const void *ap, const void *bp)
+{
+ const TranslationBlock *a = ap;
+ const TranslationBlock *b = bp;
+
+ return ((TARGET_TB_PCREL || tb_pc(a) == tb_pc(b)) &&
+ a->cs_base == b->cs_base &&
+ a->flags == b->flags &&
+ (tb_cflags(a) & ~CF_INVALID) == (tb_cflags(b) & ~CF_INVALID) &&
+ a->trace_vcpu_dstate == b->trace_vcpu_dstate &&
+ a->page_addr[0] == b->page_addr[0] &&
+ a->page_addr[1] == b->page_addr[1]);
+}
+
+void tb_htable_init(void)
+{
+ unsigned int mode = QHT_MODE_AUTO_RESIZE;
+
+ qht_init(&tb_ctx.htable, tb_cmp, CODE_GEN_HTABLE_SIZE, mode);
+}
+
+/* Set to NULL all the 'first_tb' fields in all PageDescs. */
+static void page_flush_tb_1(int level, void **lp)
+{
+ int i;
+
+ if (*lp == NULL) {
+ return;
+ }
+ if (level == 0) {
+ PageDesc *pd = *lp;
+
+ for (i = 0; i < V_L2_SIZE; ++i) {
+ page_lock(&pd[i]);
+ pd[i].first_tb = (uintptr_t)NULL;
+ page_unlock(&pd[i]);
+ }
+ } else {
+ void **pp = *lp;
+
+ for (i = 0; i < V_L2_SIZE; ++i) {
+ page_flush_tb_1(level - 1, pp + i);
+ }
+ }
+}
+
+static void page_flush_tb(void)
+{
+ int i, l1_sz = v_l1_size;
+
+ for (i = 0; i < l1_sz; i++) {
+ page_flush_tb_1(v_l2_levels, l1_map + i);
+ }
+}
+
+/* flush all the translation blocks */
+static void do_tb_flush(CPUState *cpu, run_on_cpu_data tb_flush_count)
+{
+ bool did_flush = false;
+
+ mmap_lock();
+ /* If it is already been done on request of another CPU, just retry. */
+ if (tb_ctx.tb_flush_count != tb_flush_count.host_int) {
+ goto done;
+ }
+ did_flush = true;
+
+ CPU_FOREACH(cpu) {
+ tcg_flush_jmp_cache(cpu);
+ }
+
+ qht_reset_size(&tb_ctx.htable, CODE_GEN_HTABLE_SIZE);
+ page_flush_tb();
+
+ tcg_region_reset_all();
+ /* XXX: flush processor icache at this point if cache flush is expensive */
+ qatomic_mb_set(&tb_ctx.tb_flush_count, tb_ctx.tb_flush_count + 1);
+
+done:
+ mmap_unlock();
+ if (did_flush) {
+ qemu_plugin_flush_cb();
+ }
+}
+
+void tb_flush(CPUState *cpu)
+{
+ if (tcg_enabled()) {
+ unsigned tb_flush_count = qatomic_mb_read(&tb_ctx.tb_flush_count);
+
+ if (cpu_in_exclusive_context(cpu)) {
+ do_tb_flush(cpu, RUN_ON_CPU_HOST_INT(tb_flush_count));
+ } else {
+ async_safe_run_on_cpu(cpu, do_tb_flush,
+ RUN_ON_CPU_HOST_INT(tb_flush_count));
+ }
+ }
+}
+
+/*
+ * user-mode: call with mmap_lock held
+ * !user-mode: call with @pd->lock held
+ */
+static inline void tb_page_remove(PageDesc *pd, TranslationBlock *tb)
+{
+ TranslationBlock *tb1;
+ uintptr_t *pprev;
+ unsigned int n1;
+
+ assert_page_locked(pd);
+ pprev = &pd->first_tb;
+ PAGE_FOR_EACH_TB(pd, tb1, n1) {
+ if (tb1 == tb) {
+ *pprev = tb1->page_next[n1];
+ return;
+ }
+ pprev = &tb1->page_next[n1];
+ }
+ g_assert_not_reached();
+}
+
+/* remove @orig from its @n_orig-th jump list */
+static inline void tb_remove_from_jmp_list(TranslationBlock *orig, int n_orig)
+{
+ uintptr_t ptr, ptr_locked;
+ TranslationBlock *dest;
+ TranslationBlock *tb;
+ uintptr_t *pprev;
+ int n;
+
+ /* mark the LSB of jmp_dest[] so that no further jumps can be inserted */
+ ptr = qatomic_or_fetch(&orig->jmp_dest[n_orig], 1);
+ dest = (TranslationBlock *)(ptr & ~1);
+ if (dest == NULL) {
+ return;
+ }
+
+ qemu_spin_lock(&dest->jmp_lock);
+ /*
+ * While acquiring the lock, the jump might have been removed if the
+ * destination TB was invalidated; check again.
+ */
+ ptr_locked = qatomic_read(&orig->jmp_dest[n_orig]);
+ if (ptr_locked != ptr) {
+ qemu_spin_unlock(&dest->jmp_lock);
+ /*
+ * The only possibility is that the jump was unlinked via
+ * tb_jump_unlink(dest). Seeing here another destination would be a bug,
+ * because we set the LSB above.
+ */
+ g_assert(ptr_locked == 1 && dest->cflags & CF_INVALID);
+ return;
+ }
+ /*
+ * We first acquired the lock, and since the destination pointer matches,
+ * we know for sure that @orig is in the jmp list.
+ */
+ pprev = &dest->jmp_list_head;
+ TB_FOR_EACH_JMP(dest, tb, n) {
+ if (tb == orig && n == n_orig) {
+ *pprev = tb->jmp_list_next[n];
+ /* no need to set orig->jmp_dest[n]; setting the LSB was enough */
+ qemu_spin_unlock(&dest->jmp_lock);
+ return;
+ }
+ pprev = &tb->jmp_list_next[n];
+ }
+ g_assert_not_reached();
+}
+
+/*
+ * Reset the jump entry 'n' of a TB so that it is not chained to another TB.
+ */
+void tb_reset_jump(TranslationBlock *tb, int n)
+{
+ uintptr_t addr = (uintptr_t)(tb->tc.ptr + tb->jmp_reset_offset[n]);
+ tb_set_jmp_target(tb, n, addr);
+}
+
+/* remove any jumps to the TB */
+static inline void tb_jmp_unlink(TranslationBlock *dest)
+{
+ TranslationBlock *tb;
+ int n;
+
+ qemu_spin_lock(&dest->jmp_lock);
+
+ TB_FOR_EACH_JMP(dest, tb, n) {
+ tb_reset_jump(tb, n);
+ qatomic_and(&tb->jmp_dest[n], (uintptr_t)NULL | 1);
+ /* No need to clear the list entry; setting the dest ptr is enough */
+ }
+ dest->jmp_list_head = (uintptr_t)NULL;
+
+ qemu_spin_unlock(&dest->jmp_lock);
+}
+
+static void tb_jmp_cache_inval_tb(TranslationBlock *tb)
+{
+ CPUState *cpu;
+
+ if (TARGET_TB_PCREL) {
+ /* A TB may be at any virtual address */
+ CPU_FOREACH(cpu) {
+ tcg_flush_jmp_cache(cpu);
+ }
+ } else {
+ uint32_t h = tb_jmp_cache_hash_func(tb_pc(tb));
+
+ CPU_FOREACH(cpu) {
+ CPUJumpCache *jc = cpu->tb_jmp_cache;
+
+ if (qatomic_read(&jc->array[h].tb) == tb) {
+ qatomic_set(&jc->array[h].tb, NULL);
+ }
+ }
+ }
+}
+
+/*
+ * In user-mode, call with mmap_lock held.
+ * In !user-mode, if @rm_from_page_list is set, call with the TB's pages'
+ * locks held.
+ */
+static void do_tb_phys_invalidate(TranslationBlock *tb, bool rm_from_page_list)
+{
+ PageDesc *p;
+ uint32_t h;
+ tb_page_addr_t phys_pc;
+ uint32_t orig_cflags = tb_cflags(tb);
+
+ assert_memory_lock();
+
+ /* make sure no further incoming jumps will be chained to this TB */
+ qemu_spin_lock(&tb->jmp_lock);
+ qatomic_set(&tb->cflags, tb->cflags | CF_INVALID);
+ qemu_spin_unlock(&tb->jmp_lock);
+
+ /* remove the TB from the hash list */
+ phys_pc = tb->page_addr[0];
+ h = tb_hash_func(phys_pc, (TARGET_TB_PCREL ? 0 : tb_pc(tb)),
+ tb->flags, orig_cflags, tb->trace_vcpu_dstate);
+ if (!qht_remove(&tb_ctx.htable, tb, h)) {
+ return;
+ }
+
+ /* remove the TB from the page list */
+ if (rm_from_page_list) {
+ p = page_find(tb->page_addr[0] >> TARGET_PAGE_BITS);
+ tb_page_remove(p, tb);
+ if (tb->page_addr[1] != -1) {
+ p = page_find(tb->page_addr[1] >> TARGET_PAGE_BITS);
+ tb_page_remove(p, tb);
+ }
+ }
+
+ /* remove the TB from the hash list */
+ tb_jmp_cache_inval_tb(tb);
+
+ /* suppress this TB from the two jump lists */
+ tb_remove_from_jmp_list(tb, 0);
+ tb_remove_from_jmp_list(tb, 1);
+
+ /* suppress any remaining jumps to this TB */
+ tb_jmp_unlink(tb);
+
+ qatomic_set(&tb_ctx.tb_phys_invalidate_count,
+ tb_ctx.tb_phys_invalidate_count + 1);
+}
+
+static void tb_phys_invalidate__locked(TranslationBlock *tb)
+{
+ qemu_thread_jit_write();
+ do_tb_phys_invalidate(tb, true);
+ qemu_thread_jit_execute();
+}
+
+static void page_lock_pair(PageDesc **ret_p1, tb_page_addr_t phys1,
+ PageDesc **ret_p2, tb_page_addr_t phys2, bool alloc)
+{
+ PageDesc *p1, *p2;
+ tb_page_addr_t page1;
+ tb_page_addr_t page2;
+
+ assert_memory_lock();
+ g_assert(phys1 != -1);
+
+ page1 = phys1 >> TARGET_PAGE_BITS;
+ page2 = phys2 >> TARGET_PAGE_BITS;
+
+ p1 = page_find_alloc(page1, alloc);
+ if (ret_p1) {
+ *ret_p1 = p1;
+ }
+ if (likely(phys2 == -1)) {
+ page_lock(p1);
+ return;
+ } else if (page1 == page2) {
+ page_lock(p1);
+ if (ret_p2) {
+ *ret_p2 = p1;
+ }
+ return;
+ }
+ p2 = page_find_alloc(page2, alloc);
+ if (ret_p2) {
+ *ret_p2 = p2;
+ }
+ if (page1 < page2) {
+ page_lock(p1);
+ page_lock(p2);
+ } else {
+ page_lock(p2);
+ page_lock(p1);
+ }
+}
+
+#ifdef CONFIG_USER_ONLY
+static inline void page_lock_tb(const TranslationBlock *tb) { }
+static inline void page_unlock_tb(const TranslationBlock *tb) { }
+#else
+/* lock the page(s) of a TB in the correct acquisition order */
+static void page_lock_tb(const TranslationBlock *tb)
+{
+ page_lock_pair(NULL, tb->page_addr[0], NULL, tb->page_addr[1], false);
+}
+
+static void page_unlock_tb(const TranslationBlock *tb)
+{
+ PageDesc *p1 = page_find(tb->page_addr[0] >> TARGET_PAGE_BITS);
+
+ page_unlock(p1);
+ if (unlikely(tb->page_addr[1] != -1)) {
+ PageDesc *p2 = page_find(tb->page_addr[1] >> TARGET_PAGE_BITS);
+
+ if (p2 != p1) {
+ page_unlock(p2);
+ }
+ }
+}
+#endif
+
+/*
+ * Invalidate one TB.
+ * Called with mmap_lock held in user-mode.
+ */
+void tb_phys_invalidate(TranslationBlock *tb, tb_page_addr_t page_addr)
+{
+ if (page_addr == -1 && tb->page_addr[0] != -1) {
+ page_lock_tb(tb);
+ do_tb_phys_invalidate(tb, true);
+ page_unlock_tb(tb);
+ } else {
+ do_tb_phys_invalidate(tb, false);
+ }
+}
+
+/*
+ * Add the tb in the target page and protect it if necessary.
+ * Called with mmap_lock held for user-mode emulation.
+ * Called with @p->lock held in !user-mode.
+ */
+static inline void tb_page_add(PageDesc *p, TranslationBlock *tb,
+ unsigned int n, tb_page_addr_t page_addr)
+{
+#ifndef CONFIG_USER_ONLY
+ bool page_already_protected;
+#endif
+
+ assert_page_locked(p);
+
+ tb->page_addr[n] = page_addr;
+ tb->page_next[n] = p->first_tb;
+#ifndef CONFIG_USER_ONLY
+ page_already_protected = p->first_tb != (uintptr_t)NULL;
+#endif
+ p->first_tb = (uintptr_t)tb | n;
+
+#if defined(CONFIG_USER_ONLY)
+ /* translator_loop() must have made all TB pages non-writable */
+ assert(!(p->flags & PAGE_WRITE));
+#else
+ /*
+ * If some code is already present, then the pages are already
+ * protected. So we handle the case where only the first TB is
+ * allocated in a physical page.
+ */
+ if (!page_already_protected) {
+ tlb_protect_code(page_addr);
+ }
+#endif
+}
+
+/*
+ * Add a new TB and link it to the physical page tables. phys_page2 is
+ * (-1) to indicate that only one page contains the TB.
+ *
+ * Called with mmap_lock held for user-mode emulation.
+ *
+ * Returns a pointer @tb, or a pointer to an existing TB that matches @tb.
+ * Note that in !user-mode, another thread might have already added a TB
+ * for the same block of guest code that @tb corresponds to. In that case,
+ * the caller should discard the original @tb, and use instead the returned TB.
+ */
+TranslationBlock *tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc,
+ tb_page_addr_t phys_page2)
+{
+ PageDesc *p;
+ PageDesc *p2 = NULL;
+ void *existing_tb = NULL;
+ uint32_t h;
+
+ assert_memory_lock();
+ tcg_debug_assert(!(tb->cflags & CF_INVALID));
+
+ /*
+ * Add the TB to the page list, acquiring first the pages's locks.
+ * We keep the locks held until after inserting the TB in the hash table,
+ * so that if the insertion fails we know for sure that the TBs are still
+ * in the page descriptors.
+ * Note that inserting into the hash table first isn't an option, since
+ * we can only insert TBs that are fully initialized.
+ */
+ page_lock_pair(&p, phys_pc, &p2, phys_page2, true);
+ tb_page_add(p, tb, 0, phys_pc);
+ if (p2) {
+ tb_page_add(p2, tb, 1, phys_page2);
+ } else {
+ tb->page_addr[1] = -1;
+ }
+
+ /* add in the hash table */
+ h = tb_hash_func(phys_pc, (TARGET_TB_PCREL ? 0 : tb_pc(tb)),
+ tb->flags, tb->cflags, tb->trace_vcpu_dstate);
+ qht_insert(&tb_ctx.htable, tb, h, &existing_tb);
+
+ /* remove TB from the page(s) if we couldn't insert it */
+ if (unlikely(existing_tb)) {
+ tb_page_remove(p, tb);
+ if (p2) {
+ tb_page_remove(p2, tb);
+ }
+ tb = existing_tb;
+ }
+
+ if (p2 && p2 != p) {
+ page_unlock(p2);
+ }
+ page_unlock(p);
+ return tb;
+}
+
+/*
+ * @p must be non-NULL.
+ * user-mode: call with mmap_lock held.
+ * !user-mode: call with all @pages locked.
+ */
+static void
+tb_invalidate_phys_page_range__locked(struct page_collection *pages,
+ PageDesc *p, tb_page_addr_t start,
+ tb_page_addr_t end,
+ uintptr_t retaddr)
+{
+ TranslationBlock *tb;
+ tb_page_addr_t tb_start, tb_end;
+ int n;
+#ifdef TARGET_HAS_PRECISE_SMC
+ CPUState *cpu = current_cpu;
+ CPUArchState *env = NULL;
+ bool current_tb_not_found = retaddr != 0;
+ bool current_tb_modified = false;
+ TranslationBlock *current_tb = NULL;
+ target_ulong current_pc = 0;
+ target_ulong current_cs_base = 0;
+ uint32_t current_flags = 0;
+#endif /* TARGET_HAS_PRECISE_SMC */
+
+ assert_page_locked(p);
+
+#if defined(TARGET_HAS_PRECISE_SMC)
+ if (cpu != NULL) {
+ env = cpu->env_ptr;
+ }
+#endif
+
+ /*
+ * We remove all the TBs in the range [start, end[.
+ * XXX: see if in some cases it could be faster to invalidate all the code
+ */
+ PAGE_FOR_EACH_TB(p, tb, n) {
+ assert_page_locked(p);
+ /* NOTE: this is subtle as a TB may span two physical pages */
+ if (n == 0) {
+ /* NOTE: tb_end may be after the end of the page, but
+ it is not a problem */
+ tb_start = tb->page_addr[0];
+ tb_end = tb_start + tb->size;
+ } else {
+ tb_start = tb->page_addr[1];
+ tb_end = tb_start + ((tb->page_addr[0] + tb->size)
+ & ~TARGET_PAGE_MASK);
+ }
+ if (!(tb_end <= start || tb_start >= end)) {
+#ifdef TARGET_HAS_PRECISE_SMC
+ if (current_tb_not_found) {
+ current_tb_not_found = false;
+ /* now we have a real cpu fault */
+ current_tb = tcg_tb_lookup(retaddr);
+ }
+ if (current_tb == tb &&
+ (tb_cflags(current_tb) & CF_COUNT_MASK) != 1) {
+ /*
+ * If we are modifying the current TB, we must stop
+ * its execution. We could be more precise by checking
+ * that the modification is after the current PC, but it
+ * would require a specialized function to partially
+ * restore the CPU state.
+ */
+ current_tb_modified = true;
+ cpu_restore_state_from_tb(cpu, current_tb, retaddr, true);
+ cpu_get_tb_cpu_state(env, ¤t_pc, ¤t_cs_base,
+ ¤t_flags);
+ }
+#endif /* TARGET_HAS_PRECISE_SMC */
+ tb_phys_invalidate__locked(tb);
+ }
+ }
+#if !defined(CONFIG_USER_ONLY)
+ /* if no code remaining, no need to continue to use slow writes */
+ if (!p->first_tb) {
+ tlb_unprotect_code(start);
+ }
+#endif
+#ifdef TARGET_HAS_PRECISE_SMC
+ if (current_tb_modified) {
+ page_collection_unlock(pages);
+ /* Force execution of one insn next time. */
+ cpu->cflags_next_tb = 1 | CF_NOIRQ | curr_cflags(cpu);
+ mmap_unlock();
+ cpu_loop_exit_noexc(cpu);
+ }
+#endif
+}
+
+/*
+ * Invalidate all TBs which intersect with the target physical address range
+ * [start;end[. NOTE: start and end must refer to the *same* physical page.
+ * 'is_cpu_write_access' should be true if called from a real cpu write
+ * access: the virtual CPU will exit the current TB if code is modified inside
+ * this TB.
+ *
+ * Called with mmap_lock held for user-mode emulation
+ */
+void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end)
+{
+ struct page_collection *pages;
+ PageDesc *p;
+
+ assert_memory_lock();
+
+ p = page_find(start >> TARGET_PAGE_BITS);
+ if (p == NULL) {
+ return;
+ }
+ pages = page_collection_lock(start, end);
+ tb_invalidate_phys_page_range__locked(pages, p, start, end, 0);
+ page_collection_unlock(pages);
+}
+
+/*
+ * Invalidate all TBs which intersect with the target physical address range
+ * [start;end[. NOTE: start and end may refer to *different* physical pages.
+ * 'is_cpu_write_access' should be true if called from a real cpu write
+ * access: the virtual CPU will exit the current TB if code is modified inside
+ * this TB.
+ *
+ * Called with mmap_lock held for user-mode emulation.
+ */
+#ifdef CONFIG_SOFTMMU
+void tb_invalidate_phys_range(ram_addr_t start, ram_addr_t end)
+#else
+void tb_invalidate_phys_range(target_ulong start, target_ulong end)
+#endif
+{
+ struct page_collection *pages;
+ tb_page_addr_t next;
+
+ assert_memory_lock();
+
+ pages = page_collection_lock(start, end);
+ for (next = (start & TARGET_PAGE_MASK) + TARGET_PAGE_SIZE;
+ start < end;
+ start = next, next += TARGET_PAGE_SIZE) {
+ PageDesc *pd = page_find(start >> TARGET_PAGE_BITS);
+ tb_page_addr_t bound = MIN(next, end);
+
+ if (pd == NULL) {
+ continue;
+ }
+ tb_invalidate_phys_page_range__locked(pages, pd, start, bound, 0);
+ }
+ page_collection_unlock(pages);
+}
+
+#ifdef CONFIG_SOFTMMU
+/*
+ * len must be <= 8 and start must be a multiple of len.
+ * Called via softmmu_template.h when code areas are written to with
+ * iothread mutex not held.
+ *
+ * Call with all @pages in the range [@start, @start + len[ locked.
+ */
+void tb_invalidate_phys_page_fast(struct page_collection *pages,
+ tb_page_addr_t start, int len,
+ uintptr_t retaddr)
+{
+ PageDesc *p;
+
+ assert_memory_lock();
+
+ p = page_find(start >> TARGET_PAGE_BITS);
+ if (!p) {
+ return;
+ }
+
+ assert_page_locked(p);
+ tb_invalidate_phys_page_range__locked(pages, p, start, start + len,
+ retaddr);
+}
+#else
+/*
+ * Called with mmap_lock held. If pc is not 0 then it indicates the
+ * host PC of the faulting store instruction that caused this invalidate.
+ * Returns true if the caller needs to abort execution of the current
+ * TB (because it was modified by this store and the guest CPU has
+ * precise-SMC semantics).
+ */
+bool tb_invalidate_phys_page(tb_page_addr_t addr, uintptr_t pc)
+{
+ TranslationBlock *tb;
+ PageDesc *p;
+ int n;
+#ifdef TARGET_HAS_PRECISE_SMC
+ TranslationBlock *current_tb = NULL;
+ CPUState *cpu = current_cpu;
+ CPUArchState *env = NULL;
+ int current_tb_modified = 0;
+ target_ulong current_pc = 0;
+ target_ulong current_cs_base = 0;
+ uint32_t current_flags = 0;
+#endif
+
+ assert_memory_lock();
+
+ addr &= TARGET_PAGE_MASK;
+ p = page_find(addr >> TARGET_PAGE_BITS);
+ if (!p) {
+ return false;
+ }
+
+#ifdef TARGET_HAS_PRECISE_SMC
+ if (p->first_tb && pc != 0) {
+ current_tb = tcg_tb_lookup(pc);
+ }
+ if (cpu != NULL) {
+ env = cpu->env_ptr;
+ }
+#endif
+ assert_page_locked(p);
+ PAGE_FOR_EACH_TB(p, tb, n) {
+#ifdef TARGET_HAS_PRECISE_SMC
+ if (current_tb == tb &&
+ (tb_cflags(current_tb) & CF_COUNT_MASK) != 1) {
+ /*
+ * If we are modifying the current TB, we must stop its execution.
+ * We could be more precise by checking that the modification is
+ * after the current PC, but it would require a specialized
+ * function to partially restore the CPU state.
+ */
+ current_tb_modified = 1;
+ cpu_restore_state_from_tb(cpu, current_tb, pc, true);
+ cpu_get_tb_cpu_state(env, ¤t_pc, ¤t_cs_base,
+ ¤t_flags);
+ }
+#endif /* TARGET_HAS_PRECISE_SMC */
+ tb_phys_invalidate(tb, addr);
+ }
+ p->first_tb = (uintptr_t)NULL;
+#ifdef TARGET_HAS_PRECISE_SMC
+ if (current_tb_modified) {
+ /* Force execution of one insn next time. */
+ cpu->cflags_next_tb = 1 | CF_NOIRQ | curr_cflags(cpu);
+ return true;
+ }
+#endif
+
+ return false;
+}
+#endif
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index 86848c6743..5e28e9fccd 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -109,18 +109,6 @@ struct page_collection {
struct page_entry *max;
};
-/* list iterators for lists of tagged pointers in TranslationBlock */
-#define TB_FOR_EACH_TAGGED(head, tb, n, field) \
- for (n = (head) & 1, tb = (TranslationBlock *)((head) & ~1); \
- tb; tb = (TranslationBlock *)tb->field[n], n = (uintptr_t)tb & 1, \
- tb = (TranslationBlock *)((uintptr_t)tb & ~1))
-
-#define PAGE_FOR_EACH_TB(pagedesc, tb, n) \
- TB_FOR_EACH_TAGGED((pagedesc)->first_tb, tb, n, page_next)
-
-#define TB_FOR_EACH_JMP(head_tb, tb, n) \
- TB_FOR_EACH_TAGGED((head_tb)->jmp_list_head, tb, n, jmp_list_next)
-
/*
* In system mode we want L1_MAP to be based on ram offsets,
* while in user mode we want it to be based on virtual addresses.
@@ -138,10 +126,6 @@ struct page_collection {
# define L1_MAP_ADDR_SPACE_BITS MIN(HOST_LONG_BITS, TARGET_ABI_BITS)
#endif
-/* Size of the L2 (and L3, etc) page tables. */
-#define V_L2_BITS 10
-#define V_L2_SIZE (1 << V_L2_BITS)
-
/* Make sure all possible CPU event bits fit in tb->trace_vcpu_dstate */
QEMU_BUILD_BUG_ON(CPU_TRACE_DSTATE_MAX_EVENTS >
sizeof_field(TranslationBlock, trace_vcpu_dstate)
@@ -150,18 +134,11 @@ QEMU_BUILD_BUG_ON(CPU_TRACE_DSTATE_MAX_EVENTS >
/*
* L1 Mapping properties
*/
-static int v_l1_size;
-static int v_l1_shift;
-static int v_l2_levels;
+int v_l1_size;
+int v_l1_shift;
+int v_l2_levels;
-/* The bottom level has pointers to PageDesc, and is indexed by
- * anything from 4 to (V_L2_BITS + 3) bits, depending on target page size.
- */
-#define V_L1_MIN_BITS 4
-#define V_L1_MAX_BITS (V_L2_BITS + 3)
-#define V_L1_MAX_SIZE (1 << V_L1_MAX_BITS)
-
-static void *l1_map[V_L1_MAX_SIZE];
+void *l1_map[V_L1_MAX_SIZE];
TBContext tb_ctx;
@@ -274,8 +251,8 @@ static int encode_search(TranslationBlock *tb, uint8_t *block)
* When reset_icount is true, current TB will be interrupted and
* icount should be recalculated.
*/
-static int cpu_restore_state_from_tb(CPUState *cpu, TranslationBlock *tb,
- uintptr_t searched_pc, bool reset_icount)
+int cpu_restore_state_from_tb(CPUState *cpu, TranslationBlock *tb,
+ uintptr_t searched_pc, bool reset_icount)
{
target_ulong data[TARGET_INSN_START_WORDS];
uintptr_t host_pc = (uintptr_t)tb->tc.ptr;
@@ -487,26 +464,8 @@ PageDesc *page_find_alloc(tb_page_addr_t index, bool alloc)
return pd + (index & (V_L2_SIZE - 1));
}
-static void page_lock_pair(PageDesc **ret_p1, tb_page_addr_t phys1,
- PageDesc **ret_p2, tb_page_addr_t phys2, bool alloc);
-
/* In user-mode page locks aren't used; mmap_lock is enough */
#ifdef CONFIG_USER_ONLY
-
-#define assert_page_locked(pd) tcg_debug_assert(have_mmap_lock())
-
-static inline void page_lock(PageDesc *pd)
-{ }
-
-static inline void page_unlock(PageDesc *pd)
-{ }
-
-static inline void page_lock_tb(const TranslationBlock *tb)
-{ }
-
-static inline void page_unlock_tb(const TranslationBlock *tb)
-{ }
-
struct page_collection *
page_collection_lock(tb_page_addr_t start, tb_page_addr_t end)
{
@@ -555,8 +514,7 @@ static void page_unlock__debug(const PageDesc *pd)
g_assert(removed);
}
-static void
-do_assert_page_locked(const PageDesc *pd, const char *file, int line)
+void do_assert_page_locked(const PageDesc *pd, const char *file, int line)
{
if (unlikely(!page_is_locked(pd))) {
error_report("assert_page_lock: PageDesc %p not locked @ %s:%d",
@@ -565,8 +523,6 @@ do_assert_page_locked(const PageDesc *pd, const char *file, int line)
}
}
-#define assert_page_locked(pd) do_assert_page_locked(pd, __FILE__, __LINE__)
-
void assert_no_pages_locked(void)
{
ht_pages_locked_debug_init();
@@ -575,50 +531,23 @@ void assert_no_pages_locked(void)
#else /* !CONFIG_DEBUG_TCG */
-#define assert_page_locked(pd)
-
-static inline void page_lock__debug(const PageDesc *pd)
-{
-}
-
-static inline void page_unlock__debug(const PageDesc *pd)
-{
-}
+static inline void page_lock__debug(const PageDesc *pd) { }
+static inline void page_unlock__debug(const PageDesc *pd) { }
#endif /* CONFIG_DEBUG_TCG */
-static inline void page_lock(PageDesc *pd)
+void page_lock(PageDesc *pd)
{
page_lock__debug(pd);
qemu_spin_lock(&pd->lock);
}
-static inline void page_unlock(PageDesc *pd)
+void page_unlock(PageDesc *pd)
{
qemu_spin_unlock(&pd->lock);
page_unlock__debug(pd);
}
-/* lock the page(s) of a TB in the correct acquisition order */
-static inline void page_lock_tb(const TranslationBlock *tb)
-{
- page_lock_pair(NULL, tb->page_addr[0], NULL, tb->page_addr[1], false);
-}
-
-static inline void page_unlock_tb(const TranslationBlock *tb)
-{
- PageDesc *p1 = page_find(tb->page_addr[0] >> TARGET_PAGE_BITS);
-
- page_unlock(p1);
- if (unlikely(tb->page_addr[1] != -1)) {
- PageDesc *p2 = page_find(tb->page_addr[1] >> TARGET_PAGE_BITS);
-
- if (p2 != p1) {
- page_unlock(p2);
- }
- }
-}
-
static inline struct page_entry *
page_entry_new(PageDesc *pd, tb_page_addr_t index)
{
@@ -790,434 +719,6 @@ void page_collection_unlock(struct page_collection *set)
#endif /* !CONFIG_USER_ONLY */
-static void page_lock_pair(PageDesc **ret_p1, tb_page_addr_t phys1,
- PageDesc **ret_p2, tb_page_addr_t phys2, bool alloc)
-{
- PageDesc *p1, *p2;
- tb_page_addr_t page1;
- tb_page_addr_t page2;
-
- assert_memory_lock();
- g_assert(phys1 != -1);
-
- page1 = phys1 >> TARGET_PAGE_BITS;
- page2 = phys2 >> TARGET_PAGE_BITS;
-
- p1 = page_find_alloc(page1, alloc);
- if (ret_p1) {
- *ret_p1 = p1;
- }
- if (likely(phys2 == -1)) {
- page_lock(p1);
- return;
- } else if (page1 == page2) {
- page_lock(p1);
- if (ret_p2) {
- *ret_p2 = p1;
- }
- return;
- }
- p2 = page_find_alloc(page2, alloc);
- if (ret_p2) {
- *ret_p2 = p2;
- }
- if (page1 < page2) {
- page_lock(p1);
- page_lock(p2);
- } else {
- page_lock(p2);
- page_lock(p1);
- }
-}
-
-static bool tb_cmp(const void *ap, const void *bp)
-{
- const TranslationBlock *a = ap;
- const TranslationBlock *b = bp;
-
- return ((TARGET_TB_PCREL || tb_pc(a) == tb_pc(b)) &&
- a->cs_base == b->cs_base &&
- a->flags == b->flags &&
- (tb_cflags(a) & ~CF_INVALID) == (tb_cflags(b) & ~CF_INVALID) &&
- a->trace_vcpu_dstate == b->trace_vcpu_dstate &&
- a->page_addr[0] == b->page_addr[0] &&
- a->page_addr[1] == b->page_addr[1]);
-}
-
-void tb_htable_init(void)
-{
- unsigned int mode = QHT_MODE_AUTO_RESIZE;
-
- qht_init(&tb_ctx.htable, tb_cmp, CODE_GEN_HTABLE_SIZE, mode);
-}
-
-/* Set to NULL all the 'first_tb' fields in all PageDescs. */
-static void page_flush_tb_1(int level, void **lp)
-{
- int i;
-
- if (*lp == NULL) {
- return;
- }
- if (level == 0) {
- PageDesc *pd = *lp;
-
- for (i = 0; i < V_L2_SIZE; ++i) {
- page_lock(&pd[i]);
- pd[i].first_tb = (uintptr_t)NULL;
- page_unlock(&pd[i]);
- }
- } else {
- void **pp = *lp;
-
- for (i = 0; i < V_L2_SIZE; ++i) {
- page_flush_tb_1(level - 1, pp + i);
- }
- }
-}
-
-static void page_flush_tb(void)
-{
- int i, l1_sz = v_l1_size;
-
- for (i = 0; i < l1_sz; i++) {
- page_flush_tb_1(v_l2_levels, l1_map + i);
- }
-}
-
-/* flush all the translation blocks */
-static void do_tb_flush(CPUState *cpu, run_on_cpu_data tb_flush_count)
-{
- bool did_flush = false;
-
- mmap_lock();
- /* If it is already been done on request of another CPU,
- * just retry.
- */
- if (tb_ctx.tb_flush_count != tb_flush_count.host_int) {
- goto done;
- }
- did_flush = true;
-
- CPU_FOREACH(cpu) {
- tcg_flush_jmp_cache(cpu);
- }
-
- qht_reset_size(&tb_ctx.htable, CODE_GEN_HTABLE_SIZE);
- page_flush_tb();
-
- tcg_region_reset_all();
- /* XXX: flush processor icache at this point if cache flush is
- expensive */
- qatomic_mb_set(&tb_ctx.tb_flush_count, tb_ctx.tb_flush_count + 1);
-
-done:
- mmap_unlock();
- if (did_flush) {
- qemu_plugin_flush_cb();
- }
-}
-
-void tb_flush(CPUState *cpu)
-{
- if (tcg_enabled()) {
- unsigned tb_flush_count = qatomic_mb_read(&tb_ctx.tb_flush_count);
-
- if (cpu_in_exclusive_context(cpu)) {
- do_tb_flush(cpu, RUN_ON_CPU_HOST_INT(tb_flush_count));
- } else {
- async_safe_run_on_cpu(cpu, do_tb_flush,
- RUN_ON_CPU_HOST_INT(tb_flush_count));
- }
- }
-}
-
-/*
- * user-mode: call with mmap_lock held
- * !user-mode: call with @pd->lock held
- */
-static inline void tb_page_remove(PageDesc *pd, TranslationBlock *tb)
-{
- TranslationBlock *tb1;
- uintptr_t *pprev;
- unsigned int n1;
-
- assert_page_locked(pd);
- pprev = &pd->first_tb;
- PAGE_FOR_EACH_TB(pd, tb1, n1) {
- if (tb1 == tb) {
- *pprev = tb1->page_next[n1];
- return;
- }
- pprev = &tb1->page_next[n1];
- }
- g_assert_not_reached();
-}
-
-/* remove @orig from its @n_orig-th jump list */
-static inline void tb_remove_from_jmp_list(TranslationBlock *orig, int n_orig)
-{
- uintptr_t ptr, ptr_locked;
- TranslationBlock *dest;
- TranslationBlock *tb;
- uintptr_t *pprev;
- int n;
-
- /* mark the LSB of jmp_dest[] so that no further jumps can be inserted */
- ptr = qatomic_or_fetch(&orig->jmp_dest[n_orig], 1);
- dest = (TranslationBlock *)(ptr & ~1);
- if (dest == NULL) {
- return;
- }
-
- qemu_spin_lock(&dest->jmp_lock);
- /*
- * While acquiring the lock, the jump might have been removed if the
- * destination TB was invalidated; check again.
- */
- ptr_locked = qatomic_read(&orig->jmp_dest[n_orig]);
- if (ptr_locked != ptr) {
- qemu_spin_unlock(&dest->jmp_lock);
- /*
- * The only possibility is that the jump was unlinked via
- * tb_jump_unlink(dest). Seeing here another destination would be a bug,
- * because we set the LSB above.
- */
- g_assert(ptr_locked == 1 && dest->cflags & CF_INVALID);
- return;
- }
- /*
- * We first acquired the lock, and since the destination pointer matches,
- * we know for sure that @orig is in the jmp list.
- */
- pprev = &dest->jmp_list_head;
- TB_FOR_EACH_JMP(dest, tb, n) {
- if (tb == orig && n == n_orig) {
- *pprev = tb->jmp_list_next[n];
- /* no need to set orig->jmp_dest[n]; setting the LSB was enough */
- qemu_spin_unlock(&dest->jmp_lock);
- return;
- }
- pprev = &tb->jmp_list_next[n];
- }
- g_assert_not_reached();
-}
-
-/* reset the jump entry 'n' of a TB so that it is not chained to
- another TB */
-static inline void tb_reset_jump(TranslationBlock *tb, int n)
-{
- uintptr_t addr = (uintptr_t)(tb->tc.ptr + tb->jmp_reset_offset[n]);
- tb_set_jmp_target(tb, n, addr);
-}
-
-/* remove any jumps to the TB */
-static inline void tb_jmp_unlink(TranslationBlock *dest)
-{
- TranslationBlock *tb;
- int n;
-
- qemu_spin_lock(&dest->jmp_lock);
-
- TB_FOR_EACH_JMP(dest, tb, n) {
- tb_reset_jump(tb, n);
- qatomic_and(&tb->jmp_dest[n], (uintptr_t)NULL | 1);
- /* No need to clear the list entry; setting the dest ptr is enough */
- }
- dest->jmp_list_head = (uintptr_t)NULL;
-
- qemu_spin_unlock(&dest->jmp_lock);
-}
-
-static void tb_jmp_cache_inval_tb(TranslationBlock *tb)
-{
- CPUState *cpu;
-
- if (TARGET_TB_PCREL) {
- /* A TB may be at any virtual address */
- CPU_FOREACH(cpu) {
- tcg_flush_jmp_cache(cpu);
- }
- } else {
- uint32_t h = tb_jmp_cache_hash_func(tb_pc(tb));
-
- CPU_FOREACH(cpu) {
- CPUJumpCache *jc = cpu->tb_jmp_cache;
-
- if (qatomic_read(&jc->array[h].tb) == tb) {
- qatomic_set(&jc->array[h].tb, NULL);
- }
- }
- }
-}
-
-/*
- * In user-mode, call with mmap_lock held.
- * In !user-mode, if @rm_from_page_list is set, call with the TB's pages'
- * locks held.
- */
-static void do_tb_phys_invalidate(TranslationBlock *tb, bool rm_from_page_list)
-{
- PageDesc *p;
- uint32_t h;
- tb_page_addr_t phys_pc;
- uint32_t orig_cflags = tb_cflags(tb);
-
- assert_memory_lock();
-
- /* make sure no further incoming jumps will be chained to this TB */
- qemu_spin_lock(&tb->jmp_lock);
- qatomic_set(&tb->cflags, tb->cflags | CF_INVALID);
- qemu_spin_unlock(&tb->jmp_lock);
-
- /* remove the TB from the hash list */
- phys_pc = tb->page_addr[0];
- h = tb_hash_func(phys_pc, (TARGET_TB_PCREL ? 0 : tb_pc(tb)),
- tb->flags, orig_cflags, tb->trace_vcpu_dstate);
- if (!qht_remove(&tb_ctx.htable, tb, h)) {
- return;
- }
-
- /* remove the TB from the page list */
- if (rm_from_page_list) {
- p = page_find(tb->page_addr[0] >> TARGET_PAGE_BITS);
- tb_page_remove(p, tb);
- if (tb->page_addr[1] != -1) {
- p = page_find(tb->page_addr[1] >> TARGET_PAGE_BITS);
- tb_page_remove(p, tb);
- }
- }
-
- /* remove the TB from the hash list */
- tb_jmp_cache_inval_tb(tb);
-
- /* suppress this TB from the two jump lists */
- tb_remove_from_jmp_list(tb, 0);
- tb_remove_from_jmp_list(tb, 1);
-
- /* suppress any remaining jumps to this TB */
- tb_jmp_unlink(tb);
-
- qatomic_set(&tb_ctx.tb_phys_invalidate_count,
- tb_ctx.tb_phys_invalidate_count + 1);
-}
-
-static void tb_phys_invalidate__locked(TranslationBlock *tb)
-{
- qemu_thread_jit_write();
- do_tb_phys_invalidate(tb, true);
- qemu_thread_jit_execute();
-}
-
-/* invalidate one TB
- *
- * Called with mmap_lock held in user-mode.
- */
-void tb_phys_invalidate(TranslationBlock *tb, tb_page_addr_t page_addr)
-{
- if (page_addr == -1 && tb->page_addr[0] != -1) {
- page_lock_tb(tb);
- do_tb_phys_invalidate(tb, true);
- page_unlock_tb(tb);
- } else {
- do_tb_phys_invalidate(tb, false);
- }
-}
-
-/* add the tb in the target page and protect it if necessary
- *
- * Called with mmap_lock held for user-mode emulation.
- * Called with @p->lock held in !user-mode.
- */
-static inline void tb_page_add(PageDesc *p, TranslationBlock *tb,
- unsigned int n, tb_page_addr_t page_addr)
-{
-#ifndef CONFIG_USER_ONLY
- bool page_already_protected;
-#endif
-
- assert_page_locked(p);
-
- tb->page_addr[n] = page_addr;
- tb->page_next[n] = p->first_tb;
-#ifndef CONFIG_USER_ONLY
- page_already_protected = p->first_tb != (uintptr_t)NULL;
-#endif
- p->first_tb = (uintptr_t)tb | n;
-
-#if defined(CONFIG_USER_ONLY)
- /* translator_loop() must have made all TB pages non-writable */
- assert(!(p->flags & PAGE_WRITE));
-#else
- /* if some code is already present, then the pages are already
- protected. So we handle the case where only the first TB is
- allocated in a physical page */
- if (!page_already_protected) {
- tlb_protect_code(page_addr);
- }
-#endif
-}
-
-/*
- * Add a new TB and link it to the physical page tables. phys_page2 is
- * (-1) to indicate that only one page contains the TB.
- *
- * Called with mmap_lock held for user-mode emulation.
- *
- * Returns a pointer @tb, or a pointer to an existing TB that matches @tb.
- * Note that in !user-mode, another thread might have already added a TB
- * for the same block of guest code that @tb corresponds to. In that case,
- * the caller should discard the original @tb, and use instead the returned TB.
- */
-static TranslationBlock *
-tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc,
- tb_page_addr_t phys_page2)
-{
- PageDesc *p;
- PageDesc *p2 = NULL;
- void *existing_tb = NULL;
- uint32_t h;
-
- assert_memory_lock();
- tcg_debug_assert(!(tb->cflags & CF_INVALID));
-
- /*
- * Add the TB to the page list, acquiring first the pages's locks.
- * We keep the locks held until after inserting the TB in the hash table,
- * so that if the insertion fails we know for sure that the TBs are still
- * in the page descriptors.
- * Note that inserting into the hash table first isn't an option, since
- * we can only insert TBs that are fully initialized.
- */
- page_lock_pair(&p, phys_pc, &p2, phys_page2, true);
- tb_page_add(p, tb, 0, phys_pc);
- if (p2) {
- tb_page_add(p2, tb, 1, phys_page2);
- } else {
- tb->page_addr[1] = -1;
- }
-
- /* add in the hash table */
- h = tb_hash_func(phys_pc, (TARGET_TB_PCREL ? 0 : tb_pc(tb)),
- tb->flags, tb->cflags, tb->trace_vcpu_dstate);
- qht_insert(&tb_ctx.htable, tb, h, &existing_tb);
-
- /* remove TB from the page(s) if we couldn't insert it */
- if (unlikely(existing_tb)) {
- tb_page_remove(p, tb);
- if (p2) {
- tb_page_remove(p2, tb);
- }
- tb = existing_tb;
- }
-
- if (p2 && p2 != p) {
- page_unlock(p2);
- }
- page_unlock(p);
- return tb;
-}
-
/* Called with mmap_lock held for user mode emulation. */
TranslationBlock *tb_gen_code(CPUState *cpu,
target_ulong pc, target_ulong cs_base,
@@ -1497,251 +998,6 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
return tb;
}
-/*
- * @p must be non-NULL.
- * user-mode: call with mmap_lock held.
- * !user-mode: call with all @pages locked.
- */
-static void
-tb_invalidate_phys_page_range__locked(struct page_collection *pages,
- PageDesc *p, tb_page_addr_t start,
- tb_page_addr_t end,
- uintptr_t retaddr)
-{
- TranslationBlock *tb;
- tb_page_addr_t tb_start, tb_end;
- int n;
-#ifdef TARGET_HAS_PRECISE_SMC
- CPUState *cpu = current_cpu;
- CPUArchState *env = NULL;
- bool current_tb_not_found = retaddr != 0;
- bool current_tb_modified = false;
- TranslationBlock *current_tb = NULL;
- target_ulong current_pc = 0;
- target_ulong current_cs_base = 0;
- uint32_t current_flags = 0;
-#endif /* TARGET_HAS_PRECISE_SMC */
-
- assert_page_locked(p);
-
-#if defined(TARGET_HAS_PRECISE_SMC)
- if (cpu != NULL) {
- env = cpu->env_ptr;
- }
-#endif
-
- /* we remove all the TBs in the range [start, end[ */
- /* XXX: see if in some cases it could be faster to invalidate all
- the code */
- PAGE_FOR_EACH_TB(p, tb, n) {
- assert_page_locked(p);
- /* NOTE: this is subtle as a TB may span two physical pages */
- if (n == 0) {
- /* NOTE: tb_end may be after the end of the page, but
- it is not a problem */
- tb_start = tb->page_addr[0];
- tb_end = tb_start + tb->size;
- } else {
- tb_start = tb->page_addr[1];
- tb_end = tb_start + ((tb->page_addr[0] + tb->size)
- & ~TARGET_PAGE_MASK);
- }
- if (!(tb_end <= start || tb_start >= end)) {
-#ifdef TARGET_HAS_PRECISE_SMC
- if (current_tb_not_found) {
- current_tb_not_found = false;
- /* now we have a real cpu fault */
- current_tb = tcg_tb_lookup(retaddr);
- }
- if (current_tb == tb &&
- (tb_cflags(current_tb) & CF_COUNT_MASK) != 1) {
- /*
- * If we are modifying the current TB, we must stop
- * its execution. We could be more precise by checking
- * that the modification is after the current PC, but it
- * would require a specialized function to partially
- * restore the CPU state.
- */
- current_tb_modified = true;
- cpu_restore_state_from_tb(cpu, current_tb, retaddr, true);
- cpu_get_tb_cpu_state(env, ¤t_pc, ¤t_cs_base,
- ¤t_flags);
- }
-#endif /* TARGET_HAS_PRECISE_SMC */
- tb_phys_invalidate__locked(tb);
- }
- }
-#if !defined(CONFIG_USER_ONLY)
- /* if no code remaining, no need to continue to use slow writes */
- if (!p->first_tb) {
- tlb_unprotect_code(start);
- }
-#endif
-#ifdef TARGET_HAS_PRECISE_SMC
- if (current_tb_modified) {
- page_collection_unlock(pages);
- /* Force execution of one insn next time. */
- cpu->cflags_next_tb = 1 | CF_NOIRQ | curr_cflags(cpu);
- mmap_unlock();
- cpu_loop_exit_noexc(cpu);
- }
-#endif
-}
-
-/*
- * Invalidate all TBs which intersect with the target physical address range
- * [start;end[. NOTE: start and end must refer to the *same* physical page.
- * 'is_cpu_write_access' should be true if called from a real cpu write
- * access: the virtual CPU will exit the current TB if code is modified inside
- * this TB.
- *
- * Called with mmap_lock held for user-mode emulation
- */
-void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end)
-{
- struct page_collection *pages;
- PageDesc *p;
-
- assert_memory_lock();
-
- p = page_find(start >> TARGET_PAGE_BITS);
- if (p == NULL) {
- return;
- }
- pages = page_collection_lock(start, end);
- tb_invalidate_phys_page_range__locked(pages, p, start, end, 0);
- page_collection_unlock(pages);
-}
-
-/*
- * Invalidate all TBs which intersect with the target physical address range
- * [start;end[. NOTE: start and end may refer to *different* physical pages.
- * 'is_cpu_write_access' should be true if called from a real cpu write
- * access: the virtual CPU will exit the current TB if code is modified inside
- * this TB.
- *
- * Called with mmap_lock held for user-mode emulation.
- */
-#ifdef CONFIG_SOFTMMU
-void tb_invalidate_phys_range(ram_addr_t start, ram_addr_t end)
-#else
-void tb_invalidate_phys_range(target_ulong start, target_ulong end)
-#endif
-{
- struct page_collection *pages;
- tb_page_addr_t next;
-
- assert_memory_lock();
-
- pages = page_collection_lock(start, end);
- for (next = (start & TARGET_PAGE_MASK) + TARGET_PAGE_SIZE;
- start < end;
- start = next, next += TARGET_PAGE_SIZE) {
- PageDesc *pd = page_find(start >> TARGET_PAGE_BITS);
- tb_page_addr_t bound = MIN(next, end);
-
- if (pd == NULL) {
- continue;
- }
- tb_invalidate_phys_page_range__locked(pages, pd, start, bound, 0);
- }
- page_collection_unlock(pages);
-}
-
-#ifdef CONFIG_SOFTMMU
-/* len must be <= 8 and start must be a multiple of len.
- * Called via softmmu_template.h when code areas are written to with
- * iothread mutex not held.
- *
- * Call with all @pages in the range [@start, @start + len[ locked.
- */
-void tb_invalidate_phys_page_fast(struct page_collection *pages,
- tb_page_addr_t start, int len,
- uintptr_t retaddr)
-{
- PageDesc *p;
-
- assert_memory_lock();
-
- p = page_find(start >> TARGET_PAGE_BITS);
- if (!p) {
- return;
- }
-
- assert_page_locked(p);
- tb_invalidate_phys_page_range__locked(pages, p, start, start + len,
- retaddr);
-}
-#else
-/* Called with mmap_lock held. If pc is not 0 then it indicates the
- * host PC of the faulting store instruction that caused this invalidate.
- * Returns true if the caller needs to abort execution of the current
- * TB (because it was modified by this store and the guest CPU has
- * precise-SMC semantics).
- */
-static bool tb_invalidate_phys_page(tb_page_addr_t addr, uintptr_t pc)
-{
- TranslationBlock *tb;
- PageDesc *p;
- int n;
-#ifdef TARGET_HAS_PRECISE_SMC
- TranslationBlock *current_tb = NULL;
- CPUState *cpu = current_cpu;
- CPUArchState *env = NULL;
- int current_tb_modified = 0;
- target_ulong current_pc = 0;
- target_ulong current_cs_base = 0;
- uint32_t current_flags = 0;
-#endif
-
- assert_memory_lock();
-
- addr &= TARGET_PAGE_MASK;
- p = page_find(addr >> TARGET_PAGE_BITS);
- if (!p) {
- return false;
- }
-
-#ifdef TARGET_HAS_PRECISE_SMC
- if (p->first_tb && pc != 0) {
- current_tb = tcg_tb_lookup(pc);
- }
- if (cpu != NULL) {
- env = cpu->env_ptr;
- }
-#endif
- assert_page_locked(p);
- PAGE_FOR_EACH_TB(p, tb, n) {
-#ifdef TARGET_HAS_PRECISE_SMC
- if (current_tb == tb &&
- (tb_cflags(current_tb) & CF_COUNT_MASK) != 1) {
- /* If we are modifying the current TB, we must stop
- its execution. We could be more precise by checking
- that the modification is after the current PC, but it
- would require a specialized function to partially
- restore the CPU state */
-
- current_tb_modified = 1;
- cpu_restore_state_from_tb(cpu, current_tb, pc, true);
- cpu_get_tb_cpu_state(env, ¤t_pc, ¤t_cs_base,
- ¤t_flags);
- }
-#endif /* TARGET_HAS_PRECISE_SMC */
- tb_phys_invalidate(tb, addr);
- }
- p->first_tb = (uintptr_t)NULL;
-#ifdef TARGET_HAS_PRECISE_SMC
- if (current_tb_modified) {
- /* Force execution of one insn next time. */
- cpu->cflags_next_tb = 1 | CF_NOIRQ | curr_cflags(cpu);
- return true;
- }
-#endif
-
- return false;
-}
-#endif
-
/* user-mode: call with mmap_lock held */
void tb_check_watchpoint(CPUState *cpu, uintptr_t retaddr)
{
diff --git a/accel/tcg/meson.build b/accel/tcg/meson.build
index 7a0a79d731..75e1dffb4d 100644
--- a/accel/tcg/meson.build
+++ b/accel/tcg/meson.build
@@ -3,6 +3,7 @@ tcg_ss.add(files(
'tcg-all.c',
'cpu-exec-common.c',
'cpu-exec.c',
+ 'tb-maint.c',
'tcg-runtime-gvec.c',
'tcg-runtime.c',
'translate-all.c',
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 12/47] accel/tcg: Move assert_no_pages_locked to internal.h
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (10 preceding siblings ...)
2022-10-26 2:10 ` [PULL 11/47] accel/tcg: Split out tb-maint.c Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 13/47] accel/tcg: Drop cpu_get_tb_cpu_state from TARGET_HAS_PRECISE_SMC Richard Henderson
` (35 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Alex Bennée
There are no users outside of accel/tcg; this function
does not need to be defined in exec-all.h.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
accel/tcg/internal.h | 5 +++++
include/exec/exec-all.h | 8 --------
2 files changed, 5 insertions(+), 8 deletions(-)
diff --git a/accel/tcg/internal.h b/accel/tcg/internal.h
index a77b110b78..1a704ee14f 100644
--- a/accel/tcg/internal.h
+++ b/accel/tcg/internal.h
@@ -90,6 +90,11 @@ void do_assert_page_locked(const PageDesc *pd, const char *file, int line);
void page_lock(PageDesc *pd);
void page_unlock(PageDesc *pd);
#endif
+#if !defined(CONFIG_USER_ONLY) && defined(CONFIG_DEBUG_TCG)
+void assert_no_pages_locked(void);
+#else
+static inline void assert_no_pages_locked(void) { }
+#endif
TranslationBlock *tb_gen_code(CPUState *cpu, target_ulong pc,
target_ulong cs_base, uint32_t flags,
diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index e5f8b224a5..b5bde1b56a 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -642,14 +642,6 @@ extern __thread uintptr_t tci_tb_ptr;
smaller than 4 bytes, so we don't worry about special-casing this. */
#define GETPC_ADJ 2
-#if !defined(CONFIG_USER_ONLY) && defined(CONFIG_DEBUG_TCG)
-void assert_no_pages_locked(void);
-#else
-static inline void assert_no_pages_locked(void)
-{
-}
-#endif
-
#if !defined(CONFIG_USER_ONLY)
/**
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 13/47] accel/tcg: Drop cpu_get_tb_cpu_state from TARGET_HAS_PRECISE_SMC
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (11 preceding siblings ...)
2022-10-26 2:10 ` [PULL 12/47] accel/tcg: Move assert_no_pages_locked to internal.h Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 14/47] accel/tcg: Remove duplicate store to tb->page_addr[] Richard Henderson
` (34 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Alex Bennée
The results of the calls to cpu_get_tb_cpu_state,
current_{pc,cs_base,flags}, are not used.
In tb_invalidate_phys_page, use bool for current_tb_modified.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
accel/tcg/tb-maint.c | 25 ++-----------------------
1 file changed, 2 insertions(+), 23 deletions(-)
diff --git a/accel/tcg/tb-maint.c b/accel/tcg/tb-maint.c
index 66c1900ae6..9af5cb49e0 100644
--- a/accel/tcg/tb-maint.c
+++ b/accel/tcg/tb-maint.c
@@ -502,23 +502,13 @@ tb_invalidate_phys_page_range__locked(struct page_collection *pages,
int n;
#ifdef TARGET_HAS_PRECISE_SMC
CPUState *cpu = current_cpu;
- CPUArchState *env = NULL;
bool current_tb_not_found = retaddr != 0;
bool current_tb_modified = false;
TranslationBlock *current_tb = NULL;
- target_ulong current_pc = 0;
- target_ulong current_cs_base = 0;
- uint32_t current_flags = 0;
#endif /* TARGET_HAS_PRECISE_SMC */
assert_page_locked(p);
-#if defined(TARGET_HAS_PRECISE_SMC)
- if (cpu != NULL) {
- env = cpu->env_ptr;
- }
-#endif
-
/*
* We remove all the TBs in the range [start, end[.
* XXX: see if in some cases it could be faster to invalidate all the code
@@ -554,8 +544,6 @@ tb_invalidate_phys_page_range__locked(struct page_collection *pages,
*/
current_tb_modified = true;
cpu_restore_state_from_tb(cpu, current_tb, retaddr, true);
- cpu_get_tb_cpu_state(env, ¤t_pc, ¤t_cs_base,
- ¤t_flags);
}
#endif /* TARGET_HAS_PRECISE_SMC */
tb_phys_invalidate__locked(tb);
@@ -679,11 +667,7 @@ bool tb_invalidate_phys_page(tb_page_addr_t addr, uintptr_t pc)
#ifdef TARGET_HAS_PRECISE_SMC
TranslationBlock *current_tb = NULL;
CPUState *cpu = current_cpu;
- CPUArchState *env = NULL;
- int current_tb_modified = 0;
- target_ulong current_pc = 0;
- target_ulong current_cs_base = 0;
- uint32_t current_flags = 0;
+ bool current_tb_modified = false;
#endif
assert_memory_lock();
@@ -698,9 +682,6 @@ bool tb_invalidate_phys_page(tb_page_addr_t addr, uintptr_t pc)
if (p->first_tb && pc != 0) {
current_tb = tcg_tb_lookup(pc);
}
- if (cpu != NULL) {
- env = cpu->env_ptr;
- }
#endif
assert_page_locked(p);
PAGE_FOR_EACH_TB(p, tb, n) {
@@ -713,10 +694,8 @@ bool tb_invalidate_phys_page(tb_page_addr_t addr, uintptr_t pc)
* after the current PC, but it would require a specialized
* function to partially restore the CPU state.
*/
- current_tb_modified = 1;
+ current_tb_modified = true;
cpu_restore_state_from_tb(cpu, current_tb, pc, true);
- cpu_get_tb_cpu_state(env, ¤t_pc, ¤t_cs_base,
- ¤t_flags);
}
#endif /* TARGET_HAS_PRECISE_SMC */
tb_phys_invalidate(tb, addr);
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 14/47] accel/tcg: Remove duplicate store to tb->page_addr[]
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (12 preceding siblings ...)
2022-10-26 2:10 ` [PULL 13/47] accel/tcg: Drop cpu_get_tb_cpu_state from TARGET_HAS_PRECISE_SMC Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 15/47] accel/tcg: Introduce tb_{set_}page_addr{0,1} Richard Henderson
` (33 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Alex Bennée
When we added the fast path, we initialized page_addr[] early.
These stores in and around tb_page_add() are redundant; remove them.
Fixes: 50627f1b7b1 ("accel/tcg: Add fast path for translator_ld*")
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
accel/tcg/tb-maint.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/accel/tcg/tb-maint.c b/accel/tcg/tb-maint.c
index 9af5cb49e0..7f4e1e1299 100644
--- a/accel/tcg/tb-maint.c
+++ b/accel/tcg/tb-maint.c
@@ -405,7 +405,6 @@ static inline void tb_page_add(PageDesc *p, TranslationBlock *tb,
assert_page_locked(p);
- tb->page_addr[n] = page_addr;
tb->page_next[n] = p->first_tb;
#ifndef CONFIG_USER_ONLY
page_already_protected = p->first_tb != (uintptr_t)NULL;
@@ -461,8 +460,6 @@ TranslationBlock *tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc,
tb_page_add(p, tb, 0, phys_pc);
if (p2) {
tb_page_add(p2, tb, 1, phys_page2);
- } else {
- tb->page_addr[1] = -1;
}
/* add in the hash table */
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 15/47] accel/tcg: Introduce tb_{set_}page_addr{0,1}
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (13 preceding siblings ...)
2022-10-26 2:10 ` [PULL 14/47] accel/tcg: Remove duplicate store to tb->page_addr[] Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 16/47] accel/tcg: Rename tb_invalidate_phys_page Richard Henderson
` (32 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Alex Bennée
This data structure will be replaced for user-only: add accessors.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
include/exec/exec-all.h | 22 ++++++++++++++++++++++
accel/tcg/cpu-exec.c | 9 +++++----
accel/tcg/tb-maint.c | 29 +++++++++++++++--------------
accel/tcg/translate-all.c | 16 ++++++++--------
accel/tcg/translator.c | 9 +++++----
5 files changed, 55 insertions(+), 30 deletions(-)
diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index b5bde1b56a..5900f4637b 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -610,6 +610,28 @@ static inline uint32_t tb_cflags(const TranslationBlock *tb)
return qatomic_read(&tb->cflags);
}
+static inline tb_page_addr_t tb_page_addr0(const TranslationBlock *tb)
+{
+ return tb->page_addr[0];
+}
+
+static inline tb_page_addr_t tb_page_addr1(const TranslationBlock *tb)
+{
+ return tb->page_addr[1];
+}
+
+static inline void tb_set_page_addr0(TranslationBlock *tb,
+ tb_page_addr_t addr)
+{
+ tb->page_addr[0] = addr;
+}
+
+static inline void tb_set_page_addr1(TranslationBlock *tb,
+ tb_page_addr_t addr)
+{
+ tb->page_addr[1] = addr;
+}
+
/* current cflags for hashing/comparison */
uint32_t curr_cflags(CPUState *cpu);
diff --git a/accel/tcg/cpu-exec.c b/accel/tcg/cpu-exec.c
index bb4b9e92ce..82b06c1824 100644
--- a/accel/tcg/cpu-exec.c
+++ b/accel/tcg/cpu-exec.c
@@ -187,13 +187,14 @@ static bool tb_lookup_cmp(const void *p, const void *d)
const struct tb_desc *desc = d;
if ((TARGET_TB_PCREL || tb_pc(tb) == desc->pc) &&
- tb->page_addr[0] == desc->page_addr0 &&
+ tb_page_addr0(tb) == desc->page_addr0 &&
tb->cs_base == desc->cs_base &&
tb->flags == desc->flags &&
tb->trace_vcpu_dstate == desc->trace_vcpu_dstate &&
tb_cflags(tb) == desc->cflags) {
/* check next page if needed */
- if (tb->page_addr[1] == -1) {
+ tb_page_addr_t tb_phys_page1 = tb_page_addr1(tb);
+ if (tb_phys_page1 == -1) {
return true;
} else {
tb_page_addr_t phys_page1;
@@ -210,7 +211,7 @@ static bool tb_lookup_cmp(const void *p, const void *d)
*/
virt_page1 = TARGET_PAGE_ALIGN(desc->pc);
phys_page1 = get_page_addr_code(desc->env, virt_page1);
- if (tb->page_addr[1] == phys_page1) {
+ if (tb_phys_page1 == phys_page1) {
return true;
}
}
@@ -1019,7 +1020,7 @@ int cpu_exec(CPUState *cpu)
* direct jump to a TB spanning two pages because the mapping
* for the second page can change.
*/
- if (tb->page_addr[1] != -1) {
+ if (tb_page_addr1(tb) != -1) {
last_tb = NULL;
}
#endif
diff --git a/accel/tcg/tb-maint.c b/accel/tcg/tb-maint.c
index 7f4e1e1299..15ec2f741d 100644
--- a/accel/tcg/tb-maint.c
+++ b/accel/tcg/tb-maint.c
@@ -44,8 +44,8 @@ static bool tb_cmp(const void *ap, const void *bp)
a->flags == b->flags &&
(tb_cflags(a) & ~CF_INVALID) == (tb_cflags(b) & ~CF_INVALID) &&
a->trace_vcpu_dstate == b->trace_vcpu_dstate &&
- a->page_addr[0] == b->page_addr[0] &&
- a->page_addr[1] == b->page_addr[1]);
+ tb_page_addr0(a) == tb_page_addr0(b) &&
+ tb_page_addr1(a) == tb_page_addr1(b));
}
void tb_htable_init(void)
@@ -273,7 +273,7 @@ static void do_tb_phys_invalidate(TranslationBlock *tb, bool rm_from_page_list)
qemu_spin_unlock(&tb->jmp_lock);
/* remove the TB from the hash list */
- phys_pc = tb->page_addr[0];
+ phys_pc = tb_page_addr0(tb);
h = tb_hash_func(phys_pc, (TARGET_TB_PCREL ? 0 : tb_pc(tb)),
tb->flags, orig_cflags, tb->trace_vcpu_dstate);
if (!qht_remove(&tb_ctx.htable, tb, h)) {
@@ -282,10 +282,11 @@ static void do_tb_phys_invalidate(TranslationBlock *tb, bool rm_from_page_list)
/* remove the TB from the page list */
if (rm_from_page_list) {
- p = page_find(tb->page_addr[0] >> TARGET_PAGE_BITS);
+ p = page_find(phys_pc >> TARGET_PAGE_BITS);
tb_page_remove(p, tb);
- if (tb->page_addr[1] != -1) {
- p = page_find(tb->page_addr[1] >> TARGET_PAGE_BITS);
+ phys_pc = tb_page_addr1(tb);
+ if (phys_pc != -1) {
+ p = page_find(phys_pc >> TARGET_PAGE_BITS);
tb_page_remove(p, tb);
}
}
@@ -358,16 +359,16 @@ static inline void page_unlock_tb(const TranslationBlock *tb) { }
/* lock the page(s) of a TB in the correct acquisition order */
static void page_lock_tb(const TranslationBlock *tb)
{
- page_lock_pair(NULL, tb->page_addr[0], NULL, tb->page_addr[1], false);
+ page_lock_pair(NULL, tb_page_addr0(tb), NULL, tb_page_addr1(tb), false);
}
static void page_unlock_tb(const TranslationBlock *tb)
{
- PageDesc *p1 = page_find(tb->page_addr[0] >> TARGET_PAGE_BITS);
+ PageDesc *p1 = page_find(tb_page_addr0(tb) >> TARGET_PAGE_BITS);
page_unlock(p1);
- if (unlikely(tb->page_addr[1] != -1)) {
- PageDesc *p2 = page_find(tb->page_addr[1] >> TARGET_PAGE_BITS);
+ if (unlikely(tb_page_addr1(tb) != -1)) {
+ PageDesc *p2 = page_find(tb_page_addr1(tb) >> TARGET_PAGE_BITS);
if (p2 != p1) {
page_unlock(p2);
@@ -382,7 +383,7 @@ static void page_unlock_tb(const TranslationBlock *tb)
*/
void tb_phys_invalidate(TranslationBlock *tb, tb_page_addr_t page_addr)
{
- if (page_addr == -1 && tb->page_addr[0] != -1) {
+ if (page_addr == -1 && tb_page_addr0(tb) != -1) {
page_lock_tb(tb);
do_tb_phys_invalidate(tb, true);
page_unlock_tb(tb);
@@ -516,11 +517,11 @@ tb_invalidate_phys_page_range__locked(struct page_collection *pages,
if (n == 0) {
/* NOTE: tb_end may be after the end of the page, but
it is not a problem */
- tb_start = tb->page_addr[0];
+ tb_start = tb_page_addr0(tb);
tb_end = tb_start + tb->size;
} else {
- tb_start = tb->page_addr[1];
- tb_end = tb_start + ((tb->page_addr[0] + tb->size)
+ tb_start = tb_page_addr1(tb);
+ tb_end = tb_start + ((tb_page_addr0(tb) + tb->size)
& ~TARGET_PAGE_MASK);
}
if (!(tb_end <= start || tb_start >= end)) {
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index 5e28e9fccd..bef4c56cff 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -698,9 +698,9 @@ page_collection_lock(tb_page_addr_t start, tb_page_addr_t end)
}
assert_page_locked(pd);
PAGE_FOR_EACH_TB(pd, tb, n) {
- if (page_trylock_add(set, tb->page_addr[0]) ||
- (tb->page_addr[1] != -1 &&
- page_trylock_add(set, tb->page_addr[1]))) {
+ if (page_trylock_add(set, tb_page_addr0(tb)) ||
+ (tb_page_addr1(tb) != -1 &&
+ page_trylock_add(set, tb_page_addr1(tb)))) {
/* drop all locks, and reacquire in order */
g_tree_foreach(set->tree, page_entry_unlock, NULL);
goto retry;
@@ -771,8 +771,8 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
tb->flags = flags;
tb->cflags = cflags;
tb->trace_vcpu_dstate = *cpu->trace_dstate;
- tb->page_addr[0] = phys_pc;
- tb->page_addr[1] = -1;
+ tb_set_page_addr0(tb, phys_pc);
+ tb_set_page_addr1(tb, -1);
tcg_ctx->tb_cflags = cflags;
tb_overflow:
@@ -970,7 +970,7 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
* a temporary one-insn TB, and we have nothing left to do. Return early
* before attempting to link to other TBs or add to the lookup table.
*/
- if (tb->page_addr[0] == -1) {
+ if (tb_page_addr0(tb) == -1) {
return tb;
}
@@ -985,7 +985,7 @@ TranslationBlock *tb_gen_code(CPUState *cpu,
* No explicit memory barrier is required -- tb_link_page() makes the
* TB visible in a consistent state.
*/
- existing_tb = tb_link_page(tb, tb->page_addr[0], tb->page_addr[1]);
+ existing_tb = tb_link_page(tb, tb_page_addr0(tb), tb_page_addr1(tb));
/* if the TB already exists, discard what we just translated */
if (unlikely(existing_tb != tb)) {
uintptr_t orig_aligned = (uintptr_t)gen_code_buf;
@@ -1140,7 +1140,7 @@ static gboolean tb_tree_stats_iter(gpointer key, gpointer value, gpointer data)
if (tb->size > tst->max_target_size) {
tst->max_target_size = tb->size;
}
- if (tb->page_addr[1] != -1) {
+ if (tb_page_addr1(tb) != -1) {
tst->cross_page++;
}
if (tb->jmp_reset_offset[0] != TB_JMP_RESET_OFFSET_INVALID) {
diff --git a/accel/tcg/translator.c b/accel/tcg/translator.c
index 8e78fd7a9c..061519691f 100644
--- a/accel/tcg/translator.c
+++ b/accel/tcg/translator.c
@@ -157,7 +157,7 @@ static void *translator_access(CPUArchState *env, DisasContextBase *db,
tb = db->tb;
/* Use slow path if first page is MMIO. */
- if (unlikely(tb->page_addr[0] == -1)) {
+ if (unlikely(tb_page_addr0(tb) == -1)) {
return NULL;
}
@@ -169,13 +169,14 @@ static void *translator_access(CPUArchState *env, DisasContextBase *db,
host = db->host_addr[1];
base = TARGET_PAGE_ALIGN(db->pc_first);
if (host == NULL) {
- tb->page_addr[1] =
+ tb_page_addr_t phys_page =
get_page_addr_code_hostp(env, base, &db->host_addr[1]);
+ /* We cannot handle MMIO as second page. */
+ assert(phys_page != -1);
+ tb_set_page_addr1(tb, phys_page);
#ifdef CONFIG_USER_ONLY
page_protect(end);
#endif
- /* We cannot handle MMIO as second page. */
- assert(tb->page_addr[1] != -1);
host = db->host_addr[1];
}
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 16/47] accel/tcg: Rename tb_invalidate_phys_page
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (14 preceding siblings ...)
2022-10-26 2:10 ` [PULL 15/47] accel/tcg: Introduce tb_{set_}page_addr{0,1} Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 17/47] accel/tcg: Rename tb_invalidate_phys_page_range and drop end parameter Richard Henderson
` (31 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Alex Bennée
Rename to tb_invalidate_phys_page_unwind to emphasize that
we also detect invalidating the current TB, and also to free
up that name for other usage.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
accel/tcg/internal.h | 2 +-
accel/tcg/tb-maint.c | 2 +-
accel/tcg/translate-all.c | 5 +++--
3 files changed, 5 insertions(+), 4 deletions(-)
diff --git a/accel/tcg/internal.h b/accel/tcg/internal.h
index 1a704ee14f..1227bb69bd 100644
--- a/accel/tcg/internal.h
+++ b/accel/tcg/internal.h
@@ -105,7 +105,7 @@ void tb_htable_init(void);
void tb_reset_jump(TranslationBlock *tb, int n);
TranslationBlock *tb_link_page(TranslationBlock *tb, tb_page_addr_t phys_pc,
tb_page_addr_t phys_page2);
-bool tb_invalidate_phys_page(tb_page_addr_t addr, uintptr_t pc);
+bool tb_invalidate_phys_page_unwind(tb_page_addr_t addr, uintptr_t pc);
int cpu_restore_state_from_tb(CPUState *cpu, TranslationBlock *tb,
uintptr_t searched_pc, bool reset_icount);
diff --git a/accel/tcg/tb-maint.c b/accel/tcg/tb-maint.c
index 15ec2f741d..92170cbbc1 100644
--- a/accel/tcg/tb-maint.c
+++ b/accel/tcg/tb-maint.c
@@ -657,7 +657,7 @@ void tb_invalidate_phys_page_fast(struct page_collection *pages,
* TB (because it was modified by this store and the guest CPU has
* precise-SMC semantics).
*/
-bool tb_invalidate_phys_page(tb_page_addr_t addr, uintptr_t pc)
+bool tb_invalidate_phys_page_unwind(tb_page_addr_t addr, uintptr_t pc)
{
TranslationBlock *tb;
PageDesc *p;
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index bef4c56cff..aa8d213514 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -1382,7 +1382,7 @@ void page_set_flags(target_ulong start, target_ulong end, int flags)
if (!(p->flags & PAGE_WRITE) &&
(flags & PAGE_WRITE) &&
p->first_tb) {
- tb_invalidate_phys_page(addr, 0);
+ tb_invalidate_phys_page_unwind(addr, 0);
}
if (reset_target_data) {
g_free(p->target_data);
@@ -1580,7 +1580,8 @@ int page_unprotect(target_ulong address, uintptr_t pc)
/* and since the content will be modified, we must invalidate
the corresponding translated code. */
- current_tb_invalidated |= tb_invalidate_phys_page(addr, pc);
+ current_tb_invalidated |=
+ tb_invalidate_phys_page_unwind(addr, pc);
}
mprotect((void *)g2h_untagged(host_start), qemu_host_page_size,
prot & PAGE_BITS);
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 17/47] accel/tcg: Rename tb_invalidate_phys_page_range and drop end parameter
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (15 preceding siblings ...)
2022-10-26 2:10 ` [PULL 16/47] accel/tcg: Rename tb_invalidate_phys_page Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 18/47] accel/tcg: Unify declarations of tb_invalidate_phys_range Richard Henderson
` (30 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Alex Bennée
This function is is never called with a real range,
only for a single page. Drop the second parameter
and rename to tb_invalidate_phys_page.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
include/exec/translate-all.h | 2 +-
accel/tcg/tb-maint.c | 15 ++++++++-------
cpu.c | 4 ++--
3 files changed, 11 insertions(+), 10 deletions(-)
diff --git a/include/exec/translate-all.h b/include/exec/translate-all.h
index 9f646389af..3e9cb91565 100644
--- a/include/exec/translate-all.h
+++ b/include/exec/translate-all.h
@@ -29,7 +29,7 @@ void page_collection_unlock(struct page_collection *set);
void tb_invalidate_phys_page_fast(struct page_collection *pages,
tb_page_addr_t start, int len,
uintptr_t retaddr);
-void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end);
+void tb_invalidate_phys_page(tb_page_addr_t addr);
void tb_check_watchpoint(CPUState *cpu, uintptr_t retaddr);
#ifdef CONFIG_USER_ONLY
diff --git a/accel/tcg/tb-maint.c b/accel/tcg/tb-maint.c
index 92170cbbc1..bac43774c0 100644
--- a/accel/tcg/tb-maint.c
+++ b/accel/tcg/tb-maint.c
@@ -565,25 +565,26 @@ tb_invalidate_phys_page_range__locked(struct page_collection *pages,
}
/*
- * Invalidate all TBs which intersect with the target physical address range
- * [start;end[. NOTE: start and end must refer to the *same* physical page.
- * 'is_cpu_write_access' should be true if called from a real cpu write
- * access: the virtual CPU will exit the current TB if code is modified inside
- * this TB.
+ * Invalidate all TBs which intersect with the target physical
+ * address page @addr.
*
* Called with mmap_lock held for user-mode emulation
*/
-void tb_invalidate_phys_page_range(tb_page_addr_t start, tb_page_addr_t end)
+void tb_invalidate_phys_page(tb_page_addr_t addr)
{
struct page_collection *pages;
+ tb_page_addr_t start, end;
PageDesc *p;
assert_memory_lock();
- p = page_find(start >> TARGET_PAGE_BITS);
+ p = page_find(addr >> TARGET_PAGE_BITS);
if (p == NULL) {
return;
}
+
+ start = addr & TARGET_PAGE_MASK;
+ end = start + TARGET_PAGE_SIZE;
pages = page_collection_lock(start, end);
tb_invalidate_phys_page_range__locked(pages, p, start, end, 0);
page_collection_unlock(pages);
diff --git a/cpu.c b/cpu.c
index 14365e36f3..2a09b05205 100644
--- a/cpu.c
+++ b/cpu.c
@@ -277,7 +277,7 @@ void list_cpus(const char *optarg)
void tb_invalidate_phys_addr(target_ulong addr)
{
mmap_lock();
- tb_invalidate_phys_page_range(addr, addr + 1);
+ tb_invalidate_phys_page(addr);
mmap_unlock();
}
#else
@@ -298,7 +298,7 @@ void tb_invalidate_phys_addr(AddressSpace *as, hwaddr addr, MemTxAttrs attrs)
return;
}
ram_addr = memory_region_get_ram_addr(mr) + addr;
- tb_invalidate_phys_page_range(ram_addr, ram_addr + 1);
+ tb_invalidate_phys_page(ram_addr);
}
#endif
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 18/47] accel/tcg: Unify declarations of tb_invalidate_phys_range
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (16 preceding siblings ...)
2022-10-26 2:10 ` [PULL 17/47] accel/tcg: Rename tb_invalidate_phys_page_range and drop end parameter Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 19/47] accel/tcg: Use tb_invalidate_phys_page in page_set_flags Richard Henderson
` (29 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Alex Bennée
We missed this function when we introduced tb_page_addr_t.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
include/exec/exec-all.h | 2 +-
include/exec/ram_addr.h | 2 --
accel/tcg/tb-maint.c | 13 ++-----------
3 files changed, 3 insertions(+), 14 deletions(-)
diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index 5900f4637b..5ae484e34d 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -638,12 +638,12 @@ uint32_t curr_cflags(CPUState *cpu);
/* TranslationBlock invalidate API */
#if defined(CONFIG_USER_ONLY)
void tb_invalidate_phys_addr(target_ulong addr);
-void tb_invalidate_phys_range(target_ulong start, target_ulong end);
#else
void tb_invalidate_phys_addr(AddressSpace *as, hwaddr addr, MemTxAttrs attrs);
#endif
void tb_flush(CPUState *cpu);
void tb_phys_invalidate(TranslationBlock *tb, tb_page_addr_t page_addr);
+void tb_invalidate_phys_range(tb_page_addr_t start, tb_page_addr_t end);
void tb_set_jmp_target(TranslationBlock *tb, int n, uintptr_t addr);
/* GETPC is the true target of the return instruction that we'll execute. */
diff --git a/include/exec/ram_addr.h b/include/exec/ram_addr.h
index f3e0c78161..1500680458 100644
--- a/include/exec/ram_addr.h
+++ b/include/exec/ram_addr.h
@@ -147,8 +147,6 @@ static inline void qemu_ram_block_writeback(RAMBlock *block)
#define DIRTY_CLIENTS_ALL ((1 << DIRTY_MEMORY_NUM) - 1)
#define DIRTY_CLIENTS_NOCODE (DIRTY_CLIENTS_ALL & ~(1 << DIRTY_MEMORY_CODE))
-void tb_invalidate_phys_range(ram_addr_t start, ram_addr_t end);
-
static inline bool cpu_physical_memory_get_dirty(ram_addr_t start,
ram_addr_t length,
unsigned client)
diff --git a/accel/tcg/tb-maint.c b/accel/tcg/tb-maint.c
index bac43774c0..c8e921089d 100644
--- a/accel/tcg/tb-maint.c
+++ b/accel/tcg/tb-maint.c
@@ -20,6 +20,7 @@
#include "qemu/osdep.h"
#include "exec/cputlb.h"
#include "exec/log.h"
+#include "exec/exec-all.h"
#include "exec/translate-all.h"
#include "sysemu/tcg.h"
#include "tcg/tcg.h"
@@ -27,12 +28,6 @@
#include "tb-context.h"
#include "internal.h"
-/* FIXME: tb_invalidate_phys_range is declared in different places. */
-#ifdef CONFIG_USER_ONLY
-#include "exec/exec-all.h"
-#else
-#include "exec/ram_addr.h"
-#endif
static bool tb_cmp(const void *ap, const void *bp)
{
@@ -599,11 +594,7 @@ void tb_invalidate_phys_page(tb_page_addr_t addr)
*
* Called with mmap_lock held for user-mode emulation.
*/
-#ifdef CONFIG_SOFTMMU
-void tb_invalidate_phys_range(ram_addr_t start, ram_addr_t end)
-#else
-void tb_invalidate_phys_range(target_ulong start, target_ulong end)
-#endif
+void tb_invalidate_phys_range(tb_page_addr_t start, tb_page_addr_t end)
{
struct page_collection *pages;
tb_page_addr_t next;
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 19/47] accel/tcg: Use tb_invalidate_phys_page in page_set_flags
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (17 preceding siblings ...)
2022-10-26 2:10 ` [PULL 18/47] accel/tcg: Unify declarations of tb_invalidate_phys_range Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 20/47] accel/tcg: Call tb_invalidate_phys_page for PAGE_RESET Richard Henderson
` (28 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Alex Bennée
We do not require detection of overlapping TBs here,
so use the more appropriate function.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
accel/tcg/translate-all.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index aa8d213514..8d5233fa9e 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -1382,7 +1382,7 @@ void page_set_flags(target_ulong start, target_ulong end, int flags)
if (!(p->flags & PAGE_WRITE) &&
(flags & PAGE_WRITE) &&
p->first_tb) {
- tb_invalidate_phys_page_unwind(addr, 0);
+ tb_invalidate_phys_page(addr);
}
if (reset_target_data) {
g_free(p->target_data);
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 20/47] accel/tcg: Call tb_invalidate_phys_page for PAGE_RESET
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (18 preceding siblings ...)
2022-10-26 2:10 ` [PULL 19/47] accel/tcg: Use tb_invalidate_phys_page in page_set_flags Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 21/47] accel/tcg: Use page_reset_target_data in page_set_flags Richard Henderson
` (27 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Alex Bennée
When PAGE_RESET is set, we are replacing pages with new
content, which means that we need to invalidate existing
cached data, such as TranslationBlocks. Perform the
reset invalidate while we're doing other invalidates,
which allows us to remove the separate invalidates from
the user-only mmap/munmap/mprotect routines.
In addition, restrict invalidation to PAGE_EXEC pages.
Since cdf713085131, we have validated PAGE_EXEC is present
before translation, which means we can assume that if the
bit is not present, there are no translations to invalidate.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
accel/tcg/translate-all.c | 19 +++++++++++--------
bsd-user/mmap.c | 2 --
linux-user/mmap.c | 4 ----
3 files changed, 11 insertions(+), 14 deletions(-)
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index 8d5233fa9e..478301f227 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -1352,7 +1352,7 @@ int page_get_flags(target_ulong address)
void page_set_flags(target_ulong start, target_ulong end, int flags)
{
target_ulong addr, len;
- bool reset_target_data;
+ bool reset;
/* This function should never be called with addresses outside the
guest address space. If this assert fires, it probably indicates
@@ -1369,7 +1369,7 @@ void page_set_flags(target_ulong start, target_ulong end, int flags)
if (flags & PAGE_WRITE) {
flags |= PAGE_WRITE_ORG;
}
- reset_target_data = !(flags & PAGE_VALID) || (flags & PAGE_RESET);
+ reset = !(flags & PAGE_VALID) || (flags & PAGE_RESET);
flags &= ~PAGE_RESET;
for (addr = start, len = end - start;
@@ -1377,14 +1377,17 @@ void page_set_flags(target_ulong start, target_ulong end, int flags)
len -= TARGET_PAGE_SIZE, addr += TARGET_PAGE_SIZE) {
PageDesc *p = page_find_alloc(addr >> TARGET_PAGE_BITS, true);
- /* If the write protection bit is set, then we invalidate
- the code inside. */
- if (!(p->flags & PAGE_WRITE) &&
- (flags & PAGE_WRITE) &&
- p->first_tb) {
+ /*
+ * If the page was executable, but is reset, or is no longer
+ * executable, or has become writable, then invalidate any code.
+ */
+ if ((p->flags & PAGE_EXEC)
+ && (reset ||
+ !(flags & PAGE_EXEC) ||
+ (flags & ~p->flags & PAGE_WRITE))) {
tb_invalidate_phys_page(addr);
}
- if (reset_target_data) {
+ if (reset) {
g_free(p->target_data);
p->target_data = NULL;
p->flags = flags;
diff --git a/bsd-user/mmap.c b/bsd-user/mmap.c
index e54e26de17..d6c5a344c9 100644
--- a/bsd-user/mmap.c
+++ b/bsd-user/mmap.c
@@ -663,7 +663,6 @@ abi_long target_mmap(abi_ulong start, abi_ulong len, int prot,
page_dump(stdout);
printf("\n");
#endif
- tb_invalidate_phys_range(start, start + len);
mmap_unlock();
return start;
fail:
@@ -769,7 +768,6 @@ int target_munmap(abi_ulong start, abi_ulong len)
if (ret == 0) {
page_set_flags(start, start + len, 0);
- tb_invalidate_phys_range(start, start + len);
}
mmap_unlock();
return ret;
diff --git a/linux-user/mmap.c b/linux-user/mmap.c
index 28f3bc85ed..10f5079331 100644
--- a/linux-user/mmap.c
+++ b/linux-user/mmap.c
@@ -182,7 +182,6 @@ int target_mprotect(abi_ulong start, abi_ulong len, int target_prot)
}
page_set_flags(start, start + len, page_flags);
- tb_invalidate_phys_range(start, start + len);
ret = 0;
error:
@@ -662,7 +661,6 @@ abi_long target_mmap(abi_ulong start, abi_ulong len, int target_prot,
qemu_log_unlock(f);
}
}
- tb_invalidate_phys_range(start, start + len);
mmap_unlock();
return start;
fail:
@@ -766,7 +764,6 @@ int target_munmap(abi_ulong start, abi_ulong len)
if (ret == 0) {
page_set_flags(start, start + len, 0);
- tb_invalidate_phys_range(start, start + len);
}
mmap_unlock();
return ret;
@@ -856,7 +853,6 @@ abi_long target_mremap(abi_ulong old_addr, abi_ulong old_size,
page_set_flags(new_addr, new_addr + new_size,
prot | PAGE_VALID | PAGE_RESET);
}
- tb_invalidate_phys_range(new_addr, new_addr + new_size);
mmap_unlock();
return new_addr;
}
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 21/47] accel/tcg: Use page_reset_target_data in page_set_flags
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (19 preceding siblings ...)
2022-10-26 2:10 ` [PULL 20/47] accel/tcg: Call tb_invalidate_phys_page for PAGE_RESET Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 22/47] accel/tcg: Use tb_invalidate_phys_range " Richard Henderson
` (26 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Alex Bennée
Use the existing function for clearing target data.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
accel/tcg/translate-all.c | 13 +++++--------
1 file changed, 5 insertions(+), 8 deletions(-)
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index 478301f227..41b6d5fe26 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -1370,6 +1370,9 @@ void page_set_flags(target_ulong start, target_ulong end, int flags)
flags |= PAGE_WRITE_ORG;
}
reset = !(flags & PAGE_VALID) || (flags & PAGE_RESET);
+ if (reset) {
+ page_reset_target_data(start, end);
+ }
flags &= ~PAGE_RESET;
for (addr = start, len = end - start;
@@ -1387,14 +1390,8 @@ void page_set_flags(target_ulong start, target_ulong end, int flags)
(flags & ~p->flags & PAGE_WRITE))) {
tb_invalidate_phys_page(addr);
}
- if (reset) {
- g_free(p->target_data);
- p->target_data = NULL;
- p->flags = flags;
- } else {
- /* Using mprotect on a page does not change sticky bits. */
- p->flags = (p->flags & PAGE_STICKY) | flags;
- }
+ /* Using mprotect on a page does not change sticky bits. */
+ p->flags = (reset ? 0 : p->flags & PAGE_STICKY) | flags;
}
}
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 22/47] accel/tcg: Use tb_invalidate_phys_range in page_set_flags
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (20 preceding siblings ...)
2022-10-26 2:10 ` [PULL 21/47] accel/tcg: Use page_reset_target_data in page_set_flags Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 23/47] accel/tcg: Move TARGET_PAGE_DATA_SIZE impl to user-exec.c Richard Henderson
` (25 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Alex Bennée
Flush translation blocks in bulk, rather than page-by-page.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
accel/tcg/translate-all.c | 8 ++++++--
1 file changed, 6 insertions(+), 2 deletions(-)
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index 41b6d5fe26..eea24dea96 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -1352,7 +1352,7 @@ int page_get_flags(target_ulong address)
void page_set_flags(target_ulong start, target_ulong end, int flags)
{
target_ulong addr, len;
- bool reset;
+ bool reset, inval_tb = false;
/* This function should never be called with addresses outside the
guest address space. If this assert fires, it probably indicates
@@ -1388,11 +1388,15 @@ void page_set_flags(target_ulong start, target_ulong end, int flags)
&& (reset ||
!(flags & PAGE_EXEC) ||
(flags & ~p->flags & PAGE_WRITE))) {
- tb_invalidate_phys_page(addr);
+ inval_tb = true;
}
/* Using mprotect on a page does not change sticky bits. */
p->flags = (reset ? 0 : p->flags & PAGE_STICKY) | flags;
}
+
+ if (inval_tb) {
+ tb_invalidate_phys_range(start, end);
+ }
}
void page_reset_target_data(target_ulong start, target_ulong end)
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 23/47] accel/tcg: Move TARGET_PAGE_DATA_SIZE impl to user-exec.c
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (21 preceding siblings ...)
2022-10-26 2:10 ` [PULL 22/47] accel/tcg: Use tb_invalidate_phys_range " Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 24/47] accel/tcg: Simplify page_get/alloc_target_data Richard Henderson
` (24 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Alex Bennée
Since "target data" is always user-only, move it out of
translate-all.c to user-exec.c.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
accel/tcg/translate-all.c | 50 ---------------------------------------
accel/tcg/user-exec.c | 50 +++++++++++++++++++++++++++++++++++++++
2 files changed, 50 insertions(+), 50 deletions(-)
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index eea24dea96..433fa247f4 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -1399,56 +1399,6 @@ void page_set_flags(target_ulong start, target_ulong end, int flags)
}
}
-void page_reset_target_data(target_ulong start, target_ulong end)
-{
-#ifdef TARGET_PAGE_DATA_SIZE
- target_ulong addr, len;
-
- /*
- * This function should never be called with addresses outside the
- * guest address space. If this assert fires, it probably indicates
- * a missing call to h2g_valid.
- */
- assert(end - 1 <= GUEST_ADDR_MAX);
- assert(start < end);
- assert_memory_lock();
-
- start = start & TARGET_PAGE_MASK;
- end = TARGET_PAGE_ALIGN(end);
-
- for (addr = start, len = end - start;
- len != 0;
- len -= TARGET_PAGE_SIZE, addr += TARGET_PAGE_SIZE) {
- PageDesc *p = page_find_alloc(addr >> TARGET_PAGE_BITS, 1);
-
- g_free(p->target_data);
- p->target_data = NULL;
- }
-#endif
-}
-
-#ifdef TARGET_PAGE_DATA_SIZE
-void *page_get_target_data(target_ulong address)
-{
- PageDesc *p = page_find(address >> TARGET_PAGE_BITS);
- return p ? p->target_data : NULL;
-}
-
-void *page_alloc_target_data(target_ulong address)
-{
- PageDesc *p = page_find(address >> TARGET_PAGE_BITS);
- void *ret = NULL;
-
- if (p->flags & PAGE_VALID) {
- ret = p->target_data;
- if (!ret) {
- p->target_data = ret = g_malloc0(TARGET_PAGE_DATA_SIZE);
- }
- }
- return ret;
-}
-#endif /* TARGET_PAGE_DATA_SIZE */
-
int page_check_range(target_ulong start, target_ulong len, int flags)
{
PageDesc *p;
diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c
index 521aa8b61e..927b91900f 100644
--- a/accel/tcg/user-exec.c
+++ b/accel/tcg/user-exec.c
@@ -210,6 +210,56 @@ tb_page_addr_t get_page_addr_code_hostp(CPUArchState *env, target_ulong addr,
return addr;
}
+void page_reset_target_data(target_ulong start, target_ulong end)
+{
+#ifdef TARGET_PAGE_DATA_SIZE
+ target_ulong addr, len;
+
+ /*
+ * This function should never be called with addresses outside the
+ * guest address space. If this assert fires, it probably indicates
+ * a missing call to h2g_valid.
+ */
+ assert(end - 1 <= GUEST_ADDR_MAX);
+ assert(start < end);
+ assert_memory_lock();
+
+ start = start & TARGET_PAGE_MASK;
+ end = TARGET_PAGE_ALIGN(end);
+
+ for (addr = start, len = end - start;
+ len != 0;
+ len -= TARGET_PAGE_SIZE, addr += TARGET_PAGE_SIZE) {
+ PageDesc *p = page_find_alloc(addr >> TARGET_PAGE_BITS, 1);
+
+ g_free(p->target_data);
+ p->target_data = NULL;
+ }
+#endif
+}
+
+#ifdef TARGET_PAGE_DATA_SIZE
+void *page_get_target_data(target_ulong address)
+{
+ PageDesc *p = page_find(address >> TARGET_PAGE_BITS);
+ return p ? p->target_data : NULL;
+}
+
+void *page_alloc_target_data(target_ulong address)
+{
+ PageDesc *p = page_find(address >> TARGET_PAGE_BITS);
+ void *ret = NULL;
+
+ if (p->flags & PAGE_VALID) {
+ ret = p->target_data;
+ if (!ret) {
+ p->target_data = ret = g_malloc0(TARGET_PAGE_DATA_SIZE);
+ }
+ }
+ return ret;
+}
+#endif
+
/* The softmmu versions of these helpers are in cputlb.c. */
/*
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 24/47] accel/tcg: Simplify page_get/alloc_target_data
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (22 preceding siblings ...)
2022-10-26 2:10 ` [PULL 23/47] accel/tcg: Move TARGET_PAGE_DATA_SIZE impl to user-exec.c Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 25/47] accel/tcg: Add restore_state_to_opc to TCGCPUOps Richard Henderson
` (23 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Alex Bennée
Since the only user, Arm MTE, always requires allocation,
merge the get and alloc functions to always produce a
non-null result. Also assume that the user has already
checked page validity.
Reviewed-by: Alex Bennée <alex.bennee@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
include/exec/cpu-all.h | 21 ++++++---------------
accel/tcg/user-exec.c | 16 ++++------------
target/arm/mte_helper.c | 4 ----
3 files changed, 10 insertions(+), 31 deletions(-)
diff --git a/include/exec/cpu-all.h b/include/exec/cpu-all.h
index 854adc4ac2..2eb1176538 100644
--- a/include/exec/cpu-all.h
+++ b/include/exec/cpu-all.h
@@ -281,27 +281,18 @@ void page_reset_target_data(target_ulong start, target_ulong end);
int page_check_range(target_ulong start, target_ulong len, int flags);
/**
- * page_alloc_target_data(address)
+ * page_get_target_data(address)
* @address: guest virtual address
*
- * Allocate TARGET_PAGE_DATA_SIZE bytes of out-of-band data to associate
- * with the guest page at @address. If the page is not mapped, NULL will
- * be returned. If there is existing data associated with @address,
- * no new memory will be allocated.
+ * Return TARGET_PAGE_DATA_SIZE bytes of out-of-band data to associate
+ * with the guest page at @address, allocating it if necessary. The
+ * caller should already have verified that the address is valid.
*
* The memory will be freed when the guest page is deallocated,
* e.g. with the munmap system call.
*/
-void *page_alloc_target_data(target_ulong address);
-
-/**
- * page_get_target_data(address)
- * @address: guest virtual address
- *
- * Return any out-of-bound memory assocated with the guest page
- * at @address, as per page_alloc_target_data.
- */
-void *page_get_target_data(target_ulong address);
+void *page_get_target_data(target_ulong address)
+ __attribute__((returns_nonnull));
#endif
CPUArchState *cpu_copy(CPUArchState *env);
diff --git a/accel/tcg/user-exec.c b/accel/tcg/user-exec.c
index 927b91900f..fb7d6ee9e9 100644
--- a/accel/tcg/user-exec.c
+++ b/accel/tcg/user-exec.c
@@ -242,19 +242,11 @@ void page_reset_target_data(target_ulong start, target_ulong end)
void *page_get_target_data(target_ulong address)
{
PageDesc *p = page_find(address >> TARGET_PAGE_BITS);
- return p ? p->target_data : NULL;
-}
+ void *ret = p->target_data;
-void *page_alloc_target_data(target_ulong address)
-{
- PageDesc *p = page_find(address >> TARGET_PAGE_BITS);
- void *ret = NULL;
-
- if (p->flags & PAGE_VALID) {
- ret = p->target_data;
- if (!ret) {
- p->target_data = ret = g_malloc0(TARGET_PAGE_DATA_SIZE);
- }
+ if (!ret) {
+ ret = g_malloc0(TARGET_PAGE_DATA_SIZE);
+ p->target_data = ret;
}
return ret;
}
diff --git a/target/arm/mte_helper.c b/target/arm/mte_helper.c
index a9c5fc2cb2..86b3754838 100644
--- a/target/arm/mte_helper.c
+++ b/target/arm/mte_helper.c
@@ -95,10 +95,6 @@ static uint8_t *allocation_tag_mem(CPUARMState *env, int ptr_mmu_idx,
}
tags = page_get_target_data(clean_ptr);
- if (tags == NULL) {
- tags = page_alloc_target_data(clean_ptr);
- assert(tags != NULL);
- }
index = extract32(ptr, LOG2_TAG_GRANULE + 1,
TARGET_PAGE_BITS - LOG2_TAG_GRANULE - 1);
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 25/47] accel/tcg: Add restore_state_to_opc to TCGCPUOps
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (23 preceding siblings ...)
2022-10-26 2:10 ` [PULL 24/47] accel/tcg: Simplify page_get/alloc_target_data Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-29 10:42 ` Alex Bennée
2022-10-31 17:56 ` Christian Schoenebeck
2022-10-26 2:10 ` [PULL 26/47] target/alpha: Convert to tcg_ops restore_state_to_opc Richard Henderson
` (22 subsequent siblings)
47 siblings, 2 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Claudio Fontana
Add a tcg_ops hook to replace the restore_state_to_opc
function call. Because these generic hooks cannot depend
on target-specific types, temporarily, copy the current
target_ulong data[] into uint64_t d64[].
Reviewed-by: Claudio Fontana <cfontana@suse.de>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
include/exec/exec-all.h | 2 +-
include/hw/core/tcg-cpu-ops.h | 11 +++++++++++
accel/tcg/translate-all.c | 24 ++++++++++++++++++++++--
3 files changed, 34 insertions(+), 3 deletions(-)
diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index 5ae484e34d..3b5e84240b 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -40,7 +40,7 @@ typedef ram_addr_t tb_page_addr_t;
#endif
void restore_state_to_opc(CPUArchState *env, TranslationBlock *tb,
- target_ulong *data);
+ target_ulong *data) __attribute__((weak));
/**
* cpu_restore_state:
diff --git a/include/hw/core/tcg-cpu-ops.h b/include/hw/core/tcg-cpu-ops.h
index 78c6c6635d..20e3c0ffbb 100644
--- a/include/hw/core/tcg-cpu-ops.h
+++ b/include/hw/core/tcg-cpu-ops.h
@@ -31,6 +31,17 @@ struct TCGCPUOps {
* function to restore all the state, and register it here.
*/
void (*synchronize_from_tb)(CPUState *cpu, const TranslationBlock *tb);
+ /**
+ * @restore_state_to_opc: Synchronize state from INDEX_op_start_insn
+ *
+ * This is called when we unwind state in the middle of a TB,
+ * usually before raising an exception. Set all part of the CPU
+ * state which are tracked insn-by-insn in the target-specific
+ * arguments to start_insn, passed as @data.
+ */
+ void (*restore_state_to_opc)(CPUState *cpu, const TranslationBlock *tb,
+ const uint64_t *data);
+
/** @cpu_exec_enter: Callback for cpu_exec preparation */
void (*cpu_exec_enter)(CPUState *cpu);
/** @cpu_exec_exit: Callback for cpu_exec cleanup */
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index 433fa247f4..4d8783efc7 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -256,7 +256,6 @@ int cpu_restore_state_from_tb(CPUState *cpu, TranslationBlock *tb,
{
target_ulong data[TARGET_INSN_START_WORDS];
uintptr_t host_pc = (uintptr_t)tb->tc.ptr;
- CPUArchState *env = cpu->env_ptr;
const uint8_t *p = tb->tc.ptr + tb->tc.size;
int i, j, num_insns = tb->icount;
#ifdef CONFIG_PROFILER
@@ -295,7 +294,20 @@ int cpu_restore_state_from_tb(CPUState *cpu, TranslationBlock *tb,
and shift if to the number of actually executed instructions */
cpu_neg(cpu)->icount_decr.u16.low += num_insns - i;
}
- restore_state_to_opc(env, tb, data);
+
+ {
+ const struct TCGCPUOps *ops = cpu->cc->tcg_ops;
+ __typeof(ops->restore_state_to_opc) restore = ops->restore_state_to_opc;
+ if (restore) {
+ uint64_t d64[TARGET_INSN_START_WORDS];
+ for (i = 0; i < TARGET_INSN_START_WORDS; ++i) {
+ d64[i] = data[i];
+ }
+ restore(cpu, tb, d64);
+ } else {
+ restore_state_to_opc(cpu->env_ptr, tb, data);
+ }
+ }
#ifdef CONFIG_PROFILER
qatomic_set(&prof->restore_time,
@@ -307,6 +319,14 @@ int cpu_restore_state_from_tb(CPUState *cpu, TranslationBlock *tb,
bool cpu_restore_state(CPUState *cpu, uintptr_t host_pc, bool will_exit)
{
+ /*
+ * The pc update associated with restore without exit will
+ * break the relative pc adjustments performed by TARGET_TB_PCREL.
+ */
+ if (TARGET_TB_PCREL) {
+ assert(will_exit);
+ }
+
/*
* The host_pc has to be in the rx region of the code buffer.
* If it is not we will not be able to resolve it here.
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 26/47] target/alpha: Convert to tcg_ops restore_state_to_opc
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (24 preceding siblings ...)
2022-10-26 2:10 ` [PULL 25/47] accel/tcg: Add restore_state_to_opc to TCGCPUOps Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 27/47] target/arm: " Richard Henderson
` (21 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Philippe Mathieu-Daudé
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/alpha/cpu.c | 9 +++++++++
target/alpha/translate.c | 6 ------
2 files changed, 9 insertions(+), 6 deletions(-)
diff --git a/target/alpha/cpu.c b/target/alpha/cpu.c
index 979a629d59..270ae787b1 100644
--- a/target/alpha/cpu.c
+++ b/target/alpha/cpu.c
@@ -40,6 +40,14 @@ static vaddr alpha_cpu_get_pc(CPUState *cs)
return cpu->env.pc;
}
+static void alpha_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data)
+{
+ AlphaCPU *cpu = ALPHA_CPU(cs);
+
+ cpu->env.pc = data[0];
+}
static bool alpha_cpu_has_work(CPUState *cs)
{
@@ -226,6 +234,7 @@ static const struct SysemuCPUOps alpha_sysemu_ops = {
static const struct TCGCPUOps alpha_tcg_ops = {
.initialize = alpha_translate_init,
+ .restore_state_to_opc = alpha_restore_state_to_opc,
#ifdef CONFIG_USER_ONLY
.record_sigsegv = alpha_cpu_record_sigsegv,
diff --git a/target/alpha/translate.c b/target/alpha/translate.c
index 6766350f56..f9bcdeb717 100644
--- a/target/alpha/translate.c
+++ b/target/alpha/translate.c
@@ -3049,9 +3049,3 @@ void gen_intermediate_code(CPUState *cpu, TranslationBlock *tb, int max_insns,
DisasContext dc;
translator_loop(cpu, tb, max_insns, pc, host_pc, &alpha_tr_ops, &dc.base);
}
-
-void restore_state_to_opc(CPUAlphaState *env, TranslationBlock *tb,
- target_ulong *data)
-{
- env->pc = data[0];
-}
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 27/47] target/arm: Convert to tcg_ops restore_state_to_opc
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (25 preceding siblings ...)
2022-10-26 2:10 ` [PULL 26/47] target/alpha: Convert to tcg_ops restore_state_to_opc Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 28/47] target/avr: " Richard Henderson
` (20 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Philippe Mathieu-Daudé
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/arm/cpu.c | 26 ++++++++++++++++++++++++++
target/arm/translate.c | 22 ----------------------
2 files changed, 26 insertions(+), 22 deletions(-)
diff --git a/target/arm/cpu.c b/target/arm/cpu.c
index 0bc5e9b125..0a7bfbf999 100644
--- a/target/arm/cpu.c
+++ b/target/arm/cpu.c
@@ -90,6 +90,31 @@ void arm_cpu_synchronize_from_tb(CPUState *cs,
}
}
}
+
+static void arm_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data)
+{
+ CPUARMState *env = cs->env_ptr;
+
+ if (is_a64(env)) {
+ if (TARGET_TB_PCREL) {
+ env->pc = (env->pc & TARGET_PAGE_MASK) | data[0];
+ } else {
+ env->pc = data[0];
+ }
+ env->condexec_bits = 0;
+ env->exception.syndrome = data[2] << ARM_INSN_START_WORD2_SHIFT;
+ } else {
+ if (TARGET_TB_PCREL) {
+ env->regs[15] = (env->regs[15] & TARGET_PAGE_MASK) | data[0];
+ } else {
+ env->regs[15] = data[0];
+ }
+ env->condexec_bits = data[1];
+ env->exception.syndrome = data[2] << ARM_INSN_START_WORD2_SHIFT;
+ }
+}
#endif /* CONFIG_TCG */
static bool arm_cpu_has_work(CPUState *cs)
@@ -2152,6 +2177,7 @@ static const struct TCGCPUOps arm_tcg_ops = {
.initialize = arm_translate_init,
.synchronize_from_tb = arm_cpu_synchronize_from_tb,
.debug_excp_handler = arm_debug_excp_handler,
+ .restore_state_to_opc = arm_restore_state_to_opc,
#ifdef CONFIG_USER_ONLY
.record_sigsegv = arm_cpu_record_sigsegv,
diff --git a/target/arm/translate.c b/target/arm/translate.c
index d1b868430e..74a903072f 100644
--- a/target/arm/translate.c
+++ b/target/arm/translate.c
@@ -9939,25 +9939,3 @@ void gen_intermediate_code(CPUState *cpu, TranslationBlock *tb, int max_insns,
translator_loop(cpu, tb, max_insns, pc, host_pc, ops, &dc.base);
}
-
-void restore_state_to_opc(CPUARMState *env, TranslationBlock *tb,
- target_ulong *data)
-{
- if (is_a64(env)) {
- if (TARGET_TB_PCREL) {
- env->pc = (env->pc & TARGET_PAGE_MASK) | data[0];
- } else {
- env->pc = data[0];
- }
- env->condexec_bits = 0;
- env->exception.syndrome = data[2] << ARM_INSN_START_WORD2_SHIFT;
- } else {
- if (TARGET_TB_PCREL) {
- env->regs[15] = (env->regs[15] & TARGET_PAGE_MASK) | data[0];
- } else {
- env->regs[15] = data[0];
- }
- env->condexec_bits = data[1];
- env->exception.syndrome = data[2] << ARM_INSN_START_WORD2_SHIFT;
- }
-}
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 28/47] target/avr: Convert to tcg_ops restore_state_to_opc
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (26 preceding siblings ...)
2022-10-26 2:10 ` [PULL 27/47] target/arm: " Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 29/47] target/cris: " Richard Henderson
` (19 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Philippe Mathieu-Daudé
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/avr/cpu.c | 11 +++++++++++
target/avr/translate.c | 6 ------
2 files changed, 11 insertions(+), 6 deletions(-)
diff --git a/target/avr/cpu.c b/target/avr/cpu.c
index 0d2861179d..c7295b488d 100644
--- a/target/avr/cpu.c
+++ b/target/avr/cpu.c
@@ -57,6 +57,16 @@ static void avr_cpu_synchronize_from_tb(CPUState *cs,
env->pc_w = tb_pc(tb) / 2; /* internally PC points to words */
}
+static void avr_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data)
+{
+ AVRCPU *cpu = AVR_CPU(cs);
+ CPUAVRState *env = &cpu->env;
+
+ env->pc_w = data[0];
+}
+
static void avr_cpu_reset(DeviceState *ds)
{
CPUState *cs = CPU(ds);
@@ -202,6 +212,7 @@ static const struct SysemuCPUOps avr_sysemu_ops = {
static const struct TCGCPUOps avr_tcg_ops = {
.initialize = avr_cpu_tcg_init,
.synchronize_from_tb = avr_cpu_synchronize_from_tb,
+ .restore_state_to_opc = avr_restore_state_to_opc,
.cpu_exec_interrupt = avr_cpu_exec_interrupt,
.tlb_fill = avr_cpu_tlb_fill,
.do_interrupt = avr_cpu_do_interrupt,
diff --git a/target/avr/translate.c b/target/avr/translate.c
index e65b6008c0..2bed56f135 100644
--- a/target/avr/translate.c
+++ b/target/avr/translate.c
@@ -3055,9 +3055,3 @@ void gen_intermediate_code(CPUState *cs, TranslationBlock *tb, int max_insns,
DisasContext dc = { };
translator_loop(cs, tb, max_insns, pc, host_pc, &avr_tr_ops, &dc.base);
}
-
-void restore_state_to_opc(CPUAVRState *env, TranslationBlock *tb,
- target_ulong *data)
-{
- env->pc_w = data[0];
-}
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 29/47] target/cris: Convert to tcg_ops restore_state_to_opc
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (27 preceding siblings ...)
2022-10-26 2:10 ` [PULL 28/47] target/avr: " Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:10 ` [PULL 30/47] target/hexagon: " Richard Henderson
` (18 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Philippe Mathieu-Daudé
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/cris/cpu.c | 11 +++++++++++
target/cris/translate.c | 6 ------
2 files changed, 11 insertions(+), 6 deletions(-)
diff --git a/target/cris/cpu.c b/target/cris/cpu.c
index 22f5c70f39..fb05dc6f9a 100644
--- a/target/cris/cpu.c
+++ b/target/cris/cpu.c
@@ -42,6 +42,15 @@ static vaddr cris_cpu_get_pc(CPUState *cs)
return cpu->env.pc;
}
+static void cris_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data)
+{
+ CRISCPU *cpu = CRIS_CPU(cs);
+
+ cpu->env.pc = data[0];
+}
+
static bool cris_cpu_has_work(CPUState *cs)
{
return cs->interrupt_request & (CPU_INTERRUPT_HARD | CPU_INTERRUPT_NMI);
@@ -212,6 +221,7 @@ static const struct SysemuCPUOps cris_sysemu_ops = {
static const struct TCGCPUOps crisv10_tcg_ops = {
.initialize = cris_initialize_crisv10_tcg,
+ .restore_state_to_opc = cris_restore_state_to_opc,
#ifndef CONFIG_USER_ONLY
.tlb_fill = cris_cpu_tlb_fill,
@@ -222,6 +232,7 @@ static const struct TCGCPUOps crisv10_tcg_ops = {
static const struct TCGCPUOps crisv32_tcg_ops = {
.initialize = cris_initialize_tcg,
+ .restore_state_to_opc = cris_restore_state_to_opc,
#ifndef CONFIG_USER_ONLY
.tlb_fill = cris_cpu_tlb_fill,
diff --git a/target/cris/translate.c b/target/cris/translate.c
index 73385b0b3c..fbc3fd5865 100644
--- a/target/cris/translate.c
+++ b/target/cris/translate.c
@@ -3392,9 +3392,3 @@ void cris_initialize_tcg(void)
pregnames_v32[i]);
}
}
-
-void restore_state_to_opc(CPUCRISState *env, TranslationBlock *tb,
- target_ulong *data)
-{
- env->pc = data[0];
-}
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 30/47] target/hexagon: Convert to tcg_ops restore_state_to_opc
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (28 preceding siblings ...)
2022-10-26 2:10 ` [PULL 29/47] target/cris: " Richard Henderson
@ 2022-10-26 2:10 ` Richard Henderson
2022-10-26 2:11 ` [PULL 31/47] target/hppa: " Richard Henderson
` (17 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:10 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Philippe Mathieu-Daudé
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/hexagon/cpu.c | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/target/hexagon/cpu.c b/target/hexagon/cpu.c
index fa6d722555..03221fbdc2 100644
--- a/target/hexagon/cpu.c
+++ b/target/hexagon/cpu.c
@@ -271,9 +271,13 @@ static bool hexagon_cpu_has_work(CPUState *cs)
return true;
}
-void restore_state_to_opc(CPUHexagonState *env, TranslationBlock *tb,
- target_ulong *data)
+static void hexagon_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data)
{
+ HexagonCPU *cpu = HEXAGON_CPU(cs);
+ CPUHexagonState *env = &cpu->env;
+
env->gpr[HEX_REG_PC] = data[0];
}
@@ -327,6 +331,7 @@ static void hexagon_cpu_init(Object *obj)
static const struct TCGCPUOps hexagon_tcg_ops = {
.initialize = hexagon_translate_init,
.synchronize_from_tb = hexagon_cpu_synchronize_from_tb,
+ .restore_state_to_opc = hexagon_restore_state_to_opc,
};
static void hexagon_cpu_class_init(ObjectClass *c, void *data)
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 31/47] target/hppa: Convert to tcg_ops restore_state_to_opc
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (29 preceding siblings ...)
2022-10-26 2:10 ` [PULL 30/47] target/hexagon: " Richard Henderson
@ 2022-10-26 2:11 ` Richard Henderson
2022-10-26 2:11 ` [PULL 32/47] target/i386: " Richard Henderson
` (16 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:11 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Philippe Mathieu-Daudé
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/hppa/cpu.c | 19 +++++++++++++++++++
target/hppa/translate.c | 13 -------------
2 files changed, 19 insertions(+), 13 deletions(-)
diff --git a/target/hppa/cpu.c b/target/hppa/cpu.c
index e677ca09d4..55c190280e 100644
--- a/target/hppa/cpu.c
+++ b/target/hppa/cpu.c
@@ -68,6 +68,24 @@ static void hppa_cpu_synchronize_from_tb(CPUState *cs,
cpu->env.psw_n = (tb->flags & PSW_N) != 0;
}
+static void hppa_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data)
+{
+ HPPACPU *cpu = HPPA_CPU(cs);
+
+ cpu->env.iaoq_f = data[0];
+ if (data[1] != (target_ureg)-1) {
+ cpu->env.iaoq_b = data[1];
+ }
+ /*
+ * Since we were executing the instruction at IAOQ_F, and took some
+ * sort of action that provoked the cpu_restore_state, we can infer
+ * that the instruction was not nullified.
+ */
+ cpu->env.psw_n = 0;
+}
+
static bool hppa_cpu_has_work(CPUState *cs)
{
return cs->interrupt_request & (CPU_INTERRUPT_HARD | CPU_INTERRUPT_NMI);
@@ -153,6 +171,7 @@ static const struct SysemuCPUOps hppa_sysemu_ops = {
static const struct TCGCPUOps hppa_tcg_ops = {
.initialize = hppa_translate_init,
.synchronize_from_tb = hppa_cpu_synchronize_from_tb,
+ .restore_state_to_opc = hppa_restore_state_to_opc,
#ifndef CONFIG_USER_ONLY
.tlb_fill = hppa_cpu_tlb_fill,
diff --git a/target/hppa/translate.c b/target/hppa/translate.c
index 8b861957e0..1af77473da 100644
--- a/target/hppa/translate.c
+++ b/target/hppa/translate.c
@@ -4346,16 +4346,3 @@ void gen_intermediate_code(CPUState *cs, TranslationBlock *tb, int max_insns,
DisasContext ctx;
translator_loop(cs, tb, max_insns, pc, host_pc, &hppa_tr_ops, &ctx.base);
}
-
-void restore_state_to_opc(CPUHPPAState *env, TranslationBlock *tb,
- target_ulong *data)
-{
- env->iaoq_f = data[0];
- if (data[1] != (target_ureg)-1) {
- env->iaoq_b = data[1];
- }
- /* Since we were executing the instruction at IAOQ_F, and took some
- sort of action that provoked the cpu_restore_state, we can infer
- that the instruction was not nullified. */
- env->psw_n = 0;
-}
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 32/47] target/i386: Convert to tcg_ops restore_state_to_opc
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (30 preceding siblings ...)
2022-10-26 2:11 ` [PULL 31/47] target/hppa: " Richard Henderson
@ 2022-10-26 2:11 ` Richard Henderson
2022-10-26 2:11 ` [PULL 33/47] target/loongarch: " Richard Henderson
` (15 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:11 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Philippe Mathieu-Daudé
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/i386/tcg/tcg-cpu.c | 19 +++++++++++++++++++
target/i386/tcg/translate.c | 15 ---------------
2 files changed, 19 insertions(+), 15 deletions(-)
diff --git a/target/i386/tcg/tcg-cpu.c b/target/i386/tcg/tcg-cpu.c
index 828244abe2..79ac5908f7 100644
--- a/target/i386/tcg/tcg-cpu.c
+++ b/target/i386/tcg/tcg-cpu.c
@@ -56,6 +56,24 @@ static void x86_cpu_synchronize_from_tb(CPUState *cs,
}
}
+static void x86_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data)
+{
+ X86CPU *cpu = X86_CPU(cs);
+ CPUX86State *env = &cpu->env;
+ int cc_op = data[1];
+
+ if (TARGET_TB_PCREL) {
+ env->eip = (env->eip & TARGET_PAGE_MASK) | data[0];
+ } else {
+ env->eip = data[0] - tb->cs_base;
+ }
+ if (cc_op != CC_OP_DYNAMIC) {
+ env->cc_op = cc_op;
+ }
+}
+
#ifndef CONFIG_USER_ONLY
static bool x86_debug_check_breakpoint(CPUState *cs)
{
@@ -72,6 +90,7 @@ static bool x86_debug_check_breakpoint(CPUState *cs)
static const struct TCGCPUOps x86_tcg_ops = {
.initialize = tcg_x86_init,
.synchronize_from_tb = x86_cpu_synchronize_from_tb,
+ .restore_state_to_opc = x86_restore_state_to_opc,
.cpu_exec_enter = x86_cpu_exec_enter,
.cpu_exec_exit = x86_cpu_exec_exit,
#ifdef CONFIG_USER_ONLY
diff --git a/target/i386/tcg/translate.c b/target/i386/tcg/translate.c
index 85be2e58c2..546c427c23 100644
--- a/target/i386/tcg/translate.c
+++ b/target/i386/tcg/translate.c
@@ -7023,18 +7023,3 @@ void gen_intermediate_code(CPUState *cpu, TranslationBlock *tb, int max_insns,
translator_loop(cpu, tb, max_insns, pc, host_pc, &i386_tr_ops, &dc.base);
}
-
-void restore_state_to_opc(CPUX86State *env, TranslationBlock *tb,
- target_ulong *data)
-{
- int cc_op = data[1];
-
- if (TARGET_TB_PCREL) {
- env->eip = (env->eip & TARGET_PAGE_MASK) | data[0];
- } else {
- env->eip = data[0] - tb->cs_base;
- }
- if (cc_op != CC_OP_DYNAMIC) {
- env->cc_op = cc_op;
- }
-}
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 33/47] target/loongarch: Convert to tcg_ops restore_state_to_opc
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (31 preceding siblings ...)
2022-10-26 2:11 ` [PULL 32/47] target/i386: " Richard Henderson
@ 2022-10-26 2:11 ` Richard Henderson
2022-10-26 2:11 ` [PULL 34/47] target/m68k: " Richard Henderson
` (14 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:11 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Philippe Mathieu-Daudé
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/loongarch/cpu.c | 11 +++++++++++
target/loongarch/translate.c | 6 ------
2 files changed, 11 insertions(+), 6 deletions(-)
diff --git a/target/loongarch/cpu.c b/target/loongarch/cpu.c
index 1722ed2a4d..49393d95d8 100644
--- a/target/loongarch/cpu.c
+++ b/target/loongarch/cpu.c
@@ -319,6 +319,16 @@ static void loongarch_cpu_synchronize_from_tb(CPUState *cs,
env->pc = tb_pc(tb);
}
+
+static void loongarch_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data)
+{
+ LoongArchCPU *cpu = LOONGARCH_CPU(cs);
+ CPULoongArchState *env = &cpu->env;
+
+ env->pc = data[0];
+}
#endif /* CONFIG_TCG */
static bool loongarch_cpu_has_work(CPUState *cs)
@@ -651,6 +661,7 @@ void loongarch_cpu_dump_state(CPUState *cs, FILE *f, int flags)
static struct TCGCPUOps loongarch_tcg_ops = {
.initialize = loongarch_translate_init,
.synchronize_from_tb = loongarch_cpu_synchronize_from_tb,
+ .restore_state_to_opc = loongarch_restore_state_to_opc,
#ifndef CONFIG_USER_ONLY
.tlb_fill = loongarch_cpu_tlb_fill,
diff --git a/target/loongarch/translate.c b/target/loongarch/translate.c
index 95b37ea180..6091772349 100644
--- a/target/loongarch/translate.c
+++ b/target/loongarch/translate.c
@@ -272,9 +272,3 @@ void loongarch_translate_init(void)
cpu_llval = tcg_global_mem_new(cpu_env,
offsetof(CPULoongArchState, llval), "llval");
}
-
-void restore_state_to_opc(CPULoongArchState *env, TranslationBlock *tb,
- target_ulong *data)
-{
- env->pc = data[0];
-}
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 34/47] target/m68k: Convert to tcg_ops restore_state_to_opc
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (32 preceding siblings ...)
2022-10-26 2:11 ` [PULL 33/47] target/loongarch: " Richard Henderson
@ 2022-10-26 2:11 ` Richard Henderson
2022-10-26 2:11 ` [PULL 35/47] target/microblaze: " Richard Henderson
` (13 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:11 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Philippe Mathieu-Daudé
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/m68k/cpu.c | 14 ++++++++++++++
target/m68k/translate.c | 10 ----------
2 files changed, 14 insertions(+), 10 deletions(-)
diff --git a/target/m68k/cpu.c b/target/m68k/cpu.c
index 1e902e1ef0..b67ddea2ae 100644
--- a/target/m68k/cpu.c
+++ b/target/m68k/cpu.c
@@ -38,6 +38,19 @@ static vaddr m68k_cpu_get_pc(CPUState *cs)
return cpu->env.pc;
}
+static void m68k_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data)
+{
+ M68kCPU *cpu = M68K_CPU(cs);
+ int cc_op = data[1];
+
+ cpu->env.pc = data[0];
+ if (cc_op != CC_OP_DYNAMIC) {
+ cpu->env.cc_op = cc_op;
+ }
+}
+
static bool m68k_cpu_has_work(CPUState *cs)
{
return cs->interrupt_request & CPU_INTERRUPT_HARD;
@@ -524,6 +537,7 @@ static const struct SysemuCPUOps m68k_sysemu_ops = {
static const struct TCGCPUOps m68k_tcg_ops = {
.initialize = m68k_tcg_init,
+ .restore_state_to_opc = m68k_restore_state_to_opc,
#ifndef CONFIG_USER_ONLY
.tlb_fill = m68k_cpu_tlb_fill,
diff --git a/target/m68k/translate.c b/target/m68k/translate.c
index 9df17aa4b2..5cbde4be34 100644
--- a/target/m68k/translate.c
+++ b/target/m68k/translate.c
@@ -6479,13 +6479,3 @@ void m68k_cpu_dump_state(CPUState *cs, FILE *f, int flags)
env->mmu.mmusr, env->mmu.ar);
#endif
}
-
-void restore_state_to_opc(CPUM68KState *env, TranslationBlock *tb,
- target_ulong *data)
-{
- int cc_op = data[1];
- env->pc = data[0];
- if (cc_op != CC_OP_DYNAMIC) {
- env->cc_op = cc_op;
- }
-}
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 35/47] target/microblaze: Convert to tcg_ops restore_state_to_opc
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (33 preceding siblings ...)
2022-10-26 2:11 ` [PULL 34/47] target/m68k: " Richard Henderson
@ 2022-10-26 2:11 ` Richard Henderson
2022-10-26 2:11 ` [PULL 36/47] target/mips: " Richard Henderson
` (12 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:11 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Philippe Mathieu-Daudé
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/microblaze/cpu.c | 11 +++++++++++
target/microblaze/translate.c | 7 -------
2 files changed, 11 insertions(+), 7 deletions(-)
diff --git a/target/microblaze/cpu.c b/target/microblaze/cpu.c
index c10b8ac029..89e493f3ff 100644
--- a/target/microblaze/cpu.c
+++ b/target/microblaze/cpu.c
@@ -100,6 +100,16 @@ static void mb_cpu_synchronize_from_tb(CPUState *cs,
cpu->env.iflags = tb->flags & IFLAGS_TB_MASK;
}
+static void mb_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data)
+{
+ MicroBlazeCPU *cpu = MICROBLAZE_CPU(cs);
+
+ cpu->env.pc = data[0];
+ cpu->env.iflags = data[1];
+}
+
static bool mb_cpu_has_work(CPUState *cs)
{
return cs->interrupt_request & (CPU_INTERRUPT_HARD | CPU_INTERRUPT_NMI);
@@ -373,6 +383,7 @@ static const struct SysemuCPUOps mb_sysemu_ops = {
static const struct TCGCPUOps mb_tcg_ops = {
.initialize = mb_tcg_init,
.synchronize_from_tb = mb_cpu_synchronize_from_tb,
+ .restore_state_to_opc = mb_restore_state_to_opc,
#ifndef CONFIG_USER_ONLY
.tlb_fill = mb_cpu_tlb_fill,
diff --git a/target/microblaze/translate.c b/target/microblaze/translate.c
index c5546f93aa..974f21eb31 100644
--- a/target/microblaze/translate.c
+++ b/target/microblaze/translate.c
@@ -1946,10 +1946,3 @@ void mb_tcg_init(void)
cpu_res_addr =
tcg_global_mem_new(cpu_env, offsetof(CPUMBState, res_addr), "res_addr");
}
-
-void restore_state_to_opc(CPUMBState *env, TranslationBlock *tb,
- target_ulong *data)
-{
- env->pc = data[0];
- env->iflags = data[1];
-}
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 36/47] target/mips: Convert to tcg_ops restore_state_to_opc
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (34 preceding siblings ...)
2022-10-26 2:11 ` [PULL 35/47] target/microblaze: " Richard Henderson
@ 2022-10-26 2:11 ` Richard Henderson
2022-10-26 2:11 ` [PULL 37/47] target/nios2: " Richard Henderson
` (11 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:11 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Philippe Mathieu-Daudé
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/mips/tcg/tcg-internal.h | 3 +++
target/mips/cpu.c | 1 +
target/mips/tcg/translate.c | 8 ++++++--
3 files changed, 10 insertions(+), 2 deletions(-)
diff --git a/target/mips/tcg/tcg-internal.h b/target/mips/tcg/tcg-internal.h
index 1d27fa2ff9..aef032c48d 100644
--- a/target/mips/tcg/tcg-internal.h
+++ b/target/mips/tcg/tcg-internal.h
@@ -21,6 +21,9 @@ void mips_cpu_synchronize_from_tb(CPUState *cs, const TranslationBlock *tb);
G_NORETURN void mips_cpu_do_unaligned_access(CPUState *cpu, vaddr addr,
MMUAccessType access_type, int mmu_idx,
uintptr_t retaddr);
+void mips_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data);
const char *mips_exception_name(int32_t exception);
diff --git a/target/mips/cpu.c b/target/mips/cpu.c
index da58eb8892..e997c1b9cb 100644
--- a/target/mips/cpu.c
+++ b/target/mips/cpu.c
@@ -538,6 +538,7 @@ static const struct SysemuCPUOps mips_sysemu_ops = {
static const struct TCGCPUOps mips_tcg_ops = {
.initialize = mips_tcg_init,
.synchronize_from_tb = mips_cpu_synchronize_from_tb,
+ .restore_state_to_opc = mips_restore_state_to_opc,
#if !defined(CONFIG_USER_ONLY)
.tlb_fill = mips_cpu_tlb_fill,
diff --git a/target/mips/tcg/translate.c b/target/mips/tcg/translate.c
index c3f92ea652..2f2d707a12 100644
--- a/target/mips/tcg/translate.c
+++ b/target/mips/tcg/translate.c
@@ -16229,9 +16229,13 @@ void mips_tcg_init(void)
}
}
-void restore_state_to_opc(CPUMIPSState *env, TranslationBlock *tb,
- target_ulong *data)
+void mips_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data)
{
+ MIPSCPU *cpu = MIPS_CPU(cs);
+ CPUMIPSState *env = &cpu->env;
+
env->active_tc.PC = data[0];
env->hflags &= ~MIPS_HFLAG_BMASK;
env->hflags |= data[1];
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 37/47] target/nios2: Convert to tcg_ops restore_state_to_opc
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (35 preceding siblings ...)
2022-10-26 2:11 ` [PULL 36/47] target/mips: " Richard Henderson
@ 2022-10-26 2:11 ` Richard Henderson
2022-10-26 2:11 ` [PULL 38/47] target/openrisc: " Richard Henderson
` (10 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:11 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Philippe Mathieu-Daudé
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/nios2/cpu.c | 11 +++++++++++
target/nios2/translate.c | 6 ------
2 files changed, 11 insertions(+), 6 deletions(-)
diff --git a/target/nios2/cpu.c b/target/nios2/cpu.c
index 2b28429c08..9a5351bc81 100644
--- a/target/nios2/cpu.c
+++ b/target/nios2/cpu.c
@@ -42,6 +42,16 @@ static vaddr nios2_cpu_get_pc(CPUState *cs)
return env->pc;
}
+static void nios2_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data)
+{
+ Nios2CPU *cpu = NIOS2_CPU(cs);
+ CPUNios2State *env = &cpu->env;
+
+ env->pc = data[0];
+}
+
static bool nios2_cpu_has_work(CPUState *cs)
{
return cs->interrupt_request & CPU_INTERRUPT_HARD;
@@ -346,6 +356,7 @@ static const struct SysemuCPUOps nios2_sysemu_ops = {
static const struct TCGCPUOps nios2_tcg_ops = {
.initialize = nios2_tcg_init,
+ .restore_state_to_opc = nios2_restore_state_to_opc,
#ifndef CONFIG_USER_ONLY
.tlb_fill = nios2_cpu_tlb_fill,
diff --git a/target/nios2/translate.c b/target/nios2/translate.c
index 8dc0a32c6c..4db8b47744 100644
--- a/target/nios2/translate.c
+++ b/target/nios2/translate.c
@@ -1110,9 +1110,3 @@ void nios2_tcg_init(void)
cpu_pc = tcg_global_mem_new(cpu_env,
offsetof(CPUNios2State, pc), "pc");
}
-
-void restore_state_to_opc(CPUNios2State *env, TranslationBlock *tb,
- target_ulong *data)
-{
- env->pc = data[0];
-}
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 38/47] target/openrisc: Convert to tcg_ops restore_state_to_opc
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (36 preceding siblings ...)
2022-10-26 2:11 ` [PULL 37/47] target/nios2: " Richard Henderson
@ 2022-10-26 2:11 ` Richard Henderson
2022-10-26 2:11 ` [PULL 39/47] target/ppc: " Richard Henderson
` (9 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:11 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Philippe Mathieu-Daudé
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/openrisc/cpu.c | 13 +++++++++++++
target/openrisc/translate.c | 10 ----------
2 files changed, 13 insertions(+), 10 deletions(-)
diff --git a/target/openrisc/cpu.c b/target/openrisc/cpu.c
index f6fd437785..de0176cd20 100644
--- a/target/openrisc/cpu.c
+++ b/target/openrisc/cpu.c
@@ -46,6 +46,18 @@ static void openrisc_cpu_synchronize_from_tb(CPUState *cs,
cpu->env.pc = tb_pc(tb);
}
+static void openrisc_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data)
+{
+ OpenRISCCPU *cpu = OPENRISC_CPU(cs);
+
+ cpu->env.pc = data[0];
+ cpu->env.dflag = data[1] & 1;
+ if (data[1] & 2) {
+ cpu->env.ppc = cpu->env.pc - 4;
+ }
+}
static bool openrisc_cpu_has_work(CPUState *cs)
{
@@ -203,6 +215,7 @@ static const struct SysemuCPUOps openrisc_sysemu_ops = {
static const struct TCGCPUOps openrisc_tcg_ops = {
.initialize = openrisc_translate_init,
.synchronize_from_tb = openrisc_cpu_synchronize_from_tb,
+ .restore_state_to_opc = openrisc_restore_state_to_opc,
#ifndef CONFIG_USER_ONLY
.tlb_fill = openrisc_cpu_tlb_fill,
diff --git a/target/openrisc/translate.c b/target/openrisc/translate.c
index 8154f9d744..2f3d7c5fd1 100644
--- a/target/openrisc/translate.c
+++ b/target/openrisc/translate.c
@@ -1726,13 +1726,3 @@ void openrisc_cpu_dump_state(CPUState *cs, FILE *f, int flags)
(i % 4) == 3 ? '\n' : ' ');
}
}
-
-void restore_state_to_opc(CPUOpenRISCState *env, TranslationBlock *tb,
- target_ulong *data)
-{
- env->pc = data[0];
- env->dflag = data[1] & 1;
- if (data[1] & 2) {
- env->ppc = env->pc - 4;
- }
-}
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 39/47] target/ppc: Convert to tcg_ops restore_state_to_opc
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (37 preceding siblings ...)
2022-10-26 2:11 ` [PULL 38/47] target/openrisc: " Richard Henderson
@ 2022-10-26 2:11 ` Richard Henderson
2022-10-26 2:11 ` [PULL 40/47] target/riscv: " Richard Henderson
` (8 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:11 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Philippe Mathieu-Daudé
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/ppc/cpu_init.c | 10 ++++++++++
target/ppc/translate.c | 6 ------
2 files changed, 10 insertions(+), 6 deletions(-)
diff --git a/target/ppc/cpu_init.c b/target/ppc/cpu_init.c
index 763a8431be..335351c226 100644
--- a/target/ppc/cpu_init.c
+++ b/target/ppc/cpu_init.c
@@ -7221,6 +7221,15 @@ static vaddr ppc_cpu_get_pc(CPUState *cs)
return cpu->env.nip;
}
+static void ppc_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data)
+{
+ PowerPCCPU *cpu = POWERPC_CPU(cs);
+
+ cpu->env.nip = data[0];
+}
+
static bool ppc_cpu_has_work(CPUState *cs)
{
PowerPCCPU *cpu = POWERPC_CPU(cs);
@@ -7446,6 +7455,7 @@ static const struct SysemuCPUOps ppc_sysemu_ops = {
static const struct TCGCPUOps ppc_tcg_ops = {
.initialize = ppc_translate_init,
+ .restore_state_to_opc = ppc_restore_state_to_opc,
#ifdef CONFIG_USER_ONLY
.record_sigsegv = ppc_cpu_record_sigsegv,
diff --git a/target/ppc/translate.c b/target/ppc/translate.c
index e810842925..7228857e23 100644
--- a/target/ppc/translate.c
+++ b/target/ppc/translate.c
@@ -7739,9 +7739,3 @@ void gen_intermediate_code(CPUState *cs, TranslationBlock *tb, int max_insns,
translator_loop(cs, tb, max_insns, pc, host_pc, &ppc_tr_ops, &ctx.base);
}
-
-void restore_state_to_opc(CPUPPCState *env, TranslationBlock *tb,
- target_ulong *data)
-{
- env->nip = data[0];
-}
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 40/47] target/riscv: Convert to tcg_ops restore_state_to_opc
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (38 preceding siblings ...)
2022-10-26 2:11 ` [PULL 39/47] target/ppc: " Richard Henderson
@ 2022-10-26 2:11 ` Richard Henderson
2022-10-26 2:11 ` [PULL 41/47] target/rx: " Richard Henderson
` (7 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:11 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Philippe Mathieu-Daudé
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/riscv/cpu.c | 9 +++++++--
1 file changed, 7 insertions(+), 2 deletions(-)
diff --git a/target/riscv/cpu.c b/target/riscv/cpu.c
index e6d9c706bb..d14e95c9dc 100644
--- a/target/riscv/cpu.c
+++ b/target/riscv/cpu.c
@@ -503,10 +503,14 @@ static bool riscv_cpu_has_work(CPUState *cs)
#endif
}
-void restore_state_to_opc(CPURISCVState *env, TranslationBlock *tb,
- target_ulong *data)
+static void riscv_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data)
{
+ RISCVCPU *cpu = RISCV_CPU(cs);
+ CPURISCVState *env = &cpu->env;
RISCVMXL xl = FIELD_EX32(tb->flags, TB_FLAGS, XL);
+
if (xl == MXL_RV32) {
env->pc = (int32_t)data[0];
} else {
@@ -1138,6 +1142,7 @@ static const struct SysemuCPUOps riscv_sysemu_ops = {
static const struct TCGCPUOps riscv_tcg_ops = {
.initialize = riscv_translate_init,
.synchronize_from_tb = riscv_cpu_synchronize_from_tb,
+ .restore_state_to_opc = riscv_restore_state_to_opc,
#ifndef CONFIG_USER_ONLY
.tlb_fill = riscv_cpu_tlb_fill,
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 41/47] target/rx: Convert to tcg_ops restore_state_to_opc
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (39 preceding siblings ...)
2022-10-26 2:11 ` [PULL 40/47] target/riscv: " Richard Henderson
@ 2022-10-26 2:11 ` Richard Henderson
2022-10-26 2:11 ` [PULL 42/47] target/s390x: " Richard Henderson
` (6 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:11 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Philippe Mathieu-Daudé
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/rx/cpu.c | 10 ++++++++++
target/rx/translate.c | 6 ------
2 files changed, 10 insertions(+), 6 deletions(-)
diff --git a/target/rx/cpu.c b/target/rx/cpu.c
index 2f28099723..9003c6e9fe 100644
--- a/target/rx/cpu.c
+++ b/target/rx/cpu.c
@@ -47,6 +47,15 @@ static void rx_cpu_synchronize_from_tb(CPUState *cs,
cpu->env.pc = tb_pc(tb);
}
+static void rx_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data)
+{
+ RXCPU *cpu = RX_CPU(cs);
+
+ cpu->env.pc = data[0];
+}
+
static bool rx_cpu_has_work(CPUState *cs)
{
return cs->interrupt_request &
@@ -192,6 +201,7 @@ static const struct SysemuCPUOps rx_sysemu_ops = {
static const struct TCGCPUOps rx_tcg_ops = {
.initialize = rx_translate_init,
.synchronize_from_tb = rx_cpu_synchronize_from_tb,
+ .restore_state_to_opc = rx_restore_state_to_opc,
.tlb_fill = rx_cpu_tlb_fill,
#ifndef CONFIG_USER_ONLY
diff --git a/target/rx/translate.c b/target/rx/translate.c
index ea5653bc95..87a3f54adb 100644
--- a/target/rx/translate.c
+++ b/target/rx/translate.c
@@ -2371,12 +2371,6 @@ void gen_intermediate_code(CPUState *cs, TranslationBlock *tb, int max_insns,
translator_loop(cs, tb, max_insns, pc, host_pc, &rx_tr_ops, &dc.base);
}
-void restore_state_to_opc(CPURXState *env, TranslationBlock *tb,
- target_ulong *data)
-{
- env->pc = data[0];
-}
-
#define ALLOC_REGISTER(sym, name) \
cpu_##sym = tcg_global_mem_new_i32(cpu_env, \
offsetof(CPURXState, sym), name)
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 42/47] target/s390x: Convert to tcg_ops restore_state_to_opc
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (40 preceding siblings ...)
2022-10-26 2:11 ` [PULL 41/47] target/rx: " Richard Henderson
@ 2022-10-26 2:11 ` Richard Henderson
2022-10-26 2:11 ` [PULL 43/47] target/sh4: " Richard Henderson
` (5 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:11 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Philippe Mathieu-Daudé
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/s390x/s390x-internal.h | 4 +++-
target/s390x/cpu.c | 1 +
target/s390x/tcg/translate.c | 7 +++++--
3 files changed, 9 insertions(+), 3 deletions(-)
diff --git a/target/s390x/s390x-internal.h b/target/s390x/s390x-internal.h
index b5ae0ae364..5d4361d35b 100644
--- a/target/s390x/s390x-internal.h
+++ b/target/s390x/s390x-internal.h
@@ -398,7 +398,9 @@ void handle_diag_308(CPUS390XState *env, uint64_t r1, uint64_t r3,
/* translate.c */
void s390x_translate_init(void);
-
+void s390x_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data);
/* sigp.c */
int handle_sigp(CPUS390XState *env, uint8_t order, uint64_t r1, uint64_t r3);
diff --git a/target/s390x/cpu.c b/target/s390x/cpu.c
index df00040e95..96562c516d 100644
--- a/target/s390x/cpu.c
+++ b/target/s390x/cpu.c
@@ -272,6 +272,7 @@ static void s390_cpu_reset_full(DeviceState *dev)
static const struct TCGCPUOps s390_tcg_ops = {
.initialize = s390x_translate_init,
+ .restore_state_to_opc = s390x_restore_state_to_opc,
#ifdef CONFIG_USER_ONLY
.record_sigsegv = s390_cpu_record_sigsegv,
diff --git a/target/s390x/tcg/translate.c b/target/s390x/tcg/translate.c
index 1d2dddab1c..5798928473 100644
--- a/target/s390x/tcg/translate.c
+++ b/target/s390x/tcg/translate.c
@@ -6691,9 +6691,12 @@ void gen_intermediate_code(CPUState *cs, TranslationBlock *tb, int max_insns,
translator_loop(cs, tb, max_insns, pc, host_pc, &s390x_tr_ops, &dc.base);
}
-void restore_state_to_opc(CPUS390XState *env, TranslationBlock *tb,
- target_ulong *data)
+void s390x_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data)
{
+ S390CPU *cpu = S390_CPU(cs);
+ CPUS390XState *env = &cpu->env;
int cc_op = data[1];
env->psw.addr = data[0];
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 43/47] target/sh4: Convert to tcg_ops restore_state_to_opc
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (41 preceding siblings ...)
2022-10-26 2:11 ` [PULL 42/47] target/s390x: " Richard Henderson
@ 2022-10-26 2:11 ` Richard Henderson
2022-10-26 2:11 ` [PULL 44/47] target/sparc: " Richard Henderson
` (4 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:11 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Philippe Mathieu-Daudé
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/sh4/cpu.c | 16 ++++++++++++++++
target/sh4/translate.c | 10 ----------
2 files changed, 16 insertions(+), 10 deletions(-)
diff --git a/target/sh4/cpu.c b/target/sh4/cpu.c
index 56c50530da..453268392b 100644
--- a/target/sh4/cpu.c
+++ b/target/sh4/cpu.c
@@ -50,6 +50,21 @@ static void superh_cpu_synchronize_from_tb(CPUState *cs,
cpu->env.flags = tb->flags;
}
+static void superh_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data)
+{
+ SuperHCPU *cpu = SUPERH_CPU(cs);
+
+ cpu->env.pc = data[0];
+ cpu->env.flags = data[1];
+ /*
+ * Theoretically delayed_pc should also be restored. In practice the
+ * branch instruction is re-executed after exception, so the delayed
+ * branch target will be recomputed.
+ */
+}
+
#ifndef CONFIG_USER_ONLY
static bool superh_io_recompile_replay_branch(CPUState *cs,
const TranslationBlock *tb)
@@ -243,6 +258,7 @@ static const struct SysemuCPUOps sh4_sysemu_ops = {
static const struct TCGCPUOps superh_tcg_ops = {
.initialize = sh4_translate_init,
.synchronize_from_tb = superh_cpu_synchronize_from_tb,
+ .restore_state_to_opc = superh_restore_state_to_opc,
#ifndef CONFIG_USER_ONLY
.tlb_fill = superh_cpu_tlb_fill,
diff --git a/target/sh4/translate.c b/target/sh4/translate.c
index 26231b2a5a..7db3468b01 100644
--- a/target/sh4/translate.c
+++ b/target/sh4/translate.c
@@ -2381,13 +2381,3 @@ void gen_intermediate_code(CPUState *cs, TranslationBlock *tb, int max_insns,
translator_loop(cs, tb, max_insns, pc, host_pc, &sh4_tr_ops, &ctx.base);
}
-
-void restore_state_to_opc(CPUSH4State *env, TranslationBlock *tb,
- target_ulong *data)
-{
- env->pc = data[0];
- env->flags = data[1];
- /* Theoretically delayed_pc should also be restored. In practice the
- branch instruction is re-executed after exception, so the delayed
- branch target will be recomputed. */
-}
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 44/47] target/sparc: Convert to tcg_ops restore_state_to_opc
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (42 preceding siblings ...)
2022-10-26 2:11 ` [PULL 43/47] target/sh4: " Richard Henderson
@ 2022-10-26 2:11 ` Richard Henderson
2022-10-26 2:11 ` [PULL 45/47] target/tricore: " Richard Henderson
` (3 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:11 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Philippe Mathieu-Daudé
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/sparc/cpu.h | 3 +++
target/sparc/cpu.c | 1 +
target/sparc/translate.c | 7 +++++--
3 files changed, 9 insertions(+), 2 deletions(-)
diff --git a/target/sparc/cpu.h b/target/sparc/cpu.h
index f80ea2e8cf..e478c5eb16 100644
--- a/target/sparc/cpu.h
+++ b/target/sparc/cpu.h
@@ -600,6 +600,9 @@ int sparc_cpu_memory_rw_debug(CPUState *cpu, vaddr addr,
/* translate.c */
void sparc_tcg_init(void);
+void sparc_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data);
/* cpu-exec.c */
diff --git a/target/sparc/cpu.c b/target/sparc/cpu.c
index 1f9ef7afd8..4c3d08a875 100644
--- a/target/sparc/cpu.c
+++ b/target/sparc/cpu.c
@@ -872,6 +872,7 @@ static const struct SysemuCPUOps sparc_sysemu_ops = {
static const struct TCGCPUOps sparc_tcg_ops = {
.initialize = sparc_tcg_init,
.synchronize_from_tb = sparc_cpu_synchronize_from_tb,
+ .restore_state_to_opc = sparc_restore_state_to_opc,
#ifndef CONFIG_USER_ONLY
.tlb_fill = sparc_cpu_tlb_fill,
diff --git a/target/sparc/translate.c b/target/sparc/translate.c
index 2cbbe2396a..34858eb95f 100644
--- a/target/sparc/translate.c
+++ b/target/sparc/translate.c
@@ -6011,9 +6011,12 @@ void sparc_tcg_init(void)
}
}
-void restore_state_to_opc(CPUSPARCState *env, TranslationBlock *tb,
- target_ulong *data)
+void sparc_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data)
{
+ SPARCCPU *cpu = SPARC_CPU(cs);
+ CPUSPARCState *env = &cpu->env;
target_ulong pc = data[0];
target_ulong npc = data[1];
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 45/47] target/tricore: Convert to tcg_ops restore_state_to_opc
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (43 preceding siblings ...)
2022-10-26 2:11 ` [PULL 44/47] target/sparc: " Richard Henderson
@ 2022-10-26 2:11 ` Richard Henderson
2022-10-26 2:11 ` [PULL 46/47] target/xtensa: " Richard Henderson
` (2 subsequent siblings)
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:11 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Philippe Mathieu-Daudé
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/tricore/cpu.c | 11 +++++++++++
target/tricore/translate.c | 6 ------
2 files changed, 11 insertions(+), 6 deletions(-)
diff --git a/target/tricore/cpu.c b/target/tricore/cpu.c
index ab7a1e3a6d..2c54a2825f 100644
--- a/target/tricore/cpu.c
+++ b/target/tricore/cpu.c
@@ -58,6 +58,16 @@ static void tricore_cpu_synchronize_from_tb(CPUState *cs,
env->PC = tb_pc(tb);
}
+static void tricore_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data)
+{
+ TriCoreCPU *cpu = TRICORE_CPU(cs);
+ CPUTriCoreState *env = &cpu->env;
+
+ env->PC = data[0];
+}
+
static void tricore_cpu_reset(DeviceState *dev)
{
CPUState *s = CPU(dev);
@@ -161,6 +171,7 @@ static const struct SysemuCPUOps tricore_sysemu_ops = {
static const struct TCGCPUOps tricore_tcg_ops = {
.initialize = tricore_tcg_init,
.synchronize_from_tb = tricore_cpu_synchronize_from_tb,
+ .restore_state_to_opc = tricore_restore_state_to_opc,
.tlb_fill = tricore_cpu_tlb_fill,
};
diff --git a/target/tricore/translate.c b/target/tricore/translate.c
index a0558ead71..c5b7bfbf20 100644
--- a/target/tricore/translate.c
+++ b/target/tricore/translate.c
@@ -8886,12 +8886,6 @@ void gen_intermediate_code(CPUState *cs, TranslationBlock *tb, int max_insns,
&tricore_tr_ops, &ctx.base);
}
-void
-restore_state_to_opc(CPUTriCoreState *env, TranslationBlock *tb,
- target_ulong *data)
-{
- env->PC = data[0];
-}
/*
*
* Initialization
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 46/47] target/xtensa: Convert to tcg_ops restore_state_to_opc
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (44 preceding siblings ...)
2022-10-26 2:11 ` [PULL 45/47] target/tricore: " Richard Henderson
@ 2022-10-26 2:11 ` Richard Henderson
2022-10-26 2:11 ` [PULL 47/47] accel/tcg: Remove restore_state_to_opc function Richard Henderson
2022-10-26 18:54 ` [PULL 00/47] tcg patch queue Stefan Hajnoczi
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:11 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Philippe Mathieu-Daudé
Reviewed-by: Philippe Mathieu-Daudé <philmd@linaro.org>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
target/xtensa/cpu.c | 10 ++++++++++
target/xtensa/translate.c | 6 ------
2 files changed, 10 insertions(+), 6 deletions(-)
diff --git a/target/xtensa/cpu.c b/target/xtensa/cpu.c
index cbbe0e84a2..09923301c4 100644
--- a/target/xtensa/cpu.c
+++ b/target/xtensa/cpu.c
@@ -51,6 +51,15 @@ static vaddr xtensa_cpu_get_pc(CPUState *cs)
return cpu->env.pc;
}
+static void xtensa_restore_state_to_opc(CPUState *cs,
+ const TranslationBlock *tb,
+ const uint64_t *data)
+{
+ XtensaCPU *cpu = XTENSA_CPU(cs);
+
+ cpu->env.pc = data[0];
+}
+
static bool xtensa_cpu_has_work(CPUState *cs)
{
#ifndef CONFIG_USER_ONLY
@@ -215,6 +224,7 @@ static const struct SysemuCPUOps xtensa_sysemu_ops = {
static const struct TCGCPUOps xtensa_tcg_ops = {
.initialize = xtensa_translate_init,
.debug_excp_handler = xtensa_breakpoint_handler,
+ .restore_state_to_opc = xtensa_restore_state_to_opc,
#ifndef CONFIG_USER_ONLY
.tlb_fill = xtensa_cpu_tlb_fill,
diff --git a/target/xtensa/translate.c b/target/xtensa/translate.c
index bdd4690a5c..77bcd71030 100644
--- a/target/xtensa/translate.c
+++ b/target/xtensa/translate.c
@@ -1355,12 +1355,6 @@ void xtensa_cpu_dump_state(CPUState *cs, FILE *f, int flags)
}
}
-void restore_state_to_opc(CPUXtensaState *env, TranslationBlock *tb,
- target_ulong *data)
-{
- env->pc = data[0];
-}
-
static void translate_abs(DisasContext *dc, const OpcodeArg arg[],
const uint32_t par[])
{
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* [PULL 47/47] accel/tcg: Remove restore_state_to_opc function
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (45 preceding siblings ...)
2022-10-26 2:11 ` [PULL 46/47] target/xtensa: " Richard Henderson
@ 2022-10-26 2:11 ` Richard Henderson
2022-10-26 18:54 ` [PULL 00/47] tcg patch queue Stefan Hajnoczi
47 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-26 2:11 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Claudio Fontana
All targets have been updated. Use the tcg_ops target hook
exclusively, which allows the compat code to be removed.
Reviewed-by: Claudio Fontana <cfontana@suse.de>
Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
---
include/exec/exec-all.h | 3 ---
accel/tcg/translate-all.c | 16 ++--------------
2 files changed, 2 insertions(+), 17 deletions(-)
diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
index 3b5e84240b..e948992a80 100644
--- a/include/exec/exec-all.h
+++ b/include/exec/exec-all.h
@@ -39,9 +39,6 @@ typedef ram_addr_t tb_page_addr_t;
#define TB_PAGE_ADDR_FMT RAM_ADDR_FMT
#endif
-void restore_state_to_opc(CPUArchState *env, TranslationBlock *tb,
- target_ulong *data) __attribute__((weak));
-
/**
* cpu_restore_state:
* @cpu: the vCPU state is to be restore to
diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
index 4d8783efc7..f185356a36 100644
--- a/accel/tcg/translate-all.c
+++ b/accel/tcg/translate-all.c
@@ -254,7 +254,7 @@ static int encode_search(TranslationBlock *tb, uint8_t *block)
int cpu_restore_state_from_tb(CPUState *cpu, TranslationBlock *tb,
uintptr_t searched_pc, bool reset_icount)
{
- target_ulong data[TARGET_INSN_START_WORDS];
+ uint64_t data[TARGET_INSN_START_WORDS];
uintptr_t host_pc = (uintptr_t)tb->tc.ptr;
const uint8_t *p = tb->tc.ptr + tb->tc.size;
int i, j, num_insns = tb->icount;
@@ -295,19 +295,7 @@ int cpu_restore_state_from_tb(CPUState *cpu, TranslationBlock *tb,
cpu_neg(cpu)->icount_decr.u16.low += num_insns - i;
}
- {
- const struct TCGCPUOps *ops = cpu->cc->tcg_ops;
- __typeof(ops->restore_state_to_opc) restore = ops->restore_state_to_opc;
- if (restore) {
- uint64_t d64[TARGET_INSN_START_WORDS];
- for (i = 0; i < TARGET_INSN_START_WORDS; ++i) {
- d64[i] = data[i];
- }
- restore(cpu, tb, d64);
- } else {
- restore_state_to_opc(cpu->env_ptr, tb, data);
- }
- }
+ cpu->cc->tcg_ops->restore_state_to_opc(cpu, tb, data);
#ifdef CONFIG_PROFILER
qatomic_set(&prof->restore_time,
--
2.34.1
^ permalink raw reply related [flat|nested] 56+ messages in thread
* Re: [PULL 00/47] tcg patch queue
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
` (46 preceding siblings ...)
2022-10-26 2:11 ` [PULL 47/47] accel/tcg: Remove restore_state_to_opc function Richard Henderson
@ 2022-10-26 18:54 ` Stefan Hajnoczi
47 siblings, 0 replies; 56+ messages in thread
From: Stefan Hajnoczi @ 2022-10-26 18:54 UTC (permalink / raw)
To: Richard Henderson; +Cc: qemu-devel, stefanha
[-- Attachment #1: Type: text/plain, Size: 115 bytes --]
Applied, thanks.
Please update the changelog at https://wiki.qemu.org/ChangeLog/7.2 for any user-visible changes.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PULL 25/47] accel/tcg: Add restore_state_to_opc to TCGCPUOps
2022-10-26 2:10 ` [PULL 25/47] accel/tcg: Add restore_state_to_opc to TCGCPUOps Richard Henderson
@ 2022-10-29 10:42 ` Alex Bennée
2022-10-31 0:10 ` Richard Henderson
2022-10-31 17:56 ` Christian Schoenebeck
1 sibling, 1 reply; 56+ messages in thread
From: Alex Bennée @ 2022-10-29 10:42 UTC (permalink / raw)
To: Richard Henderson; +Cc: stefanha, Claudio Fontana, qemu-devel
Richard Henderson <richard.henderson@linaro.org> writes:
> Add a tcg_ops hook to replace the restore_state_to_opc
> function call. Because these generic hooks cannot depend
> on target-specific types, temporarily, copy the current
> target_ulong data[] into uint64_t d64[].
>
> Reviewed-by: Claudio Fontana <cfontana@suse.de>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
This has triggered a regression in x86_64 stuff:
➜ make -j30
GIT ui/keycodemapdb tests/fp/berkeley-testfloat-3 tests/fp/berkeley-softfloat-3 dtc
[1/9] Generating qemu-version.h with a custom command (wrapped by meson to capture output)
🕙11:41:11 alex.bennee@hackbox2:qemu.git/builds/bisect on HEAD (8269c01) (BISECTING) [$?]
➜ ./tests/venv/bin/avocado run tests/avocado/boot_linux.py:BootLinuxX8664.test_pc_i440fx_tcg tests/avocado/boot_linux.py:BootLinuxX8664.test_pc_q35_tcg tests/avocado/linux_
initrd.py:LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16 tests/avocado/replay_kernel.py:ReplayKernelNormal.test_x86_64_pc
Fetching asset from tests/avocado/linux_initrd.py:LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16
Fetching asset from tests/avocado/replay_kernel.py:ReplayKernelNormal.test_x86_64_pc
JOB ID : 1d6ae71471e46c091ed06acc59a077c10b7b1ff9
JOB LOG : /home/alex.bennee/avocado/job-results/job-2022-10-29T11.41-1d6ae71/job.log
(1/4) tests/avocado/boot_linux.py:BootLinuxX8664.test_pc_i440fx_tcg: PASS (80.15 s)
(2/4) tests/avocado/boot_linux.py:BootLinuxX8664.test_pc_q35_tcg: PASS (69.03 s)
(3/4) tests/avocado/linux_initrd.py:LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16: PASS (14.37 s)
(4/4) tests/avocado/replay_kernel.py:ReplayKernelNormal.test_x86_64_pc: PASS (71.81 s)
RESULTS : PASS 4 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 0 | CANCEL 0
JOB TIME : 235.96 s
🕙11:45:10 alex.bennee@hackbox2:qemu.git/builds/bisect on HEAD (d292568) (BISECTING) [$?] took 3m56s
➜ ninja build
ninja: error: unknown target 'build'
🕙11:45:21 alex.bennee@hackbox2:qemu.git/builds/bisect on HEAD (d292568) (BISECTING) [$?] [🔴 ERROR]
✗ ninja
[56/56] Linking target qemu-system-x86_64
🕙11:45:29 alex.bennee@hackbox2:qemu.git/builds/bisect on HEAD (d292568) (BISECTING) [$?] took 4s
➜ ./tests/venv/bin/avocado run tests/avocado/boot_linux.py:BootLinuxX8664.test_pc_i440fx_tcg tests/avocado/boot_linux.py:BootLinuxX8664.test_pc_q35_tcg tests/avocado/linux_
initrd.py:LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16 tests/avocado/replay_kernel.py:ReplayKernelNormal.test_x86_64_pc
Fetching asset from tests/avocado/linux_initrd.py:LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16
Fetching asset from tests/avocado/replay_kernel.py:ReplayKernelNormal.test_x86_64_pc
JOB ID : a1c449facd31a2907520e6971a66a3a5529c3bd2
JOB LOG : /home/alex.bennee/avocado/job-results/job-2022-10-29T11.45-a1c449f/job.log
(1/4) tests/avocado/boot_linux.py:BootLinuxX8664.test_pc_i440fx_tcg: INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout reached\nOriginal status: ERRO
R\n{'name': '1-tests/avocado/boot_linux.py:BootLinuxX8664.test_pc_i440fx_tcg', 'logdir': '/home/alex.bennee/avocado/job-results/job-2022-10-29T11.45-a1c449f/test-result... (120.58 s)
(2/4) tests/avocado/boot_linux.py:BootLinuxX8664.test_pc_q35_tcg: INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout reached\nOriginal status: ERROR\n
{'name': '2-tests/avocado/boot_linux.py:BootLinuxX8664.test_pc_q35_tcg', 'logdir': '/home/alex.bennee/avocado/job-results/job-2022-10-29T11.45-a1c449f/test-results/2... (120.59 s)
(3/4) tests/avocado/linux_initrd.py:LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16: INTERRUPTED: Test died without reporting the status.\nRunner error occurre
d: Timeout reached\nOriginal status: ERROR\n{'name': '3-tests/avocado/linux_initrd.py:LinuxInitrd.test_with_2gib_file_should_work_with_linux_v4_16', 'logdir': '/home/alex.bennee/avocado/job-results... (311.46 s)
(4/4) tests/avocado/replay_kernel.py:ReplayKernelNormal.test_x86_64_pc: INTERRUPTED: Test interrupted by SIGTERM\nRunner error occurred: Timeout reached\nOriginal status: E
RROR\n{'name': '4-tests/avocado/replay_kernel.py:ReplayKernelNormal.test_x86_64_pc', 'logdir': '/home/alex.bennee/avocado/job-results/job-2022-10-29T11.45-a1c449f/test-res... (120.58 s)
RESULTS : PASS 0 | ERROR 0 | FAIL 0 | SKIP 0 | WARN 0 | INTERRUPT 4 | CANCEL 0
JOB TIME : 676.41 s
🕙11:56:59 alex.bennee@hackbox2:qemu.git/builds/bisect on HEAD (d292568) (BISECTING) [$?] took 11m28s [🔴 8]
✗
> ---
> include/exec/exec-all.h | 2 +-
> include/hw/core/tcg-cpu-ops.h | 11 +++++++++++
> accel/tcg/translate-all.c | 24 ++++++++++++++++++++++--
> 3 files changed, 34 insertions(+), 3 deletions(-)
>
> diff --git a/include/exec/exec-all.h b/include/exec/exec-all.h
> index 5ae484e34d..3b5e84240b 100644
> --- a/include/exec/exec-all.h
> +++ b/include/exec/exec-all.h
> @@ -40,7 +40,7 @@ typedef ram_addr_t tb_page_addr_t;
> #endif
>
> void restore_state_to_opc(CPUArchState *env, TranslationBlock *tb,
> - target_ulong *data);
> + target_ulong *data) __attribute__((weak));
>
> /**
> * cpu_restore_state:
> diff --git a/include/hw/core/tcg-cpu-ops.h b/include/hw/core/tcg-cpu-ops.h
> index 78c6c6635d..20e3c0ffbb 100644
> --- a/include/hw/core/tcg-cpu-ops.h
> +++ b/include/hw/core/tcg-cpu-ops.h
> @@ -31,6 +31,17 @@ struct TCGCPUOps {
> * function to restore all the state, and register it here.
> */
> void (*synchronize_from_tb)(CPUState *cpu, const TranslationBlock *tb);
> + /**
> + * @restore_state_to_opc: Synchronize state from INDEX_op_start_insn
> + *
> + * This is called when we unwind state in the middle of a TB,
> + * usually before raising an exception. Set all part of the CPU
> + * state which are tracked insn-by-insn in the target-specific
> + * arguments to start_insn, passed as @data.
> + */
> + void (*restore_state_to_opc)(CPUState *cpu, const TranslationBlock *tb,
> + const uint64_t *data);
> +
> /** @cpu_exec_enter: Callback for cpu_exec preparation */
> void (*cpu_exec_enter)(CPUState *cpu);
> /** @cpu_exec_exit: Callback for cpu_exec cleanup */
> diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
> index 433fa247f4..4d8783efc7 100644
> --- a/accel/tcg/translate-all.c
> +++ b/accel/tcg/translate-all.c
> @@ -256,7 +256,6 @@ int cpu_restore_state_from_tb(CPUState *cpu, TranslationBlock *tb,
> {
> target_ulong data[TARGET_INSN_START_WORDS];
> uintptr_t host_pc = (uintptr_t)tb->tc.ptr;
> - CPUArchState *env = cpu->env_ptr;
> const uint8_t *p = tb->tc.ptr + tb->tc.size;
> int i, j, num_insns = tb->icount;
> #ifdef CONFIG_PROFILER
> @@ -295,7 +294,20 @@ int cpu_restore_state_from_tb(CPUState *cpu, TranslationBlock *tb,
> and shift if to the number of actually executed instructions */
> cpu_neg(cpu)->icount_decr.u16.low += num_insns - i;
> }
> - restore_state_to_opc(env, tb, data);
> +
> + {
> + const struct TCGCPUOps *ops = cpu->cc->tcg_ops;
> + __typeof(ops->restore_state_to_opc) restore = ops->restore_state_to_opc;
> + if (restore) {
> + uint64_t d64[TARGET_INSN_START_WORDS];
> + for (i = 0; i < TARGET_INSN_START_WORDS; ++i) {
> + d64[i] = data[i];
> + }
> + restore(cpu, tb, d64);
> + } else {
> + restore_state_to_opc(cpu->env_ptr, tb, data);
> + }
> + }
>
> #ifdef CONFIG_PROFILER
> qatomic_set(&prof->restore_time,
> @@ -307,6 +319,14 @@ int cpu_restore_state_from_tb(CPUState *cpu, TranslationBlock *tb,
>
> bool cpu_restore_state(CPUState *cpu, uintptr_t host_pc, bool will_exit)
> {
> + /*
> + * The pc update associated with restore without exit will
> + * break the relative pc adjustments performed by TARGET_TB_PCREL.
> + */
> + if (TARGET_TB_PCREL) {
> + assert(will_exit);
> + }
> +
> /*
> * The host_pc has to be in the rx region of the code buffer.
> * If it is not we will not be able to resolve it here.
--
Alex Bennée
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PULL 25/47] accel/tcg: Add restore_state_to_opc to TCGCPUOps
2022-10-29 10:42 ` Alex Bennée
@ 2022-10-31 0:10 ` Richard Henderson
0 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-31 0:10 UTC (permalink / raw)
To: Alex Bennée; +Cc: stefanha, Claudio Fontana, qemu-devel
On 10/29/22 21:42, Alex Bennée wrote:
>
> Richard Henderson <richard.henderson@linaro.org> writes:
>
>> Add a tcg_ops hook to replace the restore_state_to_opc
>> function call. Because these generic hooks cannot depend
>> on target-specific types, temporarily, copy the current
>> target_ulong data[] into uint64_t d64[].
>>
>> Reviewed-by: Claudio Fontana <cfontana@suse.de>
>> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
>
> This has triggered a regression in x86_64 stuff:
Patches posted: 20221027100254.215253-1-richard.henderson@linaro.org
("[PATCH v2 0/6] tcg: Fix x86 TARGET_TB_PCREL (#1269)")
r~
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PULL 25/47] accel/tcg: Add restore_state_to_opc to TCGCPUOps
2022-10-26 2:10 ` [PULL 25/47] accel/tcg: Add restore_state_to_opc to TCGCPUOps Richard Henderson
2022-10-29 10:42 ` Alex Bennée
@ 2022-10-31 17:56 ` Christian Schoenebeck
2022-10-31 20:35 ` Richard Henderson
1 sibling, 1 reply; 56+ messages in thread
From: Christian Schoenebeck @ 2022-10-31 17:56 UTC (permalink / raw)
To: qemu-devel; +Cc: stefanha, Claudio Fontana, Richard Henderson, Alex Bennée
On Wednesday, October 26, 2022 4:10:54 AM CET Richard Henderson wrote:
> Add a tcg_ops hook to replace the restore_state_to_opc
> function call. Because these generic hooks cannot depend
> on target-specific types, temporarily, copy the current
> target_ulong data[] into uint64_t d64[].
>
> Reviewed-by: Claudio Fontana <cfontana@suse.de>
> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
> ---
[...]
> diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
> index 433fa247f4..4d8783efc7 100644
> --- a/accel/tcg/translate-all.c
> +++ b/accel/tcg/translate-all.c
[...]
> bool cpu_restore_state(CPUState *cpu, uintptr_t host_pc, bool will_exit)
> {
> + /*
> + * The pc update associated with restore without exit will
> + * break the relative pc adjustments performed by TARGET_TB_PCREL.
> + */
> + if (TARGET_TB_PCREL) {
> + assert(will_exit);
> + }
> +
> /*
> * The host_pc has to be in the rx region of the code buffer.
> * If it is not we will not be able to resolve it here.
This patch appears to break macOS host. This assertion always triggers on
Apple Silicon. Previous patch 8269c01417 runs fine. Any ideas?
BTW Richard, could you add a message-id tag to your queued TCG patches? If you
are using patchwork client then it suffices to add "msgid=on" to .pwclientrc
Best regards,
Christian Schoenebeck
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PULL 25/47] accel/tcg: Add restore_state_to_opc to TCGCPUOps
2022-10-31 17:56 ` Christian Schoenebeck
@ 2022-10-31 20:35 ` Richard Henderson
2022-10-31 20:53 ` Stefan Hajnoczi
0 siblings, 1 reply; 56+ messages in thread
From: Richard Henderson @ 2022-10-31 20:35 UTC (permalink / raw)
To: Christian Schoenebeck, qemu-devel
Cc: stefanha, Claudio Fontana, Alex Bennée
On 11/1/22 04:56, Christian Schoenebeck wrote:
> On Wednesday, October 26, 2022 4:10:54 AM CET Richard Henderson wrote:
>> Add a tcg_ops hook to replace the restore_state_to_opc
>> function call. Because these generic hooks cannot depend
>> on target-specific types, temporarily, copy the current
>> target_ulong data[] into uint64_t d64[].
>>
>> Reviewed-by: Claudio Fontana <cfontana@suse.de>
>> Signed-off-by: Richard Henderson <richard.henderson@linaro.org>
>> ---
> [...]
>> diff --git a/accel/tcg/translate-all.c b/accel/tcg/translate-all.c
>> index 433fa247f4..4d8783efc7 100644
>> --- a/accel/tcg/translate-all.c
>> +++ b/accel/tcg/translate-all.c
> [...]
>> bool cpu_restore_state(CPUState *cpu, uintptr_t host_pc, bool will_exit)
>> {
>> + /*
>> + * The pc update associated with restore without exit will
>> + * break the relative pc adjustments performed by TARGET_TB_PCREL.
>> + */
>> + if (TARGET_TB_PCREL) {
>> + assert(will_exit);
>> + }
>> +
>> /*
>> * The host_pc has to be in the rx region of the code buffer.
>> * If it is not we will not be able to resolve it here.
>
> This patch appears to break macOS host. This assertion always triggers on
> Apple Silicon. Previous patch 8269c01417 runs fine. Any ideas?
It does not previously work fine. See
https://gitlab.com/qemu-project/qemu/-/issues/1269
which also contains a link to the in-flight patches.
> BTW Richard, could you add a message-id tag to your queued TCG patches?
Sometimes I remember, but I don't use the same tooling for my own work as I do for queuing
other people's. I haven't found much value in it.
> If you
> are using patchwork client then it suffices to add "msgid=on" to .pwclientrc
I am not.
r~
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PULL 25/47] accel/tcg: Add restore_state_to_opc to TCGCPUOps
2022-10-31 20:35 ` Richard Henderson
@ 2022-10-31 20:53 ` Stefan Hajnoczi
2022-10-31 21:27 ` Mark Cave-Ayland
0 siblings, 1 reply; 56+ messages in thread
From: Stefan Hajnoczi @ 2022-10-31 20:53 UTC (permalink / raw)
To: Richard Henderson
Cc: Christian Schoenebeck, qemu-devel, stefanha, Claudio Fontana,
Alex Bennée
On Mon, 31 Oct 2022 at 16:42, Richard Henderson
<richard.henderson@linaro.org> wrote:
> On 11/1/22 04:56, Christian Schoenebeck wrote:
> > On Wednesday, October 26, 2022 4:10:54 AM CET Richard Henderson wrote:
> > BTW Richard, could you add a message-id tag to your queued TCG patches?
>
> Sometimes I remember, but I don't use the same tooling for my own work as I do for queuing
> other people's. I haven't found much value in it.
>
>
> > If you
> > are using patchwork client then it suffices to add "msgid=on" to .pwclientrc
>
> I am not.
Tools that boil down to git-am(1) need to add the -m (--message-id) flag.
Stefan
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PULL 25/47] accel/tcg: Add restore_state_to_opc to TCGCPUOps
2022-10-31 20:53 ` Stefan Hajnoczi
@ 2022-10-31 21:27 ` Mark Cave-Ayland
2022-10-31 22:03 ` Richard Henderson
0 siblings, 1 reply; 56+ messages in thread
From: Mark Cave-Ayland @ 2022-10-31 21:27 UTC (permalink / raw)
To: Stefan Hajnoczi, Richard Henderson
Cc: Christian Schoenebeck, qemu-devel, stefanha, Claudio Fontana,
Alex Bennée
On 31/10/2022 20:53, Stefan Hajnoczi wrote:
> On Mon, 31 Oct 2022 at 16:42, Richard Henderson
> <richard.henderson@linaro.org> wrote:
>> On 11/1/22 04:56, Christian Schoenebeck wrote:
>>> On Wednesday, October 26, 2022 4:10:54 AM CET Richard Henderson wrote:
>>> BTW Richard, could you add a message-id tag to your queued TCG patches?
>>
>> Sometimes I remember, but I don't use the same tooling for my own work as I do for queuing
>> other people's. I haven't found much value in it.
>>
>>
>>> If you
>>> are using patchwork client then it suffices to add "msgid=on" to .pwclientrc
>>
>> I am not.
>
> Tools that boil down to git-am(1) need to add the -m (--message-id) flag.
FWIW in my local QEMU git checkout I've run "git config am.messageid true" which
alters .git/config so that the -m flag is enabled by default for "git am" commands.
ATB,
Mark.
^ permalink raw reply [flat|nested] 56+ messages in thread
* Re: [PULL 25/47] accel/tcg: Add restore_state_to_opc to TCGCPUOps
2022-10-31 21:27 ` Mark Cave-Ayland
@ 2022-10-31 22:03 ` Richard Henderson
0 siblings, 0 replies; 56+ messages in thread
From: Richard Henderson @ 2022-10-31 22:03 UTC (permalink / raw)
To: Mark Cave-Ayland, Stefan Hajnoczi
Cc: Christian Schoenebeck, qemu-devel, stefanha, Claudio Fontana,
Alex Bennée
On 11/1/22 08:27, Mark Cave-Ayland wrote:
> On 31/10/2022 20:53, Stefan Hajnoczi wrote:
>
>> On Mon, 31 Oct 2022 at 16:42, Richard Henderson
>> <richard.henderson@linaro.org> wrote:
>>> On 11/1/22 04:56, Christian Schoenebeck wrote:
>>>> On Wednesday, October 26, 2022 4:10:54 AM CET Richard Henderson wrote:
>>>> BTW Richard, could you add a message-id tag to your queued TCG patches?
>>>
>>> Sometimes I remember, but I don't use the same tooling for my own work as I do for queuing
>>> other people's. I haven't found much value in it.
>>>
>>>
>>>> If you
>>>> are using patchwork client then it suffices to add "msgid=on" to .pwclientrc
>>>
>>> I am not.
>>
>> Tools that boil down to git-am(1) need to add the -m (--message-id) flag.
>
> FWIW in my local QEMU git checkout I've run "git config am.messageid true" which alters
> .git/config so that the -m flag is enabled by default for "git am" commands.
This all supposes that one uses git am for ones own patches, which I do not.
r~
^ permalink raw reply [flat|nested] 56+ messages in thread
end of thread, other threads:[~2022-10-31 22:04 UTC | newest]
Thread overview: 56+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2022-10-26 2:10 [PULL 00/47] tcg patch queue Richard Henderson
2022-10-26 2:10 ` [PULL 01/47] Revert "accel/tcg: Init TCG cflags in vCPU thread handler" Richard Henderson
2022-10-26 2:10 ` [PULL 02/47] tcg/loongarch64: Add direct jump support Richard Henderson
2022-10-26 2:10 ` [PULL 03/47] tcg/aarch64: Remove unused code in tcg_out_op Richard Henderson
2022-10-26 2:10 ` [PULL 04/47] accel/tcg: Add a quicker check for breakpoints Richard Henderson
2022-10-26 2:10 ` [PULL 05/47] include/qemu/osdep: Add qemu_build_assert Richard Henderson
2022-10-26 2:10 ` [PULL 06/47] include/qemu/atomic: Use qemu_build_assert Richard Henderson
2022-10-26 2:10 ` [PULL 07/47] include/qemu/thread: Use qatomic_* functions Richard Henderson
2022-10-26 2:10 ` [PULL 08/47] accel/tcg: Make page_alloc_target_data allocation constant Richard Henderson
2022-10-26 2:10 ` [PULL 09/47] accel/tcg: Remove disabled debug in translate-all.c Richard Henderson
2022-10-26 2:10 ` [PULL 10/47] accel/tcg: Split out PageDesc to internal.h Richard Henderson
2022-10-26 2:10 ` [PULL 11/47] accel/tcg: Split out tb-maint.c Richard Henderson
2022-10-26 2:10 ` [PULL 12/47] accel/tcg: Move assert_no_pages_locked to internal.h Richard Henderson
2022-10-26 2:10 ` [PULL 13/47] accel/tcg: Drop cpu_get_tb_cpu_state from TARGET_HAS_PRECISE_SMC Richard Henderson
2022-10-26 2:10 ` [PULL 14/47] accel/tcg: Remove duplicate store to tb->page_addr[] Richard Henderson
2022-10-26 2:10 ` [PULL 15/47] accel/tcg: Introduce tb_{set_}page_addr{0,1} Richard Henderson
2022-10-26 2:10 ` [PULL 16/47] accel/tcg: Rename tb_invalidate_phys_page Richard Henderson
2022-10-26 2:10 ` [PULL 17/47] accel/tcg: Rename tb_invalidate_phys_page_range and drop end parameter Richard Henderson
2022-10-26 2:10 ` [PULL 18/47] accel/tcg: Unify declarations of tb_invalidate_phys_range Richard Henderson
2022-10-26 2:10 ` [PULL 19/47] accel/tcg: Use tb_invalidate_phys_page in page_set_flags Richard Henderson
2022-10-26 2:10 ` [PULL 20/47] accel/tcg: Call tb_invalidate_phys_page for PAGE_RESET Richard Henderson
2022-10-26 2:10 ` [PULL 21/47] accel/tcg: Use page_reset_target_data in page_set_flags Richard Henderson
2022-10-26 2:10 ` [PULL 22/47] accel/tcg: Use tb_invalidate_phys_range " Richard Henderson
2022-10-26 2:10 ` [PULL 23/47] accel/tcg: Move TARGET_PAGE_DATA_SIZE impl to user-exec.c Richard Henderson
2022-10-26 2:10 ` [PULL 24/47] accel/tcg: Simplify page_get/alloc_target_data Richard Henderson
2022-10-26 2:10 ` [PULL 25/47] accel/tcg: Add restore_state_to_opc to TCGCPUOps Richard Henderson
2022-10-29 10:42 ` Alex Bennée
2022-10-31 0:10 ` Richard Henderson
2022-10-31 17:56 ` Christian Schoenebeck
2022-10-31 20:35 ` Richard Henderson
2022-10-31 20:53 ` Stefan Hajnoczi
2022-10-31 21:27 ` Mark Cave-Ayland
2022-10-31 22:03 ` Richard Henderson
2022-10-26 2:10 ` [PULL 26/47] target/alpha: Convert to tcg_ops restore_state_to_opc Richard Henderson
2022-10-26 2:10 ` [PULL 27/47] target/arm: " Richard Henderson
2022-10-26 2:10 ` [PULL 28/47] target/avr: " Richard Henderson
2022-10-26 2:10 ` [PULL 29/47] target/cris: " Richard Henderson
2022-10-26 2:10 ` [PULL 30/47] target/hexagon: " Richard Henderson
2022-10-26 2:11 ` [PULL 31/47] target/hppa: " Richard Henderson
2022-10-26 2:11 ` [PULL 32/47] target/i386: " Richard Henderson
2022-10-26 2:11 ` [PULL 33/47] target/loongarch: " Richard Henderson
2022-10-26 2:11 ` [PULL 34/47] target/m68k: " Richard Henderson
2022-10-26 2:11 ` [PULL 35/47] target/microblaze: " Richard Henderson
2022-10-26 2:11 ` [PULL 36/47] target/mips: " Richard Henderson
2022-10-26 2:11 ` [PULL 37/47] target/nios2: " Richard Henderson
2022-10-26 2:11 ` [PULL 38/47] target/openrisc: " Richard Henderson
2022-10-26 2:11 ` [PULL 39/47] target/ppc: " Richard Henderson
2022-10-26 2:11 ` [PULL 40/47] target/riscv: " Richard Henderson
2022-10-26 2:11 ` [PULL 41/47] target/rx: " Richard Henderson
2022-10-26 2:11 ` [PULL 42/47] target/s390x: " Richard Henderson
2022-10-26 2:11 ` [PULL 43/47] target/sh4: " Richard Henderson
2022-10-26 2:11 ` [PULL 44/47] target/sparc: " Richard Henderson
2022-10-26 2:11 ` [PULL 45/47] target/tricore: " Richard Henderson
2022-10-26 2:11 ` [PULL 46/47] target/xtensa: " Richard Henderson
2022-10-26 2:11 ` [PULL 47/47] accel/tcg: Remove restore_state_to_opc function Richard Henderson
2022-10-26 18:54 ` [PULL 00/47] tcg patch queue Stefan Hajnoczi
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).