From: Richard Henderson <richard.henderson@linaro.org>
To: qemu-devel@nongnu.org
Cc: qemu-arm@nongnu.org, qemu-s390x@nongnu.org,
qemu-riscv@nongnu.org, qemu-ppc@nongnu.org, git@xen0n.name,
jiaxun.yang@flygoat.com, philmd@linaro.org
Subject: [PATCH v3 00/57] tcg: Improve atomicity support
Date: Tue, 25 Apr 2023 20:30:49 +0100 [thread overview]
Message-ID: <20230425193146.2106111-1-richard.henderson@linaro.org> (raw)
v1: https://lore.kernel.org/qemu-devel/20221118094754.242910-1-richard.henderson@linaro.org/
v2: https://lore.kernel.org/qemu-devel/20230216025739.1211680-1-richard.henderson@linaro.org/
Based-on: 20230424054105.1579315-1-richard.henderson@linaro.org
("[PATCH v3 00/57] tcg: Simplify calls to load/store helpers")
The main objective here is to support Arm FEAT_LSE2, which says that any
single memory access that does not cross a 16-byte boundary is atomic.
This is the MO_ATOM_WITHIN16 control.
While I'm touching all of this, a secondary objective is to handle the
atomicity of the IBM machines. Both Power and s390x treat misaligned
accesses as atomic on the lsb of the pointer. For instance, an 8-byte
access at ptr % 8 == 4 will appear as two atomic 4-byte accesses, and
ptr % 4 == 2 will appear as four 3-byte accesses.
This is the MO_ATOM_SUBALIGN control.
By default, acceses are atomic only if aligned, which is the current
behaviour of the tcg code generator (mostly, anyway, there were bugs).
This is the MO_ATOM_IFALIGN control.
Further, one can say that a large memory access is really a set of
contiguous smaller accesses, and we need not provide more atomicity
than that (modulo MO_ATOM_WITHIN16). This is the MO_ATMAX_* control.
Changes for v3:
- More simplifications to load/store helper generation.
- All tcg backends now query atom_and_align_for_opc to
increase alignment when required for atomicity.
- 128-bit atomic load/store for aarch64, ppc64, s390x.
- Sparc64 updates.
- Unaligned load/store for loongarch64, riscv.
r~
Richard Henderson (57):
include/exec/memop: Add bits describing atomicity
accel/tcg: Add cpu_in_serial_context
accel/tcg: Introduce tlb_read_idx
accel/tcg: Reorg system mode load helpers
accel/tcg: Reorg system mode store helpers
accel/tcg: Honor atomicity of loads
accel/tcg: Honor atomicity of stores
target/loongarch: Do not include tcg-ldst.h
tcg: Unify helper_{be,le}_{ld,st}*
accel/tcg: Implement helper_{ld,st}*_mmu for user-only
tcg/tci: Use helper_{ld,st}*_mmu for user-only
tcg: Add 128-bit guest memory primitives
meson: Detect atomic128 support with optimization
tcg/i386: Add have_atomic16
accel/tcg: Use have_atomic16 in ldst_atomicity.c.inc
accel/tcg: Add aarch64 specific support in ldst_atomicity
tcg/aarch64: Detect have_lse, have_lse2 for linux
tcg/aarch64: Detect have_lse, have_lse2 for darwin
accel/tcg: Add have_lse2 support in ldst_atomicity
tcg: Introduce TCG_OPF_TYPE_MASK
tcg/i386: Use full load/store helpers in user-only mode
tcg/aarch64: Use full load/store helpers in user-only mode
tcg/ppc: Use full load/store helpers in user-only mode
tcg/loongarch64: Use full load/store helpers in user-only mode
tcg/riscv: Use full load/store helpers in user-only mode
tcg/arm: Adjust constraints on qemu_ld/st
tcg/arm: Use full load/store helpers in user-only mode
tcg/mips: Use full load/store helpers in user-only mode
tcg/s390x: Use full load/store helpers in user-only mode
tcg/sparc64: Allocate %g2 as a third temporary
tcg/sparc64: Rename tcg_out_movi_imm13 to tcg_out_movi_s13
tcg/sparc64: Rename tcg_out_movi_imm32 to tcg_out_movi_u32
tcg/sparc64: Split out tcg_out_movi_s32
tcg/sparc64: Use standard slow path for softmmu
accel/tcg: Remove helper_unaligned_{ld,st}
tcg/loongarch64: Assert the host supports unaligned accesses
tcg/loongarch64: Support softmmu unaligned accesses
tcg/riscv: Support softmmu unaligned accesses
tcg: Introduce tcg_target_has_memory_bswap
tcg: Add INDEX_op_qemu_{ld,st}_i128
tcg: Support TCG_TYPE_I128 in tcg_out_{ld,st}_helper_{args,ret}
tcg: Introduce atom_and_align_for_opc
tcg/i386: Use atom_and_align_for_opc
tcg/aarch64: Use atom_and_align_for_opc
tcg/arm: Use atom_and_align_for_opc
tcg/loongarch64: Use atom_and_align_for_opc
tcg/mips: Use atom_and_align_for_opc
tcg/ppc: Use atom_and_align_for_opc
tcg/riscv: Use atom_and_align_for_opc
tcg/s390x: Use atom_and_align_for_opc
tcg/sparc64: Use atom_and_align_for_opc
tcg/i386: Honor 64-bit atomicity in 32-bit mode
tcg/i386: Support 128-bit load/store with have_atomic16
tcg/aarch64: Rename temporaries
tcg/aarch64: Support 128-bit load/store
tcg/ppc: Support 128-bit load/store
tcg/s390x: Support 128-bit load/store
accel/tcg/internal.h | 5 +
accel/tcg/tcg-runtime.h | 3 +
include/exec/cpu-defs.h | 7 +-
include/exec/cpu_ldst.h | 26 +-
include/exec/memop.h | 36 +
include/qemu/cpuid.h | 18 +
include/tcg/tcg-ldst.h | 72 +-
include/tcg/tcg-opc.h | 8 +
include/tcg/tcg.h | 22 +-
tcg/aarch64/tcg-target-con-set.h | 2 +
tcg/aarch64/tcg-target.h | 6 +-
tcg/arm/tcg-target-con-set.h | 16 +-
tcg/arm/tcg-target-con-str.h | 5 +-
tcg/arm/tcg-target.h | 3 +-
tcg/i386/tcg-target.h | 7 +-
tcg/loongarch64/tcg-target.h | 3 +-
tcg/mips/tcg-target.h | 4 +-
tcg/ppc/tcg-target-con-set.h | 2 +
tcg/ppc/tcg-target-con-str.h | 1 +
tcg/ppc/tcg-target.h | 4 +-
tcg/riscv/tcg-target.h | 4 +-
tcg/s390x/tcg-target-con-set.h | 2 +
tcg/s390x/tcg-target.h | 4 +-
tcg/sparc64/tcg-target-con-set.h | 2 -
tcg/sparc64/tcg-target-con-str.h | 1 -
tcg/sparc64/tcg-target.h | 4 +-
tcg/tcg-internal.h | 2 +
tcg/tci/tcg-target.h | 4 +-
accel/tcg/cpu-exec-common.c | 3 +
accel/tcg/cputlb.c | 1767 ++++++++++++++++++------------
accel/tcg/tb-maint.c | 2 +-
accel/tcg/user-exec.c | 488 ++++++---
target/loongarch/csr_helper.c | 1 -
target/loongarch/iocsr_helper.c | 1 -
tcg/optimize.c | 15 +-
tcg/tcg-op.c | 265 +++--
tcg/tcg.c | 270 ++++-
tcg/tci.c | 150 +--
accel/tcg/ldst_atomicity.c.inc | 1373 +++++++++++++++++++++++
docs/devel/loads-stores.rst | 36 +-
docs/devel/tcg-ops.rst | 11 +-
meson.build | 52 +-
tcg/aarch64/tcg-target.c.inc | 384 +++++--
tcg/arm/tcg-target.c.inc | 121 +-
tcg/i386/tcg-target.c.inc | 362 ++++--
tcg/loongarch64/tcg-target.c.inc | 91 +-
tcg/mips/tcg-target.c.inc | 104 +-
tcg/ppc/tcg-target.c.inc | 261 +++--
tcg/riscv/tcg-target.c.inc | 132 +--
tcg/s390x/tcg-target.c.inc | 177 +--
tcg/sparc64/tcg-target.c.inc | 714 ++++--------
tcg/tci/tcg-target.c.inc | 8 +-
52 files changed, 4657 insertions(+), 2404 deletions(-)
create mode 100644 accel/tcg/ldst_atomicity.c.inc
--
2.34.1
next reply other threads:[~2023-04-25 19:32 UTC|newest]
Thread overview: 62+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-04-25 19:30 Richard Henderson [this message]
2023-04-25 19:30 ` [PATCH v3 01/57] include/exec/memop: Add bits describing atomicity Richard Henderson
2023-04-25 19:30 ` [PATCH v3 02/57] accel/tcg: Add cpu_in_serial_context Richard Henderson
2023-05-04 14:52 ` Peter Maydell
2023-04-25 19:30 ` [PATCH v3 03/57] accel/tcg: Introduce tlb_read_idx Richard Henderson
2023-04-25 19:30 ` [PATCH v3 04/57] accel/tcg: Reorg system mode load helpers Richard Henderson
2023-04-25 19:30 ` [PATCH v3 05/57] accel/tcg: Reorg system mode store helpers Richard Henderson
2023-04-25 19:30 ` [PATCH v3 06/57] accel/tcg: Honor atomicity of loads Richard Henderson
2023-04-25 19:30 ` [PATCH v3 07/57] accel/tcg: Honor atomicity of stores Richard Henderson
2023-04-25 19:30 ` [PATCH v3 08/57] target/loongarch: Do not include tcg-ldst.h Richard Henderson
2023-04-26 0:45 ` Song Gao
2023-04-25 19:30 ` [PATCH v3 09/57] tcg: Unify helper_{be,le}_{ld,st}* Richard Henderson
2023-04-25 19:30 ` [PATCH v3 10/57] accel/tcg: Implement helper_{ld, st}*_mmu for user-only Richard Henderson
2023-04-25 19:31 ` [PATCH v3 11/57] tcg/tci: Use helper_{ld,st}*_mmu " Richard Henderson
2023-04-25 19:31 ` [PATCH v3 12/57] tcg: Add 128-bit guest memory primitives Richard Henderson
2023-04-25 19:31 ` [PATCH v3 13/57] meson: Detect atomic128 support with optimization Richard Henderson
2023-04-25 19:31 ` [PATCH v3 14/57] tcg/i386: Add have_atomic16 Richard Henderson
2023-04-25 19:31 ` [PATCH v3 15/57] accel/tcg: Use have_atomic16 in ldst_atomicity.c.inc Richard Henderson
2023-04-25 19:31 ` [PATCH v3 16/57] accel/tcg: Add aarch64 specific support in ldst_atomicity Richard Henderson
2023-04-25 19:31 ` [PATCH v3 17/57] tcg/aarch64: Detect have_lse, have_lse2 for linux Richard Henderson
2023-04-25 19:31 ` [PATCH v3 18/57] tcg/aarch64: Detect have_lse, have_lse2 for darwin Richard Henderson
2023-04-25 19:31 ` [PATCH v3 19/57] accel/tcg: Add have_lse2 support in ldst_atomicity Richard Henderson
2023-04-25 19:31 ` [PATCH v3 20/57] tcg: Introduce TCG_OPF_TYPE_MASK Richard Henderson
2023-04-25 19:31 ` [PATCH v3 21/57] tcg/i386: Use full load/store helpers in user-only mode Richard Henderson
2023-04-25 19:31 ` [PATCH v3 22/57] tcg/aarch64: " Richard Henderson
2023-04-25 19:31 ` [PATCH v3 23/57] tcg/ppc: " Richard Henderson
2023-04-25 19:31 ` [PATCH v3 24/57] tcg/loongarch64: " Richard Henderson
2023-04-25 19:31 ` [PATCH v3 25/57] tcg/riscv: " Richard Henderson
2023-04-25 19:31 ` [PATCH v3 26/57] tcg/arm: Adjust constraints on qemu_ld/st Richard Henderson
2023-04-25 19:31 ` [PATCH v3 27/57] tcg/arm: Use full load/store helpers in user-only mode Richard Henderson
2023-04-25 19:31 ` [PATCH v3 28/57] tcg/mips: " Richard Henderson
2023-04-25 19:31 ` [PATCH v3 29/57] tcg/s390x: " Richard Henderson
2023-04-25 19:31 ` [PATCH v3 30/57] tcg/sparc64: Allocate %g2 as a third temporary Richard Henderson
2023-04-25 19:31 ` [PATCH v3 31/57] tcg/sparc64: Rename tcg_out_movi_imm13 to tcg_out_movi_s13 Richard Henderson
2023-04-25 19:31 ` [PATCH v3 32/57] tcg/sparc64: Rename tcg_out_movi_imm32 to tcg_out_movi_u32 Richard Henderson
2023-04-25 19:31 ` [PATCH v3 33/57] tcg/sparc64: Split out tcg_out_movi_s32 Richard Henderson
2023-04-25 19:31 ` [PATCH v3 34/57] tcg/sparc64: Use standard slow path for softmmu Richard Henderson
2023-04-25 19:31 ` [PATCH v3 35/57] accel/tcg: Remove helper_unaligned_{ld,st} Richard Henderson
2023-04-25 19:31 ` [PATCH v3 36/57] tcg/loongarch64: Assert the host supports unaligned accesses Richard Henderson
2023-04-25 19:31 ` [PATCH v3 37/57] tcg/loongarch64: Support softmmu " Richard Henderson
2023-04-25 19:31 ` [PATCH v3 38/57] tcg/riscv: " Richard Henderson
2023-04-25 19:31 ` [PATCH v3 39/57] tcg: Introduce tcg_target_has_memory_bswap Richard Henderson
2023-04-25 19:31 ` [PATCH v3 40/57] tcg: Add INDEX_op_qemu_{ld,st}_i128 Richard Henderson
2023-04-25 19:31 ` [PATCH v3 41/57] tcg: Support TCG_TYPE_I128 in tcg_out_{ld, st}_helper_{args, ret} Richard Henderson
2023-04-25 19:31 ` [PATCH v3 42/57] tcg: Introduce atom_and_align_for_opc Richard Henderson
2023-04-25 19:31 ` [PATCH v3 43/57] tcg/i386: Use atom_and_align_for_opc Richard Henderson
2023-04-25 19:31 ` [PATCH v3 44/57] tcg/aarch64: " Richard Henderson
2023-04-25 19:31 ` [PATCH v3 45/57] tcg/arm: " Richard Henderson
2023-04-25 19:31 ` [PATCH v3 46/57] tcg/loongarch64: " Richard Henderson
2023-04-25 19:31 ` [PATCH v3 47/57] tcg/mips: " Richard Henderson
2023-04-25 19:31 ` [PATCH v3 48/57] tcg/ppc: " Richard Henderson
2023-04-25 19:31 ` [PATCH v3 49/57] tcg/riscv: " Richard Henderson
2023-04-25 19:31 ` [PATCH v3 50/57] tcg/s390x: " Richard Henderson
2023-04-25 19:31 ` [PATCH v3 51/57] tcg/sparc64: " Richard Henderson
2023-04-25 19:31 ` [PATCH v3 52/57] tcg/i386: Honor 64-bit atomicity in 32-bit mode Richard Henderson
2023-04-25 19:31 ` [PATCH v3 53/57] tcg/i386: Support 128-bit load/store with have_atomic16 Richard Henderson
2023-04-25 19:31 ` [PATCH v3 54/57] tcg/aarch64: Rename temporaries Richard Henderson
2023-04-25 19:31 ` [PATCH v3 55/57] tcg/aarch64: Support 128-bit load/store Richard Henderson
2023-04-25 19:31 ` [PATCH v3 56/57] tcg/ppc: " Richard Henderson
2023-04-25 19:31 ` [PATCH v3 57/57] tcg/s390x: " Richard Henderson
2023-05-02 16:11 ` [PATCH v3 00/57] tcg: Improve atomicity support Peter Maydell
2023-05-02 19:19 ` Richard Henderson
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20230425193146.2106111-1-richard.henderson@linaro.org \
--to=richard.henderson@linaro.org \
--cc=git@xen0n.name \
--cc=jiaxun.yang@flygoat.com \
--cc=philmd@linaro.org \
--cc=qemu-arm@nongnu.org \
--cc=qemu-devel@nongnu.org \
--cc=qemu-ppc@nongnu.org \
--cc=qemu-riscv@nongnu.org \
--cc=qemu-s390x@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).