* [PATCH v3 0/8] riscv: Add Zalasr ISA extension support
@ 2025-09-19 7:37 Xu Lu
2025-09-19 7:37 ` [PATCH v3 1/8] riscv: add ISA extension parsing for Zalasr Xu Lu
` (8 more replies)
0 siblings, 9 replies; 15+ messages in thread
From: Xu Lu @ 2025-09-19 7:37 UTC (permalink / raw)
To: corbet, robh, krzk+dt, conor+dt, paul.walmsley, palmer, aou, alex,
will, peterz, boqun.feng, mark.rutland, parri.andrea, ajones, brs,
anup, atish.patra, pbonzini, shuah
Cc: devicetree, linux-riscv, linux-kernel, apw, joe, linux-doc, kvm,
kvm-riscv, linux-kselftest, Xu Lu
This patch adds support for the Zalasr ISA extension, which supplies the
real load acquire/store release instructions.
The specification can be found here:
https://github.com/riscv/riscv-zalasr/blob/main/chapter2.adoc
This patch seires has been tested with ltp on Qemu with Brensan's zalasr
support patch[1].
Some false positive spacing error happens during patch checking. Thus I
CCed maintainers of checkpatch.pl as well.
[1] https://lore.kernel.org/all/CAGPSXwJEdtqW=nx71oufZp64nK6tK=0rytVEcz4F-gfvCOXk2w@mail.gmail.com/
v3:
- Apply acquire/release semantics to arch_xchg/arch_cmpxchg operations
so as to ensure FENCE.TSO ordering between operations which precede the
UNLOCK+LOCK sequence and operations which follow the sequence. Thanks
to Andrea.
- Support hwprobe of Zalasr.
- Allow Zalasr extensions for Guest/VM.
v2:
- Adjust the order of Zalasr and Zalrsc in dt-bindings. Thanks to
Conor.
Xu Lu (8):
riscv: add ISA extension parsing for Zalasr
dt-bindings: riscv: Add Zalasr ISA extension description
riscv: hwprobe: Export Zalasr extension
riscv: Introduce Zalasr instructions
riscv: Use Zalasr for smp_load_acquire/smp_store_release
riscv: Apply acquire/release semantics to arch_xchg/arch_cmpxchg
operations
RISC-V: KVM: Allow Zalasr extensions for Guest/VM
KVM: riscv: selftests: Add Zalasr extensions to get-reg-list test
Documentation/arch/riscv/hwprobe.rst | 5 +-
.../devicetree/bindings/riscv/extensions.yaml | 5 +
arch/riscv/include/asm/atomic.h | 6 -
arch/riscv/include/asm/barrier.h | 91 ++++++++++--
arch/riscv/include/asm/cmpxchg.h | 136 ++++++++----------
arch/riscv/include/asm/hwcap.h | 1 +
arch/riscv/include/asm/insn-def.h | 79 ++++++++++
arch/riscv/include/uapi/asm/hwprobe.h | 1 +
arch/riscv/include/uapi/asm/kvm.h | 1 +
arch/riscv/kernel/cpufeature.c | 1 +
arch/riscv/kernel/sys_hwprobe.c | 1 +
arch/riscv/kvm/vcpu_onereg.c | 2 +
.../selftests/kvm/riscv/get-reg-list.c | 4 +
13 files changed, 242 insertions(+), 91 deletions(-)
--
2.20.1
^ permalink raw reply [flat|nested] 15+ messages in thread
* [PATCH v3 1/8] riscv: add ISA extension parsing for Zalasr
2025-09-19 7:37 [PATCH v3 0/8] riscv: Add Zalasr ISA extension support Xu Lu
@ 2025-09-19 7:37 ` Xu Lu
2025-09-19 7:37 ` [PATCH v3 2/8] dt-bindings: riscv: Add Zalasr ISA extension description Xu Lu
` (7 subsequent siblings)
8 siblings, 0 replies; 15+ messages in thread
From: Xu Lu @ 2025-09-19 7:37 UTC (permalink / raw)
To: corbet, robh, krzk+dt, conor+dt, paul.walmsley, palmer, aou, alex,
will, peterz, boqun.feng, mark.rutland, parri.andrea, ajones, brs,
anup, atish.patra, pbonzini, shuah
Cc: devicetree, linux-riscv, linux-kernel, apw, joe, linux-doc, kvm,
kvm-riscv, linux-kselftest, Xu Lu
Add parsing for Zalasr ISA extension.
Signed-off-by: Xu Lu <luxu.kernel@bytedance.com>
---
arch/riscv/include/asm/hwcap.h | 1 +
arch/riscv/kernel/cpufeature.c | 1 +
2 files changed, 2 insertions(+)
diff --git a/arch/riscv/include/asm/hwcap.h b/arch/riscv/include/asm/hwcap.h
index affd63e11b0a3..ae3852c4f2ca2 100644
--- a/arch/riscv/include/asm/hwcap.h
+++ b/arch/riscv/include/asm/hwcap.h
@@ -106,6 +106,7 @@
#define RISCV_ISA_EXT_ZAAMO 97
#define RISCV_ISA_EXT_ZALRSC 98
#define RISCV_ISA_EXT_ZICBOP 99
+#define RISCV_ISA_EXT_ZALASR 100
#define RISCV_ISA_EXT_XLINUXENVCFG 127
diff --git a/arch/riscv/kernel/cpufeature.c b/arch/riscv/kernel/cpufeature.c
index 743d53415572e..bf9d3d92bf372 100644
--- a/arch/riscv/kernel/cpufeature.c
+++ b/arch/riscv/kernel/cpufeature.c
@@ -472,6 +472,7 @@ const struct riscv_isa_ext_data riscv_isa_ext[] = {
__RISCV_ISA_EXT_DATA(zaamo, RISCV_ISA_EXT_ZAAMO),
__RISCV_ISA_EXT_DATA(zabha, RISCV_ISA_EXT_ZABHA),
__RISCV_ISA_EXT_DATA(zacas, RISCV_ISA_EXT_ZACAS),
+ __RISCV_ISA_EXT_DATA(zalasr, RISCV_ISA_EXT_ZALASR),
__RISCV_ISA_EXT_DATA(zalrsc, RISCV_ISA_EXT_ZALRSC),
__RISCV_ISA_EXT_DATA(zawrs, RISCV_ISA_EXT_ZAWRS),
__RISCV_ISA_EXT_DATA(zfa, RISCV_ISA_EXT_ZFA),
--
2.20.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH v3 2/8] dt-bindings: riscv: Add Zalasr ISA extension description
2025-09-19 7:37 [PATCH v3 0/8] riscv: Add Zalasr ISA extension support Xu Lu
2025-09-19 7:37 ` [PATCH v3 1/8] riscv: add ISA extension parsing for Zalasr Xu Lu
@ 2025-09-19 7:37 ` Xu Lu
2025-09-19 7:37 ` [PATCH v3 3/8] riscv: hwprobe: Export Zalasr extension Xu Lu
` (6 subsequent siblings)
8 siblings, 0 replies; 15+ messages in thread
From: Xu Lu @ 2025-09-19 7:37 UTC (permalink / raw)
To: corbet, robh, krzk+dt, conor+dt, paul.walmsley, palmer, aou, alex,
will, peterz, boqun.feng, mark.rutland, parri.andrea, ajones, brs,
anup, atish.patra, pbonzini, shuah
Cc: devicetree, linux-riscv, linux-kernel, apw, joe, linux-doc, kvm,
kvm-riscv, linux-kselftest, Xu Lu
Add description for the Zalasr ISA extension
Signed-off-by: Xu Lu <luxu.kernel@bytedance.com>
---
Documentation/devicetree/bindings/riscv/extensions.yaml | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/Documentation/devicetree/bindings/riscv/extensions.yaml b/Documentation/devicetree/bindings/riscv/extensions.yaml
index ede6a58ccf534..100fe53fb0731 100644
--- a/Documentation/devicetree/bindings/riscv/extensions.yaml
+++ b/Documentation/devicetree/bindings/riscv/extensions.yaml
@@ -242,6 +242,11 @@ properties:
is supported as ratified at commit 5059e0ca641c ("update to
ratified") of the riscv-zacas.
+ - const: zalasr
+ description: |
+ The standard Zalasr extension for load-acquire/store-release as frozen
+ at commit 194f0094 ("Version 0.9 for freeze") of riscv-zalasr.
+
- const: zalrsc
description: |
The standard Zalrsc extension for load-reserved/store-conditional as
--
2.20.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH v3 3/8] riscv: hwprobe: Export Zalasr extension
2025-09-19 7:37 [PATCH v3 0/8] riscv: Add Zalasr ISA extension support Xu Lu
2025-09-19 7:37 ` [PATCH v3 1/8] riscv: add ISA extension parsing for Zalasr Xu Lu
2025-09-19 7:37 ` [PATCH v3 2/8] dt-bindings: riscv: Add Zalasr ISA extension description Xu Lu
@ 2025-09-19 7:37 ` Xu Lu
2025-09-19 7:37 ` [PATCH v3 4/8] riscv: Introduce Zalasr instructions Xu Lu
` (5 subsequent siblings)
8 siblings, 0 replies; 15+ messages in thread
From: Xu Lu @ 2025-09-19 7:37 UTC (permalink / raw)
To: corbet, robh, krzk+dt, conor+dt, paul.walmsley, palmer, aou, alex,
will, peterz, boqun.feng, mark.rutland, parri.andrea, ajones, brs,
anup, atish.patra, pbonzini, shuah
Cc: devicetree, linux-riscv, linux-kernel, apw, joe, linux-doc, kvm,
kvm-riscv, linux-kselftest, Xu Lu
Export the Zalasr extension to userspace using hwprobe.
Signed-off-by: Xu Lu <luxu.kernel@bytedance.com>
---
Documentation/arch/riscv/hwprobe.rst | 5 ++++-
arch/riscv/include/uapi/asm/hwprobe.h | 1 +
arch/riscv/kernel/sys_hwprobe.c | 1 +
3 files changed, 6 insertions(+), 1 deletion(-)
diff --git a/Documentation/arch/riscv/hwprobe.rst b/Documentation/arch/riscv/hwprobe.rst
index 2aa9be272d5de..067a3595fb9d5 100644
--- a/Documentation/arch/riscv/hwprobe.rst
+++ b/Documentation/arch/riscv/hwprobe.rst
@@ -249,6 +249,9 @@ The following keys are defined:
defined in the in the RISC-V ISA manual starting from commit e87412e621f1
("integrate Zaamo and Zalrsc text (#1304)").
+ * :c:macro:`RISCV_HWPROBE_EXT_ZALASR`: The Zalasr extension is supported as
+ frozen at commit 194f0094 ("Version 0.9 for freeze") of riscv-zalasr.
+
* :c:macro:`RISCV_HWPROBE_EXT_ZALRSC`: The Zalrsc extension is supported as
defined in the in the RISC-V ISA manual starting from commit e87412e621f1
("integrate Zaamo and Zalrsc text (#1304)").
@@ -360,4 +363,4 @@ The following keys are defined:
* :c:macro:`RISCV_HWPROBE_VENDOR_EXT_XSFVFWMACCQQQ`: The Xsfvfwmaccqqq
vendor extension is supported in version 1.0 of Matrix Multiply Accumulate
- Instruction Extensions Specification.
\ No newline at end of file
+ Instruction Extensions Specification.
diff --git a/arch/riscv/include/uapi/asm/hwprobe.h b/arch/riscv/include/uapi/asm/hwprobe.h
index aaf6ad9704993..d3a65f8ff7da4 100644
--- a/arch/riscv/include/uapi/asm/hwprobe.h
+++ b/arch/riscv/include/uapi/asm/hwprobe.h
@@ -82,6 +82,7 @@ struct riscv_hwprobe {
#define RISCV_HWPROBE_EXT_ZAAMO (1ULL << 56)
#define RISCV_HWPROBE_EXT_ZALRSC (1ULL << 57)
#define RISCV_HWPROBE_EXT_ZABHA (1ULL << 58)
+#define RISCV_HWPROBE_EXT_ZALASR (1ULL << 59)
#define RISCV_HWPROBE_KEY_CPUPERF_0 5
#define RISCV_HWPROBE_MISALIGNED_UNKNOWN (0 << 0)
#define RISCV_HWPROBE_MISALIGNED_EMULATED (1 << 0)
diff --git a/arch/riscv/kernel/sys_hwprobe.c b/arch/riscv/kernel/sys_hwprobe.c
index 0b170e18a2beb..0529e692b1173 100644
--- a/arch/riscv/kernel/sys_hwprobe.c
+++ b/arch/riscv/kernel/sys_hwprobe.c
@@ -99,6 +99,7 @@ static void hwprobe_isa_ext0(struct riscv_hwprobe *pair,
EXT_KEY(ZAAMO);
EXT_KEY(ZABHA);
EXT_KEY(ZACAS);
+ EXT_KEY(ZALASR);
EXT_KEY(ZALRSC);
EXT_KEY(ZAWRS);
EXT_KEY(ZBA);
--
2.20.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH v3 4/8] riscv: Introduce Zalasr instructions
2025-09-19 7:37 [PATCH v3 0/8] riscv: Add Zalasr ISA extension support Xu Lu
` (2 preceding siblings ...)
2025-09-19 7:37 ` [PATCH v3 3/8] riscv: hwprobe: Export Zalasr extension Xu Lu
@ 2025-09-19 7:37 ` Xu Lu
2025-09-19 7:37 ` [PATCH v3 5/8] riscv: Use Zalasr for smp_load_acquire/smp_store_release Xu Lu
` (4 subsequent siblings)
8 siblings, 0 replies; 15+ messages in thread
From: Xu Lu @ 2025-09-19 7:37 UTC (permalink / raw)
To: corbet, robh, krzk+dt, conor+dt, paul.walmsley, palmer, aou, alex,
will, peterz, boqun.feng, mark.rutland, parri.andrea, ajones, brs,
anup, atish.patra, pbonzini, shuah
Cc: devicetree, linux-riscv, linux-kernel, apw, joe, linux-doc, kvm,
kvm-riscv, linux-kselftest, Xu Lu
Introduce l{b|h|w|d}.{aq|aqrl} and s{b|h|w|d}.{rl|aqrl} instruction
encodings.
Signed-off-by: Xu Lu <luxu.kernel@bytedance.com>
---
arch/riscv/include/asm/insn-def.h | 79 +++++++++++++++++++++++++++++++
1 file changed, 79 insertions(+)
diff --git a/arch/riscv/include/asm/insn-def.h b/arch/riscv/include/asm/insn-def.h
index d5adbaec1d010..3fec7e66ce50f 100644
--- a/arch/riscv/include/asm/insn-def.h
+++ b/arch/riscv/include/asm/insn-def.h
@@ -179,6 +179,7 @@
#define RV___RS1(v) __RV_REG(v)
#define RV___RS2(v) __RV_REG(v)
+#define RV_OPCODE_AMO RV_OPCODE(47)
#define RV_OPCODE_MISC_MEM RV_OPCODE(15)
#define RV_OPCODE_OP_IMM RV_OPCODE(19)
#define RV_OPCODE_SYSTEM RV_OPCODE(115)
@@ -208,6 +209,84 @@
__ASM_STR(.error "hlv.d requires 64-bit support")
#endif
+#define LB_AQ(dest, addr) \
+ INSN_R(OPCODE_AMO, FUNC3(0), FUNC7(26), \
+ RD(dest), RS1(addr), __RS2(0))
+
+#define LB_AQRL(dest, addr) \
+ INSN_R(OPCODE_AMO, FUNC3(0), FUNC7(27), \
+ RD(dest), RS1(addr), __RS2(0))
+
+#define LH_AQ(dest, addr) \
+ INSN_R(OPCODE_AMO, FUNC3(1), FUNC7(26), \
+ RD(dest), RS1(addr), __RS2(0))
+
+#define LH_AQRL(dest, addr) \
+ INSN_R(OPCODE_AMO, FUNC3(1), FUNC7(27), \
+ RD(dest), RS1(addr), __RS2(0))
+
+#define LW_AQ(dest, addr) \
+ INSN_R(OPCODE_AMO, FUNC3(2), FUNC7(26), \
+ RD(dest), RS1(addr), __RS2(0))
+
+#define LW_AQRL(dest, addr) \
+ INSN_R(OPCODE_AMO, FUNC3(2), FUNC7(27), \
+ RD(dest), RS1(addr), __RS2(0))
+
+#define SB_RL(src, addr) \
+ INSN_R(OPCODE_AMO, FUNC3(0), FUNC7(29), \
+ __RD(0), RS1(addr), RS2(src))
+
+#define SB_AQRL(src, addr) \
+ INSN_R(OPCODE_AMO, FUNC3(0), FUNC7(31), \
+ __RD(0), RS1(addr), RS2(src))
+
+#define SH_RL(src, addr) \
+ INSN_R(OPCODE_AMO, FUNC3(1), FUNC7(29), \
+ __RD(0), RS1(addr), RS2(src))
+
+#define SH_AQRL(src, addr) \
+ INSN_R(OPCODE_AMO, FUNC3(1), FUNC7(31), \
+ __RD(0), RS1(addr), RS2(src))
+
+#define SW_RL(src, addr) \
+ INSN_R(OPCODE_AMO, FUNC3(2), FUNC7(29), \
+ __RD(0), RS1(addr), RS2(src))
+
+#define SW_AQRL(src, addr) \
+ INSN_R(OPCODE_AMO, FUNC3(2), FUNC7(31), \
+ __RD(0), RS1(addr), RS2(src))
+
+#ifdef CONFIG_64BIT
+#define LD_AQ(dest, addr) \
+ INSN_R(OPCODE_AMO, FUNC3(3), FUNC7(26), \
+ RD(dest), RS1(addr), __RS2(0))
+
+#define LD_AQRL(dest, addr) \
+ INSN_R(OPCODE_AMO, FUNC3(3), FUNC7(27), \
+ RD(dest), RS1(addr), __RS2(0))
+
+#define SD_RL(src, addr) \
+ INSN_R(OPCODE_AMO, FUNC3(3), FUNC7(29), \
+ __RD(0), RS1(addr), RS2(src))
+
+#define SD_AQRL(src, addr) \
+ INSN_R(OPCODE_AMO, FUNC3(3), FUNC7(31), \
+ __RD(0), RS1(addr), RS2(src))
+#else
+#define LD_AQ(dest, addr) \
+ __ASM_STR(.error "ld.aq requires 64-bit support")
+
+#define LD_AQRL(dest, addr) \
+ __ASM_STR(.error "ld.aqrl requires 64-bit support")
+
+#define SD_RL(dest, addr) \
+ __ASM_STR(.error "sd.rl requires 64-bit support")
+
+#define SD_AQRL(dest, addr) \
+ __ASM_STR(.error "sd.aqrl requires 64-bit support")
+#endif
+
#define SINVAL_VMA(vaddr, asid) \
INSN_R(OPCODE_SYSTEM, FUNC3(0), FUNC7(11), \
__RD(0), RS1(vaddr), RS2(asid))
--
2.20.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH v3 5/8] riscv: Use Zalasr for smp_load_acquire/smp_store_release
2025-09-19 7:37 [PATCH v3 0/8] riscv: Add Zalasr ISA extension support Xu Lu
` (3 preceding siblings ...)
2025-09-19 7:37 ` [PATCH v3 4/8] riscv: Introduce Zalasr instructions Xu Lu
@ 2025-09-19 7:37 ` Xu Lu
2025-09-19 7:37 ` [PATCH v3 6/8] riscv: Apply acquire/release semantics to arch_xchg/arch_cmpxchg operations Xu Lu
` (3 subsequent siblings)
8 siblings, 0 replies; 15+ messages in thread
From: Xu Lu @ 2025-09-19 7:37 UTC (permalink / raw)
To: corbet, robh, krzk+dt, conor+dt, paul.walmsley, palmer, aou, alex,
will, peterz, boqun.feng, mark.rutland, parri.andrea, ajones, brs,
anup, atish.patra, pbonzini, shuah
Cc: devicetree, linux-riscv, linux-kernel, apw, joe, linux-doc, kvm,
kvm-riscv, linux-kselftest, Xu Lu
Replace fence instructions with Zalasr instructions during
smp_load_acquire() and smp_store_release() operations.
|----------------------------------|
| | __smp_store_release |
| |-----------------------------|
| | zalasr | !zalasr |
| rl |-----------------------------|
| | s{b|h|w|d}.rl | fence rw, w |
| | | s{b|h|w|d} |
|----------------------------------|
| | __smp_load_acquire |
| |-----------------------------|
| | zalasr | !zalasr |
| aq |-----------------------------|
| | l{b|h|w|d}.rl | l{b|h|w|d} |
| | | fence r, rw |
|----------------------------------|
Signed-off-by: Xu Lu <luxu.kernel@bytedance.com>
---
arch/riscv/include/asm/barrier.h | 91 ++++++++++++++++++++++++++++----
1 file changed, 80 insertions(+), 11 deletions(-)
diff --git a/arch/riscv/include/asm/barrier.h b/arch/riscv/include/asm/barrier.h
index b8c5726d86acb..9eaf94a028096 100644
--- a/arch/riscv/include/asm/barrier.h
+++ b/arch/riscv/include/asm/barrier.h
@@ -51,19 +51,88 @@
*/
#define smp_mb__after_spinlock() RISCV_FENCE(iorw, iorw)
-#define __smp_store_release(p, v) \
-do { \
- compiletime_assert_atomic_type(*p); \
- RISCV_FENCE(rw, w); \
- WRITE_ONCE(*p, v); \
+extern void __bad_size_call_parameter(void);
+
+#define __smp_store_release(p, v) \
+do { \
+ typeof(p) __p = (p); \
+ union { typeof(*p) __val; char __c[1]; } __u = \
+ { .__val = (__force typeof(*p)) (v) }; \
+ compiletime_assert_atomic_type(*p); \
+ switch (sizeof(*p)) { \
+ case 1: \
+ asm volatile(ALTERNATIVE("fence rw, w;\t\nsb %0, 0(%1)\t\n", \
+ SB_RL(%0, %1) "\t\nnop\t\n", \
+ 0, RISCV_ISA_EXT_ZALASR, 1) \
+ : : "r" (*(__u8 *)__u.__c), "r" (__p) \
+ : "memory"); \
+ break; \
+ case 2: \
+ asm volatile(ALTERNATIVE("fence rw, w;\t\nsh %0, 0(%1)\t\n", \
+ SH_RL(%0, %1) "\t\nnop\t\n", \
+ 0, RISCV_ISA_EXT_ZALASR, 1) \
+ : : "r" (*(__u16 *)__u.__c), "r" (__p) \
+ : "memory"); \
+ break; \
+ case 4: \
+ asm volatile(ALTERNATIVE("fence rw, w;\t\nsw %0, 0(%1)\t\n", \
+ SW_RL(%0, %1) "\t\nnop\t\n", \
+ 0, RISCV_ISA_EXT_ZALASR, 1) \
+ : : "r" (*(__u32 *)__u.__c), "r" (__p) \
+ : "memory"); \
+ break; \
+ case 8: \
+ asm volatile(ALTERNATIVE("fence rw, w;\t\nsd %0, 0(%1)\t\n", \
+ SD_RL(%0, %1) "\t\nnop\t\n", \
+ 0, RISCV_ISA_EXT_ZALASR, 1) \
+ : : "r" (*(__u64 *)__u.__c), "r" (__p) \
+ : "memory"); \
+ break; \
+ default: \
+ __bad_size_call_parameter(); \
+ break; \
+ } \
} while (0)
-#define __smp_load_acquire(p) \
-({ \
- typeof(*p) ___p1 = READ_ONCE(*p); \
- compiletime_assert_atomic_type(*p); \
- RISCV_FENCE(r, rw); \
- ___p1; \
+#define __smp_load_acquire(p) \
+({ \
+ union { typeof(*p) __val; char __c[1]; } __u; \
+ typeof(p) __p = (p); \
+ compiletime_assert_atomic_type(*p); \
+ switch (sizeof(*p)) { \
+ case 1: \
+ asm volatile(ALTERNATIVE("lb %0, 0(%1)\t\nfence r, rw\t\n", \
+ LB_AQ(%0, %1) "\t\nnop\t\n", \
+ 0, RISCV_ISA_EXT_ZALASR, 1) \
+ : "=r" (*(__u8 *)__u.__c) : "r" (__p) \
+ : "memory"); \
+ break; \
+ case 2: \
+ asm volatile(ALTERNATIVE("lh %0, 0(%1)\t\nfence r, rw\t\n", \
+ LH_AQ(%0, %1) "\t\nnop\t\n", \
+ 0, RISCV_ISA_EXT_ZALASR, 1) \
+ : "=r" (*(__u16 *)__u.__c) : "r" (__p) \
+ : "memory"); \
+ break; \
+ case 4: \
+ asm volatile(ALTERNATIVE("lw %0, 0(%1)\t\nfence r, rw\t\n", \
+ LW_AQ(%0, %1) "\t\nnop\t\n", \
+ 0, RISCV_ISA_EXT_ZALASR, 1) \
+ : "=r" (*(__u32 *)__u.__c) : "r" (__p) \
+ : "memory"); \
+ break; \
+ case 8: \
+ asm volatile(ALTERNATIVE("ld %0, 0(%1)\t\nfence r, rw\t\n", \
+ LD_AQ(%0, %1) "\t\nnop\t\n", \
+ 0, RISCV_ISA_EXT_ZALASR, 1) \
+ : "=r" (*(__u64 *)__u.__c) : "r" (__p) \
+ : "memory"); \
+ break; \
+ default: \
+ __bad_size_call_parameter(); \
+ break; \
+ } \
+ __u.__val; \
})
#ifdef CONFIG_RISCV_ISA_ZAWRS
--
2.20.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH v3 6/8] riscv: Apply acquire/release semantics to arch_xchg/arch_cmpxchg operations
2025-09-19 7:37 [PATCH v3 0/8] riscv: Add Zalasr ISA extension support Xu Lu
` (4 preceding siblings ...)
2025-09-19 7:37 ` [PATCH v3 5/8] riscv: Use Zalasr for smp_load_acquire/smp_store_release Xu Lu
@ 2025-09-19 7:37 ` Xu Lu
2025-09-20 14:52 ` kernel test robot
2025-09-19 7:37 ` [PATCH v3 7/8] RISC-V: KVM: Allow Zalasr extensions for Guest/VM Xu Lu
` (2 subsequent siblings)
8 siblings, 1 reply; 15+ messages in thread
From: Xu Lu @ 2025-09-19 7:37 UTC (permalink / raw)
To: corbet, robh, krzk+dt, conor+dt, paul.walmsley, palmer, aou, alex,
will, peterz, boqun.feng, mark.rutland, parri.andrea, ajones, brs,
anup, atish.patra, pbonzini, shuah
Cc: devicetree, linux-riscv, linux-kernel, apw, joe, linux-doc, kvm,
kvm-riscv, linux-kselftest, Xu Lu
The existing arch_xchg/arch_cmpxchg operations are implemented by
inserting fence instructions before or after atomic instructions.
This commit replaces them with real acquire/release semantics.
|----------------------------------------------------------------|
| | arch_xchg_release | arch_cmpxchg_release |
| |-----------------------------------------------------------|
| | zabha | !zabha | zabha+zacas | !(zabha+zacas) |
| rl |-----------------------------------------------------------|
| | | (fence rw, w) | | (fence rw, w) |
| | amoswap.rl | lr.w | amocas.rl | lr.w |
| | | sc.w.rl | | sc.w.rl |
|----------------------------------------------------------------|
| | arch_xchg_acquire | arch_cmpxchg_acquire |
| |-----------------------------------------------------------|
| | zabha | !zabha | zabha+zacas | !(zabha+zacas) |
| aq |-----------------------------------------------------------|
| | | lr.w.aq | | lr.w.aq |
| | amoswap.aq | sc.w | amocas.aq | sc.w |
| | | (fence r, rw) | | (fence r, rw) |
|----------------------------------------------------------------|
(fence rw, w), (fence r, rw) here means such instructions will only
be inserted when zalasr is not implemented.
Signed-off-by: Xu Lu <luxu.kernel@bytedance.com>
---
arch/riscv/include/asm/atomic.h | 6 --
arch/riscv/include/asm/cmpxchg.h | 136 ++++++++++++++-----------------
2 files changed, 63 insertions(+), 79 deletions(-)
diff --git a/arch/riscv/include/asm/atomic.h b/arch/riscv/include/asm/atomic.h
index 5b96c2f61adb5..b79a4f889f339 100644
--- a/arch/riscv/include/asm/atomic.h
+++ b/arch/riscv/include/asm/atomic.h
@@ -18,12 +18,6 @@
#include <asm/cmpxchg.h>
-#define __atomic_acquire_fence() \
- __asm__ __volatile__(RISCV_ACQUIRE_BARRIER "" ::: "memory")
-
-#define __atomic_release_fence() \
- __asm__ __volatile__(RISCV_RELEASE_BARRIER "" ::: "memory");
-
static __always_inline int arch_atomic_read(const atomic_t *v)
{
return READ_ONCE(v->counter);
diff --git a/arch/riscv/include/asm/cmpxchg.h b/arch/riscv/include/asm/cmpxchg.h
index 0b749e7102162..207fdba38d1fc 100644
--- a/arch/riscv/include/asm/cmpxchg.h
+++ b/arch/riscv/include/asm/cmpxchg.h
@@ -15,15 +15,23 @@
#include <asm/cpufeature-macros.h>
#include <asm/processor.h>
-#define __arch_xchg_masked(sc_sfx, swap_sfx, prepend, sc_append, \
- swap_append, r, p, n) \
+/*
+ * These macros are here to improve the readability of the arch_xchg_XXX()
+ * and arch_cmpxchg_XXX() macros.
+ */
+#define LR_SFX(x) x
+#define SC_SFX(x) x
+#define CAS_SFX(x) x
+#define SC_PREPEND(x) x
+#define SC_APPEND(x) x
+
+#define __arch_xchg_masked(lr_sfx, sc_sfx, swap_sfx, sc_prepend, sc_append, \
+ r, p, n) \
({ \
if (IS_ENABLED(CONFIG_RISCV_ISA_ZABHA) && \
riscv_has_extension_unlikely(RISCV_ISA_EXT_ZABHA)) { \
__asm__ __volatile__ ( \
- prepend \
" amoswap" swap_sfx " %0, %z2, %1\n" \
- swap_append \
: "=&r" (r), "+A" (*(p)) \
: "rJ" (n) \
: "memory"); \
@@ -37,14 +45,16 @@
ulong __rc; \
\
__asm__ __volatile__ ( \
- prepend \
PREFETCHW_ASM(%5) \
+ ALTERNATIVE(__nops(1), sc_prepend, \
+ 0, RISCV_ISA_EXT_ZALASR, 1) \
"0: lr.w %0, %2\n" \
" and %1, %0, %z4\n" \
" or %1, %1, %z3\n" \
" sc.w" sc_sfx " %1, %1, %2\n" \
" bnez %1, 0b\n" \
- sc_append \
+ ALTERNATIVE(__nops(1), sc_append, \
+ 0, RISCV_ISA_EXT_ZALASR, 1) \
: "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \
: "rJ" (__newx), "rJ" (~__mask), "rJ" (__ptr32b) \
: "memory"); \
@@ -53,19 +63,17 @@
} \
})
-#define __arch_xchg(sfx, prepend, append, r, p, n) \
+#define __arch_xchg(sfx, r, p, n) \
({ \
__asm__ __volatile__ ( \
- prepend \
" amoswap" sfx " %0, %2, %1\n" \
- append \
: "=r" (r), "+A" (*(p)) \
: "r" (n) \
: "memory"); \
})
-#define _arch_xchg(ptr, new, sc_sfx, swap_sfx, prepend, \
- sc_append, swap_append) \
+#define _arch_xchg(ptr, new, lr_sfx, sc_sfx, swap_sfx, \
+ sc_prepend, sc_append) \
({ \
__typeof__(ptr) __ptr = (ptr); \
__typeof__(*(__ptr)) __new = (new); \
@@ -73,22 +81,20 @@
\
switch (sizeof(*__ptr)) { \
case 1: \
- __arch_xchg_masked(sc_sfx, ".b" swap_sfx, \
- prepend, sc_append, swap_append, \
+ __arch_xchg_masked(lr_sfx, sc_sfx, ".b" swap_sfx, \
+ sc_prepend, sc_append, \
__ret, __ptr, __new); \
break; \
case 2: \
- __arch_xchg_masked(sc_sfx, ".h" swap_sfx, \
- prepend, sc_append, swap_append, \
+ __arch_xchg_masked(lr_sfx, sc_sfx, ".h" swap_sfx, \
+ sc_prepend, sc_append, \
__ret, __ptr, __new); \
break; \
case 4: \
- __arch_xchg(".w" swap_sfx, prepend, swap_append, \
- __ret, __ptr, __new); \
+ __arch_xchg(".w" swap_sfx, __ret, __ptr, __new); \
break; \
case 8: \
- __arch_xchg(".d" swap_sfx, prepend, swap_append, \
- __ret, __ptr, __new); \
+ __arch_xchg(".d" swap_sfx, __ret, __ptr, __new); \
break; \
default: \
BUILD_BUG(); \
@@ -97,17 +103,23 @@
})
#define arch_xchg_relaxed(ptr, x) \
- _arch_xchg(ptr, x, "", "", "", "", "")
+ _arch_xchg(ptr, x, LR_SFX(""), SC_SFX(""), CAS_SFX(""), \
+ SC_PREPEND(__nops(1)), SC_APPEND(__nops(1)))
#define arch_xchg_acquire(ptr, x) \
- _arch_xchg(ptr, x, "", "", "", \
- RISCV_ACQUIRE_BARRIER, RISCV_ACQUIRE_BARRIER)
+ _arch_xchg(ptr, x, LR_SFX(".aq"), SC_SFX(""), CAS_SFX(".aq"), \
+ SC_PREPEND(__nops(1)), \
+ SC_APPEND(RISCV_ACQUIRE_BARRIER))
#define arch_xchg_release(ptr, x) \
- _arch_xchg(ptr, x, "", "", RISCV_RELEASE_BARRIER, "", "")
+ _arch_xchg(ptr, x, LR_SFX(""), SC_SFX(".rl"), CAS_SFX(".rl"), \
+ SC_PREPEND(RISCV_RELEASE_BARRIER), \
+ SC_APPEND(__nops(1)))
#define arch_xchg(ptr, x) \
- _arch_xchg(ptr, x, ".rl", ".aqrl", "", RISCV_FULL_BARRIER, "")
+ _arch_xchg(ptr, x, LR_SFX(""), SC_SFX(".aqrl"), \
+ CAS_SFX(".aqrl"), SC_PREPEND(__nops(1)), \
+ SC_APPEND(__nops(1)))
#define xchg32(ptr, x) \
({ \
@@ -126,9 +138,7 @@
* store NEW in MEM. Return the initial value in MEM. Success is
* indicated by comparing RETURN with OLD.
*/
-#define __arch_cmpxchg_masked(sc_sfx, cas_sfx, \
- sc_prepend, sc_append, \
- cas_prepend, cas_append, \
+#define __arch_cmpxchg_masked(lr_sfx, sc_sfx, cas_sfx, sc_prepend, sc_append, \
r, p, o, n) \
({ \
if (IS_ENABLED(CONFIG_RISCV_ISA_ZABHA) && \
@@ -138,9 +148,7 @@
r = o; \
\
__asm__ __volatile__ ( \
- cas_prepend \
" amocas" cas_sfx " %0, %z2, %1\n" \
- cas_append \
: "+&r" (r), "+A" (*(p)) \
: "rJ" (n) \
: "memory"); \
@@ -155,15 +163,17 @@
ulong __rc; \
\
__asm__ __volatile__ ( \
- sc_prepend \
- "0: lr.w %0, %2\n" \
+ ALTERNATIVE(__nops(1), sc_prepend, \
+ 0, RISCV_ISA_EXT_ZALASR, 1) \
+ "0: lr.w" lr_sfx " %0, %2\n" \
" and %1, %0, %z5\n" \
" bne %1, %z3, 1f\n" \
" and %1, %0, %z6\n" \
" or %1, %1, %z4\n" \
" sc.w" sc_sfx " %1, %1, %2\n" \
" bnez %1, 0b\n" \
- sc_append \
+ ALTERNATIVE(__nops(1), sc_append, \
+ 0, RISCV_ISA_EXT_ZALASR, 1) \
"1:\n" \
: "=&r" (__retx), "=&r" (__rc), "+A" (*(__ptr32b)) \
: "rJ" ((long)__oldx), "rJ" (__newx), \
@@ -174,9 +184,7 @@
} \
})
-#define __arch_cmpxchg(lr_sfx, sc_sfx, cas_sfx, \
- sc_prepend, sc_append, \
- cas_prepend, cas_append, \
+#define __arch_cmpxchg(lr_sfx, sc_sfx, cas_sfx, sc_prepend, sc_append, \
r, p, co, o, n) \
({ \
if (IS_ENABLED(CONFIG_RISCV_ISA_ZACAS) && \
@@ -184,9 +192,7 @@
r = o; \
\
__asm__ __volatile__ ( \
- cas_prepend \
" amocas" cas_sfx " %0, %z2, %1\n" \
- cas_append \
: "+&r" (r), "+A" (*(p)) \
: "rJ" (n) \
: "memory"); \
@@ -194,12 +200,14 @@
register unsigned int __rc; \
\
__asm__ __volatile__ ( \
- sc_prepend \
+ ALTERNATIVE(__nops(1), sc_prepend, \
+ 0, RISCV_ISA_EXT_ZALASR, 1) \
"0: lr" lr_sfx " %0, %2\n" \
" bne %0, %z3, 1f\n" \
" sc" sc_sfx " %1, %z4, %2\n" \
" bnez %1, 0b\n" \
- sc_append \
+ ALTERNATIVE(__nops(1), sc_append, \
+ 0, RISCV_ISA_EXT_ZALASR, 1) \
"1:\n" \
: "=&r" (r), "=&r" (__rc), "+A" (*(p)) \
: "rJ" (co o), "rJ" (n) \
@@ -207,9 +215,8 @@
} \
})
-#define _arch_cmpxchg(ptr, old, new, sc_sfx, cas_sfx, \
- sc_prepend, sc_append, \
- cas_prepend, cas_append) \
+#define _arch_cmpxchg(ptr, old, new, lr_sfx, sc_sfx, cas_sfx, \
+ sc_prepend, sc_append) \
({ \
__typeof__(ptr) __ptr = (ptr); \
__typeof__(*(__ptr)) __old = (old); \
@@ -218,27 +225,23 @@
\
switch (sizeof(*__ptr)) { \
case 1: \
- __arch_cmpxchg_masked(sc_sfx, ".b" cas_sfx, \
+ __arch_cmpxchg_masked(lr_sfx, sc_sfx, ".b" cas_sfx, \
sc_prepend, sc_append, \
- cas_prepend, cas_append, \
__ret, __ptr, __old, __new); \
break; \
case 2: \
- __arch_cmpxchg_masked(sc_sfx, ".h" cas_sfx, \
+ __arch_cmpxchg_masked(lr_sfx, sc_sfx, ".h" cas_sfx, \
sc_prepend, sc_append, \
- cas_prepend, cas_append, \
__ret, __ptr, __old, __new); \
break; \
case 4: \
- __arch_cmpxchg(".w", ".w" sc_sfx, ".w" cas_sfx, \
+ __arch_cmpxchg(".w" lr_sfx, ".w" sc_sfx, ".w" cas_sfx, \
sc_prepend, sc_append, \
- cas_prepend, cas_append, \
__ret, __ptr, (long)(int)(long), __old, __new); \
break; \
case 8: \
- __arch_cmpxchg(".d", ".d" sc_sfx, ".d" cas_sfx, \
+ __arch_cmpxchg(".d" lr_sfx, ".d" sc_sfx, ".d" cas_sfx, \
sc_prepend, sc_append, \
- cas_prepend, cas_append, \
__ret, __ptr, /**/, __old, __new); \
break; \
default: \
@@ -247,40 +250,27 @@
(__typeof__(*(__ptr)))__ret; \
})
-/*
- * These macros are here to improve the readability of the arch_cmpxchg_XXX()
- * macros.
- */
-#define SC_SFX(x) x
-#define CAS_SFX(x) x
-#define SC_PREPEND(x) x
-#define SC_APPEND(x) x
-#define CAS_PREPEND(x) x
-#define CAS_APPEND(x) x
-
#define arch_cmpxchg_relaxed(ptr, o, n) \
_arch_cmpxchg((ptr), (o), (n), \
- SC_SFX(""), CAS_SFX(""), \
- SC_PREPEND(""), SC_APPEND(""), \
- CAS_PREPEND(""), CAS_APPEND(""))
+ LR_SFX(""), SC_SFX(""), CAS_SFX(""), \
+ SC_PREPEND(__nops(1)), SC_APPEND(__nops(1)))
#define arch_cmpxchg_acquire(ptr, o, n) \
_arch_cmpxchg((ptr), (o), (n), \
- SC_SFX(""), CAS_SFX(""), \
- SC_PREPEND(""), SC_APPEND(RISCV_ACQUIRE_BARRIER), \
- CAS_PREPEND(""), CAS_APPEND(RISCV_ACQUIRE_BARRIER))
+ LR_SFX(".aq"), SC_SFX(""), CAS_SFX(".aq"), \
+ SC_PREPEND(__nops(1)), \
+ SC_APPEND(RISCV_ACQUIRE_BARRIER))
#define arch_cmpxchg_release(ptr, o, n) \
_arch_cmpxchg((ptr), (o), (n), \
- SC_SFX(""), CAS_SFX(""), \
- SC_PREPEND(RISCV_RELEASE_BARRIER), SC_APPEND(""), \
- CAS_PREPEND(RISCV_RELEASE_BARRIER), CAS_APPEND(""))
+ LR_SFX(""), SC_SFX(".rl"), CAS_SFX(".rl"), \
+ SC_PREPEND(RISCV_RELEASE_BARRIER), \
+ SC_APPEND(__nops(1)))
#define arch_cmpxchg(ptr, o, n) \
_arch_cmpxchg((ptr), (o), (n), \
- SC_SFX(".rl"), CAS_SFX(".aqrl"), \
- SC_PREPEND(""), SC_APPEND(RISCV_FULL_BARRIER), \
- CAS_PREPEND(""), CAS_APPEND(""))
+ LR_SFX(""), SC_SFX(".aqrl"), CAS_SFX(".aqrl"), \
+ SC_PREPEND(__nops(1)), SC_APPEND(__nops(1)))
#define arch_cmpxchg_local(ptr, o, n) \
arch_cmpxchg_relaxed((ptr), (o), (n))
--
2.20.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH v3 7/8] RISC-V: KVM: Allow Zalasr extensions for Guest/VM
2025-09-19 7:37 [PATCH v3 0/8] riscv: Add Zalasr ISA extension support Xu Lu
` (5 preceding siblings ...)
2025-09-19 7:37 ` [PATCH v3 6/8] riscv: Apply acquire/release semantics to arch_xchg/arch_cmpxchg operations Xu Lu
@ 2025-09-19 7:37 ` Xu Lu
2025-09-19 7:37 ` [PATCH v3 8/8] KVM: riscv: selftests: Add Zalasr extensions to get-reg-list test Xu Lu
2025-09-19 10:04 ` [PATCH v3 0/8] riscv: Add Zalasr ISA extension support Andrea Parri
8 siblings, 0 replies; 15+ messages in thread
From: Xu Lu @ 2025-09-19 7:37 UTC (permalink / raw)
To: corbet, robh, krzk+dt, conor+dt, paul.walmsley, palmer, aou, alex,
will, peterz, boqun.feng, mark.rutland, parri.andrea, ajones, brs,
anup, atish.patra, pbonzini, shuah
Cc: devicetree, linux-riscv, linux-kernel, apw, joe, linux-doc, kvm,
kvm-riscv, linux-kselftest, Xu Lu
Extend the KVM ISA extension ONE_REG interface to allow KVM user space
to detect and enable Zalasr extensions for Guest/VM.
Signed-off-by: Xu Lu <luxu.kernel@bytedance.com>
---
arch/riscv/include/uapi/asm/kvm.h | 1 +
arch/riscv/kvm/vcpu_onereg.c | 2 ++
2 files changed, 3 insertions(+)
diff --git a/arch/riscv/include/uapi/asm/kvm.h b/arch/riscv/include/uapi/asm/kvm.h
index ef27d4289da11..4fbc32ef888fa 100644
--- a/arch/riscv/include/uapi/asm/kvm.h
+++ b/arch/riscv/include/uapi/asm/kvm.h
@@ -185,6 +185,7 @@ enum KVM_RISCV_ISA_EXT_ID {
KVM_RISCV_ISA_EXT_ZICCRSE,
KVM_RISCV_ISA_EXT_ZAAMO,
KVM_RISCV_ISA_EXT_ZALRSC,
+ KVM_RISCV_ISA_EXT_ZALASR,
KVM_RISCV_ISA_EXT_MAX,
};
diff --git a/arch/riscv/kvm/vcpu_onereg.c b/arch/riscv/kvm/vcpu_onereg.c
index cce6a38ea54f2..6ae5f9859f25b 100644
--- a/arch/riscv/kvm/vcpu_onereg.c
+++ b/arch/riscv/kvm/vcpu_onereg.c
@@ -50,6 +50,7 @@ static const unsigned long kvm_isa_ext_arr[] = {
KVM_ISA_EXT_ARR(ZAAMO),
KVM_ISA_EXT_ARR(ZABHA),
KVM_ISA_EXT_ARR(ZACAS),
+ KVM_ISA_EXT_ARR(ZALASR),
KVM_ISA_EXT_ARR(ZALRSC),
KVM_ISA_EXT_ARR(ZAWRS),
KVM_ISA_EXT_ARR(ZBA),
@@ -184,6 +185,7 @@ static bool kvm_riscv_vcpu_isa_disable_allowed(unsigned long ext)
case KVM_RISCV_ISA_EXT_ZAAMO:
case KVM_RISCV_ISA_EXT_ZABHA:
case KVM_RISCV_ISA_EXT_ZACAS:
+ case KVM_RISCV_ISA_EXT_ZALASR:
case KVM_RISCV_ISA_EXT_ZALRSC:
case KVM_RISCV_ISA_EXT_ZAWRS:
case KVM_RISCV_ISA_EXT_ZBA:
--
2.20.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* [PATCH v3 8/8] KVM: riscv: selftests: Add Zalasr extensions to get-reg-list test
2025-09-19 7:37 [PATCH v3 0/8] riscv: Add Zalasr ISA extension support Xu Lu
` (6 preceding siblings ...)
2025-09-19 7:37 ` [PATCH v3 7/8] RISC-V: KVM: Allow Zalasr extensions for Guest/VM Xu Lu
@ 2025-09-19 7:37 ` Xu Lu
2025-09-19 10:04 ` [PATCH v3 0/8] riscv: Add Zalasr ISA extension support Andrea Parri
8 siblings, 0 replies; 15+ messages in thread
From: Xu Lu @ 2025-09-19 7:37 UTC (permalink / raw)
To: corbet, robh, krzk+dt, conor+dt, paul.walmsley, palmer, aou, alex,
will, peterz, boqun.feng, mark.rutland, parri.andrea, ajones, brs,
anup, atish.patra, pbonzini, shuah
Cc: devicetree, linux-riscv, linux-kernel, apw, joe, linux-doc, kvm,
kvm-riscv, linux-kselftest, Xu Lu
The KVM RISC-V allows Zalasr extensions for Guest/VM so add these
extensions to get-reg-list test.
Signed-off-by: Xu Lu <luxu.kernel@bytedance.com>
---
tools/testing/selftests/kvm/riscv/get-reg-list.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/tools/testing/selftests/kvm/riscv/get-reg-list.c b/tools/testing/selftests/kvm/riscv/get-reg-list.c
index a0b7dabb50406..3020e37f621ba 100644
--- a/tools/testing/selftests/kvm/riscv/get-reg-list.c
+++ b/tools/testing/selftests/kvm/riscv/get-reg-list.c
@@ -65,6 +65,7 @@ bool filter_reg(__u64 reg)
case KVM_REG_RISCV_ISA_EXT | KVM_REG_RISCV_ISA_SINGLE | KVM_RISCV_ISA_EXT_ZAAMO:
case KVM_REG_RISCV_ISA_EXT | KVM_REG_RISCV_ISA_SINGLE | KVM_RISCV_ISA_EXT_ZABHA:
case KVM_REG_RISCV_ISA_EXT | KVM_REG_RISCV_ISA_SINGLE | KVM_RISCV_ISA_EXT_ZACAS:
+ case KVM_REG_RISCV_ISA_EXT | KVM_REG_RISCV_ISA_SINGLE | KVM_RISCV_ISA_EXT_ZALASR:
case KVM_REG_RISCV_ISA_EXT | KVM_REG_RISCV_ISA_SINGLE | KVM_RISCV_ISA_EXT_ZALRSC:
case KVM_REG_RISCV_ISA_EXT | KVM_REG_RISCV_ISA_SINGLE | KVM_RISCV_ISA_EXT_ZAWRS:
case KVM_REG_RISCV_ISA_EXT | KVM_REG_RISCV_ISA_SINGLE | KVM_RISCV_ISA_EXT_ZBA:
@@ -517,6 +518,7 @@ static const char *isa_ext_single_id_to_str(__u64 reg_off)
KVM_ISA_EXT_ARR(ZAAMO),
KVM_ISA_EXT_ARR(ZABHA),
KVM_ISA_EXT_ARR(ZACAS),
+ KVM_ISA_EXT_ARR(ZALASR),
KVM_ISA_EXT_ARR(ZALRSC),
KVM_ISA_EXT_ARR(ZAWRS),
KVM_ISA_EXT_ARR(ZBA),
@@ -1112,6 +1114,7 @@ KVM_ISA_EXT_SIMPLE_CONFIG(svvptc, SVVPTC);
KVM_ISA_EXT_SIMPLE_CONFIG(zaamo, ZAAMO);
KVM_ISA_EXT_SIMPLE_CONFIG(zabha, ZABHA);
KVM_ISA_EXT_SIMPLE_CONFIG(zacas, ZACAS);
+KVM_ISA_EXT_SIMPLE_CONFIG(zalasr, ZALASR);
KVM_ISA_EXT_SIMPLE_CONFIG(zalrsc, ZALRSC);
KVM_ISA_EXT_SIMPLE_CONFIG(zawrs, ZAWRS);
KVM_ISA_EXT_SIMPLE_CONFIG(zba, ZBA);
@@ -1187,6 +1190,7 @@ struct vcpu_reg_list *vcpu_configs[] = {
&config_zabha,
&config_zacas,
&config_zalrsc,
+ &config_zalasr,
&config_zawrs,
&config_zba,
&config_zbb,
--
2.20.1
^ permalink raw reply related [flat|nested] 15+ messages in thread
* Re: [PATCH v3 0/8] riscv: Add Zalasr ISA extension support
2025-09-19 7:37 [PATCH v3 0/8] riscv: Add Zalasr ISA extension support Xu Lu
` (7 preceding siblings ...)
2025-09-19 7:37 ` [PATCH v3 8/8] KVM: riscv: selftests: Add Zalasr extensions to get-reg-list test Xu Lu
@ 2025-09-19 10:04 ` Andrea Parri
2025-09-19 10:39 ` [External] " Xu Lu
8 siblings, 1 reply; 15+ messages in thread
From: Andrea Parri @ 2025-09-19 10:04 UTC (permalink / raw)
To: Xu Lu
Cc: corbet, robh, krzk+dt, conor+dt, paul.walmsley, palmer, aou, alex,
will, peterz, boqun.feng, mark.rutland, ajones, brs, anup,
atish.patra, pbonzini, shuah, devicetree, linux-riscv,
linux-kernel, apw, joe, linux-doc, kvm, kvm-riscv,
linux-kselftest
On Fri, Sep 19, 2025 at 03:37:06PM +0800, Xu Lu wrote:
> This patch adds support for the Zalasr ISA extension, which supplies the
> real load acquire/store release instructions.
>
> The specification can be found here:
> https://github.com/riscv/riscv-zalasr/blob/main/chapter2.adoc
>
> This patch seires has been tested with ltp on Qemu with Brensan's zalasr
> support patch[1].
>
> Some false positive spacing error happens during patch checking. Thus I
> CCed maintainers of checkpatch.pl as well.
>
> [1] https://lore.kernel.org/all/CAGPSXwJEdtqW=nx71oufZp64nK6tK=0rytVEcz4F-gfvCOXk2w@mail.gmail.com/
>
> v3:
> - Apply acquire/release semantics to arch_xchg/arch_cmpxchg operations
> so as to ensure FENCE.TSO ordering between operations which precede the
> UNLOCK+LOCK sequence and operations which follow the sequence. Thanks
> to Andrea.
> - Support hwprobe of Zalasr.
> - Allow Zalasr extensions for Guest/VM.
>
> v2:
> - Adjust the order of Zalasr and Zalrsc in dt-bindings. Thanks to
> Conor.
>
> Xu Lu (8):
> riscv: add ISA extension parsing for Zalasr
> dt-bindings: riscv: Add Zalasr ISA extension description
> riscv: hwprobe: Export Zalasr extension
> riscv: Introduce Zalasr instructions
> riscv: Use Zalasr for smp_load_acquire/smp_store_release
> riscv: Apply acquire/release semantics to arch_xchg/arch_cmpxchg
> operations
> RISC-V: KVM: Allow Zalasr extensions for Guest/VM
> KVM: riscv: selftests: Add Zalasr extensions to get-reg-list test
>
> Documentation/arch/riscv/hwprobe.rst | 5 +-
> .../devicetree/bindings/riscv/extensions.yaml | 5 +
> arch/riscv/include/asm/atomic.h | 6 -
> arch/riscv/include/asm/barrier.h | 91 ++++++++++--
> arch/riscv/include/asm/cmpxchg.h | 136 ++++++++----------
> arch/riscv/include/asm/hwcap.h | 1 +
> arch/riscv/include/asm/insn-def.h | 79 ++++++++++
> arch/riscv/include/uapi/asm/hwprobe.h | 1 +
> arch/riscv/include/uapi/asm/kvm.h | 1 +
> arch/riscv/kernel/cpufeature.c | 1 +
> arch/riscv/kernel/sys_hwprobe.c | 1 +
> arch/riscv/kvm/vcpu_onereg.c | 2 +
> .../selftests/kvm/riscv/get-reg-list.c | 4 +
> 13 files changed, 242 insertions(+), 91 deletions(-)
I wouldn't have rushed this submission while the discussion on v2 seems
so much alive; IAC, to add and link to that discussion, this version
(not a review, just looking at this diff stat) is changing the fastpath
read_unlock()
read_lock()
from something like
fence rw,w
amodadd.w
amoadd.w
fence r,rw
to
fence rw,rw
amoadd.w
amoadd.w
fence rw,rw
no matter Zalasr or !Zalasr. Similarly for other atomic operations with
release or acquire semantics. I guess the change was not intentional?
If it was intentional, it should be properly mentioned in the changelog.
Andrea
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [External] Re: [PATCH v3 0/8] riscv: Add Zalasr ISA extension support
2025-09-19 10:04 ` [PATCH v3 0/8] riscv: Add Zalasr ISA extension support Andrea Parri
@ 2025-09-19 10:39 ` Xu Lu
2025-09-19 10:53 ` Xu Lu
0 siblings, 1 reply; 15+ messages in thread
From: Xu Lu @ 2025-09-19 10:39 UTC (permalink / raw)
To: Andrea Parri
Cc: corbet, robh, krzk+dt, conor+dt, paul.walmsley, palmer, aou, alex,
will, peterz, boqun.feng, mark.rutland, ajones, brs, anup,
atish.patra, pbonzini, shuah, devicetree, linux-riscv,
linux-kernel, apw, joe, linux-doc, kvm, kvm-riscv,
linux-kselftest
Hi Andrea,
On Fri, Sep 19, 2025 at 6:04 PM Andrea Parri <parri.andrea@gmail.com> wrote:
>
> On Fri, Sep 19, 2025 at 03:37:06PM +0800, Xu Lu wrote:
> > This patch adds support for the Zalasr ISA extension, which supplies the
> > real load acquire/store release instructions.
> >
> > The specification can be found here:
> > https://github.com/riscv/riscv-zalasr/blob/main/chapter2.adoc
> >
> > This patch seires has been tested with ltp on Qemu with Brensan's zalasr
> > support patch[1].
> >
> > Some false positive spacing error happens during patch checking. Thus I
> > CCed maintainers of checkpatch.pl as well.
> >
> > [1] https://lore.kernel.org/all/CAGPSXwJEdtqW=nx71oufZp64nK6tK=0rytVEcz4F-gfvCOXk2w@mail.gmail.com/
> >
> > v3:
> > - Apply acquire/release semantics to arch_xchg/arch_cmpxchg operations
> > so as to ensure FENCE.TSO ordering between operations which precede the
> > UNLOCK+LOCK sequence and operations which follow the sequence. Thanks
> > to Andrea.
> > - Support hwprobe of Zalasr.
> > - Allow Zalasr extensions for Guest/VM.
> >
> > v2:
> > - Adjust the order of Zalasr and Zalrsc in dt-bindings. Thanks to
> > Conor.
> >
> > Xu Lu (8):
> > riscv: add ISA extension parsing for Zalasr
> > dt-bindings: riscv: Add Zalasr ISA extension description
> > riscv: hwprobe: Export Zalasr extension
> > riscv: Introduce Zalasr instructions
> > riscv: Use Zalasr for smp_load_acquire/smp_store_release
> > riscv: Apply acquire/release semantics to arch_xchg/arch_cmpxchg
> > operations
> > RISC-V: KVM: Allow Zalasr extensions for Guest/VM
> > KVM: riscv: selftests: Add Zalasr extensions to get-reg-list test
> >
> > Documentation/arch/riscv/hwprobe.rst | 5 +-
> > .../devicetree/bindings/riscv/extensions.yaml | 5 +
> > arch/riscv/include/asm/atomic.h | 6 -
> > arch/riscv/include/asm/barrier.h | 91 ++++++++++--
> > arch/riscv/include/asm/cmpxchg.h | 136 ++++++++----------
> > arch/riscv/include/asm/hwcap.h | 1 +
> > arch/riscv/include/asm/insn-def.h | 79 ++++++++++
> > arch/riscv/include/uapi/asm/hwprobe.h | 1 +
> > arch/riscv/include/uapi/asm/kvm.h | 1 +
> > arch/riscv/kernel/cpufeature.c | 1 +
> > arch/riscv/kernel/sys_hwprobe.c | 1 +
> > arch/riscv/kvm/vcpu_onereg.c | 2 +
> > .../selftests/kvm/riscv/get-reg-list.c | 4 +
> > 13 files changed, 242 insertions(+), 91 deletions(-)
>
> I wouldn't have rushed this submission while the discussion on v2 seems
> so much alive; IAC, to add and link to that discussion, this version
Thanks. This version is sent out to show my solution to the FENCE.TSO
problem you pointed out before. I will continue to improve it. Look
forward to more suggestions from you.
> (not a review, just looking at this diff stat) is changing the fastpath
>
> read_unlock()
> read_lock()
>
> from something like
>
> fence rw,w
> amodadd.w
> amoadd.w
> fence r,rw
>
> to
>
> fence rw,rw
> amoadd.w
> amoadd.w
> fence rw,rw
>
> no matter Zalasr or !Zalasr. Similarly for other atomic operations with
> release or acquire semantics. I guess the change was not intentional?
> If it was intentional, it should be properly mentioned in the changelog.
Sorry about that. It is intended. The atomic operation before
__atomic_acquire_fence or operation after __atomic_release_fence can
be just a single ld or sd instruction instead of amocas or amoswap. In
such cases, when the store release operation becomes 'sd.rl', the
__atomic_acquire_fence via 'fence r, rw' can not ensure FENCE.TSO
anymore. Thus I replace it with 'fence rw, rw'.
I will make it a separate commit and provide more messages in the
changelog. Maybe alternative mechanism can be applied to accelerate
it.
Best Regards,
Xu Lu
>
> Andrea
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [External] Re: [PATCH v3 0/8] riscv: Add Zalasr ISA extension support
2025-09-19 10:39 ` [External] " Xu Lu
@ 2025-09-19 10:53 ` Xu Lu
2025-09-19 11:06 ` Andrea Parri
0 siblings, 1 reply; 15+ messages in thread
From: Xu Lu @ 2025-09-19 10:53 UTC (permalink / raw)
To: Andrea Parri
Cc: corbet, robh, krzk+dt, conor+dt, paul.walmsley, palmer, aou, alex,
will, peterz, boqun.feng, mark.rutland, ajones, brs, anup,
atish.patra, pbonzini, shuah, devicetree, linux-riscv,
linux-kernel, apw, joe, linux-doc, kvm, kvm-riscv,
linux-kselftest
On Fri, Sep 19, 2025 at 6:39 PM Xu Lu <luxu.kernel@bytedance.com> wrote:
>
> Hi Andrea,
>
> On Fri, Sep 19, 2025 at 6:04 PM Andrea Parri <parri.andrea@gmail.com> wrote:
> >
> > On Fri, Sep 19, 2025 at 03:37:06PM +0800, Xu Lu wrote:
> > > This patch adds support for the Zalasr ISA extension, which supplies the
> > > real load acquire/store release instructions.
> > >
> > > The specification can be found here:
> > > https://github.com/riscv/riscv-zalasr/blob/main/chapter2.adoc
> > >
> > > This patch seires has been tested with ltp on Qemu with Brensan's zalasr
> > > support patch[1].
> > >
> > > Some false positive spacing error happens during patch checking. Thus I
> > > CCed maintainers of checkpatch.pl as well.
> > >
> > > [1] https://lore.kernel.org/all/CAGPSXwJEdtqW=nx71oufZp64nK6tK=0rytVEcz4F-gfvCOXk2w@mail.gmail.com/
> > >
> > > v3:
> > > - Apply acquire/release semantics to arch_xchg/arch_cmpxchg operations
> > > so as to ensure FENCE.TSO ordering between operations which precede the
> > > UNLOCK+LOCK sequence and operations which follow the sequence. Thanks
> > > to Andrea.
> > > - Support hwprobe of Zalasr.
> > > - Allow Zalasr extensions for Guest/VM.
> > >
> > > v2:
> > > - Adjust the order of Zalasr and Zalrsc in dt-bindings. Thanks to
> > > Conor.
> > >
> > > Xu Lu (8):
> > > riscv: add ISA extension parsing for Zalasr
> > > dt-bindings: riscv: Add Zalasr ISA extension description
> > > riscv: hwprobe: Export Zalasr extension
> > > riscv: Introduce Zalasr instructions
> > > riscv: Use Zalasr for smp_load_acquire/smp_store_release
> > > riscv: Apply acquire/release semantics to arch_xchg/arch_cmpxchg
> > > operations
> > > RISC-V: KVM: Allow Zalasr extensions for Guest/VM
> > > KVM: riscv: selftests: Add Zalasr extensions to get-reg-list test
> > >
> > > Documentation/arch/riscv/hwprobe.rst | 5 +-
> > > .../devicetree/bindings/riscv/extensions.yaml | 5 +
> > > arch/riscv/include/asm/atomic.h | 6 -
> > > arch/riscv/include/asm/barrier.h | 91 ++++++++++--
> > > arch/riscv/include/asm/cmpxchg.h | 136 ++++++++----------
> > > arch/riscv/include/asm/hwcap.h | 1 +
> > > arch/riscv/include/asm/insn-def.h | 79 ++++++++++
> > > arch/riscv/include/uapi/asm/hwprobe.h | 1 +
> > > arch/riscv/include/uapi/asm/kvm.h | 1 +
> > > arch/riscv/kernel/cpufeature.c | 1 +
> > > arch/riscv/kernel/sys_hwprobe.c | 1 +
> > > arch/riscv/kvm/vcpu_onereg.c | 2 +
> > > .../selftests/kvm/riscv/get-reg-list.c | 4 +
> > > 13 files changed, 242 insertions(+), 91 deletions(-)
> >
> > I wouldn't have rushed this submission while the discussion on v2 seems
> > so much alive; IAC, to add and link to that discussion, this version
>
> Thanks. This version is sent out to show my solution to the FENCE.TSO
> problem you pointed out before. I will continue to improve it. Look
> forward to more suggestions from you.
>
> > (not a review, just looking at this diff stat) is changing the fastpath
> >
> > read_unlock()
> > read_lock()
> >
> > from something like
> >
> > fence rw,w
> > amodadd.w
> > amoadd.w
> > fence r,rw
> >
> > to
> >
> > fence rw,rw
> > amoadd.w
> > amoadd.w
> > fence rw,rw
> >
> > no matter Zalasr or !Zalasr. Similarly for other atomic operations with
> > release or acquire semantics. I guess the change was not intentional?
> > If it was intentional, it should be properly mentioned in the changelog.
>
> Sorry about that. It is intended. The atomic operation before
> __atomic_acquire_fence or operation after __atomic_release_fence can
> be just a single ld or sd instruction instead of amocas or amoswap. In
> such cases, when the store release operation becomes 'sd.rl', the
> __atomic_acquire_fence via 'fence r, rw' can not ensure FENCE.TSO
> anymore. Thus I replace it with 'fence rw, rw'.
This is also the common implementation on other architectures who use
aq/rl instructions like ARM. And you certainly already knew it~
>
> I will make it a separate commit and provide more messages in the
> changelog. Maybe alternative mechanism can be applied to accelerate
> it.
>
> Best Regards,
> Xu Lu
>
> >
> > Andrea
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [External] Re: [PATCH v3 0/8] riscv: Add Zalasr ISA extension support
2025-09-19 10:53 ` Xu Lu
@ 2025-09-19 11:06 ` Andrea Parri
2025-09-19 12:12 ` Xu Lu
0 siblings, 1 reply; 15+ messages in thread
From: Andrea Parri @ 2025-09-19 11:06 UTC (permalink / raw)
To: Xu Lu
Cc: corbet, robh, krzk+dt, conor+dt, paul.walmsley, palmer, aou, alex,
will, peterz, boqun.feng, mark.rutland, ajones, brs, anup,
atish.patra, pbonzini, shuah, devicetree, linux-riscv,
linux-kernel, apw, joe, linux-doc, kvm, kvm-riscv,
linux-kselftest
> > > (not a review, just looking at this diff stat) is changing the fastpath
> > >
> > > read_unlock()
> > > read_lock()
> > >
> > > from something like
> > >
> > > fence rw,w
> > > amodadd.w
> > > amoadd.w
> > > fence r,rw
> > >
> > > to
> > >
> > > fence rw,rw
> > > amoadd.w
> > > amoadd.w
> > > fence rw,rw
> > >
> > > no matter Zalasr or !Zalasr. Similarly for other atomic operations with
> > > release or acquire semantics. I guess the change was not intentional?
> > > If it was intentional, it should be properly mentioned in the changelog.
> >
> > Sorry about that. It is intended. The atomic operation before
> > __atomic_acquire_fence or operation after __atomic_release_fence can
> > be just a single ld or sd instruction instead of amocas or amoswap. In
> > such cases, when the store release operation becomes 'sd.rl', the
> > __atomic_acquire_fence via 'fence r, rw' can not ensure FENCE.TSO
> > anymore. Thus I replace it with 'fence rw, rw'.
But you could apply similar changes you performed for xchg & cmpxchg: use
.AQ and .RL for other atomic RMW operations as well, no? AFAICS, that is
what arm64 actually does in arch/arm64/include/asm/atomic_{ll_sc,lse}.h .
Andrea
> This is also the common implementation on other architectures who use
> aq/rl instructions like ARM. And you certainly already knew it~
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [External] Re: [PATCH v3 0/8] riscv: Add Zalasr ISA extension support
2025-09-19 11:06 ` Andrea Parri
@ 2025-09-19 12:12 ` Xu Lu
0 siblings, 0 replies; 15+ messages in thread
From: Xu Lu @ 2025-09-19 12:12 UTC (permalink / raw)
To: Andrea Parri
Cc: corbet, robh, krzk+dt, conor+dt, paul.walmsley, palmer, aou, alex,
will, peterz, boqun.feng, mark.rutland, ajones, brs, anup,
atish.patra, pbonzini, shuah, devicetree, linux-riscv,
linux-kernel, apw, joe, linux-doc, kvm, kvm-riscv,
linux-kselftest
On Fri, Sep 19, 2025 at 7:06 PM Andrea Parri <parri.andrea@gmail.com> wrote:
>
> > > > (not a review, just looking at this diff stat) is changing the fastpath
> > > >
> > > > read_unlock()
> > > > read_lock()
> > > >
> > > > from something like
> > > >
> > > > fence rw,w
> > > > amodadd.w
> > > > amoadd.w
> > > > fence r,rw
> > > >
> > > > to
> > > >
> > > > fence rw,rw
> > > > amoadd.w
> > > > amoadd.w
> > > > fence rw,rw
> > > >
> > > > no matter Zalasr or !Zalasr. Similarly for other atomic operations with
> > > > release or acquire semantics. I guess the change was not intentional?
> > > > If it was intentional, it should be properly mentioned in the changelog.
> > >
> > > Sorry about that. It is intended. The atomic operation before
> > > __atomic_acquire_fence or operation after __atomic_release_fence can
> > > be just a single ld or sd instruction instead of amocas or amoswap. In
> > > such cases, when the store release operation becomes 'sd.rl', the
> > > __atomic_acquire_fence via 'fence r, rw' can not ensure FENCE.TSO
> > > anymore. Thus I replace it with 'fence rw, rw'.
>
> But you could apply similar changes you performed for xchg & cmpxchg: use
> .AQ and .RL for other atomic RMW operations as well, no? AFAICS, that is
> what arm64 actually does in arch/arm64/include/asm/atomic_{ll_sc,lse}.h .
I see. I will study the implementation of ARM and refine my patch. Thanks a lot.
Best regards,
Xu Lu
>
> Andrea
>
>
> > This is also the common implementation on other architectures who use
> > aq/rl instructions like ARM. And you certainly already knew it~
^ permalink raw reply [flat|nested] 15+ messages in thread
* Re: [PATCH v3 6/8] riscv: Apply acquire/release semantics to arch_xchg/arch_cmpxchg operations
2025-09-19 7:37 ` [PATCH v3 6/8] riscv: Apply acquire/release semantics to arch_xchg/arch_cmpxchg operations Xu Lu
@ 2025-09-20 14:52 ` kernel test robot
0 siblings, 0 replies; 15+ messages in thread
From: kernel test robot @ 2025-09-20 14:52 UTC (permalink / raw)
To: Xu Lu, corbet, robh, krzk+dt, conor+dt, paul.walmsley, palmer,
aou, alex, will, peterz, boqun.feng, mark.rutland, parri.andrea,
ajones, brs, anup, atish.patra, pbonzini, shuah
Cc: llvm, oe-kbuild-all, devicetree, linux-riscv, linux-kernel, apw,
joe, linux-doc, kvm, kvm-riscv, linux-kselftest, Xu Lu
Hi Xu,
kernel test robot noticed the following build errors:
[auto build test ERROR on robh/for-next]
[also build test ERROR on kvm/queue kvm/next linus/master v6.17-rc6]
[cannot apply to kvm/linux-next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch#_base_tree_information]
url: https://github.com/intel-lab-lkp/linux/commits/Xu-Lu/riscv-add-ISA-extension-parsing-for-Zalasr/20250919-154304
base: https://git.kernel.org/pub/scm/linux/kernel/git/robh/linux.git for-next
patch link: https://lore.kernel.org/r/20250919073714.83063-7-luxu.kernel%40bytedance.com
patch subject: [PATCH v3 6/8] riscv: Apply acquire/release semantics to arch_xchg/arch_cmpxchg operations
config: riscv-randconfig-002-20250920 (https://download.01.org/0day-ci/archive/20250920/202509202249.rOR3GJbT-lkp@intel.com/config)
compiler: clang version 22.0.0git (https://github.com/llvm/llvm-project 7c861bcedf61607b6c087380ac711eb7ff918ca6)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250920/202509202249.rOR3GJbT-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202509202249.rOR3GJbT-lkp@intel.com/
All errors (new ones prefixed by >>):
In file included from lib/objpool.c:3:
>> include/linux/objpool.h:156:7: error: invalid .org offset '1528' (at offset '1532')
156 | if (try_cmpxchg_release(&slot->head, &head, head + 1))
| ^
include/linux/atomic/atomic-instrumented.h:4899:2: note: expanded from macro 'try_cmpxchg_release'
4899 | raw_try_cmpxchg_release(__ai_ptr, __ai_oldp, __VA_ARGS__); \
| ^
include/linux/atomic/atomic-arch-fallback.h:228:9: note: expanded from macro 'raw_try_cmpxchg_release'
228 | ___r = raw_cmpxchg_release((_ptr), ___o, (_new)); \
| ^
include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
77 | #define raw_cmpxchg_release arch_cmpxchg_release
| ^
note: (skipping 4 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
104 | __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
| ^
arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
94 | ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
| ^
arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
81 | ".org . - (887b - 886b) + (889b - 888b)\n" \
| ^
<inline asm>:27:6: note: instantiated into assembly here
27 | .org . - (887b - 886b) + (889b - 888b)
| ^
In file included from lib/objpool.c:3:
>> include/linux/objpool.h:156:7: error: invalid .org offset '1528' (at offset '1532')
156 | if (try_cmpxchg_release(&slot->head, &head, head + 1))
| ^
include/linux/atomic/atomic-instrumented.h:4899:2: note: expanded from macro 'try_cmpxchg_release'
4899 | raw_try_cmpxchg_release(__ai_ptr, __ai_oldp, __VA_ARGS__); \
| ^
include/linux/atomic/atomic-arch-fallback.h:228:9: note: expanded from macro 'raw_try_cmpxchg_release'
228 | ___r = raw_cmpxchg_release((_ptr), ___o, (_new)); \
| ^
include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
77 | #define raw_cmpxchg_release arch_cmpxchg_release
| ^
note: (skipping 4 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
104 | __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
| ^
arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
94 | ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
| ^
arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
81 | ".org . - (887b - 886b) + (889b - 888b)\n" \
| ^
<inline asm>:27:6: note: instantiated into assembly here
27 | .org . - (887b - 886b) + (889b - 888b)
| ^
2 errors generated.
--
>> lib/generic-radix-tree.c:53:12: error: invalid .org offset '1850' (at offset '1854')
53 | if ((v = cmpxchg_release(&radix->root, r, new_root)) == r) {
| ^
include/linux/atomic/atomic-instrumented.h:4803:2: note: expanded from macro 'cmpxchg_release'
4803 | raw_cmpxchg_release(__ai_ptr, __VA_ARGS__); \
| ^
include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
77 | #define raw_cmpxchg_release arch_cmpxchg_release
| ^
arch/riscv/include/asm/cmpxchg.h:265:2: note: expanded from macro 'arch_cmpxchg_release'
265 | _arch_cmpxchg((ptr), (o), (n), \
| ^
note: (skipping 3 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
104 | __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
| ^
arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
94 | ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
| ^
arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
81 | ".org . - (887b - 886b) + (889b - 888b)\n" \
| ^
<inline asm>:27:6: note: instantiated into assembly here
27 | .org . - (887b - 886b) + (889b - 888b)
| ^
lib/generic-radix-tree.c:74:14: error: invalid .org offset '1862' (at offset '1866')
74 | if (!(n = cmpxchg_release(p, NULL, new_node)))
| ^
include/linux/atomic/atomic-instrumented.h:4803:2: note: expanded from macro 'cmpxchg_release'
4803 | raw_cmpxchg_release(__ai_ptr, __VA_ARGS__); \
| ^
include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
77 | #define raw_cmpxchg_release arch_cmpxchg_release
| ^
arch/riscv/include/asm/cmpxchg.h:265:2: note: expanded from macro 'arch_cmpxchg_release'
265 | _arch_cmpxchg((ptr), (o), (n), \
| ^
note: (skipping 3 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
104 | __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
| ^
arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
94 | ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
| ^
arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
81 | ".org . - (887b - 886b) + (889b - 888b)\n" \
| ^
<inline asm>:27:6: note: instantiated into assembly here
27 | .org . - (887b - 886b) + (889b - 888b)
| ^
>> lib/generic-radix-tree.c:53:12: error: invalid .org offset '1850' (at offset '1854')
53 | if ((v = cmpxchg_release(&radix->root, r, new_root)) == r) {
| ^
include/linux/atomic/atomic-instrumented.h:4803:2: note: expanded from macro 'cmpxchg_release'
4803 | raw_cmpxchg_release(__ai_ptr, __VA_ARGS__); \
| ^
include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
77 | #define raw_cmpxchg_release arch_cmpxchg_release
| ^
arch/riscv/include/asm/cmpxchg.h:265:2: note: expanded from macro 'arch_cmpxchg_release'
265 | _arch_cmpxchg((ptr), (o), (n), \
| ^
note: (skipping 3 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
104 | __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
| ^
arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
94 | ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
| ^
arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
81 | ".org . - (887b - 886b) + (889b - 888b)\n" \
| ^
<inline asm>:27:6: note: instantiated into assembly here
27 | .org . - (887b - 886b) + (889b - 888b)
| ^
lib/generic-radix-tree.c:74:14: error: invalid .org offset '1862' (at offset '1866')
74 | if (!(n = cmpxchg_release(p, NULL, new_node)))
| ^
include/linux/atomic/atomic-instrumented.h:4803:2: note: expanded from macro 'cmpxchg_release'
4803 | raw_cmpxchg_release(__ai_ptr, __VA_ARGS__); \
| ^
include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
77 | #define raw_cmpxchg_release arch_cmpxchg_release
| ^
arch/riscv/include/asm/cmpxchg.h:265:2: note: expanded from macro 'arch_cmpxchg_release'
265 | _arch_cmpxchg((ptr), (o), (n), \
| ^
note: (skipping 3 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
104 | __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
| ^
arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
94 | ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
| ^
arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
81 | ".org . - (887b - 886b) + (889b - 888b)\n" \
| ^
<inline asm>:27:6: note: instantiated into assembly here
27 | .org . - (887b - 886b) + (889b - 888b)
| ^
4 errors generated.
--
In file included from lib/refcount.c:6:
In file included from include/linux/mutex.h:17:
In file included from include/linux/lockdep.h:14:
In file included from include/linux/smp.h:13:
In file included from include/linux/cpumask.h:14:
In file included from include/linux/atomic.h:80:
>> include/linux/atomic/atomic-arch-fallback.h:2083:9: error: invalid .org offset '528' (at offset '532')
2083 | return raw_cmpxchg_release(&v->counter, old, new);
| ^
include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
77 | #define raw_cmpxchg_release arch_cmpxchg_release
| ^
arch/riscv/include/asm/cmpxchg.h:265:2: note: expanded from macro 'arch_cmpxchg_release'
265 | _arch_cmpxchg((ptr), (o), (n), \
| ^
arch/riscv/include/asm/cmpxchg.h:238:3: note: expanded from macro '_arch_cmpxchg'
238 | __arch_cmpxchg(".w" lr_sfx, ".w" sc_sfx, ".w" cas_sfx, \
| ^
note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
104 | __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
| ^
arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
94 | ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
| ^
arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
81 | ".org . - (887b - 886b) + (889b - 888b)\n" \
| ^
<inline asm>:27:6: note: instantiated into assembly here
27 | .org . - (887b - 886b) + (889b - 888b)
| ^
In file included from lib/refcount.c:6:
In file included from include/linux/mutex.h:17:
In file included from include/linux/lockdep.h:14:
In file included from include/linux/smp.h:13:
In file included from include/linux/cpumask.h:14:
In file included from include/linux/atomic.h:80:
include/linux/atomic/atomic-arch-fallback.h:2083:9: error: invalid .org offset '540' (at offset '544')
2083 | return raw_cmpxchg_release(&v->counter, old, new);
| ^
include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
77 | #define raw_cmpxchg_release arch_cmpxchg_release
| ^
arch/riscv/include/asm/cmpxchg.h:265:2: note: expanded from macro 'arch_cmpxchg_release'
265 | _arch_cmpxchg((ptr), (o), (n), \
| ^
arch/riscv/include/asm/cmpxchg.h:238:3: note: expanded from macro '_arch_cmpxchg'
238 | __arch_cmpxchg(".w" lr_sfx, ".w" sc_sfx, ".w" cas_sfx, \
| ^
note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
104 | __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
| ^
arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
94 | ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
| ^
arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
81 | ".org . - (887b - 886b) + (889b - 888b)\n" \
| ^
<inline asm>:27:6: note: instantiated into assembly here
27 | .org . - (887b - 886b) + (889b - 888b)
| ^
In file included from lib/refcount.c:6:
In file included from include/linux/mutex.h:17:
In file included from include/linux/lockdep.h:14:
In file included from include/linux/smp.h:13:
In file included from include/linux/cpumask.h:14:
In file included from include/linux/atomic.h:80:
>> include/linux/atomic/atomic-arch-fallback.h:2083:9: error: invalid .org offset '528' (at offset '532')
2083 | return raw_cmpxchg_release(&v->counter, old, new);
| ^
include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
77 | #define raw_cmpxchg_release arch_cmpxchg_release
| ^
arch/riscv/include/asm/cmpxchg.h:265:2: note: expanded from macro 'arch_cmpxchg_release'
265 | _arch_cmpxchg((ptr), (o), (n), \
| ^
arch/riscv/include/asm/cmpxchg.h:238:3: note: expanded from macro '_arch_cmpxchg'
238 | __arch_cmpxchg(".w" lr_sfx, ".w" sc_sfx, ".w" cas_sfx, \
| ^
note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
104 | __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
| ^
arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
94 | ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
| ^
arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
81 | ".org . - (887b - 886b) + (889b - 888b)\n" \
| ^
<inline asm>:27:6: note: instantiated into assembly here
27 | .org . - (887b - 886b) + (889b - 888b)
| ^
In file included from lib/refcount.c:6:
In file included from include/linux/mutex.h:17:
In file included from include/linux/lockdep.h:14:
In file included from include/linux/smp.h:13:
In file included from include/linux/cpumask.h:14:
In file included from include/linux/atomic.h:80:
include/linux/atomic/atomic-arch-fallback.h:2083:9: error: invalid .org offset '540' (at offset '544')
2083 | return raw_cmpxchg_release(&v->counter, old, new);
| ^
include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
77 | #define raw_cmpxchg_release arch_cmpxchg_release
| ^
arch/riscv/include/asm/cmpxchg.h:265:2: note: expanded from macro 'arch_cmpxchg_release'
265 | _arch_cmpxchg((ptr), (o), (n), \
| ^
arch/riscv/include/asm/cmpxchg.h:238:3: note: expanded from macro '_arch_cmpxchg'
238 | __arch_cmpxchg(".w" lr_sfx, ".w" sc_sfx, ".w" cas_sfx, \
| ^
note: (skipping 2 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
104 | __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
| ^
arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
94 | ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
| ^
arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
81 | ".org . - (887b - 886b) + (889b - 888b)\n" \
| ^
<inline asm>:27:6: note: instantiated into assembly here
27 | .org . - (887b - 886b) + (889b - 888b)
| ^
4 errors generated.
--
>> fs/overlayfs/file.c:147:10: error: invalid .org offset '3640' (at offset '3644')
147 | old = cmpxchg_release(&of->upperfile, NULL, upperfile);
| ^
include/linux/atomic/atomic-instrumented.h:4803:2: note: expanded from macro 'cmpxchg_release'
4803 | raw_cmpxchg_release(__ai_ptr, __VA_ARGS__); \
| ^
include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
77 | #define raw_cmpxchg_release arch_cmpxchg_release
| ^
arch/riscv/include/asm/cmpxchg.h:265:2: note: expanded from macro 'arch_cmpxchg_release'
265 | _arch_cmpxchg((ptr), (o), (n), \
| ^
note: (skipping 3 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
104 | __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
| ^
arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
94 | ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
| ^
arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
81 | ".org . - (887b - 886b) + (889b - 888b)\n" \
| ^
<inline asm>:27:6: note: instantiated into assembly here
27 | .org . - (887b - 886b) + (889b - 888b)
| ^
>> fs/overlayfs/file.c:147:10: error: invalid .org offset '3640' (at offset '3644')
147 | old = cmpxchg_release(&of->upperfile, NULL, upperfile);
| ^
include/linux/atomic/atomic-instrumented.h:4803:2: note: expanded from macro 'cmpxchg_release'
4803 | raw_cmpxchg_release(__ai_ptr, __VA_ARGS__); \
| ^
include/linux/atomic/atomic-arch-fallback.h:77:29: note: expanded from macro 'raw_cmpxchg_release'
77 | #define raw_cmpxchg_release arch_cmpxchg_release
| ^
arch/riscv/include/asm/cmpxchg.h:265:2: note: expanded from macro 'arch_cmpxchg_release'
265 | _arch_cmpxchg((ptr), (o), (n), \
| ^
note: (skipping 3 expansions in backtrace; use -fmacro-backtrace-limit=0 to see all)
arch/riscv/include/asm/alternative-macros.h:104:2: note: expanded from macro '_ALTERNATIVE_CFG'
104 | __ALTERNATIVE_CFG(old_c, new_c, vendor_id, patch_id, IS_ENABLED(CONFIG_k))
| ^
arch/riscv/include/asm/alternative-macros.h:94:2: note: expanded from macro '__ALTERNATIVE_CFG'
94 | ALT_NEW_CONTENT(vendor_id, patch_id, enable, new_c)
| ^
arch/riscv/include/asm/alternative-macros.h:81:3: note: expanded from macro 'ALT_NEW_CONTENT'
81 | ".org . - (887b - 886b) + (889b - 888b)\n" \
| ^
<inline asm>:27:6: note: instantiated into assembly here
27 | .org . - (887b - 886b) + (889b - 888b)
| ^
2 errors generated.
..
vim +156 include/linux/objpool.h
b4edb8d2d4647a wuqiang.matt 2023-10-17 100
b4edb8d2d4647a wuqiang.matt 2023-10-17 101 /**
b4edb8d2d4647a wuqiang.matt 2023-10-17 102 * objpool_init() - initialize objpool and pre-allocated objects
b4edb8d2d4647a wuqiang.matt 2023-10-17 103 * @pool: the object pool to be initialized, declared by caller
b4edb8d2d4647a wuqiang.matt 2023-10-17 104 * @nr_objs: total objects to be pre-allocated by this object pool
b4edb8d2d4647a wuqiang.matt 2023-10-17 105 * @object_size: size of an object (should be > 0)
b4edb8d2d4647a wuqiang.matt 2023-10-17 106 * @gfp: flags for memory allocation (via kmalloc or vmalloc)
b4edb8d2d4647a wuqiang.matt 2023-10-17 107 * @context: user context for object initialization callback
b4edb8d2d4647a wuqiang.matt 2023-10-17 108 * @objinit: object initialization callback for extra setup
b4edb8d2d4647a wuqiang.matt 2023-10-17 109 * @release: cleanup callback for extra cleanup task
b4edb8d2d4647a wuqiang.matt 2023-10-17 110 *
b4edb8d2d4647a wuqiang.matt 2023-10-17 111 * return value: 0 for success, otherwise error code
b4edb8d2d4647a wuqiang.matt 2023-10-17 112 *
b4edb8d2d4647a wuqiang.matt 2023-10-17 113 * All pre-allocated objects are to be zeroed after memory allocation.
b4edb8d2d4647a wuqiang.matt 2023-10-17 114 * Caller could do extra initialization in objinit callback. objinit()
b4edb8d2d4647a wuqiang.matt 2023-10-17 115 * will be called just after slot allocation and called only once for
b4edb8d2d4647a wuqiang.matt 2023-10-17 116 * each object. After that the objpool won't touch any content of the
b4edb8d2d4647a wuqiang.matt 2023-10-17 117 * objects. It's caller's duty to perform reinitialization after each
b4edb8d2d4647a wuqiang.matt 2023-10-17 118 * pop (object allocation) or do clearance before each push (object
b4edb8d2d4647a wuqiang.matt 2023-10-17 119 * reclamation).
b4edb8d2d4647a wuqiang.matt 2023-10-17 120 */
b4edb8d2d4647a wuqiang.matt 2023-10-17 121 int objpool_init(struct objpool_head *pool, int nr_objs, int object_size,
b4edb8d2d4647a wuqiang.matt 2023-10-17 122 gfp_t gfp, void *context, objpool_init_obj_cb objinit,
b4edb8d2d4647a wuqiang.matt 2023-10-17 123 objpool_fini_cb release);
b4edb8d2d4647a wuqiang.matt 2023-10-17 124
a3b00f10da808b Andrii Nakryiko 2024-04-24 125 /* try to retrieve object from slot */
a3b00f10da808b Andrii Nakryiko 2024-04-24 126 static inline void *__objpool_try_get_slot(struct objpool_head *pool, int cpu)
a3b00f10da808b Andrii Nakryiko 2024-04-24 127 {
a3b00f10da808b Andrii Nakryiko 2024-04-24 128 struct objpool_slot *slot = pool->cpu_slots[cpu];
a3b00f10da808b Andrii Nakryiko 2024-04-24 129 /* load head snapshot, other cpus may change it */
a3b00f10da808b Andrii Nakryiko 2024-04-24 130 uint32_t head = smp_load_acquire(&slot->head);
a3b00f10da808b Andrii Nakryiko 2024-04-24 131
a3b00f10da808b Andrii Nakryiko 2024-04-24 132 while (head != READ_ONCE(slot->last)) {
a3b00f10da808b Andrii Nakryiko 2024-04-24 133 void *obj;
a3b00f10da808b Andrii Nakryiko 2024-04-24 134
a3b00f10da808b Andrii Nakryiko 2024-04-24 135 /*
a3b00f10da808b Andrii Nakryiko 2024-04-24 136 * data visibility of 'last' and 'head' could be out of
a3b00f10da808b Andrii Nakryiko 2024-04-24 137 * order since memory updating of 'last' and 'head' are
a3b00f10da808b Andrii Nakryiko 2024-04-24 138 * performed in push() and pop() independently
a3b00f10da808b Andrii Nakryiko 2024-04-24 139 *
a3b00f10da808b Andrii Nakryiko 2024-04-24 140 * before any retrieving attempts, pop() must guarantee
a3b00f10da808b Andrii Nakryiko 2024-04-24 141 * 'last' is behind 'head', that is to say, there must
a3b00f10da808b Andrii Nakryiko 2024-04-24 142 * be available objects in slot, which could be ensured
a3b00f10da808b Andrii Nakryiko 2024-04-24 143 * by condition 'last != head && last - head <= nr_objs'
a3b00f10da808b Andrii Nakryiko 2024-04-24 144 * that is equivalent to 'last - head - 1 < nr_objs' as
a3b00f10da808b Andrii Nakryiko 2024-04-24 145 * 'last' and 'head' are both unsigned int32
a3b00f10da808b Andrii Nakryiko 2024-04-24 146 */
a3b00f10da808b Andrii Nakryiko 2024-04-24 147 if (READ_ONCE(slot->last) - head - 1 >= pool->nr_objs) {
a3b00f10da808b Andrii Nakryiko 2024-04-24 148 head = READ_ONCE(slot->head);
a3b00f10da808b Andrii Nakryiko 2024-04-24 149 continue;
a3b00f10da808b Andrii Nakryiko 2024-04-24 150 }
a3b00f10da808b Andrii Nakryiko 2024-04-24 151
a3b00f10da808b Andrii Nakryiko 2024-04-24 152 /* obj must be retrieved before moving forward head */
a3b00f10da808b Andrii Nakryiko 2024-04-24 153 obj = READ_ONCE(slot->entries[head & slot->mask]);
a3b00f10da808b Andrii Nakryiko 2024-04-24 154
a3b00f10da808b Andrii Nakryiko 2024-04-24 155 /* move head forward to mark it's consumption */
a3b00f10da808b Andrii Nakryiko 2024-04-24 @156 if (try_cmpxchg_release(&slot->head, &head, head + 1))
a3b00f10da808b Andrii Nakryiko 2024-04-24 157 return obj;
a3b00f10da808b Andrii Nakryiko 2024-04-24 158 }
a3b00f10da808b Andrii Nakryiko 2024-04-24 159
a3b00f10da808b Andrii Nakryiko 2024-04-24 160 return NULL;
a3b00f10da808b Andrii Nakryiko 2024-04-24 161 }
a3b00f10da808b Andrii Nakryiko 2024-04-24 162
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 15+ messages in thread
end of thread, other threads:[~2025-09-20 14:52 UTC | newest]
Thread overview: 15+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-09-19 7:37 [PATCH v3 0/8] riscv: Add Zalasr ISA extension support Xu Lu
2025-09-19 7:37 ` [PATCH v3 1/8] riscv: add ISA extension parsing for Zalasr Xu Lu
2025-09-19 7:37 ` [PATCH v3 2/8] dt-bindings: riscv: Add Zalasr ISA extension description Xu Lu
2025-09-19 7:37 ` [PATCH v3 3/8] riscv: hwprobe: Export Zalasr extension Xu Lu
2025-09-19 7:37 ` [PATCH v3 4/8] riscv: Introduce Zalasr instructions Xu Lu
2025-09-19 7:37 ` [PATCH v3 5/8] riscv: Use Zalasr for smp_load_acquire/smp_store_release Xu Lu
2025-09-19 7:37 ` [PATCH v3 6/8] riscv: Apply acquire/release semantics to arch_xchg/arch_cmpxchg operations Xu Lu
2025-09-20 14:52 ` kernel test robot
2025-09-19 7:37 ` [PATCH v3 7/8] RISC-V: KVM: Allow Zalasr extensions for Guest/VM Xu Lu
2025-09-19 7:37 ` [PATCH v3 8/8] KVM: riscv: selftests: Add Zalasr extensions to get-reg-list test Xu Lu
2025-09-19 10:04 ` [PATCH v3 0/8] riscv: Add Zalasr ISA extension support Andrea Parri
2025-09-19 10:39 ` [External] " Xu Lu
2025-09-19 10:53 ` Xu Lu
2025-09-19 11:06 ` Andrea Parri
2025-09-19 12:12 ` Xu Lu
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).