* [PATCH v4 0/3] riscv: vdso: Implement __vdso_futex_robust_try_unlock() @ 2026-07-22 9:06 ` Nam Cao 0 siblings, 0 replies; 8+ messages in thread From: Nam Cao @ 2026-07-22 9:06 UTC (permalink / raw) To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Alexandre Ghiti, linux-riscv, linux-kernel, Thomas Weißschuh, André Almeida, Mathieu Desnoyers, Thomas Gleixner, Peter Zijlstra, Sebastian Andrzej Siewior Cc: Nam Cao This series adds new vdso futex entries, allowing user to address the robust futex race condition regarding clearing the robust_list_head::list_op_pending pointer. Thomas wrote a summary of the problem: https://lore.kernel.org/lkml/20260602084648.462672743@kernel.org/ The core infrastructure has been implemented along with x86 support. This series adds RISC-V support. The first 4 patches prepare compat vdso Makefile. The final patch is the main show. v4: - fix minor whitespace issue - stop touching -march in Makefile, instead use .option arch v3: - add missing include guard - switch to if/else for compare-and-exchange vs load-reserved/store-conditional v2: - address Sashiko's comments Nam Cao (3): riscv: compat_vdso: switch to standard kbuild rule riscv: compat_vdso: Allow *.c source files riscv: vdso: Implement __vdso_futex_robust_try_unlock() arch/riscv/Kconfig | 1 + arch/riscv/include/asm/cpufeature-macros.h | 2 + arch/riscv/include/asm/futex_robust.h | 16 ++++ arch/riscv/include/asm/vdso/futex.h | 14 ++++ arch/riscv/kernel/compat_vdso/Makefile | 16 ++-- arch/riscv/kernel/compat_vdso/futex.c | 3 + arch/riscv/kernel/vdso.c | 61 +++++++++++++++ arch/riscv/kernel/vdso/Makefile | 7 +- arch/riscv/kernel/vdso/futex.c | 87 ++++++++++++++++++++++ arch/riscv/kernel/vdso/vdso.lds.S | 8 ++ 10 files changed, 206 insertions(+), 9 deletions(-) create mode 100644 arch/riscv/include/asm/futex_robust.h create mode 100644 arch/riscv/include/asm/vdso/futex.h create mode 100644 arch/riscv/kernel/compat_vdso/futex.c create mode 100644 arch/riscv/kernel/vdso/futex.c -- 2.47.3 ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v4 0/3] riscv: vdso: Implement __vdso_futex_robust_try_unlock() @ 2026-07-22 9:06 ` Nam Cao 0 siblings, 0 replies; 8+ messages in thread From: Nam Cao @ 2026-07-22 9:06 UTC (permalink / raw) To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Alexandre Ghiti, linux-riscv, linux-kernel, Thomas Weißschuh, André Almeida, Mathieu Desnoyers, Thomas Gleixner, Peter Zijlstra, Sebastian Andrzej Siewior Cc: Nam Cao This series adds new vdso futex entries, allowing user to address the robust futex race condition regarding clearing the robust_list_head::list_op_pending pointer. Thomas wrote a summary of the problem: https://lore.kernel.org/lkml/20260602084648.462672743@kernel.org/ The core infrastructure has been implemented along with x86 support. This series adds RISC-V support. The first 4 patches prepare compat vdso Makefile. The final patch is the main show. v4: - fix minor whitespace issue - stop touching -march in Makefile, instead use .option arch v3: - add missing include guard - switch to if/else for compare-and-exchange vs load-reserved/store-conditional v2: - address Sashiko's comments Nam Cao (3): riscv: compat_vdso: switch to standard kbuild rule riscv: compat_vdso: Allow *.c source files riscv: vdso: Implement __vdso_futex_robust_try_unlock() arch/riscv/Kconfig | 1 + arch/riscv/include/asm/cpufeature-macros.h | 2 + arch/riscv/include/asm/futex_robust.h | 16 ++++ arch/riscv/include/asm/vdso/futex.h | 14 ++++ arch/riscv/kernel/compat_vdso/Makefile | 16 ++-- arch/riscv/kernel/compat_vdso/futex.c | 3 + arch/riscv/kernel/vdso.c | 61 +++++++++++++++ arch/riscv/kernel/vdso/Makefile | 7 +- arch/riscv/kernel/vdso/futex.c | 87 ++++++++++++++++++++++ arch/riscv/kernel/vdso/vdso.lds.S | 8 ++ 10 files changed, 206 insertions(+), 9 deletions(-) create mode 100644 arch/riscv/include/asm/futex_robust.h create mode 100644 arch/riscv/include/asm/vdso/futex.h create mode 100644 arch/riscv/kernel/compat_vdso/futex.c create mode 100644 arch/riscv/kernel/vdso/futex.c -- 2.47.3 _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv ^ permalink raw reply [flat|nested] 8+ messages in thread
* [PATCH v4 1/3] riscv: compat_vdso: switch to standard kbuild rule 2026-07-22 9:06 ` Nam Cao @ 2026-07-22 9:06 ` Nam Cao -1 siblings, 0 replies; 8+ messages in thread From: Nam Cao @ 2026-07-22 9:06 UTC (permalink / raw) To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Alexandre Ghiti, linux-riscv, linux-kernel, Thomas Weißschuh, André Almeida, Mathieu Desnoyers, Thomas Gleixner, Peter Zijlstra, Sebastian Andrzej Siewior Cc: Nam Cao The compat_vdso Makefile has a custom build rule for *.S files. This is unnecessary, and makes it hard to extend this Makefile to support building *.c files. Switch to standard kbuild rule instead. Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de> Signed-off-by: Nam Cao <namcao@linutronix.de> --- arch/riscv/kernel/compat_vdso/Makefile | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/arch/riscv/kernel/compat_vdso/Makefile b/arch/riscv/kernel/compat_vdso/Makefile index 24e37d1ef7ec..7ec9dd47c4fa 100644 --- a/arch/riscv/kernel/compat_vdso/Makefile +++ b/arch/riscv/kernel/compat_vdso/Makefile @@ -8,7 +8,6 @@ compat_vdso-syms = rt_sigreturn compat_vdso-syms += getcpu compat_vdso-syms += flush_icache -COMPAT_CC := $(CC) COMPAT_LD := $(LD) # binutils 2.35 does not support the zifencei extension, but in the ISA @@ -43,8 +42,7 @@ $(obj)/compat_vdso.so.dbg: $(obj)/compat_vdso.lds $(obj-compat_vdso) FORCE LDFLAGS_compat_vdso.so.dbg = -shared -S -soname=linux-compat_vdso.so.1 \ --build-id=sha1 --hash-style=both --eh-frame-hdr -$(obj-compat_vdso): %.o: %.S FORCE - $(call if_changed_dep,compat_vdsoas) +$(obj-compat_vdso): KBUILD_AFLAGS += $(COMPAT_CC_FLAGS) # strip rule for the .so file $(obj)/%.so: OBJCOPYFLAGS := -S @@ -66,7 +64,3 @@ quiet_cmd_compat_vdsold = VDSOLD $@ cmd_compat_vdsold = $(COMPAT_LD) $(ld_flags) $(COMPAT_LD_FLAGS) -T $(filter-out FORCE,$^) -o $@.tmp && \ $(OBJCOPY) $(patsubst %, -G __compat_vdso_%, $(compat_vdso-syms)) $@.tmp $@ && \ rm $@.tmp - -# actual build commands -quiet_cmd_compat_vdsoas = VDSOAS $@ - cmd_compat_vdsoas = $(COMPAT_CC) $(a_flags) $(COMPAT_CC_FLAGS) -c -o $@ $< -- 2.47.3 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v4 1/3] riscv: compat_vdso: switch to standard kbuild rule @ 2026-07-22 9:06 ` Nam Cao 0 siblings, 0 replies; 8+ messages in thread From: Nam Cao @ 2026-07-22 9:06 UTC (permalink / raw) To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Alexandre Ghiti, linux-riscv, linux-kernel, Thomas Weißschuh, André Almeida, Mathieu Desnoyers, Thomas Gleixner, Peter Zijlstra, Sebastian Andrzej Siewior Cc: Nam Cao The compat_vdso Makefile has a custom build rule for *.S files. This is unnecessary, and makes it hard to extend this Makefile to support building *.c files. Switch to standard kbuild rule instead. Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de> Signed-off-by: Nam Cao <namcao@linutronix.de> --- arch/riscv/kernel/compat_vdso/Makefile | 8 +------- 1 file changed, 1 insertion(+), 7 deletions(-) diff --git a/arch/riscv/kernel/compat_vdso/Makefile b/arch/riscv/kernel/compat_vdso/Makefile index 24e37d1ef7ec..7ec9dd47c4fa 100644 --- a/arch/riscv/kernel/compat_vdso/Makefile +++ b/arch/riscv/kernel/compat_vdso/Makefile @@ -8,7 +8,6 @@ compat_vdso-syms = rt_sigreturn compat_vdso-syms += getcpu compat_vdso-syms += flush_icache -COMPAT_CC := $(CC) COMPAT_LD := $(LD) # binutils 2.35 does not support the zifencei extension, but in the ISA @@ -43,8 +42,7 @@ $(obj)/compat_vdso.so.dbg: $(obj)/compat_vdso.lds $(obj-compat_vdso) FORCE LDFLAGS_compat_vdso.so.dbg = -shared -S -soname=linux-compat_vdso.so.1 \ --build-id=sha1 --hash-style=both --eh-frame-hdr -$(obj-compat_vdso): %.o: %.S FORCE - $(call if_changed_dep,compat_vdsoas) +$(obj-compat_vdso): KBUILD_AFLAGS += $(COMPAT_CC_FLAGS) # strip rule for the .so file $(obj)/%.so: OBJCOPYFLAGS := -S @@ -66,7 +64,3 @@ quiet_cmd_compat_vdsold = VDSOLD $@ cmd_compat_vdsold = $(COMPAT_LD) $(ld_flags) $(COMPAT_LD_FLAGS) -T $(filter-out FORCE,$^) -o $@.tmp && \ $(OBJCOPY) $(patsubst %, -G __compat_vdso_%, $(compat_vdso-syms)) $@.tmp $@ && \ rm $@.tmp - -# actual build commands -quiet_cmd_compat_vdsoas = VDSOAS $@ - cmd_compat_vdsoas = $(COMPAT_CC) $(a_flags) $(COMPAT_CC_FLAGS) -c -o $@ $< -- 2.47.3 _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv ^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v4 2/3] riscv: compat_vdso: Allow *.c source files 2026-07-22 9:06 ` Nam Cao @ 2026-07-22 9:06 ` Nam Cao -1 siblings, 0 replies; 8+ messages in thread From: Nam Cao @ 2026-07-22 9:06 UTC (permalink / raw) To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Alexandre Ghiti, linux-riscv, linux-kernel, Thomas Weißschuh, André Almeida, Mathieu Desnoyers, Thomas Gleixner, Peter Zijlstra, Sebastian Andrzej Siewior Cc: Nam Cao Extend to support building *.c files. This allows adding futex.c in a follow-up commit. Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de> Signed-off-by: Nam Cao <namcao@linutronix.de> --- arch/riscv/kernel/compat_vdso/Makefile | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/riscv/kernel/compat_vdso/Makefile b/arch/riscv/kernel/compat_vdso/Makefile index 7ec9dd47c4fa..03d128080c6a 100644 --- a/arch/riscv/kernel/compat_vdso/Makefile +++ b/arch/riscv/kernel/compat_vdso/Makefile @@ -43,6 +43,7 @@ LDFLAGS_compat_vdso.so.dbg = -shared -S -soname=linux-compat_vdso.so.1 \ --build-id=sha1 --hash-style=both --eh-frame-hdr $(obj-compat_vdso): KBUILD_AFLAGS += $(COMPAT_CC_FLAGS) +$(obj-compat_vdso): KBUILD_CFLAGS += $(COMPAT_CC_FLAGS) # strip rule for the .so file $(obj)/%.so: OBJCOPYFLAGS := -S -- 2.47.3 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v4 2/3] riscv: compat_vdso: Allow *.c source files @ 2026-07-22 9:06 ` Nam Cao 0 siblings, 0 replies; 8+ messages in thread From: Nam Cao @ 2026-07-22 9:06 UTC (permalink / raw) To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Alexandre Ghiti, linux-riscv, linux-kernel, Thomas Weißschuh, André Almeida, Mathieu Desnoyers, Thomas Gleixner, Peter Zijlstra, Sebastian Andrzej Siewior Cc: Nam Cao Extend to support building *.c files. This allows adding futex.c in a follow-up commit. Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de> Signed-off-by: Nam Cao <namcao@linutronix.de> --- arch/riscv/kernel/compat_vdso/Makefile | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/riscv/kernel/compat_vdso/Makefile b/arch/riscv/kernel/compat_vdso/Makefile index 7ec9dd47c4fa..03d128080c6a 100644 --- a/arch/riscv/kernel/compat_vdso/Makefile +++ b/arch/riscv/kernel/compat_vdso/Makefile @@ -43,6 +43,7 @@ LDFLAGS_compat_vdso.so.dbg = -shared -S -soname=linux-compat_vdso.so.1 \ --build-id=sha1 --hash-style=both --eh-frame-hdr $(obj-compat_vdso): KBUILD_AFLAGS += $(COMPAT_CC_FLAGS) +$(obj-compat_vdso): KBUILD_CFLAGS += $(COMPAT_CC_FLAGS) # strip rule for the .so file $(obj)/%.so: OBJCOPYFLAGS := -S -- 2.47.3 _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv ^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v4 3/3] riscv: vdso: Implement __vdso_futex_robust_try_unlock() 2026-07-22 9:06 ` Nam Cao @ 2026-07-22 9:06 ` Nam Cao -1 siblings, 0 replies; 8+ messages in thread From: Nam Cao @ 2026-07-22 9:06 UTC (permalink / raw) To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Alexandre Ghiti, linux-riscv, linux-kernel, Thomas Weißschuh, André Almeida, Mathieu Desnoyers, Thomas Gleixner, Peter Zijlstra, Sebastian Andrzej Siewior Cc: Nam Cao This is the RISC-V port for __vdso_futex_robust_try_unlock(). It is based on the x86's implementation in commit 61cfc8e372d1 ("x86/vdso: Prepare for robust futex unlock support") and commit a2274cc0091e ("x86/vdso: Implement __vdso_futex_robust_try_unlock()"). When the FUTEX_ROBUST_UNLOCK mechanism is used for unlocking (PI-)futexes, the unlock sequence in userspace looks like this: 1) robust_list_set_op_pending(mutex); 2) robust_list_remove(mutex); lval = gettid(); 3) if (atomic_try_cmpxchg(&mutex->lock, lval, 0)) 4) robust_list_clear_op_pending(); else 5) sys_futex(OP,...FUTEX_ROBUST_UNLOCK); That still leaves a minimal race window between #3 and #4 where the mutex could be acquired by some other task which observes that it is the last user and: 1) unmaps the mutex memory 2) maps a different file, which ends up covering the same address When then the original task exits before reaching #4 then the kernel robust list handling observes the pending op entry and tries to fix up user space. In case that the newly mapped data contains the TID of the exiting thread at the address of the mutex/futex the kernel will set the owner died bit in that memory and therefore corrupt unrelated data. Provide a VDSO function which exposes the critical section window in the VDSO symbol table. The resulting addresses are updated in the task's mm when the VDSO is (re)map()'ed. The core code detects when a task was interrupted within the critical section and is about to deliver a signal. It then invokes an architecture specific function which determines whether the pending op pointer has to be cleared or not. The unlock assembly sequence on 64-bit is: mv a5, a0 ; move LOCK to a5 .Lretry lr.w a0, (a5) ; load *LOCK bne a0, a1, 24 .Lend ; if (*LOCK != TID) then exit sc.w.rl t0, zero, (a5) ; try changing *LOCK from TID to zero .Lstart bnez t0, .Lretry ; retry if store-conditional failed sd zero, 0(a2) ; clear POP .Lend sext.w a0,a0 ret So if the kernel sees that the user task is between .Lstart and .Lend, t0 can be checked to determine if the pending op pointer (POP) should be cleared. When the kernel is built with ZACAS, another unlock assembly sequence is provided using ZACAS instructions. If the CPU supports ZACAS, that alternative sequence is used and the critical section is also changed to that. There are two entry points to handle the different robust list pending op pointer sizes: __vdso_futex_robust_list64_try_unlock() __vdso_futex_robust_list32_try_unlock() The 32-bit VDSO and compat VDSO provides __vdso_futex_robust_list32_try_unlock(). Unlike x86, the RISC-V 64-bit VDSO provides only provides __vdso_futex_robust_list64_try_unlock(). Because RISC-V threads can only have one list_op_pending pointer type. Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de> Signed-off-by: Nam Cao <namcao@linutronix.de> --- This depends on compat vdso patching: https://lore.kernel.org/all/20260630-riscv-vdso32-alternative-v1-0-a32fd89b7b1c@linutronix.de/ --- arch/riscv/Kconfig | 1 + arch/riscv/include/asm/cpufeature-macros.h | 2 + arch/riscv/include/asm/futex_robust.h | 16 ++++ arch/riscv/include/asm/vdso/futex.h | 14 ++++ arch/riscv/kernel/compat_vdso/Makefile | 7 +- arch/riscv/kernel/compat_vdso/futex.c | 3 + arch/riscv/kernel/vdso.c | 61 +++++++++++++++ arch/riscv/kernel/vdso/Makefile | 7 +- arch/riscv/kernel/vdso/futex.c | 87 ++++++++++++++++++++++ arch/riscv/kernel/vdso/vdso.lds.S | 8 ++ 10 files changed, 204 insertions(+), 2 deletions(-) create mode 100644 arch/riscv/include/asm/futex_robust.h create mode 100644 arch/riscv/include/asm/vdso/futex.h create mode 100644 arch/riscv/kernel/compat_vdso/futex.c create mode 100644 arch/riscv/kernel/vdso/futex.c diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index c2c2f5f3ed5e..588963bef804 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -166,6 +166,7 @@ config RISCV select HAVE_FUNCTION_GRAPH_TRACER if HAVE_DYNAMIC_FTRACE_WITH_ARGS select HAVE_FUNCTION_GRAPH_FREGS select HAVE_FUNCTION_TRACER if HAVE_DYNAMIC_FTRACE + select HAVE_FUTEX_ROBUST_UNLOCK select HAVE_EBPF_JIT if MMU select HAVE_GENERIC_TIF_BITS select HAVE_GUP_FAST if MMU diff --git a/arch/riscv/include/asm/cpufeature-macros.h b/arch/riscv/include/asm/cpufeature-macros.h index a8103edbf51f..f522c6c3cd97 100644 --- a/arch/riscv/include/asm/cpufeature-macros.h +++ b/arch/riscv/include/asm/cpufeature-macros.h @@ -6,6 +6,8 @@ #ifndef _ASM_CPUFEATURE_MACROS_H #define _ASM_CPUFEATURE_MACROS_H +#include <linux/types.h> + #include <asm/hwcap.h> #include <asm/alternative-macros.h> diff --git a/arch/riscv/include/asm/futex_robust.h b/arch/riscv/include/asm/futex_robust.h new file mode 100644 index 000000000000..16ec58e925c8 --- /dev/null +++ b/arch/riscv/include/asm/futex_robust.h @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_RISCV_FUTEX_ROBUST_H +#define _ASM_RISCV_FUTEX_ROBUST_H + +#include <asm/ptrace.h> +#include <asm/vdso/futex.h> + +static inline void __user *arch_futex_robust_unlock_get_pop(struct pt_regs *regs) +{ + if (cpu_supports_zacas()) + return (regs->a0 == regs->a1) ? (void __user *)regs->a2 : NULL; + else + return (regs->t0 == 0) ? (void __user *)regs->a2 : NULL; +} + +#endif /* _ASM_RISCV_FUTEX_ROBUST_H */ diff --git a/arch/riscv/include/asm/vdso/futex.h b/arch/riscv/include/asm/vdso/futex.h new file mode 100644 index 000000000000..9e173f45f3e2 --- /dev/null +++ b/arch/riscv/include/asm/vdso/futex.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __ASM_VDSO_FUTEX_H +#define __ASM_VDSO_FUTEX_H + +#include <asm/cpufeature-macros.h> + +static inline bool cpu_supports_zacas(void) +{ + return IS_ENABLED(CONFIG_RISCV_ISA_ZACAS) && IS_ENABLED(CONFIG_TOOLCHAIN_HAS_ZACAS) && + riscv_has_extension_unlikely(RISCV_ISA_EXT_ZACAS); +} + +#endif /* __ASM_VDSO_FUTEX_H */ diff --git a/arch/riscv/kernel/compat_vdso/Makefile b/arch/riscv/kernel/compat_vdso/Makefile index 03d128080c6a..81307063945d 100644 --- a/arch/riscv/kernel/compat_vdso/Makefile +++ b/arch/riscv/kernel/compat_vdso/Makefile @@ -24,7 +24,12 @@ COMPAT_CC_FLAGS += $(call cc-option,-mno-riscv-attribute) COMPAT_CC_FLAGS += $(call as-option,-Wa$(comma)-mno-arch-attr) # Files to link into the compat_vdso -obj-compat_vdso = $(patsubst %, %.o, $(compat_vdso-syms)) note.o +obj-compat_vdso := $(patsubst %, %.o, $(compat_vdso-syms)) note.o + +ifdef CONFIG_FUTEX_ROBUST_UNLOCK +compat_vdso-syms += __vdso_futex_robust_list32_try_unlock +obj-compat_vdso += futex.o +endif # Build rules targets := $(obj-compat_vdso) compat_vdso.so compat_vdso.so.dbg compat_vdso.lds diff --git a/arch/riscv/kernel/compat_vdso/futex.c b/arch/riscv/kernel/compat_vdso/futex.c new file mode 100644 index 000000000000..ad05ded43955 --- /dev/null +++ b/arch/riscv/kernel/compat_vdso/futex.c @@ -0,0 +1,3 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include "../vdso/futex.c" diff --git a/arch/riscv/kernel/vdso.c b/arch/riscv/kernel/vdso.c index 9c2f5e442338..2610363f9a02 100644 --- a/arch/riscv/kernel/vdso.c +++ b/arch/riscv/kernel/vdso.c @@ -11,6 +11,7 @@ #include <linux/slab.h> #include <linux/binfmts.h> #include <linux/err.h> +#include <linux/futex.h> #include <asm/page.h> #include <asm/vdso.h> #include <linux/vdso_datastore.h> @@ -33,11 +34,69 @@ static struct __vdso_info vdso_info; static struct __vdso_info compat_vdso_info; #endif +#define _CONCAT3(a, b, c) a ## b ## c +#define CONCAT3(a, b, c) _CONCAT3(a, b, c) + +#if defined(CONFIG_RISCV_ISA_ZACAS) && defined(CONFIG_TOOLCHAIN_HAS_ZACAS) +#define FUTEX_CAS_SET_VDSO_CS_RANGE(vdso, fd, idx, xlen, symbol) \ +({ \ + void *start = symbol(vdso, CONCAT3(futex_list, xlen, _try_unlock_cs_cas_start)); \ + void *end = symbol(vdso, CONCAT3(futex_list, xlen, _try_unlock_cs_cas_end)); \ + \ + futex_set_vdso_cs_range(fd, idx, (uintptr_t)start, (uintptr_t)end, xlen == 32); \ +}) +#else +#define FUTEX_CAS_SET_VDSO_CS_RANGE(...) BUILD_BUG() +#endif + +#define FUTEX_LRSC_SET_VDSO_CS_RANGE(vdso, fd, idx, xlen, symbol) \ +({ \ + void *start = symbol(vdso, CONCAT3(futex_list, xlen, _try_unlock_cs_lrsc_start)); \ + void *end = symbol(vdso, CONCAT3(futex_list, xlen, _try_unlock_cs_lrsc_end)); \ + \ + futex_set_vdso_cs_range(fd, idx, (uintptr_t)start, (uintptr_t)end, xlen == 32); \ +}) + +#define FUTEX_SET_VDSO_CS_RANGE(vdso, fd, idx, xlen, symbol) \ +({ \ + if (cpu_supports_zacas()) \ + FUTEX_CAS_SET_VDSO_CS_RANGE(vdso, fd, idx, xlen, symbol); \ + else \ + FUTEX_LRSC_SET_VDSO_CS_RANGE(vdso, fd, idx, xlen, symbol); \ +}) + +#ifdef CONFIG_FUTEX_ROBUST_UNLOCK + +/* Allow parameters to expand first */ +#define __VDSO_SYMBOL(vdso, name) VDSO_SYMBOL(vdso, name) +#define __COMPAT_VDSO_SYMBOL(vdso, name) COMPAT_VDSO_SYMBOL(vdso, name) + +static void vdso_futex_robust_unlock_update_ips(void) +{ + unsigned long vdso = (unsigned long) current->mm->context.vdso; + struct futex_mm_data *fd = ¤t->mm->futex; + + futex_reset_cs_ranges(fd); + + if (!is_compat_task()) + FUTEX_SET_VDSO_CS_RANGE(vdso, fd, 0, __riscv_xlen, __VDSO_SYMBOL); + +#ifdef CONFIG_COMPAT + if (is_compat_task()) + FUTEX_SET_VDSO_CS_RANGE(vdso, fd, 0, 32, __COMPAT_VDSO_SYMBOL); +#endif +} +#else +static inline void vdso_futex_robust_unlock_update_ips(void) {} +#endif + static int vdso_mremap(const struct vm_special_mapping *sm, struct vm_area_struct *new_vma) { current->mm->context.vdso = (void *)new_vma->vm_start; + vdso_futex_robust_unlock_update_ips(); + return 0; } @@ -147,6 +206,8 @@ static int __setup_additional_pages(struct mm_struct *mm, if (IS_ERR(ret)) goto up_fail; + vdso_futex_robust_unlock_update_ips(); + return 0; up_fail: diff --git a/arch/riscv/kernel/vdso/Makefile b/arch/riscv/kernel/vdso/Makefile index 8dbf2532a573..8da2f605bde4 100644 --- a/arch/riscv/kernel/vdso/Makefile +++ b/arch/riscv/kernel/vdso/Makefile @@ -27,12 +27,17 @@ asflags-y += -DVDSO_CFI=1 endif # Files to link into the vdso -obj-vdso = $(patsubst %, %.o, $(vdso-syms)) note.o +obj-vdso := $(patsubst %, %.o, $(vdso-syms)) note.o ifdef CONFIG_VDSO_GETRANDOM obj-vdso += vgetrandom-chacha.o endif +ifdef CONFIG_FUTEX_ROBUST_UNLOCK +obj-vdso += futex.o +vdso-syms += __vdso_futex_robust_list$(BITS)_try_unlock +endif + ccflags-y := -fno-stack-protector ccflags-y += -DDISABLE_BRANCH_PROFILING ccflags-y += -fno-builtin diff --git a/arch/riscv/kernel/vdso/futex.c b/arch/riscv/kernel/vdso/futex.c new file mode 100644 index 000000000000..761bf2a209a6 --- /dev/null +++ b/arch/riscv/kernel/vdso/futex.c @@ -0,0 +1,87 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include <linux/stringify.h> +#include <vdso/futex.h> +#include <asm/hwcap.h> +#include <asm/cpufeature-macros.h> +#include <asm/vdso/futex.h> + +#define LABEL(pop_size, which) __stringify(__vdso_futex_list##pop_size##_try_unlock_cs_##which) + +#define futex_robust_try_unlock_cas(pop_size, store_pop, lock, tid, pop) \ +({ \ + /* \ + * arch_futex_robust_unlock_get_pop() assumes the variables are in \ + * those registers. So make sure. \ + * \ + * tid and pop are in a1 and a2 at function entry according to the \ + * calling convention, so it likely still works if we remove _tid \ + * and _pop. But technically compiler is allowed to move tid and pop \ + * to different registers, and _tid and _pop do not generate any \ + * extra instructions so it does not hurt to keep them. \ + */ \ + register __u32 ret asm ("a0") = tid; \ + register __u32 _tid asm ("a1") = tid; \ + register void *_pop asm ("a2") = pop; \ + \ + asm volatile ( \ + " .option push\n" \ + " .option arch, +zacas\n" \ + " amocas.w.rl %[ret], zero, (%[lock])\n" \ + " .option pop\n" \ + LABEL(pop_size, cas_start)":" \ + " bne %[ret], %[tid], "LABEL(pop_size, cas_end)"\n" \ + " "store_pop" zero, (%[pop])\n" \ + LABEL(pop_size, cas_end)":" \ + : [ret] "+&r" (ret) \ + : [tid] "r" (_tid), \ + [lock] "r" (lock), \ + [pop] "r" (_pop) \ + : "memory" \ + ); \ + \ + ret; \ +}) + +#define futex_robust_try_unlock_lrsc(pop_size, store_pop, lock, tid, pop) \ +({ \ + register void *_pop asm ("a2") = pop; \ + __u32 ret; \ + \ + asm volatile ( \ + "1: lr.w %[ret], (%[lock])\n" \ + " bne %[ret], %[tid], "LABEL(pop_size, lrsc_end)"\n" \ + " sc.w.rl t0, x0, (%[lock])\n" \ + LABEL(pop_size, lrsc_start)":" \ + " bnez t0, 1b\n" \ + " "store_pop" zero, (%[pop])\n" \ + LABEL(pop_size, lrsc_end)":" \ + : [ret] "=&r" (ret) \ + : [tid] "r" (tid), \ + [lock] "r" (lock), \ + [pop] "r" (_pop) \ + : "t0", "memory" \ + ); \ + \ + ret; \ +}) + + +#if __riscv_xlen == 64 +__u32 __vdso_futex_robust_list64_try_unlock(__u32 *lock, __u32 tid, __u64 *pop) +{ + if (cpu_supports_zacas()) + return futex_robust_try_unlock_cas(64, "sd", lock, tid, pop); + else + return futex_robust_try_unlock_lrsc(64, "sd", lock, tid, pop); +} +#endif + +#if __riscv_xlen == 32 +__u32 __vdso_futex_robust_list32_try_unlock(__u32 *lock, __u32 tid, __u32 *pop) +{ + if (cpu_supports_zacas()) + return futex_robust_try_unlock_cas(32, "sw", lock, tid, pop); + else + return futex_robust_try_unlock_lrsc(32, "sw", lock, tid, pop); +} +#endif diff --git a/arch/riscv/kernel/vdso/vdso.lds.S b/arch/riscv/kernel/vdso/vdso.lds.S index c29ef12a63bb..9acfa9ef5bab 100644 --- a/arch/riscv/kernel/vdso/vdso.lds.S +++ b/arch/riscv/kernel/vdso/vdso.lds.S @@ -82,6 +82,14 @@ VERSION #endif #if defined(CONFIG_VDSO_GETRANDOM) && !defined(COMPAT_VDSO) __vdso_getrandom; +#endif + +#ifdef CONFIG_FUTEX_ROBUST_UNLOCK +#if defined(CONFIG_32BIT) || defined(COMPAT_VDSO) + __vdso_futex_robust_list32_try_unlock; +#else + __vdso_futex_robust_list64_try_unlock; +#endif #endif local: *; }; -- 2.47.3 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* [PATCH v4 3/3] riscv: vdso: Implement __vdso_futex_robust_try_unlock() @ 2026-07-22 9:06 ` Nam Cao 0 siblings, 0 replies; 8+ messages in thread From: Nam Cao @ 2026-07-22 9:06 UTC (permalink / raw) To: Paul Walmsley, Palmer Dabbelt, Albert Ou, Alexandre Ghiti, linux-riscv, linux-kernel, Thomas Weißschuh, André Almeida, Mathieu Desnoyers, Thomas Gleixner, Peter Zijlstra, Sebastian Andrzej Siewior Cc: Nam Cao This is the RISC-V port for __vdso_futex_robust_try_unlock(). It is based on the x86's implementation in commit 61cfc8e372d1 ("x86/vdso: Prepare for robust futex unlock support") and commit a2274cc0091e ("x86/vdso: Implement __vdso_futex_robust_try_unlock()"). When the FUTEX_ROBUST_UNLOCK mechanism is used for unlocking (PI-)futexes, the unlock sequence in userspace looks like this: 1) robust_list_set_op_pending(mutex); 2) robust_list_remove(mutex); lval = gettid(); 3) if (atomic_try_cmpxchg(&mutex->lock, lval, 0)) 4) robust_list_clear_op_pending(); else 5) sys_futex(OP,...FUTEX_ROBUST_UNLOCK); That still leaves a minimal race window between #3 and #4 where the mutex could be acquired by some other task which observes that it is the last user and: 1) unmaps the mutex memory 2) maps a different file, which ends up covering the same address When then the original task exits before reaching #4 then the kernel robust list handling observes the pending op entry and tries to fix up user space. In case that the newly mapped data contains the TID of the exiting thread at the address of the mutex/futex the kernel will set the owner died bit in that memory and therefore corrupt unrelated data. Provide a VDSO function which exposes the critical section window in the VDSO symbol table. The resulting addresses are updated in the task's mm when the VDSO is (re)map()'ed. The core code detects when a task was interrupted within the critical section and is about to deliver a signal. It then invokes an architecture specific function which determines whether the pending op pointer has to be cleared or not. The unlock assembly sequence on 64-bit is: mv a5, a0 ; move LOCK to a5 .Lretry lr.w a0, (a5) ; load *LOCK bne a0, a1, 24 .Lend ; if (*LOCK != TID) then exit sc.w.rl t0, zero, (a5) ; try changing *LOCK from TID to zero .Lstart bnez t0, .Lretry ; retry if store-conditional failed sd zero, 0(a2) ; clear POP .Lend sext.w a0,a0 ret So if the kernel sees that the user task is between .Lstart and .Lend, t0 can be checked to determine if the pending op pointer (POP) should be cleared. When the kernel is built with ZACAS, another unlock assembly sequence is provided using ZACAS instructions. If the CPU supports ZACAS, that alternative sequence is used and the critical section is also changed to that. There are two entry points to handle the different robust list pending op pointer sizes: __vdso_futex_robust_list64_try_unlock() __vdso_futex_robust_list32_try_unlock() The 32-bit VDSO and compat VDSO provides __vdso_futex_robust_list32_try_unlock(). Unlike x86, the RISC-V 64-bit VDSO provides only provides __vdso_futex_robust_list64_try_unlock(). Because RISC-V threads can only have one list_op_pending pointer type. Reviewed-by: Thomas Weißschuh <thomas.weissschuh@linutronix.de> Signed-off-by: Nam Cao <namcao@linutronix.de> --- This depends on compat vdso patching: https://lore.kernel.org/all/20260630-riscv-vdso32-alternative-v1-0-a32fd89b7b1c@linutronix.de/ --- arch/riscv/Kconfig | 1 + arch/riscv/include/asm/cpufeature-macros.h | 2 + arch/riscv/include/asm/futex_robust.h | 16 ++++ arch/riscv/include/asm/vdso/futex.h | 14 ++++ arch/riscv/kernel/compat_vdso/Makefile | 7 +- arch/riscv/kernel/compat_vdso/futex.c | 3 + arch/riscv/kernel/vdso.c | 61 +++++++++++++++ arch/riscv/kernel/vdso/Makefile | 7 +- arch/riscv/kernel/vdso/futex.c | 87 ++++++++++++++++++++++ arch/riscv/kernel/vdso/vdso.lds.S | 8 ++ 10 files changed, 204 insertions(+), 2 deletions(-) create mode 100644 arch/riscv/include/asm/futex_robust.h create mode 100644 arch/riscv/include/asm/vdso/futex.h create mode 100644 arch/riscv/kernel/compat_vdso/futex.c create mode 100644 arch/riscv/kernel/vdso/futex.c diff --git a/arch/riscv/Kconfig b/arch/riscv/Kconfig index c2c2f5f3ed5e..588963bef804 100644 --- a/arch/riscv/Kconfig +++ b/arch/riscv/Kconfig @@ -166,6 +166,7 @@ config RISCV select HAVE_FUNCTION_GRAPH_TRACER if HAVE_DYNAMIC_FTRACE_WITH_ARGS select HAVE_FUNCTION_GRAPH_FREGS select HAVE_FUNCTION_TRACER if HAVE_DYNAMIC_FTRACE + select HAVE_FUTEX_ROBUST_UNLOCK select HAVE_EBPF_JIT if MMU select HAVE_GENERIC_TIF_BITS select HAVE_GUP_FAST if MMU diff --git a/arch/riscv/include/asm/cpufeature-macros.h b/arch/riscv/include/asm/cpufeature-macros.h index a8103edbf51f..f522c6c3cd97 100644 --- a/arch/riscv/include/asm/cpufeature-macros.h +++ b/arch/riscv/include/asm/cpufeature-macros.h @@ -6,6 +6,8 @@ #ifndef _ASM_CPUFEATURE_MACROS_H #define _ASM_CPUFEATURE_MACROS_H +#include <linux/types.h> + #include <asm/hwcap.h> #include <asm/alternative-macros.h> diff --git a/arch/riscv/include/asm/futex_robust.h b/arch/riscv/include/asm/futex_robust.h new file mode 100644 index 000000000000..16ec58e925c8 --- /dev/null +++ b/arch/riscv/include/asm/futex_robust.h @@ -0,0 +1,16 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +#ifndef _ASM_RISCV_FUTEX_ROBUST_H +#define _ASM_RISCV_FUTEX_ROBUST_H + +#include <asm/ptrace.h> +#include <asm/vdso/futex.h> + +static inline void __user *arch_futex_robust_unlock_get_pop(struct pt_regs *regs) +{ + if (cpu_supports_zacas()) + return (regs->a0 == regs->a1) ? (void __user *)regs->a2 : NULL; + else + return (regs->t0 == 0) ? (void __user *)regs->a2 : NULL; +} + +#endif /* _ASM_RISCV_FUTEX_ROBUST_H */ diff --git a/arch/riscv/include/asm/vdso/futex.h b/arch/riscv/include/asm/vdso/futex.h new file mode 100644 index 000000000000..9e173f45f3e2 --- /dev/null +++ b/arch/riscv/include/asm/vdso/futex.h @@ -0,0 +1,14 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#ifndef __ASM_VDSO_FUTEX_H +#define __ASM_VDSO_FUTEX_H + +#include <asm/cpufeature-macros.h> + +static inline bool cpu_supports_zacas(void) +{ + return IS_ENABLED(CONFIG_RISCV_ISA_ZACAS) && IS_ENABLED(CONFIG_TOOLCHAIN_HAS_ZACAS) && + riscv_has_extension_unlikely(RISCV_ISA_EXT_ZACAS); +} + +#endif /* __ASM_VDSO_FUTEX_H */ diff --git a/arch/riscv/kernel/compat_vdso/Makefile b/arch/riscv/kernel/compat_vdso/Makefile index 03d128080c6a..81307063945d 100644 --- a/arch/riscv/kernel/compat_vdso/Makefile +++ b/arch/riscv/kernel/compat_vdso/Makefile @@ -24,7 +24,12 @@ COMPAT_CC_FLAGS += $(call cc-option,-mno-riscv-attribute) COMPAT_CC_FLAGS += $(call as-option,-Wa$(comma)-mno-arch-attr) # Files to link into the compat_vdso -obj-compat_vdso = $(patsubst %, %.o, $(compat_vdso-syms)) note.o +obj-compat_vdso := $(patsubst %, %.o, $(compat_vdso-syms)) note.o + +ifdef CONFIG_FUTEX_ROBUST_UNLOCK +compat_vdso-syms += __vdso_futex_robust_list32_try_unlock +obj-compat_vdso += futex.o +endif # Build rules targets := $(obj-compat_vdso) compat_vdso.so compat_vdso.so.dbg compat_vdso.lds diff --git a/arch/riscv/kernel/compat_vdso/futex.c b/arch/riscv/kernel/compat_vdso/futex.c new file mode 100644 index 000000000000..ad05ded43955 --- /dev/null +++ b/arch/riscv/kernel/compat_vdso/futex.c @@ -0,0 +1,3 @@ +// SPDX-License-Identifier: GPL-2.0-only + +#include "../vdso/futex.c" diff --git a/arch/riscv/kernel/vdso.c b/arch/riscv/kernel/vdso.c index 9c2f5e442338..2610363f9a02 100644 --- a/arch/riscv/kernel/vdso.c +++ b/arch/riscv/kernel/vdso.c @@ -11,6 +11,7 @@ #include <linux/slab.h> #include <linux/binfmts.h> #include <linux/err.h> +#include <linux/futex.h> #include <asm/page.h> #include <asm/vdso.h> #include <linux/vdso_datastore.h> @@ -33,11 +34,69 @@ static struct __vdso_info vdso_info; static struct __vdso_info compat_vdso_info; #endif +#define _CONCAT3(a, b, c) a ## b ## c +#define CONCAT3(a, b, c) _CONCAT3(a, b, c) + +#if defined(CONFIG_RISCV_ISA_ZACAS) && defined(CONFIG_TOOLCHAIN_HAS_ZACAS) +#define FUTEX_CAS_SET_VDSO_CS_RANGE(vdso, fd, idx, xlen, symbol) \ +({ \ + void *start = symbol(vdso, CONCAT3(futex_list, xlen, _try_unlock_cs_cas_start)); \ + void *end = symbol(vdso, CONCAT3(futex_list, xlen, _try_unlock_cs_cas_end)); \ + \ + futex_set_vdso_cs_range(fd, idx, (uintptr_t)start, (uintptr_t)end, xlen == 32); \ +}) +#else +#define FUTEX_CAS_SET_VDSO_CS_RANGE(...) BUILD_BUG() +#endif + +#define FUTEX_LRSC_SET_VDSO_CS_RANGE(vdso, fd, idx, xlen, symbol) \ +({ \ + void *start = symbol(vdso, CONCAT3(futex_list, xlen, _try_unlock_cs_lrsc_start)); \ + void *end = symbol(vdso, CONCAT3(futex_list, xlen, _try_unlock_cs_lrsc_end)); \ + \ + futex_set_vdso_cs_range(fd, idx, (uintptr_t)start, (uintptr_t)end, xlen == 32); \ +}) + +#define FUTEX_SET_VDSO_CS_RANGE(vdso, fd, idx, xlen, symbol) \ +({ \ + if (cpu_supports_zacas()) \ + FUTEX_CAS_SET_VDSO_CS_RANGE(vdso, fd, idx, xlen, symbol); \ + else \ + FUTEX_LRSC_SET_VDSO_CS_RANGE(vdso, fd, idx, xlen, symbol); \ +}) + +#ifdef CONFIG_FUTEX_ROBUST_UNLOCK + +/* Allow parameters to expand first */ +#define __VDSO_SYMBOL(vdso, name) VDSO_SYMBOL(vdso, name) +#define __COMPAT_VDSO_SYMBOL(vdso, name) COMPAT_VDSO_SYMBOL(vdso, name) + +static void vdso_futex_robust_unlock_update_ips(void) +{ + unsigned long vdso = (unsigned long) current->mm->context.vdso; + struct futex_mm_data *fd = ¤t->mm->futex; + + futex_reset_cs_ranges(fd); + + if (!is_compat_task()) + FUTEX_SET_VDSO_CS_RANGE(vdso, fd, 0, __riscv_xlen, __VDSO_SYMBOL); + +#ifdef CONFIG_COMPAT + if (is_compat_task()) + FUTEX_SET_VDSO_CS_RANGE(vdso, fd, 0, 32, __COMPAT_VDSO_SYMBOL); +#endif +} +#else +static inline void vdso_futex_robust_unlock_update_ips(void) {} +#endif + static int vdso_mremap(const struct vm_special_mapping *sm, struct vm_area_struct *new_vma) { current->mm->context.vdso = (void *)new_vma->vm_start; + vdso_futex_robust_unlock_update_ips(); + return 0; } @@ -147,6 +206,8 @@ static int __setup_additional_pages(struct mm_struct *mm, if (IS_ERR(ret)) goto up_fail; + vdso_futex_robust_unlock_update_ips(); + return 0; up_fail: diff --git a/arch/riscv/kernel/vdso/Makefile b/arch/riscv/kernel/vdso/Makefile index 8dbf2532a573..8da2f605bde4 100644 --- a/arch/riscv/kernel/vdso/Makefile +++ b/arch/riscv/kernel/vdso/Makefile @@ -27,12 +27,17 @@ asflags-y += -DVDSO_CFI=1 endif # Files to link into the vdso -obj-vdso = $(patsubst %, %.o, $(vdso-syms)) note.o +obj-vdso := $(patsubst %, %.o, $(vdso-syms)) note.o ifdef CONFIG_VDSO_GETRANDOM obj-vdso += vgetrandom-chacha.o endif +ifdef CONFIG_FUTEX_ROBUST_UNLOCK +obj-vdso += futex.o +vdso-syms += __vdso_futex_robust_list$(BITS)_try_unlock +endif + ccflags-y := -fno-stack-protector ccflags-y += -DDISABLE_BRANCH_PROFILING ccflags-y += -fno-builtin diff --git a/arch/riscv/kernel/vdso/futex.c b/arch/riscv/kernel/vdso/futex.c new file mode 100644 index 000000000000..761bf2a209a6 --- /dev/null +++ b/arch/riscv/kernel/vdso/futex.c @@ -0,0 +1,87 @@ +// SPDX-License-Identifier: GPL-2.0-only +#include <linux/stringify.h> +#include <vdso/futex.h> +#include <asm/hwcap.h> +#include <asm/cpufeature-macros.h> +#include <asm/vdso/futex.h> + +#define LABEL(pop_size, which) __stringify(__vdso_futex_list##pop_size##_try_unlock_cs_##which) + +#define futex_robust_try_unlock_cas(pop_size, store_pop, lock, tid, pop) \ +({ \ + /* \ + * arch_futex_robust_unlock_get_pop() assumes the variables are in \ + * those registers. So make sure. \ + * \ + * tid and pop are in a1 and a2 at function entry according to the \ + * calling convention, so it likely still works if we remove _tid \ + * and _pop. But technically compiler is allowed to move tid and pop \ + * to different registers, and _tid and _pop do not generate any \ + * extra instructions so it does not hurt to keep them. \ + */ \ + register __u32 ret asm ("a0") = tid; \ + register __u32 _tid asm ("a1") = tid; \ + register void *_pop asm ("a2") = pop; \ + \ + asm volatile ( \ + " .option push\n" \ + " .option arch, +zacas\n" \ + " amocas.w.rl %[ret], zero, (%[lock])\n" \ + " .option pop\n" \ + LABEL(pop_size, cas_start)":" \ + " bne %[ret], %[tid], "LABEL(pop_size, cas_end)"\n" \ + " "store_pop" zero, (%[pop])\n" \ + LABEL(pop_size, cas_end)":" \ + : [ret] "+&r" (ret) \ + : [tid] "r" (_tid), \ + [lock] "r" (lock), \ + [pop] "r" (_pop) \ + : "memory" \ + ); \ + \ + ret; \ +}) + +#define futex_robust_try_unlock_lrsc(pop_size, store_pop, lock, tid, pop) \ +({ \ + register void *_pop asm ("a2") = pop; \ + __u32 ret; \ + \ + asm volatile ( \ + "1: lr.w %[ret], (%[lock])\n" \ + " bne %[ret], %[tid], "LABEL(pop_size, lrsc_end)"\n" \ + " sc.w.rl t0, x0, (%[lock])\n" \ + LABEL(pop_size, lrsc_start)":" \ + " bnez t0, 1b\n" \ + " "store_pop" zero, (%[pop])\n" \ + LABEL(pop_size, lrsc_end)":" \ + : [ret] "=&r" (ret) \ + : [tid] "r" (tid), \ + [lock] "r" (lock), \ + [pop] "r" (_pop) \ + : "t0", "memory" \ + ); \ + \ + ret; \ +}) + + +#if __riscv_xlen == 64 +__u32 __vdso_futex_robust_list64_try_unlock(__u32 *lock, __u32 tid, __u64 *pop) +{ + if (cpu_supports_zacas()) + return futex_robust_try_unlock_cas(64, "sd", lock, tid, pop); + else + return futex_robust_try_unlock_lrsc(64, "sd", lock, tid, pop); +} +#endif + +#if __riscv_xlen == 32 +__u32 __vdso_futex_robust_list32_try_unlock(__u32 *lock, __u32 tid, __u32 *pop) +{ + if (cpu_supports_zacas()) + return futex_robust_try_unlock_cas(32, "sw", lock, tid, pop); + else + return futex_robust_try_unlock_lrsc(32, "sw", lock, tid, pop); +} +#endif diff --git a/arch/riscv/kernel/vdso/vdso.lds.S b/arch/riscv/kernel/vdso/vdso.lds.S index c29ef12a63bb..9acfa9ef5bab 100644 --- a/arch/riscv/kernel/vdso/vdso.lds.S +++ b/arch/riscv/kernel/vdso/vdso.lds.S @@ -82,6 +82,14 @@ VERSION #endif #if defined(CONFIG_VDSO_GETRANDOM) && !defined(COMPAT_VDSO) __vdso_getrandom; +#endif + +#ifdef CONFIG_FUTEX_ROBUST_UNLOCK +#if defined(CONFIG_32BIT) || defined(COMPAT_VDSO) + __vdso_futex_robust_list32_try_unlock; +#else + __vdso_futex_robust_list64_try_unlock; +#endif #endif local: *; }; -- 2.47.3 _______________________________________________ linux-riscv mailing list linux-riscv@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-riscv ^ permalink raw reply related [flat|nested] 8+ messages in thread
end of thread, other threads:[~2026-07-22 9:08 UTC | newest] Thread overview: 8+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-07-22 9:06 [PATCH v4 0/3] riscv: vdso: Implement __vdso_futex_robust_try_unlock() Nam Cao 2026-07-22 9:06 ` Nam Cao 2026-07-22 9:06 ` [PATCH v4 1/3] riscv: compat_vdso: switch to standard kbuild rule Nam Cao 2026-07-22 9:06 ` Nam Cao 2026-07-22 9:06 ` [PATCH v4 2/3] riscv: compat_vdso: Allow *.c source files Nam Cao 2026-07-22 9:06 ` Nam Cao 2026-07-22 9:06 ` [PATCH v4 3/3] riscv: vdso: Implement __vdso_futex_robust_try_unlock() Nam Cao 2026-07-22 9:06 ` Nam Cao
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.