* [PATCH 0/4] entry: Move ret_from_fork() to C and inline syscall_exit_to_user_mode()
@ 2025-01-22 22:56 Charlie Jenkins
2025-01-22 22:56 ` [PATCH 1/4] riscv: entry: Convert ret_from_fork() to C Charlie Jenkins
` (3 more replies)
0 siblings, 4 replies; 14+ messages in thread
From: Charlie Jenkins @ 2025-01-22 22:56 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Huacai Chen, WANG Xuerui,
Thomas Gleixner, Peter Zijlstra, Andy Lutomirski, Alexandre Ghiti
Cc: linux-riscv, linux-kernel, loongarch, Charlie Jenkins
Similar to commit 221a164035fd ("entry: Move
syscall_enter_from_user_mode() to header file"), move
syscall_exit_to_user_mode() to the header file as well.
Testing was done with the byte-unixbench [1] syscall benchmark (which
calls getpid) and QEMU. On riscv I measured a 7.09246% improvement, on
x86 a 2.98843% improvement, on loongarch a 6.07954% improvement, and on
s390 a 11.1328% improvement.
Since this is on QEMU, I know these numbers are not perfect, but they
show a trend of general improvement across all architectures that use
the generic entry code.
[1] https://github.com/kdlucas/byte-unixbench
Signed-off-by: Charlie Jenkins <charlie@rivosinc.com>
---
Charlie Jenkins (4):
riscv: entry: Convert ret_from_fork() to C
riscv: entry: Split ret_from_fork() into user and kernel
loongarch: entry: Migrate ret_from_fork() to C
entry: Inline syscall_exit_to_user_mode()
arch/loongarch/include/asm/switch_to.h | 8 ++++++
arch/loongarch/kernel/entry.S | 22 +++++++--------
arch/loongarch/kernel/process.c | 36 ++++++++++++++++++++----
arch/riscv/include/asm/asm-prototypes.h | 2 ++
arch/riscv/kernel/entry.S | 20 ++++++++------
arch/riscv/kernel/process.c | 21 ++++++++++++--
include/linux/entry-common.h | 43 +++++++++++++++++++++++++++--
kernel/entry/common.c | 49 +--------------------------------
8 files changed, 122 insertions(+), 79 deletions(-)
---
base-commit: ffd294d346d185b70e28b1a28abe367bbfe53c04
change-id: 20240402-riscv_optimize_entry-583843420325
--
- Charlie
^ permalink raw reply [flat|nested] 14+ messages in thread
* [PATCH 1/4] riscv: entry: Convert ret_from_fork() to C
2025-01-22 22:56 [PATCH 0/4] entry: Move ret_from_fork() to C and inline syscall_exit_to_user_mode() Charlie Jenkins
@ 2025-01-22 22:56 ` Charlie Jenkins
2025-01-24 7:09 ` Alexandre Ghiti
2025-01-25 23:30 ` Maciej W. Rozycki
2025-01-22 22:56 ` [PATCH 2/4] riscv: entry: Split ret_from_fork() into user and kernel Charlie Jenkins
` (2 subsequent siblings)
3 siblings, 2 replies; 14+ messages in thread
From: Charlie Jenkins @ 2025-01-22 22:56 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Huacai Chen, WANG Xuerui,
Thomas Gleixner, Peter Zijlstra, Andy Lutomirski, Alexandre Ghiti
Cc: linux-riscv, linux-kernel, loongarch, Charlie Jenkins
Move the main section of ret_from_fork() to C to allow inlining of
syscall_exit_to_user_mode().
Signed-off-by: Charlie Jenkins <charlie@rivosinc.com>
---
arch/riscv/include/asm/asm-prototypes.h | 1 +
arch/riscv/kernel/entry.S | 15 ++++++---------
arch/riscv/kernel/process.c | 14 ++++++++++++--
3 files changed, 19 insertions(+), 11 deletions(-)
diff --git a/arch/riscv/include/asm/asm-prototypes.h b/arch/riscv/include/asm/asm-prototypes.h
index cd627ec289f163a630b73dd03dd52a6b28692997..733ff609778797001006c33bba9e3cc5b1f15387 100644
--- a/arch/riscv/include/asm/asm-prototypes.h
+++ b/arch/riscv/include/asm/asm-prototypes.h
@@ -52,6 +52,7 @@ DECLARE_DO_ERROR_INFO(do_trap_ecall_s);
DECLARE_DO_ERROR_INFO(do_trap_ecall_m);
DECLARE_DO_ERROR_INFO(do_trap_break);
+asmlinkage void ret_from_fork(void *fn_arg, int (*fn)(void *), struct pt_regs *regs);
asmlinkage void handle_bad_stack(struct pt_regs *regs);
asmlinkage void do_page_fault(struct pt_regs *regs);
asmlinkage void do_irq(struct pt_regs *regs);
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index 33a5a9f2a0d4e1eeccfb3621b9e518b88e1b0704..9225c322279aa90e737b1d7144db084319cf8103 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -319,17 +319,14 @@ SYM_CODE_END(handle_kernel_stack_overflow)
ASM_NOKPROBE(handle_kernel_stack_overflow)
#endif
-SYM_CODE_START(ret_from_fork)
+SYM_CODE_START(ret_from_fork_asm)
call schedule_tail
- beqz s0, 1f /* not from kernel thread */
- /* Call fn(arg) */
- move a0, s1
- jalr s0
-1:
- move a0, sp /* pt_regs */
- call syscall_exit_to_user_mode
+ move a0, s1 /* fn */
+ move a1, s0 /* fn_arg */
+ move a2, sp /* pt_regs */
+ call ret_from_fork
j ret_from_exception
-SYM_CODE_END(ret_from_fork)
+SYM_CODE_END(ret_from_fork_asm)
#ifdef CONFIG_IRQ_STACKS
/*
diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c
index 58b6482c2bf662bf5224ca50c8e21a68760a6b41..0d07e6d8f6b57beba438dbba5e8c74a014582bee 100644
--- a/arch/riscv/kernel/process.c
+++ b/arch/riscv/kernel/process.c
@@ -17,7 +17,9 @@
#include <linux/ptrace.h>
#include <linux/uaccess.h>
#include <linux/personality.h>
+#include <linux/entry-common.h>
+#include <asm/asm-prototypes.h>
#include <asm/unistd.h>
#include <asm/processor.h>
#include <asm/csr.h>
@@ -36,7 +38,7 @@ unsigned long __stack_chk_guard __read_mostly;
EXPORT_SYMBOL(__stack_chk_guard);
#endif
-extern asmlinkage void ret_from_fork(void);
+extern asmlinkage void ret_from_fork_asm(void);
void noinstr arch_cpu_idle(void)
{
@@ -206,6 +208,14 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
return 0;
}
+asmlinkage void ret_from_fork(void *fn_arg, int (*fn)(void *), struct pt_regs *regs)
+{
+ if (unlikely(fn))
+ fn(fn_arg);
+
+ syscall_exit_to_user_mode(regs);
+}
+
int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
{
unsigned long clone_flags = args->flags;
@@ -242,7 +252,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
p->thread.riscv_v_flags = 0;
if (has_vector())
riscv_v_thread_alloc(p);
- p->thread.ra = (unsigned long)ret_from_fork;
+ p->thread.ra = (unsigned long)ret_from_fork_asm;
p->thread.sp = (unsigned long)childregs; /* kernel sp */
return 0;
}
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH 2/4] riscv: entry: Split ret_from_fork() into user and kernel
2025-01-22 22:56 [PATCH 0/4] entry: Move ret_from_fork() to C and inline syscall_exit_to_user_mode() Charlie Jenkins
2025-01-22 22:56 ` [PATCH 1/4] riscv: entry: Convert ret_from_fork() to C Charlie Jenkins
@ 2025-01-22 22:56 ` Charlie Jenkins
2025-01-22 22:56 ` [PATCH 3/4] loongarch: entry: Migrate ret_from_fork() to C Charlie Jenkins
2025-01-22 22:56 ` [PATCH 4/4] entry: Inline syscall_exit_to_user_mode() Charlie Jenkins
3 siblings, 0 replies; 14+ messages in thread
From: Charlie Jenkins @ 2025-01-22 22:56 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Huacai Chen, WANG Xuerui,
Thomas Gleixner, Peter Zijlstra, Andy Lutomirski, Alexandre Ghiti
Cc: linux-riscv, linux-kernel, loongarch, Charlie Jenkins
This function was unified into a single function in commit ab9164dae273
("riscv: entry: Consolidate ret_from_kernel_thread into ret_from_fork").
However that imposed a performance degradation. Partially reverting this
commit to have ret_from_fork() split again results in a 1% increase on
the number of times fork is able to be called per second.
Signed-off-by: Charlie Jenkins <charlie@rivosinc.com>
---
arch/riscv/include/asm/asm-prototypes.h | 3 ++-
arch/riscv/kernel/entry.S | 13 ++++++++++---
arch/riscv/kernel/process.c | 17 +++++++++++------
3 files changed, 23 insertions(+), 10 deletions(-)
diff --git a/arch/riscv/include/asm/asm-prototypes.h b/arch/riscv/include/asm/asm-prototypes.h
index 733ff609778797001006c33bba9e3cc5b1f15387..bfc8ea5f9319b19449ec59493b45b926df888832 100644
--- a/arch/riscv/include/asm/asm-prototypes.h
+++ b/arch/riscv/include/asm/asm-prototypes.h
@@ -52,7 +52,8 @@ DECLARE_DO_ERROR_INFO(do_trap_ecall_s);
DECLARE_DO_ERROR_INFO(do_trap_ecall_m);
DECLARE_DO_ERROR_INFO(do_trap_break);
-asmlinkage void ret_from_fork(void *fn_arg, int (*fn)(void *), struct pt_regs *regs);
+asmlinkage void ret_from_fork_kernel(void *fn_arg, int (*fn)(void *), struct pt_regs *regs);
+asmlinkage void ret_from_fork_user(struct pt_regs *regs);
asmlinkage void handle_bad_stack(struct pt_regs *regs);
asmlinkage void do_page_fault(struct pt_regs *regs);
asmlinkage void do_irq(struct pt_regs *regs);
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index 9225c322279aa90e737b1d7144db084319cf8103..9386ef7444267f0b9bf8a0550f4e31deaeb85881 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -319,14 +319,21 @@ SYM_CODE_END(handle_kernel_stack_overflow)
ASM_NOKPROBE(handle_kernel_stack_overflow)
#endif
-SYM_CODE_START(ret_from_fork_asm)
+SYM_CODE_START(ret_from_fork_kernel_asm)
call schedule_tail
move a0, s1 /* fn */
move a1, s0 /* fn_arg */
move a2, sp /* pt_regs */
- call ret_from_fork
+ call ret_from_fork_kernel
j ret_from_exception
-SYM_CODE_END(ret_from_fork_asm)
+SYM_CODE_END(ret_from_fork_kernel_asm)
+
+SYM_CODE_START(ret_from_fork_user_asm)
+ call schedule_tail
+ move a0, sp /* pt_regs */
+ call ret_from_fork_user
+ j ret_from_exception
+SYM_CODE_END(ret_from_fork_user_asm)
#ifdef CONFIG_IRQ_STACKS
/*
diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c
index 0d07e6d8f6b57beba438dbba5e8c74a014582bee..5f15236cb526bd9fe61636ed372b4b76c94df946 100644
--- a/arch/riscv/kernel/process.c
+++ b/arch/riscv/kernel/process.c
@@ -38,7 +38,8 @@ unsigned long __stack_chk_guard __read_mostly;
EXPORT_SYMBOL(__stack_chk_guard);
#endif
-extern asmlinkage void ret_from_fork_asm(void);
+extern asmlinkage void ret_from_fork_kernel_asm(void);
+extern asmlinkage void ret_from_fork_user_asm(void);
void noinstr arch_cpu_idle(void)
{
@@ -208,14 +209,18 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
return 0;
}
-asmlinkage void ret_from_fork(void *fn_arg, int (*fn)(void *), struct pt_regs *regs)
+asmlinkage void ret_from_fork_kernel(void *fn_arg, int (*fn)(void *), struct pt_regs *regs)
{
- if (unlikely(fn))
- fn(fn_arg);
+ fn(fn_arg);
syscall_exit_to_user_mode(regs);
}
+asmlinkage void ret_from_fork_user(struct pt_regs *regs)
+{
+ syscall_exit_to_user_mode(regs);
+}
+
int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
{
unsigned long clone_flags = args->flags;
@@ -238,6 +243,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
p->thread.s[0] = (unsigned long)args->fn;
p->thread.s[1] = (unsigned long)args->fn_arg;
+ p->thread.ra = (unsigned long)ret_from_fork_kernel_asm;
} else {
*childregs = *(current_pt_regs());
/* Turn off status.VS */
@@ -247,12 +253,11 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
if (clone_flags & CLONE_SETTLS)
childregs->tp = tls;
childregs->a0 = 0; /* Return value of fork() */
- p->thread.s[0] = 0;
+ p->thread.ra = (unsigned long)ret_from_fork_user_asm;
}
p->thread.riscv_v_flags = 0;
if (has_vector())
riscv_v_thread_alloc(p);
- p->thread.ra = (unsigned long)ret_from_fork_asm;
p->thread.sp = (unsigned long)childregs; /* kernel sp */
return 0;
}
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH 3/4] loongarch: entry: Migrate ret_from_fork() to C
2025-01-22 22:56 [PATCH 0/4] entry: Move ret_from_fork() to C and inline syscall_exit_to_user_mode() Charlie Jenkins
2025-01-22 22:56 ` [PATCH 1/4] riscv: entry: Convert ret_from_fork() to C Charlie Jenkins
2025-01-22 22:56 ` [PATCH 2/4] riscv: entry: Split ret_from_fork() into user and kernel Charlie Jenkins
@ 2025-01-22 22:56 ` Charlie Jenkins
2025-01-23 17:12 ` kernel test robot
2025-01-22 22:56 ` [PATCH 4/4] entry: Inline syscall_exit_to_user_mode() Charlie Jenkins
3 siblings, 1 reply; 14+ messages in thread
From: Charlie Jenkins @ 2025-01-22 22:56 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Huacai Chen, WANG Xuerui,
Thomas Gleixner, Peter Zijlstra, Andy Lutomirski, Alexandre Ghiti
Cc: linux-riscv, linux-kernel, loongarch, Charlie Jenkins
Loongarch is the only architecture that calls
syscall_exit_to_user_mode() from asm. Move the call into C so that this
function can be inlined across all architectures.
Signed-off-by: Charlie Jenkins <charlie@rivosinc.com>
---
arch/loongarch/include/asm/switch_to.h | 8 ++++++++
arch/loongarch/kernel/entry.S | 22 ++++++++++-----------
arch/loongarch/kernel/process.c | 36 ++++++++++++++++++++++++++++------
3 files changed, 48 insertions(+), 18 deletions(-)
diff --git a/arch/loongarch/include/asm/switch_to.h b/arch/loongarch/include/asm/switch_to.h
index 5b225aff3ba21aa06d0713bc8e73e1b941389630..a1c5576f1fd145670e13038bec6dd390486099ab 100644
--- a/arch/loongarch/include/asm/switch_to.h
+++ b/arch/loongarch/include/asm/switch_to.h
@@ -26,6 +26,14 @@ extern asmlinkage struct task_struct *__switch_to(struct task_struct *prev,
struct task_struct *next, struct thread_info *next_ti,
void *sched_ra, void *sched_cfa);
+void noinstr __no_stack_protector ret_from_kernel_thread(struct task_struct *prev,
+ struct pt_regs *regs,
+ int (*fn)(void *),
+ void *fn_arg);
+
+void noinstr __no_stack_protector ret_from_fork(struct task_struct *prev,
+ struct pt_regs *regs);
+
/*
* For newly created kernel threads switch_to() will return to
* ret_from_kernel_thread, newly created user threads to ret_from_fork.
diff --git a/arch/loongarch/kernel/entry.S b/arch/loongarch/kernel/entry.S
index 48e7e34e355e83eae8165957ba2eac05a8bf17df..2abc29e573810e000f2fef4646ddca0dbb80eabe 100644
--- a/arch/loongarch/kernel/entry.S
+++ b/arch/loongarch/kernel/entry.S
@@ -77,24 +77,22 @@ SYM_CODE_START(handle_syscall)
SYM_CODE_END(handle_syscall)
_ASM_NOKPROBE(handle_syscall)
-SYM_CODE_START(ret_from_fork)
+SYM_CODE_START(ret_from_fork_asm)
UNWIND_HINT_REGS
- bl schedule_tail # a0 = struct task_struct *prev
- move a0, sp
- bl syscall_exit_to_user_mode
+ move a1, sp
+ bl ret_from_fork
RESTORE_STATIC
RESTORE_SOME
RESTORE_SP_AND_RET
-SYM_CODE_END(ret_from_fork)
+SYM_CODE_END(ret_from_fork_asm)
-SYM_CODE_START(ret_from_kernel_thread)
+SYM_CODE_START(ret_from_kernel_thread_asm)
UNWIND_HINT_REGS
- bl schedule_tail # a0 = struct task_struct *prev
- move a0, s1
- jirl ra, s0, 0
- move a0, sp
- bl syscall_exit_to_user_mode
+ move a1, sp
+ move a2, s0
+ move a3, s1
+ bl ret_from_kernel_thread
RESTORE_STATIC
RESTORE_SOME
RESTORE_SP_AND_RET
-SYM_CODE_END(ret_from_kernel_thread)
+SYM_CODE_END(ret_from_kernel_thread_asm)
diff --git a/arch/loongarch/kernel/process.c b/arch/loongarch/kernel/process.c
index 6e58f65455c7ca3eae2e88ed852c8655a6701e5c..9bb069ef363fefed1f2115e6572b31c2b76bb414 100644
--- a/arch/loongarch/kernel/process.c
+++ b/arch/loongarch/kernel/process.c
@@ -14,6 +14,7 @@
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/errno.h>
+#include <linux/entry-common.h>
#include <linux/sched.h>
#include <linux/sched/debug.h>
#include <linux/sched/task.h>
@@ -47,6 +48,7 @@
#include <asm/pgtable.h>
#include <asm/processor.h>
#include <asm/reg.h>
+#include <asm/switch_to.h>
#include <asm/unwind.h>
#include <asm/vdso.h>
@@ -63,8 +65,12 @@ EXPORT_SYMBOL(__stack_chk_guard);
unsigned long boot_option_idle_override = IDLE_NO_OVERRIDE;
EXPORT_SYMBOL(boot_option_idle_override);
-asmlinkage void ret_from_fork(void);
-asmlinkage void ret_from_kernel_thread(void);
+asmlinkage void restore_and_ret(void);
+asmlinkage void ret_from_fork_asm(void);
+void noinstr __no_stack_protector ret_from_kernel_thread(struct task_struct *prev,
+ struct pt_regs *regs,
+ int (*fn)(void *),
+ void *fn_arg)
void start_thread(struct pt_regs *regs, unsigned long pc, unsigned long sp)
{
@@ -138,6 +144,24 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
return 0;
}
+void noinstr __no_stack_protector ret_from_kernel_thread(struct task_struct *prev,
+ struct pt_regs *regs,
+ int (*fn)(void *),
+ void *fn_arg)
+{
+ schedule_tail(prev);
+
+ fn(fn_arg);
+
+ syscall_exit_to_user_mode(regs);
+}
+
+void noinstr __no_stack_protector ret_from_fork(struct task_struct *prev, struct pt_regs *regs)
+{
+ schedule_tail(prev);
+ syscall_exit_to_user_mode(regs);
+}
+
/*
* Copy architecture-specific thread state
*/
@@ -165,8 +189,8 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
p->thread.reg03 = childksp;
p->thread.reg23 = (unsigned long)args->fn;
p->thread.reg24 = (unsigned long)args->fn_arg;
- p->thread.reg01 = (unsigned long)ret_from_kernel_thread;
- p->thread.sched_ra = (unsigned long)ret_from_kernel_thread;
+ p->thread.reg01 = (unsigned long)ret_from_kernel_thread_asm;
+ p->thread.sched_ra = (unsigned long)ret_from_kernel_thread_asm;
memset(childregs, 0, sizeof(struct pt_regs));
childregs->csr_euen = p->thread.csr_euen;
childregs->csr_crmd = p->thread.csr_crmd;
@@ -182,8 +206,8 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
childregs->regs[3] = usp;
p->thread.reg03 = (unsigned long) childregs;
- p->thread.reg01 = (unsigned long) ret_from_fork;
- p->thread.sched_ra = (unsigned long) ret_from_fork;
+ p->thread.reg01 = (unsigned long) ret_from_fork_asm;
+ p->thread.sched_ra = (unsigned long) ret_from_fork_asm;
/*
* New tasks lose permission to use the fpu. This accelerates context
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* [PATCH 4/4] entry: Inline syscall_exit_to_user_mode()
2025-01-22 22:56 [PATCH 0/4] entry: Move ret_from_fork() to C and inline syscall_exit_to_user_mode() Charlie Jenkins
` (2 preceding siblings ...)
2025-01-22 22:56 ` [PATCH 3/4] loongarch: entry: Migrate ret_from_fork() to C Charlie Jenkins
@ 2025-01-22 22:56 ` Charlie Jenkins
2025-01-23 17:12 ` kernel test robot
2025-01-23 17:34 ` kernel test robot
3 siblings, 2 replies; 14+ messages in thread
From: Charlie Jenkins @ 2025-01-22 22:56 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Huacai Chen, WANG Xuerui,
Thomas Gleixner, Peter Zijlstra, Andy Lutomirski, Alexandre Ghiti
Cc: linux-riscv, linux-kernel, loongarch, Charlie Jenkins
Architectures using the generic entry code can be optimized by having
syscall_exit_to_user_mode inlined.
Signed-off-by: Charlie Jenkins <charlie@rivosinc.com>
---
include/linux/entry-common.h | 43 ++++++++++++++++++++++++++++++++++++--
kernel/entry/common.c | 49 +-------------------------------------------
2 files changed, 42 insertions(+), 50 deletions(-)
diff --git a/include/linux/entry-common.h b/include/linux/entry-common.h
index fc61d0205c97084acc89c8e45e088946f5e6d9b2..a46861ffd6858fadf4014c387e8f2f216a879c25 100644
--- a/include/linux/entry-common.h
+++ b/include/linux/entry-common.h
@@ -14,6 +14,7 @@
#include <linux/kmsan.h>
#include <asm/entry-common.h>
+#include <asm/syscall.h>
/*
* Define dummy _TIF work flags if not defined by the architecture or for
@@ -366,6 +367,15 @@ static __always_inline void exit_to_user_mode(void)
lockdep_hardirqs_on(CALLER_ADDR0);
}
+/**
+ * syscall_exit_work - Handle work before returning to user mode
+ * @regs: Pointer to current pt_regs
+ * @work: Current thread syscall work
+ *
+ * Do one-time syscall specific work.
+ */
+void syscall_exit_work(struct pt_regs *regs, unsigned long work);
+
/**
* syscall_exit_to_user_mode_work - Handle work before returning to user mode
* @regs: Pointer to currents pt_regs
@@ -379,7 +389,30 @@ static __always_inline void exit_to_user_mode(void)
* make the final state transitions. Interrupts must stay disabled between
* return from this function and the invocation of exit_to_user_mode().
*/
-void syscall_exit_to_user_mode_work(struct pt_regs *regs);
+static __always_inline void syscall_exit_to_user_mode_work(struct pt_regs *regs)
+{
+ unsigned long work = READ_ONCE(current_thread_info()->syscall_work);
+ unsigned long nr = syscall_get_nr(current, regs);
+
+ CT_WARN_ON(ct_state() != CONTEXT_KERNEL);
+
+ if (IS_ENABLED(CONFIG_PROVE_LOCKING)) {
+ if (WARN(irqs_disabled(), "syscall %lu left IRQs disabled", nr))
+ local_irq_enable();
+ }
+
+ rseq_syscall(regs);
+
+ /*
+ * Do one-time syscall specific work. If these work items are
+ * enabled, we want to run them exactly once per syscall exit with
+ * interrupts enabled.
+ */
+ if (unlikely(work & SYSCALL_WORK_EXIT))
+ syscall_exit_work(regs, work);
+ local_irq_disable_exit_to_user();
+ exit_to_user_mode_prepare(regs);
+}
/**
* syscall_exit_to_user_mode - Handle work before returning to user mode
@@ -410,7 +443,13 @@ void syscall_exit_to_user_mode_work(struct pt_regs *regs);
* exit_to_user_mode(). This function is preferred unless there is a
* compelling architectural reason to use the separate functions.
*/
-void syscall_exit_to_user_mode(struct pt_regs *regs);
+static __always_inline void syscall_exit_to_user_mode(struct pt_regs *regs)
+{
+ instrumentation_begin();
+ syscall_exit_to_user_mode_work(regs);
+ instrumentation_end();
+ exit_to_user_mode();
+}
/**
* irqentry_enter_from_user_mode - Establish state before invoking the irq handler
diff --git a/kernel/entry/common.c b/kernel/entry/common.c
index e33691d5adf7aab4af54cf2bf8e5ef5bd6ad1424..f55e421fb196dd5f9d4e34dd85ae096c774cf879 100644
--- a/kernel/entry/common.c
+++ b/kernel/entry/common.c
@@ -146,7 +146,7 @@ static inline bool report_single_step(unsigned long work)
return work & SYSCALL_WORK_SYSCALL_EXIT_TRAP;
}
-static void syscall_exit_work(struct pt_regs *regs, unsigned long work)
+void syscall_exit_work(struct pt_regs *regs, unsigned long work)
{
bool step;
@@ -173,53 +173,6 @@ static void syscall_exit_work(struct pt_regs *regs, unsigned long work)
ptrace_report_syscall_exit(regs, step);
}
-/*
- * Syscall specific exit to user mode preparation. Runs with interrupts
- * enabled.
- */
-static void syscall_exit_to_user_mode_prepare(struct pt_regs *regs)
-{
- unsigned long work = READ_ONCE(current_thread_info()->syscall_work);
- unsigned long nr = syscall_get_nr(current, regs);
-
- CT_WARN_ON(ct_state() != CT_STATE_KERNEL);
-
- if (IS_ENABLED(CONFIG_PROVE_LOCKING)) {
- if (WARN(irqs_disabled(), "syscall %lu left IRQs disabled", nr))
- local_irq_enable();
- }
-
- rseq_syscall(regs);
-
- /*
- * Do one-time syscall specific work. If these work items are
- * enabled, we want to run them exactly once per syscall exit with
- * interrupts enabled.
- */
- if (unlikely(work & SYSCALL_WORK_EXIT))
- syscall_exit_work(regs, work);
-}
-
-static __always_inline void __syscall_exit_to_user_mode_work(struct pt_regs *regs)
-{
- syscall_exit_to_user_mode_prepare(regs);
- local_irq_disable_exit_to_user();
- exit_to_user_mode_prepare(regs);
-}
-
-void syscall_exit_to_user_mode_work(struct pt_regs *regs)
-{
- __syscall_exit_to_user_mode_work(regs);
-}
-
-__visible noinstr void syscall_exit_to_user_mode(struct pt_regs *regs)
-{
- instrumentation_begin();
- __syscall_exit_to_user_mode_work(regs);
- instrumentation_end();
- exit_to_user_mode();
-}
-
noinstr void irqentry_enter_from_user_mode(struct pt_regs *regs)
{
enter_from_user_mode(regs);
--
2.43.0
^ permalink raw reply related [flat|nested] 14+ messages in thread
* Re: [PATCH 4/4] entry: Inline syscall_exit_to_user_mode()
2025-01-22 22:56 ` [PATCH 4/4] entry: Inline syscall_exit_to_user_mode() Charlie Jenkins
@ 2025-01-23 17:12 ` kernel test robot
2025-01-23 17:34 ` kernel test robot
1 sibling, 0 replies; 14+ messages in thread
From: kernel test robot @ 2025-01-23 17:12 UTC (permalink / raw)
To: Charlie Jenkins, Paul Walmsley, Palmer Dabbelt, Huacai Chen,
WANG Xuerui, Thomas Gleixner, Peter Zijlstra, Andy Lutomirski,
Alexandre Ghiti
Cc: llvm, oe-kbuild-all, linux-riscv, linux-kernel, loongarch,
Charlie Jenkins
Hi Charlie,
kernel test robot noticed the following build errors:
[auto build test ERROR on ffd294d346d185b70e28b1a28abe367bbfe53c04]
url: https://github.com/intel-lab-lkp/linux/commits/Charlie-Jenkins/riscv-entry-Convert-ret_from_fork-to-C/20250123-065957
base: ffd294d346d185b70e28b1a28abe367bbfe53c04
patch link: https://lore.kernel.org/r/20250122-riscv_optimize_entry-v1-4-4ee95559cfd0%40rivosinc.com
patch subject: [PATCH 4/4] entry: Inline syscall_exit_to_user_mode()
config: x86_64-randconfig-071-20250123 (https://download.01.org/0day-ci/archive/20250124/202501240036.hxoscXJM-lkp@intel.com/config)
compiler: clang version 19.1.3 (https://github.com/llvm/llvm-project ab51eccf88f5321e7c60591c5546b254b6afab99)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250124/202501240036.hxoscXJM-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202501240036.hxoscXJM-lkp@intel.com/
All errors (new ones prefixed by >>):
In file included from arch/x86/kernel/ptrace.c:40:
In file included from arch/x86/include/asm/traps.h:9:
In file included from arch/x86/include/asm/idtentry.h:11:
>> include/linux/entry-common.h:397:27: error: use of undeclared identifier 'CONTEXT_KERNEL'
397 | CT_WARN_ON(ct_state() != CONTEXT_KERNEL);
| ^
1 error generated.
vim +/CONTEXT_KERNEL +397 include/linux/entry-common.h
378
379 /**
380 * syscall_exit_to_user_mode_work - Handle work before returning to user mode
381 * @regs: Pointer to currents pt_regs
382 *
383 * Same as step 1 and 2 of syscall_exit_to_user_mode() but without calling
384 * exit_to_user_mode() to perform the final transition to user mode.
385 *
386 * Calling convention is the same as for syscall_exit_to_user_mode() and it
387 * returns with all work handled and interrupts disabled. The caller must
388 * invoke exit_to_user_mode() before actually switching to user mode to
389 * make the final state transitions. Interrupts must stay disabled between
390 * return from this function and the invocation of exit_to_user_mode().
391 */
392 static __always_inline void syscall_exit_to_user_mode_work(struct pt_regs *regs)
393 {
394 unsigned long work = READ_ONCE(current_thread_info()->syscall_work);
395 unsigned long nr = syscall_get_nr(current, regs);
396
> 397 CT_WARN_ON(ct_state() != CONTEXT_KERNEL);
398
399 if (IS_ENABLED(CONFIG_PROVE_LOCKING)) {
400 if (WARN(irqs_disabled(), "syscall %lu left IRQs disabled", nr))
401 local_irq_enable();
402 }
403
404 rseq_syscall(regs);
405
406 /*
407 * Do one-time syscall specific work. If these work items are
408 * enabled, we want to run them exactly once per syscall exit with
409 * interrupts enabled.
410 */
411 if (unlikely(work & SYSCALL_WORK_EXIT))
412 syscall_exit_work(regs, work);
413 local_irq_disable_exit_to_user();
414 exit_to_user_mode_prepare(regs);
415 }
416
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 3/4] loongarch: entry: Migrate ret_from_fork() to C
2025-01-22 22:56 ` [PATCH 3/4] loongarch: entry: Migrate ret_from_fork() to C Charlie Jenkins
@ 2025-01-23 17:12 ` kernel test robot
0 siblings, 0 replies; 14+ messages in thread
From: kernel test robot @ 2025-01-23 17:12 UTC (permalink / raw)
To: Charlie Jenkins, Paul Walmsley, Palmer Dabbelt, Huacai Chen,
WANG Xuerui, Thomas Gleixner, Peter Zijlstra, Andy Lutomirski,
Alexandre Ghiti
Cc: oe-kbuild-all, linux-riscv, linux-kernel, loongarch,
Charlie Jenkins
Hi Charlie,
kernel test robot noticed the following build errors:
[auto build test ERROR on ffd294d346d185b70e28b1a28abe367bbfe53c04]
url: https://github.com/intel-lab-lkp/linux/commits/Charlie-Jenkins/riscv-entry-Convert-ret_from_fork-to-C/20250123-065957
base: ffd294d346d185b70e28b1a28abe367bbfe53c04
patch link: https://lore.kernel.org/r/20250122-riscv_optimize_entry-v1-3-4ee95559cfd0%40rivosinc.com
patch subject: [PATCH 3/4] loongarch: entry: Migrate ret_from_fork() to C
config: loongarch-allnoconfig (https://download.01.org/0day-ci/archive/20250124/202501240042.hEXt9u9r-lkp@intel.com/config)
compiler: loongarch64-linux-gcc (GCC) 14.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250124/202501240042.hEXt9u9r-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202501240042.hEXt9u9r-lkp@intel.com/
All errors (new ones prefixed by >>):
arch/loongarch/kernel/process.c: In function 'ret_from_kernel_thread':
>> arch/loongarch/kernel/process.c:76:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
76 | {
| ^
arch/loongarch/kernel/process.c:105:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
105 | {
| ^
arch/loongarch/kernel/process.c:110:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
110 | {
| ^
arch/loongarch/kernel/process.c:114:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
114 | {
| ^
arch/loongarch/kernel/process.c:151:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
151 | {
| ^
arch/loongarch/kernel/process.c:160:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
160 | {
| ^
arch/loongarch/kernel/process.c:169:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
169 | {
| ^
arch/loongarch/kernel/process.c:234:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
234 | {
| ^
arch/loongarch/kernel/process.c:257:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
257 | {
| ^
arch/loongarch/kernel/process.c:279:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
279 | {
| ^
arch/loongarch/kernel/process.c:296:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
296 | {
| ^
arch/loongarch/kernel/process.c:317:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
317 | {
| ^
arch/loongarch/kernel/process.c:338:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
338 | {
| ^
In file included from include/asm-generic/percpu.h:7,
from arch/loongarch/include/asm/percpu.h:184,
from include/linux/irqflags.h:19,
from include/linux/spinlock.h:59,
from include/linux/sched.h:2171,
from include/linux/ratelimit.h:6,
from include/linux/dev_printk.h:16,
from include/linux/device.h:15,
from include/linux/node.h:18,
from include/linux/cpu.h:17,
from arch/loongarch/kernel/process.c:13:
>> arch/loongarch/kernel/process.c:345:43: error: storage class specified for parameter 'backtrace_csd'
345 | static DEFINE_PER_CPU(call_single_data_t, backtrace_csd);
| ^~~~~~~~~~~~~
include/linux/percpu-defs.h:104:44: note: in definition of macro 'DEFINE_PER_CPU_SECTION'
104 | __PCPU_ATTRS(sec) __typeof__(type) name
| ^~~~
arch/loongarch/kernel/process.c:345:8: note: in expansion of macro 'DEFINE_PER_CPU'
345 | static DEFINE_PER_CPU(call_single_data_t, backtrace_csd);
| ^~~~~~~~~~~~~~
>> arch/loongarch/kernel/process.c:345:43: error: section attribute not allowed for 'backtrace_csd'
345 | static DEFINE_PER_CPU(call_single_data_t, backtrace_csd);
| ^~~~~~~~~~~~~
include/linux/percpu-defs.h:104:44: note: in definition of macro 'DEFINE_PER_CPU_SECTION'
104 | __PCPU_ATTRS(sec) __typeof__(type) name
| ^~~~
arch/loongarch/kernel/process.c:345:8: note: in expansion of macro 'DEFINE_PER_CPU'
345 | static DEFINE_PER_CPU(call_single_data_t, backtrace_csd);
| ^~~~~~~~~~~~~~
>> arch/loongarch/kernel/process.c:346:23: error: storage class specified for parameter 'backtrace_csd_busy'
346 | static struct cpumask backtrace_csd_busy;
| ^~~~~~~~~~~~~~~~~~
arch/loongarch/kernel/process.c:349:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
349 | {
| ^
arch/loongarch/kernel/process.c:355:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
355 | {
| ^
arch/loongarch/kernel/process.c:379:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
379 | {
| ^
arch/loongarch/kernel/process.c:385:1: error: expected '=', ',', ';', 'asm' or '__attribute__' before '{' token
385 | {
| ^
>> arch/loongarch/kernel/process.c:70:35: error: old-style parameter declarations in prototyped function definition
70 | void noinstr __no_stack_protector ret_from_kernel_thread(struct task_struct *prev,
| ^~~~~~~~~~~~~~~~~~~~~~
>> arch/loongarch/kernel/process.c:402: error: expected '{' at end of input
vim +76 arch/loongarch/kernel/process.c
803b0fc5c3f2baa Huacai Chen 2022-05-31 @13 #include <linux/cpu.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 14 #include <linux/init.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 15 #include <linux/kernel.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 16 #include <linux/errno.h>
5c881018646d666 Charlie Jenkins 2025-01-22 17 #include <linux/entry-common.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 18 #include <linux/sched.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 19 #include <linux/sched/debug.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 20 #include <linux/sched/task.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 21 #include <linux/sched/task_stack.h>
edffa33c7bb5a73 Qing Zhang 2023-02-25 22 #include <linux/hw_breakpoint.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 23 #include <linux/mm.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 24 #include <linux/stddef.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 25 #include <linux/unistd.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 26 #include <linux/export.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 27 #include <linux/ptrace.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 28 #include <linux/mman.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 29 #include <linux/personality.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 30 #include <linux/sys.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 31 #include <linux/completion.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 32 #include <linux/kallsyms.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 33 #include <linux/random.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 34 #include <linux/prctl.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 35 #include <linux/nmi.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 36
803b0fc5c3f2baa Huacai Chen 2022-05-31 37 #include <asm/asm.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 38 #include <asm/bootinfo.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 39 #include <asm/cpu.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 40 #include <asm/elf.h>
c718a0bad75ccef Bibo Mao 2023-09-20 41 #include <asm/exec.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 42 #include <asm/fpu.h>
bd3c5798484aa9a Qi Hu 2023-09-06 43 #include <asm/lbt.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 44 #include <asm/io.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 45 #include <asm/irq.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 46 #include <asm/irq_regs.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 47 #include <asm/loongarch.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 48 #include <asm/pgtable.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 49 #include <asm/processor.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 50 #include <asm/reg.h>
5c881018646d666 Charlie Jenkins 2025-01-22 51 #include <asm/switch_to.h>
49232773d8233ed Qing Zhang 2022-08-06 52 #include <asm/unwind.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 53 #include <asm/vdso.h>
803b0fc5c3f2baa Huacai Chen 2022-05-31 54
09f33601bf940f9 Huacai Chen 2022-12-10 55 #ifdef CONFIG_STACKPROTECTOR
09f33601bf940f9 Huacai Chen 2022-12-10 56 #include <linux/stackprotector.h>
09f33601bf940f9 Huacai Chen 2022-12-10 57 unsigned long __stack_chk_guard __read_mostly;
09f33601bf940f9 Huacai Chen 2022-12-10 58 EXPORT_SYMBOL(__stack_chk_guard);
09f33601bf940f9 Huacai Chen 2022-12-10 59 #endif
09f33601bf940f9 Huacai Chen 2022-12-10 60
803b0fc5c3f2baa Huacai Chen 2022-05-31 61 /*
803b0fc5c3f2baa Huacai Chen 2022-05-31 62 * Idle related variables and functions
803b0fc5c3f2baa Huacai Chen 2022-05-31 63 */
803b0fc5c3f2baa Huacai Chen 2022-05-31 64
803b0fc5c3f2baa Huacai Chen 2022-05-31 65 unsigned long boot_option_idle_override = IDLE_NO_OVERRIDE;
803b0fc5c3f2baa Huacai Chen 2022-05-31 66 EXPORT_SYMBOL(boot_option_idle_override);
803b0fc5c3f2baa Huacai Chen 2022-05-31 67
5c881018646d666 Charlie Jenkins 2025-01-22 68 asmlinkage void restore_and_ret(void);
5c881018646d666 Charlie Jenkins 2025-01-22 69 asmlinkage void ret_from_fork_asm(void);
5c881018646d666 Charlie Jenkins 2025-01-22 @70 void noinstr __no_stack_protector ret_from_kernel_thread(struct task_struct *prev,
5c881018646d666 Charlie Jenkins 2025-01-22 71 struct pt_regs *regs,
5c881018646d666 Charlie Jenkins 2025-01-22 72 int (*fn)(void *),
5c881018646d666 Charlie Jenkins 2025-01-22 73 void *fn_arg)
803b0fc5c3f2baa Huacai Chen 2022-05-31 74
803b0fc5c3f2baa Huacai Chen 2022-05-31 75 void start_thread(struct pt_regs *regs, unsigned long pc, unsigned long sp)
803b0fc5c3f2baa Huacai Chen 2022-05-31 @76 {
803b0fc5c3f2baa Huacai Chen 2022-05-31 77 unsigned long crmd;
803b0fc5c3f2baa Huacai Chen 2022-05-31 78 unsigned long prmd;
803b0fc5c3f2baa Huacai Chen 2022-05-31 79 unsigned long euen;
803b0fc5c3f2baa Huacai Chen 2022-05-31 80
803b0fc5c3f2baa Huacai Chen 2022-05-31 81 /* New thread loses kernel privileges. */
803b0fc5c3f2baa Huacai Chen 2022-05-31 82 crmd = regs->csr_crmd & ~(PLV_MASK);
803b0fc5c3f2baa Huacai Chen 2022-05-31 83 crmd |= PLV_USER;
803b0fc5c3f2baa Huacai Chen 2022-05-31 84 regs->csr_crmd = crmd;
803b0fc5c3f2baa Huacai Chen 2022-05-31 85
803b0fc5c3f2baa Huacai Chen 2022-05-31 86 prmd = regs->csr_prmd & ~(PLV_MASK);
803b0fc5c3f2baa Huacai Chen 2022-05-31 87 prmd |= PLV_USER;
803b0fc5c3f2baa Huacai Chen 2022-05-31 88 regs->csr_prmd = prmd;
803b0fc5c3f2baa Huacai Chen 2022-05-31 89
803b0fc5c3f2baa Huacai Chen 2022-05-31 90 euen = regs->csr_euen & ~(CSR_EUEN_FPEN);
803b0fc5c3f2baa Huacai Chen 2022-05-31 91 regs->csr_euen = euen;
803b0fc5c3f2baa Huacai Chen 2022-05-31 92 lose_fpu(0);
bd3c5798484aa9a Qi Hu 2023-09-06 93 lose_lbt(0);
c2396651309eba2 Xi Ruoyao 2024-01-17 94 current->thread.fpu.fcsr = boot_cpu_data.fpu_csr0;
803b0fc5c3f2baa Huacai Chen 2022-05-31 95
803b0fc5c3f2baa Huacai Chen 2022-05-31 96 clear_thread_flag(TIF_LSX_CTX_LIVE);
803b0fc5c3f2baa Huacai Chen 2022-05-31 97 clear_thread_flag(TIF_LASX_CTX_LIVE);
bd3c5798484aa9a Qi Hu 2023-09-06 98 clear_thread_flag(TIF_LBT_CTX_LIVE);
803b0fc5c3f2baa Huacai Chen 2022-05-31 99 clear_used_math();
803b0fc5c3f2baa Huacai Chen 2022-05-31 100 regs->csr_era = pc;
803b0fc5c3f2baa Huacai Chen 2022-05-31 101 regs->regs[3] = sp;
803b0fc5c3f2baa Huacai Chen 2022-05-31 102 }
803b0fc5c3f2baa Huacai Chen 2022-05-31 103
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 4/4] entry: Inline syscall_exit_to_user_mode()
2025-01-22 22:56 ` [PATCH 4/4] entry: Inline syscall_exit_to_user_mode() Charlie Jenkins
2025-01-23 17:12 ` kernel test robot
@ 2025-01-23 17:34 ` kernel test robot
1 sibling, 0 replies; 14+ messages in thread
From: kernel test robot @ 2025-01-23 17:34 UTC (permalink / raw)
To: Charlie Jenkins, Paul Walmsley, Palmer Dabbelt, Huacai Chen,
WANG Xuerui, Thomas Gleixner, Peter Zijlstra, Andy Lutomirski,
Alexandre Ghiti
Cc: oe-kbuild-all, linux-riscv, linux-kernel, loongarch,
Charlie Jenkins
Hi Charlie,
kernel test robot noticed the following build errors:
[auto build test ERROR on ffd294d346d185b70e28b1a28abe367bbfe53c04]
url: https://github.com/intel-lab-lkp/linux/commits/Charlie-Jenkins/riscv-entry-Convert-ret_from_fork-to-C/20250123-065957
base: ffd294d346d185b70e28b1a28abe367bbfe53c04
patch link: https://lore.kernel.org/r/20250122-riscv_optimize_entry-v1-4-4ee95559cfd0%40rivosinc.com
patch subject: [PATCH 4/4] entry: Inline syscall_exit_to_user_mode()
config: x86_64-randconfig-072-20250123 (https://download.01.org/0day-ci/archive/20250124/202501240101.mSdGpEoH-lkp@intel.com/config)
compiler: gcc-12 (Debian 12.2.0-14) 12.2.0
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20250124/202501240101.mSdGpEoH-lkp@intel.com/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of
the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <lkp@intel.com>
| Closes: https://lore.kernel.org/oe-kbuild-all/202501240101.mSdGpEoH-lkp@intel.com/
All errors (new ones prefixed by >>):
In file included from arch/x86/include/asm/bug.h:99,
from include/linux/bug.h:5,
from include/linux/thread_info.h:13,
from include/linux/sched.h:14,
from arch/x86/mm/fault.c:7:
include/linux/entry-common.h: In function 'syscall_exit_to_user_mode_work':
>> include/linux/entry-common.h:397:34: error: 'CONTEXT_KERNEL' undeclared (first use in this function); did you mean 'PERF_CONTEXT_KERNEL'?
397 | CT_WARN_ON(ct_state() != CONTEXT_KERNEL);
| ^~~~~~~~~~~~~~
include/asm-generic/bug.h:123:32: note: in definition of macro 'WARN_ON'
123 | int __ret_warn_on = !!(condition); \
| ^~~~~~~~~
include/linux/entry-common.h:397:9: note: in expansion of macro 'CT_WARN_ON'
397 | CT_WARN_ON(ct_state() != CONTEXT_KERNEL);
| ^~~~~~~~~~
include/linux/entry-common.h:397:34: note: each undeclared identifier is reported only once for each function it appears in
397 | CT_WARN_ON(ct_state() != CONTEXT_KERNEL);
| ^~~~~~~~~~~~~~
include/asm-generic/bug.h:123:32: note: in definition of macro 'WARN_ON'
123 | int __ret_warn_on = !!(condition); \
| ^~~~~~~~~
include/linux/entry-common.h:397:9: note: in expansion of macro 'CT_WARN_ON'
397 | CT_WARN_ON(ct_state() != CONTEXT_KERNEL);
| ^~~~~~~~~~
vim +397 include/linux/entry-common.h
378
379 /**
380 * syscall_exit_to_user_mode_work - Handle work before returning to user mode
381 * @regs: Pointer to currents pt_regs
382 *
383 * Same as step 1 and 2 of syscall_exit_to_user_mode() but without calling
384 * exit_to_user_mode() to perform the final transition to user mode.
385 *
386 * Calling convention is the same as for syscall_exit_to_user_mode() and it
387 * returns with all work handled and interrupts disabled. The caller must
388 * invoke exit_to_user_mode() before actually switching to user mode to
389 * make the final state transitions. Interrupts must stay disabled between
390 * return from this function and the invocation of exit_to_user_mode().
391 */
392 static __always_inline void syscall_exit_to_user_mode_work(struct pt_regs *regs)
393 {
394 unsigned long work = READ_ONCE(current_thread_info()->syscall_work);
395 unsigned long nr = syscall_get_nr(current, regs);
396
> 397 CT_WARN_ON(ct_state() != CONTEXT_KERNEL);
398
399 if (IS_ENABLED(CONFIG_PROVE_LOCKING)) {
400 if (WARN(irqs_disabled(), "syscall %lu left IRQs disabled", nr))
401 local_irq_enable();
402 }
403
404 rseq_syscall(regs);
405
406 /*
407 * Do one-time syscall specific work. If these work items are
408 * enabled, we want to run them exactly once per syscall exit with
409 * interrupts enabled.
410 */
411 if (unlikely(work & SYSCALL_WORK_EXIT))
412 syscall_exit_work(regs, work);
413 local_irq_disable_exit_to_user();
414 exit_to_user_mode_prepare(regs);
415 }
416
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 1/4] riscv: entry: Convert ret_from_fork() to C
2025-01-22 22:56 ` [PATCH 1/4] riscv: entry: Convert ret_from_fork() to C Charlie Jenkins
@ 2025-01-24 7:09 ` Alexandre Ghiti
2025-01-24 7:56 ` Charlie Jenkins
2025-01-25 23:30 ` Maciej W. Rozycki
1 sibling, 1 reply; 14+ messages in thread
From: Alexandre Ghiti @ 2025-01-24 7:09 UTC (permalink / raw)
To: Charlie Jenkins, Paul Walmsley, Palmer Dabbelt, Huacai Chen,
WANG Xuerui, Thomas Gleixner, Peter Zijlstra, Andy Lutomirski,
Alexandre Ghiti
Cc: linux-riscv, linux-kernel, loongarch
Hi Charlie,
On 22/01/2025 23:56, Charlie Jenkins wrote:
> Move the main section of ret_from_fork() to C to allow inlining of
> syscall_exit_to_user_mode().
>
> Signed-off-by: Charlie Jenkins <charlie@rivosinc.com>
> ---
> arch/riscv/include/asm/asm-prototypes.h | 1 +
> arch/riscv/kernel/entry.S | 15 ++++++---------
> arch/riscv/kernel/process.c | 14 ++++++++++++--
> 3 files changed, 19 insertions(+), 11 deletions(-)
>
> diff --git a/arch/riscv/include/asm/asm-prototypes.h b/arch/riscv/include/asm/asm-prototypes.h
> index cd627ec289f163a630b73dd03dd52a6b28692997..733ff609778797001006c33bba9e3cc5b1f15387 100644
> --- a/arch/riscv/include/asm/asm-prototypes.h
> +++ b/arch/riscv/include/asm/asm-prototypes.h
> @@ -52,6 +52,7 @@ DECLARE_DO_ERROR_INFO(do_trap_ecall_s);
> DECLARE_DO_ERROR_INFO(do_trap_ecall_m);
> DECLARE_DO_ERROR_INFO(do_trap_break);
>
> +asmlinkage void ret_from_fork(void *fn_arg, int (*fn)(void *), struct pt_regs *regs);
> asmlinkage void handle_bad_stack(struct pt_regs *regs);
> asmlinkage void do_page_fault(struct pt_regs *regs);
> asmlinkage void do_irq(struct pt_regs *regs);
> diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
> index 33a5a9f2a0d4e1eeccfb3621b9e518b88e1b0704..9225c322279aa90e737b1d7144db084319cf8103 100644
> --- a/arch/riscv/kernel/entry.S
> +++ b/arch/riscv/kernel/entry.S
> @@ -319,17 +319,14 @@ SYM_CODE_END(handle_kernel_stack_overflow)
> ASM_NOKPROBE(handle_kernel_stack_overflow)
> #endif
>
> -SYM_CODE_START(ret_from_fork)
> +SYM_CODE_START(ret_from_fork_asm)
> call schedule_tail
> - beqz s0, 1f /* not from kernel thread */
> - /* Call fn(arg) */
> - move a0, s1
> - jalr s0
> -1:
> - move a0, sp /* pt_regs */
> - call syscall_exit_to_user_mode
> + move a0, s1 /* fn */
> + move a1, s0 /* fn_arg */
You pass the arguments in the right order but the comments are wrong (s1
contains fn_arg and s0 fn).
> + move a2, sp /* pt_regs */
> + call ret_from_fork
> j ret_from_exception
> -SYM_CODE_END(ret_from_fork)
> +SYM_CODE_END(ret_from_fork_asm)
>
> #ifdef CONFIG_IRQ_STACKS
> /*
> diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c
> index 58b6482c2bf662bf5224ca50c8e21a68760a6b41..0d07e6d8f6b57beba438dbba5e8c74a014582bee 100644
> --- a/arch/riscv/kernel/process.c
> +++ b/arch/riscv/kernel/process.c
> @@ -17,7 +17,9 @@
> #include <linux/ptrace.h>
> #include <linux/uaccess.h>
> #include <linux/personality.h>
> +#include <linux/entry-common.h>
>
> +#include <asm/asm-prototypes.h>
> #include <asm/unistd.h>
> #include <asm/processor.h>
> #include <asm/csr.h>
> @@ -36,7 +38,7 @@ unsigned long __stack_chk_guard __read_mostly;
> EXPORT_SYMBOL(__stack_chk_guard);
> #endif
>
> -extern asmlinkage void ret_from_fork(void);
> +extern asmlinkage void ret_from_fork_asm(void);
>
> void noinstr arch_cpu_idle(void)
> {
> @@ -206,6 +208,14 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
> return 0;
> }
>
> +asmlinkage void ret_from_fork(void *fn_arg, int (*fn)(void *), struct pt_regs *regs)
> +{
> + if (unlikely(fn))
> + fn(fn_arg);
> +
> + syscall_exit_to_user_mode(regs);
> +}
> +
> int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
> {
> unsigned long clone_flags = args->flags;
> @@ -242,7 +252,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
> p->thread.riscv_v_flags = 0;
> if (has_vector())
> riscv_v_thread_alloc(p);
> - p->thread.ra = (unsigned long)ret_from_fork;
> + p->thread.ra = (unsigned long)ret_from_fork_asm;
> p->thread.sp = (unsigned long)childregs; /* kernel sp */
> return 0;
> }
>
With the comment fix, you can add:
Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com>
Thanks,
Alex
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 1/4] riscv: entry: Convert ret_from_fork() to C
2025-01-24 7:09 ` Alexandre Ghiti
@ 2025-01-24 7:56 ` Charlie Jenkins
0 siblings, 0 replies; 14+ messages in thread
From: Charlie Jenkins @ 2025-01-24 7:56 UTC (permalink / raw)
To: Alexandre Ghiti
Cc: Paul Walmsley, Palmer Dabbelt, Huacai Chen, WANG Xuerui,
Thomas Gleixner, Peter Zijlstra, Andy Lutomirski, Alexandre Ghiti,
linux-riscv, linux-kernel, loongarch
On Fri, Jan 24, 2025 at 08:09:16AM +0100, Alexandre Ghiti wrote:
> Hi Charlie,
>
> On 22/01/2025 23:56, Charlie Jenkins wrote:
> > Move the main section of ret_from_fork() to C to allow inlining of
> > syscall_exit_to_user_mode().
> >
> > Signed-off-by: Charlie Jenkins <charlie@rivosinc.com>
> > ---
> > arch/riscv/include/asm/asm-prototypes.h | 1 +
> > arch/riscv/kernel/entry.S | 15 ++++++---------
> > arch/riscv/kernel/process.c | 14 ++++++++++++--
> > 3 files changed, 19 insertions(+), 11 deletions(-)
> >
> > diff --git a/arch/riscv/include/asm/asm-prototypes.h b/arch/riscv/include/asm/asm-prototypes.h
> > index cd627ec289f163a630b73dd03dd52a6b28692997..733ff609778797001006c33bba9e3cc5b1f15387 100644
> > --- a/arch/riscv/include/asm/asm-prototypes.h
> > +++ b/arch/riscv/include/asm/asm-prototypes.h
> > @@ -52,6 +52,7 @@ DECLARE_DO_ERROR_INFO(do_trap_ecall_s);
> > DECLARE_DO_ERROR_INFO(do_trap_ecall_m);
> > DECLARE_DO_ERROR_INFO(do_trap_break);
> > +asmlinkage void ret_from_fork(void *fn_arg, int (*fn)(void *), struct pt_regs *regs);
> > asmlinkage void handle_bad_stack(struct pt_regs *regs);
> > asmlinkage void do_page_fault(struct pt_regs *regs);
> > asmlinkage void do_irq(struct pt_regs *regs);
> > diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
> > index 33a5a9f2a0d4e1eeccfb3621b9e518b88e1b0704..9225c322279aa90e737b1d7144db084319cf8103 100644
> > --- a/arch/riscv/kernel/entry.S
> > +++ b/arch/riscv/kernel/entry.S
> > @@ -319,17 +319,14 @@ SYM_CODE_END(handle_kernel_stack_overflow)
> > ASM_NOKPROBE(handle_kernel_stack_overflow)
> > #endif
> > -SYM_CODE_START(ret_from_fork)
> > +SYM_CODE_START(ret_from_fork_asm)
> > call schedule_tail
> > - beqz s0, 1f /* not from kernel thread */
> > - /* Call fn(arg) */
> > - move a0, s1
> > - jalr s0
> > -1:
> > - move a0, sp /* pt_regs */
> > - call syscall_exit_to_user_mode
> > + move a0, s1 /* fn */
> > + move a1, s0 /* fn_arg */
>
>
> You pass the arguments in the right order but the comments are wrong (s1
> contains fn_arg and s0 fn).
Oh thank you! I originally had it in this order but then moved it around
to reduce the amount of register shuffling that is needed on the C side,
I will fix in the next version.
- Charlie
>
>
> > + move a2, sp /* pt_regs */
> > + call ret_from_fork
> > j ret_from_exception
> > -SYM_CODE_END(ret_from_fork)
> > +SYM_CODE_END(ret_from_fork_asm)
> > #ifdef CONFIG_IRQ_STACKS
> > /*
> > diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c
> > index 58b6482c2bf662bf5224ca50c8e21a68760a6b41..0d07e6d8f6b57beba438dbba5e8c74a014582bee 100644
> > --- a/arch/riscv/kernel/process.c
> > +++ b/arch/riscv/kernel/process.c
> > @@ -17,7 +17,9 @@
> > #include <linux/ptrace.h>
> > #include <linux/uaccess.h>
> > #include <linux/personality.h>
> > +#include <linux/entry-common.h>
> > +#include <asm/asm-prototypes.h>
> > #include <asm/unistd.h>
> > #include <asm/processor.h>
> > #include <asm/csr.h>
> > @@ -36,7 +38,7 @@ unsigned long __stack_chk_guard __read_mostly;
> > EXPORT_SYMBOL(__stack_chk_guard);
> > #endif
> > -extern asmlinkage void ret_from_fork(void);
> > +extern asmlinkage void ret_from_fork_asm(void);
> > void noinstr arch_cpu_idle(void)
> > {
> > @@ -206,6 +208,14 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
> > return 0;
> > }
> > +asmlinkage void ret_from_fork(void *fn_arg, int (*fn)(void *), struct pt_regs *regs)
> > +{
> > + if (unlikely(fn))
> > + fn(fn_arg);
> > +
> > + syscall_exit_to_user_mode(regs);
> > +}
> > +
> > int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
> > {
> > unsigned long clone_flags = args->flags;
> > @@ -242,7 +252,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
> > p->thread.riscv_v_flags = 0;
> > if (has_vector())
> > riscv_v_thread_alloc(p);
> > - p->thread.ra = (unsigned long)ret_from_fork;
> > + p->thread.ra = (unsigned long)ret_from_fork_asm;
> > p->thread.sp = (unsigned long)childregs; /* kernel sp */
> > return 0;
> > }
> >
>
> With the comment fix, you can add:
>
> Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com>
>
> Thanks,
>
> Alex
>
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 1/4] riscv: entry: Convert ret_from_fork() to C
2025-01-22 22:56 ` [PATCH 1/4] riscv: entry: Convert ret_from_fork() to C Charlie Jenkins
2025-01-24 7:09 ` Alexandre Ghiti
@ 2025-01-25 23:30 ` Maciej W. Rozycki
2025-01-26 19:51 ` Charlie Jenkins
1 sibling, 1 reply; 14+ messages in thread
From: Maciej W. Rozycki @ 2025-01-25 23:30 UTC (permalink / raw)
To: Charlie Jenkins
Cc: Paul Walmsley, Palmer Dabbelt, Huacai Chen, WANG Xuerui,
Thomas Gleixner, Peter Zijlstra, Andy Lutomirski, Alexandre Ghiti,
linux-riscv, linux-kernel, loongarch
On Wed, 22 Jan 2025, Charlie Jenkins wrote:
> diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
> index 33a5a9f2a0d4e1eeccfb3621b9e518b88e1b0704..9225c322279aa90e737b1d7144db084319cf8103 100644
> --- a/arch/riscv/kernel/entry.S
> +++ b/arch/riscv/kernel/entry.S
> @@ -319,17 +319,14 @@ SYM_CODE_END(handle_kernel_stack_overflow)
> ASM_NOKPROBE(handle_kernel_stack_overflow)
> #endif
>
> -SYM_CODE_START(ret_from_fork)
> +SYM_CODE_START(ret_from_fork_asm)
> call schedule_tail
> - beqz s0, 1f /* not from kernel thread */
> - /* Call fn(arg) */
> - move a0, s1
> - jalr s0
> -1:
> - move a0, sp /* pt_regs */
> - call syscall_exit_to_user_mode
> + move a0, s1 /* fn */
> + move a1, s0 /* fn_arg */
> + move a2, sp /* pt_regs */
> + call ret_from_fork
> j ret_from_exception
Wouldn't it make sense to save a jump here and make it a tail call, i.e.:
la ra, ret_from_exception
tail ret_from_fork
?
Maciej
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 1/4] riscv: entry: Convert ret_from_fork() to C
2025-01-25 23:30 ` Maciej W. Rozycki
@ 2025-01-26 19:51 ` Charlie Jenkins
2025-01-27 20:20 ` Maciej W. Rozycki
0 siblings, 1 reply; 14+ messages in thread
From: Charlie Jenkins @ 2025-01-26 19:51 UTC (permalink / raw)
To: Maciej W. Rozycki
Cc: Paul Walmsley, Palmer Dabbelt, Huacai Chen, WANG Xuerui,
Thomas Gleixner, Peter Zijlstra, Andy Lutomirski, Alexandre Ghiti,
linux-riscv, linux-kernel, loongarch
On Sat, Jan 25, 2025 at 11:30:09PM +0000, Maciej W. Rozycki wrote:
> On Wed, 22 Jan 2025, Charlie Jenkins wrote:
>
> > diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
> > index 33a5a9f2a0d4e1eeccfb3621b9e518b88e1b0704..9225c322279aa90e737b1d7144db084319cf8103 100644
> > --- a/arch/riscv/kernel/entry.S
> > +++ b/arch/riscv/kernel/entry.S
> > @@ -319,17 +319,14 @@ SYM_CODE_END(handle_kernel_stack_overflow)
> > ASM_NOKPROBE(handle_kernel_stack_overflow)
> > #endif
> >
> > -SYM_CODE_START(ret_from_fork)
> > +SYM_CODE_START(ret_from_fork_asm)
> > call schedule_tail
> > - beqz s0, 1f /* not from kernel thread */
> > - /* Call fn(arg) */
> > - move a0, s1
> > - jalr s0
> > -1:
> > - move a0, sp /* pt_regs */
> > - call syscall_exit_to_user_mode
> > + move a0, s1 /* fn */
> > + move a1, s0 /* fn_arg */
> > + move a2, sp /* pt_regs */
> > + call ret_from_fork
> > j ret_from_exception
>
> Wouldn't it make sense to save a jump here and make it a tail call, i.e.:
>
> la ra, ret_from_exception
> tail ret_from_fork
>
I don't believe so due to the return address stack. It was shown in this
patch [1] that a 7% performance improvement can be seen on existing
riscv hardware by performing the extra jump. Doing tail calls should be
avoided on riscv since the hardware can be expected to predict the
return address incorrectly every time if the return address is manually
changed.
- Charlie
> ?
>
> Maciej
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 1/4] riscv: entry: Convert ret_from_fork() to C
2025-01-26 19:51 ` Charlie Jenkins
@ 2025-01-27 20:20 ` Maciej W. Rozycki
2025-01-27 20:55 ` Charlie Jenkins
0 siblings, 1 reply; 14+ messages in thread
From: Maciej W. Rozycki @ 2025-01-27 20:20 UTC (permalink / raw)
To: Charlie Jenkins
Cc: Paul Walmsley, Palmer Dabbelt, Huacai Chen, WANG Xuerui,
Thomas Gleixner, Peter Zijlstra, Andy Lutomirski, Alexandre Ghiti,
linux-riscv, linux-kernel, loongarch
On Sun, 26 Jan 2025, Charlie Jenkins wrote:
> > Wouldn't it make sense to save a jump here and make it a tail call, i.e.:
> >
> > la ra, ret_from_exception
> > tail ret_from_fork
> >
>
> I don't believe so due to the return address stack. It was shown in this
> patch [1] that a 7% performance improvement can be seen on existing
> riscv hardware by performing the extra jump. Doing tail calls should be
> avoided on riscv since the hardware can be expected to predict the
> return address incorrectly every time if the return address is manually
> changed.
Fair enough, thanks. Though no link to said patch given.
Maciej
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: [PATCH 1/4] riscv: entry: Convert ret_from_fork() to C
2025-01-27 20:20 ` Maciej W. Rozycki
@ 2025-01-27 20:55 ` Charlie Jenkins
0 siblings, 0 replies; 14+ messages in thread
From: Charlie Jenkins @ 2025-01-27 20:55 UTC (permalink / raw)
To: Maciej W. Rozycki
Cc: Paul Walmsley, Palmer Dabbelt, Huacai Chen, WANG Xuerui,
Thomas Gleixner, Peter Zijlstra, Andy Lutomirski, Alexandre Ghiti,
linux-riscv, linux-kernel, loongarch
On Mon, Jan 27, 2025 at 08:20:56PM +0000, Maciej W. Rozycki wrote:
> On Sun, 26 Jan 2025, Charlie Jenkins wrote:
>
> > > Wouldn't it make sense to save a jump here and make it a tail call, i.e.:
> > >
> > > la ra, ret_from_exception
> > > tail ret_from_fork
> > >
> >
> > I don't believe so due to the return address stack. It was shown in this
> > patch [1] that a 7% performance improvement can be seen on existing
> > riscv hardware by performing the extra jump. Doing tail calls should be
> > avoided on riscv since the hardware can be expected to predict the
> > return address incorrectly every time if the return address is manually
> > changed.
>
> Fair enough, thanks. Though no link to said patch given.
Oops! Here it is:
https://lore.kernel.org/linux-riscv/20240607061335.2197383-1-cyrilbur@tenstorrent.com/.
>
> Maciej
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2025-01-27 20:55 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-01-22 22:56 [PATCH 0/4] entry: Move ret_from_fork() to C and inline syscall_exit_to_user_mode() Charlie Jenkins
2025-01-22 22:56 ` [PATCH 1/4] riscv: entry: Convert ret_from_fork() to C Charlie Jenkins
2025-01-24 7:09 ` Alexandre Ghiti
2025-01-24 7:56 ` Charlie Jenkins
2025-01-25 23:30 ` Maciej W. Rozycki
2025-01-26 19:51 ` Charlie Jenkins
2025-01-27 20:20 ` Maciej W. Rozycki
2025-01-27 20:55 ` Charlie Jenkins
2025-01-22 22:56 ` [PATCH 2/4] riscv: entry: Split ret_from_fork() into user and kernel Charlie Jenkins
2025-01-22 22:56 ` [PATCH 3/4] loongarch: entry: Migrate ret_from_fork() to C Charlie Jenkins
2025-01-23 17:12 ` kernel test robot
2025-01-22 22:56 ` [PATCH 4/4] entry: Inline syscall_exit_to_user_mode() Charlie Jenkins
2025-01-23 17:12 ` kernel test robot
2025-01-23 17:34 ` kernel test robot
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox