* [PATCH v4 0/4] entry: Move ret_from_fork() to C and inline syscall_exit_to_user_mode()
@ 2025-01-28 5:33 Charlie Jenkins
2025-01-28 5:33 ` [PATCH v4 1/4] riscv: entry: Convert ret_from_fork() to C Charlie Jenkins
` (3 more replies)
0 siblings, 4 replies; 6+ messages in thread
From: Charlie Jenkins @ 2025-01-28 5:33 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Huacai Chen, WANG Xuerui,
Thomas Gleixner, Peter Zijlstra, Andy Lutomirski, Alexandre Ghiti
Cc: linux-riscv, linux-kernel, loongarch, Charlie Jenkins
Similar to commit 221a164035fd ("entry: Move
syscall_enter_from_user_mode() to header file"), move
syscall_exit_to_user_mode() to the header file as well.
Testing was done with the byte-unixbench [1] syscall benchmark (which
calls getpid) and QEMU. On riscv I measured a 7.09246% improvement, on
x86 a 2.98843% improvement, on loongarch a 6.07954% improvement, and on
s390 a 11.1328% improvement.
Since this is on QEMU, I know these numbers are not perfect, but they
show a trend of general improvement across all architectures that use
the generic entry code.
[1] https://github.com/kdlucas/byte-unixbench
Signed-off-by: Charlie Jenkins <charlie@rivosinc.com>
---
Changes in v4:
- I had messed up warning for ct_state() on rebase, correct that issue
- Link to v3: https://lore.kernel.org/r/20250124-riscv_optimize_entry-v3-0-869f36b9e43b@rivosinc.com
Changes in v3:
- Fixup comment to properly reflect args (Alex)
- Fix prototypes for loongarch (Huacai)
- Link to v2: https://lore.kernel.org/r/20250123-riscv_optimize_entry-v2-0-7c259492d508@rivosinc.com
Changes in v2:
- Fixup compilation issues for loongarch
- Fixup compilation issues with CONFIG_CONTEXT_TRACKING_USER
- Link to v1: https://lore.kernel.org/r/20250122-riscv_optimize_entry-v1-0-4ee95559cfd0@rivosinc.com
---
Charlie Jenkins (4):
riscv: entry: Convert ret_from_fork() to C
riscv: entry: Split ret_from_fork() into user and kernel
LoongArch: entry: Migrate ret_from_fork() to C
entry: Inline syscall_exit_to_user_mode()
arch/loongarch/include/asm/asm-prototypes.h | 8 +++++
arch/loongarch/kernel/entry.S | 22 ++++++-------
arch/loongarch/kernel/process.c | 33 +++++++++++++++----
arch/riscv/include/asm/asm-prototypes.h | 2 ++
arch/riscv/kernel/entry.S | 20 +++++++-----
arch/riscv/kernel/process.c | 21 +++++++++++--
include/linux/entry-common.h | 43 +++++++++++++++++++++++--
kernel/entry/common.c | 49 +----------------------------
8 files changed, 119 insertions(+), 79 deletions(-)
---
base-commit: ffd294d346d185b70e28b1a28abe367bbfe53c04
change-id: 20240402-riscv_optimize_entry-583843420325
--
- Charlie
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PATCH v4 1/4] riscv: entry: Convert ret_from_fork() to C
2025-01-28 5:33 [PATCH v4 0/4] entry: Move ret_from_fork() to C and inline syscall_exit_to_user_mode() Charlie Jenkins
@ 2025-01-28 5:33 ` Charlie Jenkins
2025-01-28 5:33 ` [PATCH v4 2/4] riscv: entry: Split ret_from_fork() into user and kernel Charlie Jenkins
` (2 subsequent siblings)
3 siblings, 0 replies; 6+ messages in thread
From: Charlie Jenkins @ 2025-01-28 5:33 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Huacai Chen, WANG Xuerui,
Thomas Gleixner, Peter Zijlstra, Andy Lutomirski, Alexandre Ghiti
Cc: linux-riscv, linux-kernel, loongarch, Charlie Jenkins
Move the main section of ret_from_fork() to C to allow inlining of
syscall_exit_to_user_mode().
Signed-off-by: Charlie Jenkins <charlie@rivosinc.com>
Reviewed-by: Alexandre Ghiti <alexghiti@rivosinc.com>
---
arch/riscv/include/asm/asm-prototypes.h | 1 +
arch/riscv/kernel/entry.S | 15 ++++++---------
arch/riscv/kernel/process.c | 14 ++++++++++++--
3 files changed, 19 insertions(+), 11 deletions(-)
diff --git a/arch/riscv/include/asm/asm-prototypes.h b/arch/riscv/include/asm/asm-prototypes.h
index cd627ec289f163a630b73dd03dd52a6b28692997..733ff609778797001006c33bba9e3cc5b1f15387 100644
--- a/arch/riscv/include/asm/asm-prototypes.h
+++ b/arch/riscv/include/asm/asm-prototypes.h
@@ -52,6 +52,7 @@ DECLARE_DO_ERROR_INFO(do_trap_ecall_s);
DECLARE_DO_ERROR_INFO(do_trap_ecall_m);
DECLARE_DO_ERROR_INFO(do_trap_break);
+asmlinkage void ret_from_fork(void *fn_arg, int (*fn)(void *), struct pt_regs *regs);
asmlinkage void handle_bad_stack(struct pt_regs *regs);
asmlinkage void do_page_fault(struct pt_regs *regs);
asmlinkage void do_irq(struct pt_regs *regs);
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index 33a5a9f2a0d4e1eeccfb3621b9e518b88e1b0704..b2dc5e7c7b3a843fa4aa02eba2a911eb3ce31d1f 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -319,17 +319,14 @@ SYM_CODE_END(handle_kernel_stack_overflow)
ASM_NOKPROBE(handle_kernel_stack_overflow)
#endif
-SYM_CODE_START(ret_from_fork)
+SYM_CODE_START(ret_from_fork_asm)
call schedule_tail
- beqz s0, 1f /* not from kernel thread */
- /* Call fn(arg) */
- move a0, s1
- jalr s0
-1:
- move a0, sp /* pt_regs */
- call syscall_exit_to_user_mode
+ move a0, s1 /* fn_arg */
+ move a1, s0 /* fn */
+ move a2, sp /* pt_regs */
+ call ret_from_fork
j ret_from_exception
-SYM_CODE_END(ret_from_fork)
+SYM_CODE_END(ret_from_fork_asm)
#ifdef CONFIG_IRQ_STACKS
/*
diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c
index 58b6482c2bf662bf5224ca50c8e21a68760a6b41..0d07e6d8f6b57beba438dbba5e8c74a014582bee 100644
--- a/arch/riscv/kernel/process.c
+++ b/arch/riscv/kernel/process.c
@@ -17,7 +17,9 @@
#include <linux/ptrace.h>
#include <linux/uaccess.h>
#include <linux/personality.h>
+#include <linux/entry-common.h>
+#include <asm/asm-prototypes.h>
#include <asm/unistd.h>
#include <asm/processor.h>
#include <asm/csr.h>
@@ -36,7 +38,7 @@ unsigned long __stack_chk_guard __read_mostly;
EXPORT_SYMBOL(__stack_chk_guard);
#endif
-extern asmlinkage void ret_from_fork(void);
+extern asmlinkage void ret_from_fork_asm(void);
void noinstr arch_cpu_idle(void)
{
@@ -206,6 +208,14 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
return 0;
}
+asmlinkage void ret_from_fork(void *fn_arg, int (*fn)(void *), struct pt_regs *regs)
+{
+ if (unlikely(fn))
+ fn(fn_arg);
+
+ syscall_exit_to_user_mode(regs);
+}
+
int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
{
unsigned long clone_flags = args->flags;
@@ -242,7 +252,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
p->thread.riscv_v_flags = 0;
if (has_vector())
riscv_v_thread_alloc(p);
- p->thread.ra = (unsigned long)ret_from_fork;
+ p->thread.ra = (unsigned long)ret_from_fork_asm;
p->thread.sp = (unsigned long)childregs; /* kernel sp */
return 0;
}
--
2.43.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v4 2/4] riscv: entry: Split ret_from_fork() into user and kernel
2025-01-28 5:33 [PATCH v4 0/4] entry: Move ret_from_fork() to C and inline syscall_exit_to_user_mode() Charlie Jenkins
2025-01-28 5:33 ` [PATCH v4 1/4] riscv: entry: Convert ret_from_fork() to C Charlie Jenkins
@ 2025-01-28 5:33 ` Charlie Jenkins
2025-01-28 5:33 ` [PATCH v4 3/4] LoongArch: entry: Migrate ret_from_fork() to C Charlie Jenkins
2025-01-28 5:33 ` [PATCH v4 4/4] entry: Inline syscall_exit_to_user_mode() Charlie Jenkins
3 siblings, 0 replies; 6+ messages in thread
From: Charlie Jenkins @ 2025-01-28 5:33 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Huacai Chen, WANG Xuerui,
Thomas Gleixner, Peter Zijlstra, Andy Lutomirski, Alexandre Ghiti
Cc: linux-riscv, linux-kernel, loongarch, Charlie Jenkins
This function was unified into a single function in commit ab9164dae273
("riscv: entry: Consolidate ret_from_kernel_thread into ret_from_fork").
However that imposed a performance degradation. Partially reverting this
commit to have ret_from_fork() split again results in a 1% increase on
the number of times fork is able to be called per second.
Signed-off-by: Charlie Jenkins <charlie@rivosinc.com>
---
arch/riscv/include/asm/asm-prototypes.h | 3 ++-
arch/riscv/kernel/entry.S | 13 ++++++++++---
arch/riscv/kernel/process.c | 17 +++++++++++------
3 files changed, 23 insertions(+), 10 deletions(-)
diff --git a/arch/riscv/include/asm/asm-prototypes.h b/arch/riscv/include/asm/asm-prototypes.h
index 733ff609778797001006c33bba9e3cc5b1f15387..bfc8ea5f9319b19449ec59493b45b926df888832 100644
--- a/arch/riscv/include/asm/asm-prototypes.h
+++ b/arch/riscv/include/asm/asm-prototypes.h
@@ -52,7 +52,8 @@ DECLARE_DO_ERROR_INFO(do_trap_ecall_s);
DECLARE_DO_ERROR_INFO(do_trap_ecall_m);
DECLARE_DO_ERROR_INFO(do_trap_break);
-asmlinkage void ret_from_fork(void *fn_arg, int (*fn)(void *), struct pt_regs *regs);
+asmlinkage void ret_from_fork_kernel(void *fn_arg, int (*fn)(void *), struct pt_regs *regs);
+asmlinkage void ret_from_fork_user(struct pt_regs *regs);
asmlinkage void handle_bad_stack(struct pt_regs *regs);
asmlinkage void do_page_fault(struct pt_regs *regs);
asmlinkage void do_irq(struct pt_regs *regs);
diff --git a/arch/riscv/kernel/entry.S b/arch/riscv/kernel/entry.S
index b2dc5e7c7b3a843fa4aa02eba2a911eb3ce31d1f..0fb338000c6dc0358742cd03497fa54b9e9d1aec 100644
--- a/arch/riscv/kernel/entry.S
+++ b/arch/riscv/kernel/entry.S
@@ -319,14 +319,21 @@ SYM_CODE_END(handle_kernel_stack_overflow)
ASM_NOKPROBE(handle_kernel_stack_overflow)
#endif
-SYM_CODE_START(ret_from_fork_asm)
+SYM_CODE_START(ret_from_fork_kernel_asm)
call schedule_tail
move a0, s1 /* fn_arg */
move a1, s0 /* fn */
move a2, sp /* pt_regs */
- call ret_from_fork
+ call ret_from_fork_kernel
j ret_from_exception
-SYM_CODE_END(ret_from_fork_asm)
+SYM_CODE_END(ret_from_fork_kernel_asm)
+
+SYM_CODE_START(ret_from_fork_user_asm)
+ call schedule_tail
+ move a0, sp /* pt_regs */
+ call ret_from_fork_user
+ j ret_from_exception
+SYM_CODE_END(ret_from_fork_user_asm)
#ifdef CONFIG_IRQ_STACKS
/*
diff --git a/arch/riscv/kernel/process.c b/arch/riscv/kernel/process.c
index 0d07e6d8f6b57beba438dbba5e8c74a014582bee..5f15236cb526bd9fe61636ed372b4b76c94df946 100644
--- a/arch/riscv/kernel/process.c
+++ b/arch/riscv/kernel/process.c
@@ -38,7 +38,8 @@ unsigned long __stack_chk_guard __read_mostly;
EXPORT_SYMBOL(__stack_chk_guard);
#endif
-extern asmlinkage void ret_from_fork_asm(void);
+extern asmlinkage void ret_from_fork_kernel_asm(void);
+extern asmlinkage void ret_from_fork_user_asm(void);
void noinstr arch_cpu_idle(void)
{
@@ -208,14 +209,18 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
return 0;
}
-asmlinkage void ret_from_fork(void *fn_arg, int (*fn)(void *), struct pt_regs *regs)
+asmlinkage void ret_from_fork_kernel(void *fn_arg, int (*fn)(void *), struct pt_regs *regs)
{
- if (unlikely(fn))
- fn(fn_arg);
+ fn(fn_arg);
syscall_exit_to_user_mode(regs);
}
+asmlinkage void ret_from_fork_user(struct pt_regs *regs)
+{
+ syscall_exit_to_user_mode(regs);
+}
+
int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
{
unsigned long clone_flags = args->flags;
@@ -238,6 +243,7 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
p->thread.s[0] = (unsigned long)args->fn;
p->thread.s[1] = (unsigned long)args->fn_arg;
+ p->thread.ra = (unsigned long)ret_from_fork_kernel_asm;
} else {
*childregs = *(current_pt_regs());
/* Turn off status.VS */
@@ -247,12 +253,11 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
if (clone_flags & CLONE_SETTLS)
childregs->tp = tls;
childregs->a0 = 0; /* Return value of fork() */
- p->thread.s[0] = 0;
+ p->thread.ra = (unsigned long)ret_from_fork_user_asm;
}
p->thread.riscv_v_flags = 0;
if (has_vector())
riscv_v_thread_alloc(p);
- p->thread.ra = (unsigned long)ret_from_fork_asm;
p->thread.sp = (unsigned long)childregs; /* kernel sp */
return 0;
}
--
2.43.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v4 3/4] LoongArch: entry: Migrate ret_from_fork() to C
2025-01-28 5:33 [PATCH v4 0/4] entry: Move ret_from_fork() to C and inline syscall_exit_to_user_mode() Charlie Jenkins
2025-01-28 5:33 ` [PATCH v4 1/4] riscv: entry: Convert ret_from_fork() to C Charlie Jenkins
2025-01-28 5:33 ` [PATCH v4 2/4] riscv: entry: Split ret_from_fork() into user and kernel Charlie Jenkins
@ 2025-01-28 5:33 ` Charlie Jenkins
2025-01-28 5:33 ` [PATCH v4 4/4] entry: Inline syscall_exit_to_user_mode() Charlie Jenkins
3 siblings, 0 replies; 6+ messages in thread
From: Charlie Jenkins @ 2025-01-28 5:33 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Huacai Chen, WANG Xuerui,
Thomas Gleixner, Peter Zijlstra, Andy Lutomirski, Alexandre Ghiti
Cc: linux-riscv, linux-kernel, loongarch, Charlie Jenkins
LoongArch is the only architecture that calls
syscall_exit_to_user_mode() from asm. Move the call into C so that this
function can be inlined across all architectures.
Signed-off-by: Charlie Jenkins <charlie@rivosinc.com>
---
arch/loongarch/include/asm/asm-prototypes.h | 8 +++++++
arch/loongarch/kernel/entry.S | 22 +++++++++----------
arch/loongarch/kernel/process.c | 33 +++++++++++++++++++++++------
3 files changed, 45 insertions(+), 18 deletions(-)
diff --git a/arch/loongarch/include/asm/asm-prototypes.h b/arch/loongarch/include/asm/asm-prototypes.h
index 51f224bcfc654228ae423e9a066b25b35102a5b9..704066b4f7368be15be960fadbcd6c2574bbf6c0 100644
--- a/arch/loongarch/include/asm/asm-prototypes.h
+++ b/arch/loongarch/include/asm/asm-prototypes.h
@@ -12,3 +12,11 @@ __int128_t __ashlti3(__int128_t a, int b);
__int128_t __ashrti3(__int128_t a, int b);
__int128_t __lshrti3(__int128_t a, int b);
#endif
+
+asmlinkage void noinstr __no_stack_protector ret_from_fork(struct task_struct *prev,
+ struct pt_regs *regs);
+
+asmlinkage void noinstr __no_stack_protector ret_from_kernel_thread(struct task_struct *prev,
+ struct pt_regs *regs,
+ int (*fn)(void *),
+ void *fn_arg);
diff --git a/arch/loongarch/kernel/entry.S b/arch/loongarch/kernel/entry.S
index 48e7e34e355e83eae8165957ba2eac05a8bf17df..2abc29e573810e000f2fef4646ddca0dbb80eabe 100644
--- a/arch/loongarch/kernel/entry.S
+++ b/arch/loongarch/kernel/entry.S
@@ -77,24 +77,22 @@ SYM_CODE_START(handle_syscall)
SYM_CODE_END(handle_syscall)
_ASM_NOKPROBE(handle_syscall)
-SYM_CODE_START(ret_from_fork)
+SYM_CODE_START(ret_from_fork_asm)
UNWIND_HINT_REGS
- bl schedule_tail # a0 = struct task_struct *prev
- move a0, sp
- bl syscall_exit_to_user_mode
+ move a1, sp
+ bl ret_from_fork
RESTORE_STATIC
RESTORE_SOME
RESTORE_SP_AND_RET
-SYM_CODE_END(ret_from_fork)
+SYM_CODE_END(ret_from_fork_asm)
-SYM_CODE_START(ret_from_kernel_thread)
+SYM_CODE_START(ret_from_kernel_thread_asm)
UNWIND_HINT_REGS
- bl schedule_tail # a0 = struct task_struct *prev
- move a0, s1
- jirl ra, s0, 0
- move a0, sp
- bl syscall_exit_to_user_mode
+ move a1, sp
+ move a2, s0
+ move a3, s1
+ bl ret_from_kernel_thread
RESTORE_STATIC
RESTORE_SOME
RESTORE_SP_AND_RET
-SYM_CODE_END(ret_from_kernel_thread)
+SYM_CODE_END(ret_from_kernel_thread_asm)
diff --git a/arch/loongarch/kernel/process.c b/arch/loongarch/kernel/process.c
index 6e58f65455c7ca3eae2e88ed852c8655a6701e5c..98bc60d7c550fcc0225e8452f81a7d6cd7888015 100644
--- a/arch/loongarch/kernel/process.c
+++ b/arch/loongarch/kernel/process.c
@@ -14,6 +14,7 @@
#include <linux/init.h>
#include <linux/kernel.h>
#include <linux/errno.h>
+#include <linux/entry-common.h>
#include <linux/sched.h>
#include <linux/sched/debug.h>
#include <linux/sched/task.h>
@@ -33,6 +34,7 @@
#include <linux/prctl.h>
#include <linux/nmi.h>
+#include <asm/asm-prototypes.h>
#include <asm/asm.h>
#include <asm/bootinfo.h>
#include <asm/cpu.h>
@@ -47,6 +49,7 @@
#include <asm/pgtable.h>
#include <asm/processor.h>
#include <asm/reg.h>
+#include <asm/switch_to.h>
#include <asm/unwind.h>
#include <asm/vdso.h>
@@ -63,8 +66,9 @@ EXPORT_SYMBOL(__stack_chk_guard);
unsigned long boot_option_idle_override = IDLE_NO_OVERRIDE;
EXPORT_SYMBOL(boot_option_idle_override);
-asmlinkage void ret_from_fork(void);
-asmlinkage void ret_from_kernel_thread(void);
+asmlinkage void restore_and_ret(void);
+asmlinkage void ret_from_fork_asm(void);
+asmlinkage void ret_from_kernel_thread_asm(void);
void start_thread(struct pt_regs *regs, unsigned long pc, unsigned long sp)
{
@@ -138,6 +142,23 @@ int arch_dup_task_struct(struct task_struct *dst, struct task_struct *src)
return 0;
}
+asmlinkage void noinstr __no_stack_protector ret_from_fork(struct task_struct *prev,
+ struct pt_regs *regs)
+{
+ schedule_tail(prev);
+ syscall_exit_to_user_mode(regs);
+}
+
+asmlinkage void noinstr __no_stack_protector ret_from_kernel_thread(struct task_struct *prev,
+ struct pt_regs *regs,
+ int (*fn)(void *),
+ void *fn_arg)
+{
+ schedule_tail(prev);
+ fn(fn_arg);
+ syscall_exit_to_user_mode(regs);
+}
+
/*
* Copy architecture-specific thread state
*/
@@ -165,8 +186,8 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
p->thread.reg03 = childksp;
p->thread.reg23 = (unsigned long)args->fn;
p->thread.reg24 = (unsigned long)args->fn_arg;
- p->thread.reg01 = (unsigned long)ret_from_kernel_thread;
- p->thread.sched_ra = (unsigned long)ret_from_kernel_thread;
+ p->thread.reg01 = (unsigned long)ret_from_kernel_thread_asm;
+ p->thread.sched_ra = (unsigned long)ret_from_kernel_thread_asm;
memset(childregs, 0, sizeof(struct pt_regs));
childregs->csr_euen = p->thread.csr_euen;
childregs->csr_crmd = p->thread.csr_crmd;
@@ -182,8 +203,8 @@ int copy_thread(struct task_struct *p, const struct kernel_clone_args *args)
childregs->regs[3] = usp;
p->thread.reg03 = (unsigned long) childregs;
- p->thread.reg01 = (unsigned long) ret_from_fork;
- p->thread.sched_ra = (unsigned long) ret_from_fork;
+ p->thread.reg01 = (unsigned long) ret_from_fork_asm;
+ p->thread.sched_ra = (unsigned long) ret_from_fork_asm;
/*
* New tasks lose permission to use the fpu. This accelerates context
--
2.43.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH v4 4/4] entry: Inline syscall_exit_to_user_mode()
2025-01-28 5:33 [PATCH v4 0/4] entry: Move ret_from_fork() to C and inline syscall_exit_to_user_mode() Charlie Jenkins
` (2 preceding siblings ...)
2025-01-28 5:33 ` [PATCH v4 3/4] LoongArch: entry: Migrate ret_from_fork() to C Charlie Jenkins
@ 2025-01-28 5:33 ` Charlie Jenkins
2025-02-05 8:13 ` kernel test robot
3 siblings, 1 reply; 6+ messages in thread
From: Charlie Jenkins @ 2025-01-28 5:33 UTC (permalink / raw)
To: Paul Walmsley, Palmer Dabbelt, Huacai Chen, WANG Xuerui,
Thomas Gleixner, Peter Zijlstra, Andy Lutomirski, Alexandre Ghiti
Cc: linux-riscv, linux-kernel, loongarch, Charlie Jenkins
Architectures using the generic entry code can be optimized by having
syscall_exit_to_user_mode inlined.
Signed-off-by: Charlie Jenkins <charlie@rivosinc.com>
---
include/linux/entry-common.h | 43 ++++++++++++++++++++++++++++++++++++--
kernel/entry/common.c | 49 +-------------------------------------------
2 files changed, 42 insertions(+), 50 deletions(-)
diff --git a/include/linux/entry-common.h b/include/linux/entry-common.h
index fc61d0205c97084acc89c8e45e088946f5e6d9b2..f94f3fdf15fc0091223cc9f7b823970302e67312 100644
--- a/include/linux/entry-common.h
+++ b/include/linux/entry-common.h
@@ -14,6 +14,7 @@
#include <linux/kmsan.h>
#include <asm/entry-common.h>
+#include <asm/syscall.h>
/*
* Define dummy _TIF work flags if not defined by the architecture or for
@@ -366,6 +367,15 @@ static __always_inline void exit_to_user_mode(void)
lockdep_hardirqs_on(CALLER_ADDR0);
}
+/**
+ * syscall_exit_work - Handle work before returning to user mode
+ * @regs: Pointer to current pt_regs
+ * @work: Current thread syscall work
+ *
+ * Do one-time syscall specific work.
+ */
+void syscall_exit_work(struct pt_regs *regs, unsigned long work);
+
/**
* syscall_exit_to_user_mode_work - Handle work before returning to user mode
* @regs: Pointer to currents pt_regs
@@ -379,7 +389,30 @@ static __always_inline void exit_to_user_mode(void)
* make the final state transitions. Interrupts must stay disabled between
* return from this function and the invocation of exit_to_user_mode().
*/
-void syscall_exit_to_user_mode_work(struct pt_regs *regs);
+static __always_inline void syscall_exit_to_user_mode_work(struct pt_regs *regs)
+{
+ unsigned long work = READ_ONCE(current_thread_info()->syscall_work);
+ unsigned long nr = syscall_get_nr(current, regs);
+
+ CT_WARN_ON(ct_state() != CT_STATE_KERNEL);
+
+ if (IS_ENABLED(CONFIG_PROVE_LOCKING)) {
+ if (WARN(irqs_disabled(), "syscall %lu left IRQs disabled", nr))
+ local_irq_enable();
+ }
+
+ rseq_syscall(regs);
+
+ /*
+ * Do one-time syscall specific work. If these work items are
+ * enabled, we want to run them exactly once per syscall exit with
+ * interrupts enabled.
+ */
+ if (unlikely(work & SYSCALL_WORK_EXIT))
+ syscall_exit_work(regs, work);
+ local_irq_disable_exit_to_user();
+ exit_to_user_mode_prepare(regs);
+}
/**
* syscall_exit_to_user_mode - Handle work before returning to user mode
@@ -410,7 +443,13 @@ void syscall_exit_to_user_mode_work(struct pt_regs *regs);
* exit_to_user_mode(). This function is preferred unless there is a
* compelling architectural reason to use the separate functions.
*/
-void syscall_exit_to_user_mode(struct pt_regs *regs);
+static __always_inline void syscall_exit_to_user_mode(struct pt_regs *regs)
+{
+ instrumentation_begin();
+ syscall_exit_to_user_mode_work(regs);
+ instrumentation_end();
+ exit_to_user_mode();
+}
/**
* irqentry_enter_from_user_mode - Establish state before invoking the irq handler
diff --git a/kernel/entry/common.c b/kernel/entry/common.c
index e33691d5adf7aab4af54cf2bf8e5ef5bd6ad1424..f55e421fb196dd5f9d4e34dd85ae096c774cf879 100644
--- a/kernel/entry/common.c
+++ b/kernel/entry/common.c
@@ -146,7 +146,7 @@ static inline bool report_single_step(unsigned long work)
return work & SYSCALL_WORK_SYSCALL_EXIT_TRAP;
}
-static void syscall_exit_work(struct pt_regs *regs, unsigned long work)
+void syscall_exit_work(struct pt_regs *regs, unsigned long work)
{
bool step;
@@ -173,53 +173,6 @@ static void syscall_exit_work(struct pt_regs *regs, unsigned long work)
ptrace_report_syscall_exit(regs, step);
}
-/*
- * Syscall specific exit to user mode preparation. Runs with interrupts
- * enabled.
- */
-static void syscall_exit_to_user_mode_prepare(struct pt_regs *regs)
-{
- unsigned long work = READ_ONCE(current_thread_info()->syscall_work);
- unsigned long nr = syscall_get_nr(current, regs);
-
- CT_WARN_ON(ct_state() != CT_STATE_KERNEL);
-
- if (IS_ENABLED(CONFIG_PROVE_LOCKING)) {
- if (WARN(irqs_disabled(), "syscall %lu left IRQs disabled", nr))
- local_irq_enable();
- }
-
- rseq_syscall(regs);
-
- /*
- * Do one-time syscall specific work. If these work items are
- * enabled, we want to run them exactly once per syscall exit with
- * interrupts enabled.
- */
- if (unlikely(work & SYSCALL_WORK_EXIT))
- syscall_exit_work(regs, work);
-}
-
-static __always_inline void __syscall_exit_to_user_mode_work(struct pt_regs *regs)
-{
- syscall_exit_to_user_mode_prepare(regs);
- local_irq_disable_exit_to_user();
- exit_to_user_mode_prepare(regs);
-}
-
-void syscall_exit_to_user_mode_work(struct pt_regs *regs)
-{
- __syscall_exit_to_user_mode_work(regs);
-}
-
-__visible noinstr void syscall_exit_to_user_mode(struct pt_regs *regs)
-{
- instrumentation_begin();
- __syscall_exit_to_user_mode_work(regs);
- instrumentation_end();
- exit_to_user_mode();
-}
-
noinstr void irqentry_enter_from_user_mode(struct pt_regs *regs)
{
enter_from_user_mode(regs);
--
2.43.0
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH v4 4/4] entry: Inline syscall_exit_to_user_mode()
2025-01-28 5:33 ` [PATCH v4 4/4] entry: Inline syscall_exit_to_user_mode() Charlie Jenkins
@ 2025-02-05 8:13 ` kernel test robot
0 siblings, 0 replies; 6+ messages in thread
From: kernel test robot @ 2025-02-05 8:13 UTC (permalink / raw)
To: Charlie Jenkins
Cc: oe-lkp, lkp, linux-kernel, Paul Walmsley, Palmer Dabbelt,
Huacai Chen, WANG Xuerui, Thomas Gleixner, Peter Zijlstra,
Andy Lutomirski, Alexandre Ghiti, linux-riscv, loongarch,
Charlie Jenkins, oliver.sang
Hello,
kernel test robot noticed a 1.9% improvement of stress-ng.seek.ops_per_sec on:
commit: c1bc35dd5bf6c7fa86a936a4fbe3b8d92fbf8641 ("[PATCH v4 4/4] entry: Inline syscall_exit_to_user_mode()")
url: https://github.com/intel-lab-lkp/linux/commits/Charlie-Jenkins/riscv-entry-Convert-ret_from_fork-to-C/20250128-133636
patch link: https://lore.kernel.org/all/20250127-riscv_optimize_entry-v4-4-868cf7702dc9@rivosinc.com/
patch subject: [PATCH v4 4/4] entry: Inline syscall_exit_to_user_mode()
testcase: stress-ng
config: x86_64-rhel-9.4
compiler: gcc-12
test machine: 64 threads 2 sockets Intel(R) Xeon(R) Gold 6346 CPU @ 3.10GHz (Ice Lake) with 256G memory
parameters:
nr_threads: 100%
testtime: 60s
test: seek
cpufreq_governor: performance
In addition to that, the commit also has significant impact on the following tests:
+------------------+--------------------------------------------------------------------------------+
| testcase: change | stress-ng: stress-ng.context.swapcontext_calls_per_sec 1.9% improvement |
| test machine | 384 threads 2 sockets Intel(R) Xeon(R) 6972P (Granite Rapids) with 128G memory |
| test parameters | cpufreq_governor=performance |
| | nr_threads=100% |
| | test=context |
| | testtime=60s |
+------------------+--------------------------------------------------------------------------------+
Details are as below:
-------------------------------------------------------------------------------------------------->
The kernel config and materials to reproduce are available at:
https://download.01.org/0day-ci/archive/20250205/202502051555.85ae6844-lkp@intel.com
=========================================================================================
compiler/cpufreq_governor/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime:
gcc-12/performance/x86_64-rhel-9.4/100%/debian-12-x86_64-20240206.cgz/lkp-icl-2sp8/seek/stress-ng/60s
commit:
37c1871b51 ("LoongArch: entry: Migrate ret_from_fork() to C")
c1bc35dd5b ("entry: Inline syscall_exit_to_user_mode()")
37c1871b51766a66 c1bc35dd5bf6c7fa86a936a4fbe
---------------- ---------------------------
%stddev %change %stddev
\ | \
104886 ± 19% +19.3% 125157 ± 17% numa-meminfo.node1.Slab
2583 ± 39% +75.4% 4531 ± 40% proc-vmstat.numa_hint_faults_local
179842 +0.6% 180945 vmstat.system.in
177.18 -2.6% 172.49 stress-ng.seek.nanosecs_per_seek
1.223e+09 +1.9% 1.246e+09 stress-ng.seek.ops
20376380 +1.9% 20771261 stress-ng.seek.ops_per_sec
1.05 ± 20% -100.0% 0.00 perf-sched.sch_delay.avg.ms.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
13.11 ± 28% -100.0% 0.00 perf-sched.sch_delay.max.ms.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
3.12 ± 21% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
2785 ± 14% -100.0% 0.00 perf-sched.wait_and_delay.count.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
836.20 ± 43% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
2.07 ± 27% -100.0% 0.00 perf-sched.wait_time.avg.ms.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
834.79 ± 44% -100.0% 0.00 perf-sched.wait_time.max.ms.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
2.04 +3.4% 2.11 perf-stat.i.MPKI
3.682e+08 +2.0% 3.754e+08 perf-stat.i.cache-misses
4.637e+08 +1.8% 4.721e+08 perf-stat.i.cache-references
1.23 +1.5% 1.25 perf-stat.i.cpi
603.02 -1.9% 591.60 perf-stat.i.cycles-between-cache-misses
1.798e+11 -1.4% 1.772e+11 perf-stat.i.instructions
0.82 -1.4% 0.80 perf-stat.i.ipc
3902 +1.8% 3972 ± 2% perf-stat.i.minor-faults
3902 +1.8% 3972 ± 2% perf-stat.i.page-faults
2.05 +3.4% 2.12 perf-stat.overall.MPKI
1.23 +1.5% 1.25 perf-stat.overall.cpi
602.25 -1.9% 590.74 perf-stat.overall.cycles-between-cache-misses
0.81 -1.4% 0.80 perf-stat.overall.ipc
3.623e+08 +1.9% 3.693e+08 perf-stat.ps.cache-misses
4.562e+08 +1.8% 4.645e+08 perf-stat.ps.cache-references
1.769e+11 -1.4% 1.743e+11 perf-stat.ps.instructions
3826 +1.8% 3893 ± 2% perf-stat.ps.minor-faults
3826 +1.8% 3893 ± 2% perf-stat.ps.page-faults
1.085e+13 -2.0% 1.063e+13 perf-stat.total.instructions
10.62 ± 2% -0.6 10.02 ± 3% perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.llseek.stress_run
9.46 ± 2% -0.5 8.94 ± 3% perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.llseek.stress_run
0.63 +0.0 0.66 ± 3% perf-profile.calltrace.cycles-pp.x64_sys_call.do_syscall_64.entry_SYSCALL_64_after_hwframe.llseek
1.61 +0.0 1.64 perf-profile.calltrace.cycles-pp.copy_page_from_iter_atomic.generic_perform_write.generic_file_write_iter.vfs_write.ksys_write
2.78 +0.1 2.85 perf-profile.calltrace.cycles-pp.__filemap_get_folio.simple_write_begin.generic_perform_write.generic_file_write_iter.vfs_write
2.94 +0.1 3.02 perf-profile.calltrace.cycles-pp.simple_write_begin.generic_perform_write.generic_file_write_iter.vfs_write.ksys_write
8.58 +0.2 8.77 perf-profile.calltrace.cycles-pp.copy_page_to_iter.filemap_read.vfs_read.ksys_read.do_syscall_64
8.37 +0.2 8.56 perf-profile.calltrace.cycles-pp._copy_to_iter.copy_page_to_iter.filemap_read.vfs_read.ksys_read
8.96 +0.2 9.17 perf-profile.calltrace.cycles-pp.folio_unlock.simple_write_end.generic_perform_write.generic_file_write_iter.vfs_write
9.53 +0.2 9.75 perf-profile.calltrace.cycles-pp.simple_write_end.generic_perform_write.generic_file_write_iter.vfs_write.ksys_write
12.86 +0.3 13.15 perf-profile.calltrace.cycles-pp.filemap_read.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe
14.08 +0.3 14.42 perf-profile.calltrace.cycles-pp.vfs_read.ksys_read.do_syscall_64.entry_SYSCALL_64_after_hwframe.read
15.98 +0.3 16.32 perf-profile.calltrace.cycles-pp.generic_perform_write.generic_file_write_iter.vfs_write.ksys_write.do_syscall_64
19.18 +0.4 19.55 perf-profile.calltrace.cycles-pp.generic_file_write_iter.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe
20.30 +0.4 20.67 perf-profile.calltrace.cycles-pp.vfs_write.ksys_write.do_syscall_64.entry_SYSCALL_64_after_hwframe.write
7.39 -7.4 0.00 perf-profile.children.cycles-pp.syscall_exit_to_user_mode
54.31 -0.7 53.60 perf-profile.children.cycles-pp.llseek
56.77 -0.3 56.42 perf-profile.children.cycles-pp.do_syscall_64
59.25 -0.3 58.95 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
0.12 ± 3% +0.0 0.15 ± 13% perf-profile.children.cycles-pp.generic_file_read_iter
1.73 +0.0 1.77 perf-profile.children.cycles-pp.x64_sys_call
1.97 +0.1 2.02 perf-profile.children.cycles-pp.filemap_get_entry
2.84 +0.1 2.92 perf-profile.children.cycles-pp.__filemap_get_folio
2.97 +0.1 3.05 perf-profile.children.cycles-pp.simple_write_begin
6.98 +0.1 7.09 perf-profile.children.cycles-pp.entry_SYSRETQ_unsafe_stack
1.96 +0.1 2.08 ± 5% perf-profile.children.cycles-pp.stress_shim_lseek
8.92 +0.1 9.06 perf-profile.children.cycles-pp.entry_SYSCALL_64
8.40 +0.2 8.58 perf-profile.children.cycles-pp._copy_to_iter
8.61 +0.2 8.80 perf-profile.children.cycles-pp.copy_page_to_iter
8.97 +0.2 9.19 perf-profile.children.cycles-pp.folio_unlock
9.57 +0.2 9.80 perf-profile.children.cycles-pp.simple_write_end
19.10 +0.3 19.38 perf-profile.children.cycles-pp.read
12.94 +0.3 13.24 perf-profile.children.cycles-pp.filemap_read
25.30 +0.3 25.62 perf-profile.children.cycles-pp.write
14.14 +0.3 14.48 perf-profile.children.cycles-pp.vfs_read
16.12 +0.3 16.47 perf-profile.children.cycles-pp.generic_perform_write
14.72 +0.4 15.08 perf-profile.children.cycles-pp.ksys_read
19.25 +0.4 19.62 perf-profile.children.cycles-pp.generic_file_write_iter
20.95 +0.4 21.33 perf-profile.children.cycles-pp.ksys_write
20.40 +0.4 20.78 perf-profile.children.cycles-pp.vfs_write
6.38 -6.4 0.00 perf-profile.self.cycles-pp.syscall_exit_to_user_mode
0.63 +0.0 0.65 perf-profile.self.cycles-pp.__filemap_get_folio
2.20 +0.0 2.23 perf-profile.self.cycles-pp.entry_SYSCALL_64
2.45 +0.0 2.48 perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.97 +0.0 1.00 perf-profile.self.cycles-pp.filemap_read
1.51 +0.0 1.56 perf-profile.self.cycles-pp.x64_sys_call
1.54 +0.0 1.59 perf-profile.self.cycles-pp.filemap_get_read_batch
6.54 +0.1 6.64 perf-profile.self.cycles-pp.llseek
6.74 +0.1 6.85 perf-profile.self.cycles-pp.entry_SYSRETQ_unsafe_stack
8.35 +0.2 8.54 perf-profile.self.cycles-pp._copy_to_iter
8.93 +0.2 9.14 perf-profile.self.cycles-pp.folio_unlock
3.91 +6.1 9.96 perf-profile.self.cycles-pp.do_syscall_64
***************************************************************************************************
lkp-gnr-2ap2: 384 threads 2 sockets Intel(R) Xeon(R) 6972P (Granite Rapids) with 128G memory
=========================================================================================
compiler/cpufreq_governor/kconfig/nr_threads/rootfs/tbox_group/test/testcase/testtime:
gcc-12/performance/x86_64-rhel-9.4/100%/debian-12-x86_64-20240206.cgz/lkp-gnr-2ap2/context/stress-ng/60s
commit:
37c1871b51 ("LoongArch: entry: Migrate ret_from_fork() to C")
c1bc35dd5b ("entry: Inline syscall_exit_to_user_mode()")
37c1871b51766a66 c1bc35dd5bf6c7fa86a936a4fbe
---------------- ---------------------------
%stddev %change %stddev
\ | \
933000 ± 10% +30.5% 1217543 ± 18% proc-vmstat.pgfree
40.25 ± 37% +70.8% 68.75 ± 37% sched_debug.cpu.nr_uninterruptible.max
1.063e+08 +1.9% 1.083e+08 stress-ng.context.ops
1771139 +1.9% 1805148 stress-ng.context.ops_per_sec
4608060 +1.9% 4696809 stress-ng.context.swapcontext_calls_per_sec
0.06 ± 24% -100.0% 0.00 perf-sched.sch_delay.avg.ms.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
4.53 ± 59% -100.0% 0.00 perf-sched.sch_delay.max.ms.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
217.64 ± 10% -17.8% 178.86 ± 17% perf-sched.wait_and_delay.avg.ms.pipe_read.vfs_read.ksys_read.do_syscall_64
0.67 ± 83% -100.0% 0.00 perf-sched.wait_and_delay.avg.ms.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
3262 ± 3% -100.0% 0.00 perf-sched.wait_and_delay.count.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
505.60 ± 97% -100.0% 0.00 perf-sched.wait_and_delay.max.ms.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
217.59 ± 10% -18.1% 178.22 ± 17% perf-sched.wait_time.avg.ms.pipe_read.vfs_read.ksys_read.do_syscall_64
0.61 ± 91% -100.0% 0.00 perf-sched.wait_time.avg.ms.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
502.72 ± 98% -100.0% 0.00 perf-sched.wait_time.max.ms.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.[unknown]
1.197e+11 -4.4% 1.145e+11 perf-stat.i.branch-instructions
1.48 +0.1 1.57 perf-stat.i.branch-miss-rate%
1.761e+09 +1.5% 1.788e+09 perf-stat.i.branch-misses
2.06 +4.1% 2.15 perf-stat.i.cpi
6.404e+11 -4.3% 6.129e+11 perf-stat.i.instructions
0.49 -3.9% 0.47 perf-stat.i.ipc
1.47 +0.1 1.56 perf-stat.overall.branch-miss-rate%
2.06 +4.1% 2.15 perf-stat.overall.cpi
0.48 -3.9% 0.47 perf-stat.overall.ipc
1.178e+11 -4.4% 1.126e+11 perf-stat.ps.branch-instructions
1.732e+09 +1.5% 1.758e+09 perf-stat.ps.branch-misses
6.3e+11 -4.3% 6.029e+11 perf-stat.ps.instructions
3.849e+13 -3.5% 3.716e+13 perf-stat.total.instructions
6.12 -6.1 0.00 perf-profile.calltrace.cycles-pp.syscall_exit_to_user_mode.do_syscall_64.entry_SYSCALL_64_after_hwframe.swapcontext
33.80 -0.7 33.14 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_after_hwframe.swapcontext
31.62 -0.5 31.12 perf-profile.calltrace.cycles-pp.do_syscall_64.entry_SYSCALL_64_after_hwframe.swapcontext
90.78 -0.3 90.49 perf-profile.calltrace.cycles-pp.swapcontext
1.40 -0.1 1.30 perf-profile.calltrace.cycles-pp.syscall_return_via_sysret.swapcontext
1.44 -0.0 1.40 perf-profile.calltrace.cycles-pp.sigprocmask.__x64_sys_rt_sigprocmask.do_syscall_64.entry_SYSCALL_64_after_hwframe.swapcontext
0.57 +0.0 0.61 perf-profile.calltrace.cycles-pp.entry_SYSRETQ_unsafe_stack.swapcontext
0.72 +0.0 0.77 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64_safe_stack.swapcontext
2.21 +0.1 2.28 perf-profile.calltrace.cycles-pp.stress_thread2
2.20 +0.1 2.28 perf-profile.calltrace.cycles-pp.stress_thread3
2.15 +0.1 2.24 perf-profile.calltrace.cycles-pp.stress_thread1
7.38 +0.1 7.48 perf-profile.calltrace.cycles-pp._copy_to_user.__x64_sys_rt_sigprocmask.do_syscall_64.entry_SYSCALL_64_after_hwframe.swapcontext
8.90 +0.1 9.00 perf-profile.calltrace.cycles-pp._copy_from_user.__x64_sys_rt_sigprocmask.do_syscall_64.entry_SYSCALL_64_after_hwframe.swapcontext
1.26 +0.1 1.37 perf-profile.calltrace.cycles-pp.x64_sys_call.do_syscall_64.entry_SYSCALL_64_after_hwframe.swapcontext
21.14 +0.3 21.49 perf-profile.calltrace.cycles-pp.__x64_sys_rt_sigprocmask.do_syscall_64.entry_SYSCALL_64_after_hwframe.swapcontext
22.96 +0.5 23.48 perf-profile.calltrace.cycles-pp.entry_SYSCALL_64.swapcontext
6.45 -6.4 0.00 perf-profile.children.cycles-pp.syscall_exit_to_user_mode
32.36 -0.7 31.64 perf-profile.children.cycles-pp.do_syscall_64
34.18 -0.7 33.52 perf-profile.children.cycles-pp.entry_SYSCALL_64_after_hwframe
96.11 -0.1 96.00 perf-profile.children.cycles-pp.swapcontext
1.59 -0.1 1.50 perf-profile.children.cycles-pp.syscall_return_via_sysret
1.54 -0.0 1.51 perf-profile.children.cycles-pp.sigprocmask
0.74 +0.1 0.79 perf-profile.children.cycles-pp.entry_SYSCALL_64_safe_stack
1.72 +0.1 1.78 perf-profile.children.cycles-pp.stress_thread3
1.70 +0.1 1.75 perf-profile.children.cycles-pp.stress_thread1
1.72 +0.1 1.78 perf-profile.children.cycles-pp.stress_thread2
7.64 +0.1 7.76 perf-profile.children.cycles-pp._copy_to_user
1.44 +0.1 1.58 perf-profile.children.cycles-pp.x64_sys_call
9.59 +0.2 9.74 perf-profile.children.cycles-pp._copy_from_user
7.18 +0.2 7.35 perf-profile.children.cycles-pp.entry_SYSRETQ_unsafe_stack
12.65 +0.3 12.92 perf-profile.children.cycles-pp.entry_SYSCALL_64
21.19 +0.3 21.50 perf-profile.children.cycles-pp.__x64_sys_rt_sigprocmask
5.45 -5.5 0.00 perf-profile.self.cycles-pp.syscall_exit_to_user_mode
1.59 -0.1 1.50 perf-profile.self.cycles-pp.syscall_return_via_sysret
1.39 -0.0 1.36 perf-profile.self.cycles-pp.sigprocmask
2.32 +0.0 2.35 perf-profile.self.cycles-pp.entry_SYSCALL_64
1.17 +0.0 1.20 perf-profile.self.cycles-pp.stress_thread3
1.18 +0.0 1.21 perf-profile.self.cycles-pp.stress_thread2
1.17 +0.0 1.20 perf-profile.self.cycles-pp.stress_thread1
2.83 +0.0 2.87 perf-profile.self.cycles-pp.__x64_sys_rt_sigprocmask
2.00 +0.1 2.05 perf-profile.self.cycles-pp.entry_SYSCALL_64_after_hwframe
0.73 +0.1 0.79 perf-profile.self.cycles-pp.entry_SYSCALL_64_safe_stack
7.50 +0.1 7.62 perf-profile.self.cycles-pp._copy_to_user
9.20 +0.1 9.34 perf-profile.self.cycles-pp._copy_from_user
1.22 +0.1 1.37 perf-profile.self.cycles-pp.x64_sys_call
6.99 +0.2 7.15 perf-profile.self.cycles-pp.entry_SYSRETQ_unsafe_stack
49.94 +0.4 50.34 perf-profile.self.cycles-pp.swapcontext
3.36 +5.2 8.51 perf-profile.self.cycles-pp.do_syscall_64
Disclaimer:
Results have been estimated based on internal Intel analysis and are provided
for informational purposes only. Any difference in system hardware or software
design or configuration may affect actual performance.
--
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2025-02-05 8:13 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-01-28 5:33 [PATCH v4 0/4] entry: Move ret_from_fork() to C and inline syscall_exit_to_user_mode() Charlie Jenkins
2025-01-28 5:33 ` [PATCH v4 1/4] riscv: entry: Convert ret_from_fork() to C Charlie Jenkins
2025-01-28 5:33 ` [PATCH v4 2/4] riscv: entry: Split ret_from_fork() into user and kernel Charlie Jenkins
2025-01-28 5:33 ` [PATCH v4 3/4] LoongArch: entry: Migrate ret_from_fork() to C Charlie Jenkins
2025-01-28 5:33 ` [PATCH v4 4/4] entry: Inline syscall_exit_to_user_mode() Charlie Jenkins
2025-02-05 8:13 ` kernel test robot
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox