Re: 8aeb879baf12 - significant system call latency regression, bisected

["H. Peter Anvin" <hpa@zytor.com>]

[-- Attachment #1: Type: text/plain, Size: 1222 bytes --]

On 2026-06-14 17:19, H. Peter Anvin wrote:
> 
> OK, so v7.1 was released with this sizable performance regression. That 
> begs the question how to deal with it.
> 
> One option that might be reasonable for -stable is to simply add back 16 
> bytes of NOPs into the assembly file. However, that is obviously not a 
> long term fix.
> 

Okay, here is a hack that actually generates the proper alignment, and 
it DOES in fact fix the performance regression.

It uses the same hack as the Makefile to deal with function alignment 
with a prefix: it adds unnecessary NOPs so that the pre-alignment and 
post-alignment are the same. At the end of the day this really ought to 
be fixed in gcc.

This is not meant to be a final patch; this should go in a header file 
and be cleaned up etc, but I wanted to confirm that it does, in fact, 
fix the regression and that the alignment of x64_sys_call is the root 
cause of the problem.

PeterZ: at some point you and I talked about the following:

- Should x64_sys_call() be noinstr?
- If so, any reason we can't inline it into do_syscall_64()?
- Since we no longer use the sys_call_table[] as a jump table,
   do we actually need array_index_nospec()? in do_syscall_x64|32?

	-hpa

[-- Attachment #2: diff --]
[-- Type: text/plain, Size: 1428 bytes --]

diff --git a/arch/x86/entry/syscall_64.c b/arch/x86/entry/syscall_64.c
index 71f032504e73..337e3e53d262 100644
--- a/arch/x86/entry/syscall_64.c
+++ b/arch/x86/entry/syscall_64.c
@@ -9,6 +9,14 @@
 #include <linux/nospec.h>
 #include <asm/syscall.h>
 
+#ifdef CONFIG_CALL_PADDING
+# define _pfe(x) __attribute((patchable_function_entry(x,x)))
+#else
+# define _pfe(x)
+#endif
+#define _align_func(x) __aligned(x) _pfe(x-CONFIG_FUNCTION_ALIGNMENT+CONFIG_FUNCTION_PADDING_BYTES)
+#define align_func(x) _align_func((x) < CONFIG_FUNCTION_ALIGNMENT ? CONFIG_FUNCTION_ALIGNMENT : (x))
+
 #define __SYSCALL(nr, sym) extern long __x64_##sym(const struct pt_regs *);
 #define __SYSCALL_NORETURN(nr, sym) extern long __noreturn __x64_##sym(const struct pt_regs *);
 #include <asm/syscalls_64.h>
@@ -32,7 +40,7 @@ const sys_call_ptr_t sys_call_table[] = {
 #undef  __SYSCALL
 
 #define __SYSCALL(nr, sym) case nr: return __x64_##sym(regs);
-long x64_sys_call(const struct pt_regs *regs, unsigned int nr)
+long align_func(32) x64_sys_call(const struct pt_regs *regs, unsigned int nr)
 {
 	switch (nr) {
 	#include <asm/syscalls_64.h>
@@ -41,7 +49,7 @@ long x64_sys_call(const struct pt_regs *regs, unsigned int nr)
 }
 
 #ifdef CONFIG_X86_X32_ABI
-long x32_sys_call(const struct pt_regs *regs, unsigned int nr)
+long align_func(32) x32_sys_call(const struct pt_regs *regs, unsigned int nr)
 {
 	switch (nr) {
 	#include <asm/syscalls_x32.h>