* Re: [patch 02/18] randomize_kstack: Provide add_random_kstack_offset_irqsoff()
From: Mukesh Kumar Chaurasiya @ 2026-07-09 18:34 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, Peter Zijlstra, Kees Cook, Michael Ellerman,
Shrikanth Hegde, linuxppc-dev, Huacai Chen, loongarch,
Paul Walmsley, Palmer Dabbelt, linux-riscv, Sven Schnelle,
linux-s390, x86, Mark Rutland, Jinjie Ruan, Andy Lutomirski,
Oleg Nesterov, Richard Henderson, Russell King, Catalin Marinas,
Guo Ren, Geert Uytterhoeven, Thomas Bogendoerfer, Helge Deller,
Yoshinori Sato, Richard Weinberger, Chris Zankel,
linux-arm-kernel, linux-alpha, linux-csky, linux-m68k, linux-mips,
linux-parisc, linux-sh, linux-um, Arnd Bergmann, Vineet Gupta,
Will Deacon, Brian Cain, Michal Simek, Dinh Nguyen,
David S. Miller, Andreas Larsson, linux-snps-arc, linux-hexagon,
linux-openrisc, sparclinux, linux-arch, Michal Suchánek,
Jonathan Corbet, linux-doc
In-Reply-To: <20260707190253.768842729@kernel.org>
On Tue, Jul 07, 2026 at 09:06:02PM +0200, Thomas Gleixner wrote:
> add_random_kstack_offset() uses get/put_cpu_var() which is pointless
> overhead when it is invoked from low level entry code with interrupts
> disabled.
>
> Provide a irqsoff() variant, which avoids that.
>
> Signed-off-by: Thomas Gleixner <tglx@kernel.org>
> Cc: Kees Cook <kees@kernel.org>
> ---
> include/linux/randomize_kstack.h | 19 +++++++++++++++++++
> 1 file changed, 19 insertions(+)
>
> --- a/include/linux/randomize_kstack.h
> +++ b/include/linux/randomize_kstack.h
> @@ -77,8 +77,27 @@ static __always_inline u32 get_kstack_of
> } \
> } while (0)
>
> +/**
> + * add_random_kstack_offset_irqsoff - Increase stack utilization by a random offset.
> + *
> + * This should be used in the syscall entry path after user registers have been
> + * stored to the stack. Interrupts must be still disabled.
> + */
> +#define add_random_kstack_offset_irqsoff() \
> +do { \
> + lockdep_assert_irqs_disabled(); \
> + if (static_branch_maybe(CONFIG_RANDOMIZE_KSTACK_OFFSET_DEFAULT, \
> + &randomize_kstack_offset)) { \
> + u32 offset = prandom_u32_state(raw_cpu_ptr(&kstack_rnd_state)); \
> + u8 *ptr = __kstack_alloca(KSTACK_OFFSET_MAX(offset)); \
> + /* Keep allocation even after "ptr" loses scope. */ \
> + asm volatile("" :: "r"(ptr) : "memory"); \
> + } \
> +} while (0)
> +
> #else /* CONFIG_RANDOMIZE_KSTACK_OFFSET */
> #define add_random_kstack_offset() do { } while (0)
> +#define add_random_kstack_offset_irqsoff() do { } while (0)
> #endif /* CONFIG_RANDOMIZE_KSTACK_OFFSET */
>
> #endif
>
LGTM.
Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
>
> _______________________________________________
> linux-snps-arc mailing list
> linux-snps-arc@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-snps-arc
^ permalink raw reply
* Re: [patch 04/18] loongarch/syscall: Use syscall_enter_from_user_mode_randomize_stack()
From: Mukesh Kumar Chaurasiya @ 2026-07-09 18:40 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, Peter Zijlstra, Huacai Chen, loongarch, Michael Ellerman,
Shrikanth Hegde, linuxppc-dev, Kees Cook, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Sven Schnelle, linux-s390, x86,
Mark Rutland, Jinjie Ruan, Andy Lutomirski, Oleg Nesterov,
Richard Henderson, Russell King, Catalin Marinas, Guo Ren,
Geert Uytterhoeven, Thomas Bogendoerfer, Helge Deller,
Yoshinori Sato, Richard Weinberger, Chris Zankel,
linux-arm-kernel, linux-alpha, linux-csky, linux-m68k, linux-mips,
linux-parisc, linux-sh, linux-um, Arnd Bergmann, Vineet Gupta,
Will Deacon, Brian Cain, Michal Simek, Dinh Nguyen,
David S. Miller, Andreas Larsson, linux-snps-arc, linux-hexagon,
linux-openrisc, sparclinux, linux-arch, Michal Suchánek,
Jonathan Corbet, linux-doc
In-Reply-To: <20260707190253.865955911@kernel.org>
On Tue, Jul 07, 2026 at 09:06:11PM +0200, Thomas Gleixner wrote:
> syscall_enter_from_user_mode_randomize_stack() replaces
> syscall_enter_from_user_mode() and the subsequent invocation of
> add_random_kstack_offset().
>
> The advantage is that it applies the stack randomization right after
> enter_from_user_mode() and thereby avoids the overhead of get/put_cpu_var()
> as that code is invoked with interrupts disabled.
>
> No functional change.
>
> Signed-off-by: Thomas Gleixner <tglx@kernel.org>
> Cc: Huacai Chen <chenhuacai@kernel.org>
> Cc: loongarch@lists.linux.dev
> ---
> arch/loongarch/kernel/syscall.c | 5 +----
> 1 file changed, 1 insertion(+), 4 deletions(-)
>
> --- a/arch/loongarch/kernel/syscall.c
> +++ b/arch/loongarch/kernel/syscall.c
> @@ -11,7 +11,6 @@
> #include <linux/linkage.h>
> #include <linux/nospec.h>
> #include <linux/objtool.h>
> -#include <linux/randomize_kstack.h>
> #include <linux/syscalls.h>
> #include <linux/unistd.h>
>
> @@ -70,9 +69,7 @@ void noinstr __no_stack_protector do_sys
> regs->orig_a0 = regs->regs[4];
> regs->regs[4] = -ENOSYS;
>
> - nr = syscall_enter_from_user_mode(regs, nr);
> -
> - add_random_kstack_offset();
> + nr = syscall_enter_from_user_mode_randomize_stack(regs, nr);
>
> if (nr < NR_syscalls) {
> syscall_fn = sys_call_table[array_index_nospec(nr, NR_syscalls)];
>
>
Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
> _______________________________________________
> linux-snps-arc mailing list
> linux-snps-arc@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-snps-arc
^ permalink raw reply
* Re: [patch 05/18] powerpc/syscall: Use syscall_enter_from_user_mode_randomize_stack()
From: Mukesh Kumar Chaurasiya @ 2026-07-09 18:41 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, Peter Zijlstra, Michael Ellerman, Shrikanth Hegde,
linuxppc-dev, Kees Cook, Huacai Chen, loongarch, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Sven Schnelle, linux-s390, x86,
Mark Rutland, Jinjie Ruan, Andy Lutomirski, Oleg Nesterov,
Richard Henderson, Russell King, Catalin Marinas, Guo Ren,
Geert Uytterhoeven, Thomas Bogendoerfer, Helge Deller,
Yoshinori Sato, Richard Weinberger, Chris Zankel,
linux-arm-kernel, linux-alpha, linux-csky, linux-m68k, linux-mips,
linux-parisc, linux-sh, linux-um, Arnd Bergmann, Vineet Gupta,
Will Deacon, Brian Cain, Michal Simek, Dinh Nguyen,
David S. Miller, Andreas Larsson, linux-snps-arc, linux-hexagon,
linux-openrisc, sparclinux, linux-arch, Michal Suchánek,
Jonathan Corbet, linux-doc
In-Reply-To: <20260707190253.918861529@kernel.org>
On Tue, Jul 07, 2026 at 09:06:15PM +0200, Thomas Gleixner wrote:
> syscall_enter_from_user_mode_randomize_stack() replaces
> syscall_enter_from_user_mode() and the subsequent invocation of
> add_random_kstack_offset().
>
> The advantage is that it applies the stack randomization right after
> enter_from_user_mode() and thereby avoids the overhead of get/put_cpu_var()
> as that code is invoked with interrupts disabled.
>
> No functional change.
>
> Signed-off-by: Thomas Gleixner <tglx@kernel.org>
> Cc: Michael Ellerman <mpe@ellerman.id.au>
> Cc: Shrikanth Hegde <sshegde@linux.ibm.com>
> Cc: linuxppc-dev@lists.ozlabs.org
> ---
> arch/powerpc/kernel/syscall.c | 4 +---
> 1 file changed, 1 insertion(+), 3 deletions(-)
>
> --- a/arch/powerpc/kernel/syscall.c
> +++ b/arch/powerpc/kernel/syscall.c
> @@ -2,7 +2,6 @@
>
> #include <linux/compat.h>
> #include <linux/context_tracking.h>
> -#include <linux/randomize_kstack.h>
> #include <linux/entry-common.h>
>
> #include <asm/interrupt.h>
> @@ -19,8 +18,7 @@ notrace long system_call_exception(struc
> long ret;
> syscall_fn f;
>
> - r0 = syscall_enter_from_user_mode(regs, r0);
> - add_random_kstack_offset();
> + r0 = syscall_enter_from_user_mode_randomize_stack(regs, r0);
>
> if (unlikely(r0 >= NR_syscalls)) {
> if (unlikely(trap_is_unsupported_scv(regs))) {
>
Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
^ permalink raw reply
* Re: [patch 06/18] riscv/syscall: Use syscall_enter_from_user_mode_randomize_stack()
From: Mukesh Kumar Chaurasiya @ 2026-07-09 18:42 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, Peter Zijlstra, Paul Walmsley, Palmer Dabbelt, linux-riscv,
Michael Ellerman, Shrikanth Hegde, linuxppc-dev, Kees Cook,
Huacai Chen, loongarch, Sven Schnelle, linux-s390, x86,
Mark Rutland, Jinjie Ruan, Andy Lutomirski, Oleg Nesterov,
Richard Henderson, Russell King, Catalin Marinas, Guo Ren,
Geert Uytterhoeven, Thomas Bogendoerfer, Helge Deller,
Yoshinori Sato, Richard Weinberger, Chris Zankel,
linux-arm-kernel, linux-alpha, linux-csky, linux-m68k, linux-mips,
linux-parisc, linux-sh, linux-um, Arnd Bergmann, Vineet Gupta,
Will Deacon, Brian Cain, Michal Simek, Dinh Nguyen,
David S. Miller, Andreas Larsson, linux-snps-arc, linux-hexagon,
linux-openrisc, sparclinux, linux-arch, Michal Suchánek,
Jonathan Corbet, linux-doc
In-Reply-To: <20260707190253.974626922@kernel.org>
On Tue, Jul 07, 2026 at 09:06:19PM +0200, Thomas Gleixner wrote:
> syscall_enter_from_user_mode_randomize_stack() replaces
> syscall_enter_from_user_mode() and the subsequent invocation of
> add_random_kstack_offset().
>
> The advantage is that it applies the stack randomization right after
> enter_from_user_mode() and thereby avoids the overhead of get/put_cpu_var()
> as that code is invoked with interrupts disabled.
>
> No functional change.
>
> Signed-off-by: Thomas Gleixner <tglx@kernel.org>
> Cc: Paul Walmsley <pjw@kernel.org>
> Cc: Palmer Dabbelt <palmer@dabbelt.com>
> Cc: linux-riscv@lists.infradead.org
> ---
> arch/riscv/kernel/traps.c | 5 +----
> 1 file changed, 1 insertion(+), 4 deletions(-)
>
> --- a/arch/riscv/kernel/traps.c
> +++ b/arch/riscv/kernel/traps.c
> @@ -7,7 +7,6 @@
> #include <linux/kernel.h>
> #include <linux/init.h>
> #include <linux/irqflags.h>
> -#include <linux/randomize_kstack.h>
> #include <linux/sched.h>
> #include <linux/sched/debug.h>
> #include <linux/sched/signal.h>
> @@ -333,9 +332,7 @@ void do_trap_ecall_u(struct pt_regs *reg
>
> riscv_v_vstate_discard(regs);
>
> - syscall = syscall_enter_from_user_mode(regs, syscall);
> -
> - add_random_kstack_offset();
> + syscall = syscall_enter_from_user_mode_randomize_stack(regs, syscall);
>
> if (syscall >= 0 && syscall < NR_syscalls) {
> syscall = array_index_nospec(syscall, NR_syscalls);
>
Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
^ permalink raw reply
* Re: [patch 07/18] s390/syscall: Use enter_from_user_mode_randomize_stack()
From: Mukesh Kumar Chaurasiya @ 2026-07-09 18:43 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, Peter Zijlstra, Sven Schnelle, linux-s390, Michael Ellerman,
Shrikanth Hegde, linuxppc-dev, Kees Cook, Huacai Chen, loongarch,
Paul Walmsley, Palmer Dabbelt, linux-riscv, x86, Mark Rutland,
Jinjie Ruan, Andy Lutomirski, Oleg Nesterov, Richard Henderson,
Russell King, Catalin Marinas, Guo Ren, Geert Uytterhoeven,
Thomas Bogendoerfer, Helge Deller, Yoshinori Sato,
Richard Weinberger, Chris Zankel, linux-arm-kernel, linux-alpha,
linux-csky, linux-m68k, linux-mips, linux-parisc, linux-sh,
linux-um, Arnd Bergmann, Vineet Gupta, Will Deacon, Brian Cain,
Michal Simek, Dinh Nguyen, David S. Miller, Andreas Larsson,
linux-snps-arc, linux-hexagon, linux-openrisc, sparclinux,
linux-arch, Michal Suchánek, Jonathan Corbet, linux-doc
In-Reply-To: <20260707190254.030598804@kernel.org>
On Tue, Jul 07, 2026 at 09:06:23PM +0200, Thomas Gleixner wrote:
> enter_from_user_mode_randomize_stack() replaces enter_from_user_mode() and
> the subsequent invocation of add_random_kstack_offset_irqsoff().
>
> As a bonus this avoids the overhead of get/put_cpu_var() in
> add_random_kstack_offset().
>
> No functional change.
>
> Signed-off-by: Thomas Gleixner <tglx@kernel.org>
> Cc: Sven Schnelle <svens@linux.ibm.com>
> Cc: linux-s390@vger.kernel.org
> ---
> arch/s390/kernel/syscall.c | 4 ++--
> 1 file changed, 2 insertions(+), 2 deletions(-)
>
> --- a/arch/s390/kernel/syscall.c
> +++ b/arch/s390/kernel/syscall.c
> @@ -97,8 +97,8 @@ void noinstr __do_syscall(struct pt_regs
> {
> unsigned long nr;
>
> - enter_from_user_mode(regs);
> - add_random_kstack_offset();
> + enter_from_user_mode_randomize_stack(regs);
> +
> regs->psw = get_lowcore()->svc_old_psw;
> regs->int_code = get_lowcore()->svc_int_code;
> update_timer_sys();
>
Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
^ permalink raw reply
* Re: [patch 08/18] x86/syscall: Use [syscall_]enter_from_user_mode_randomize_stack()
From: Mukesh Kumar Chaurasiya @ 2026-07-09 18:45 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, Peter Zijlstra, x86, Michael Ellerman, Shrikanth Hegde,
linuxppc-dev, Kees Cook, Huacai Chen, loongarch, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Sven Schnelle, linux-s390,
Mark Rutland, Jinjie Ruan, Andy Lutomirski, Oleg Nesterov,
Richard Henderson, Russell King, Catalin Marinas, Guo Ren,
Geert Uytterhoeven, Thomas Bogendoerfer, Helge Deller,
Yoshinori Sato, Richard Weinberger, Chris Zankel,
linux-arm-kernel, linux-alpha, linux-csky, linux-m68k, linux-mips,
linux-parisc, linux-sh, linux-um, Arnd Bergmann, Vineet Gupta,
Will Deacon, Brian Cain, Michal Simek, Dinh Nguyen,
David S. Miller, Andreas Larsson, linux-snps-arc, linux-hexagon,
linux-openrisc, sparclinux, linux-arch, Michal Suchánek,
Jonathan Corbet, linux-doc
In-Reply-To: <20260707190254.079478122@kernel.org>
On Tue, Jul 07, 2026 at 09:06:27PM +0200, Thomas Gleixner wrote:
> These functions integrate the stack randomization.
>
> syscall_enter_from_user_mode_randomize_stack() has the advantage that the
> randomization happens early right after enter_from_user_mode().
>
> In both cases also the overhead of get/put_cpu_var() in
> add_random_kstack_offset() is avoided.
>
> No functional change.
>
> Signed-off-by: Thomas Gleixner <tglx@kernel.org>
> Cc: x86@kernel.org
> ---
> arch/x86/entry/syscall_32.c | 19 +++++--------------
> arch/x86/entry/syscall_64.c | 3 +--
> arch/x86/include/asm/entry-common.h | 1 -
> 3 files changed, 6 insertions(+), 17 deletions(-)
>
> --- a/arch/x86/entry/syscall_32.c
> +++ b/arch/x86/entry/syscall_32.c
> @@ -142,10 +142,9 @@ static __always_inline bool int80_is_ext
> * int80_is_external() below which calls into the APIC driver.
> * Identical for soft and external interrupts.
> */
> - enter_from_user_mode(regs);
> + enter_from_user_mode_randomize_stack(regs);
>
> instrumentation_begin();
> - add_random_kstack_offset();
>
> /* Validate that this is a soft interrupt to the extent possible */
> if (unlikely(int80_is_external()))
> @@ -210,11 +209,9 @@ DEFINE_FREDENTRY_RAW(int80_emulation)
> {
> int nr;
>
> - enter_from_user_mode(regs);
> + enter_from_user_mode_randomize_stack(regs);
>
> instrumentation_begin();
> - add_random_kstack_offset();
> -
> /*
> * FRED pushed 0 into regs::orig_ax and regs::ax contains the
> * syscall number.
> @@ -252,10 +249,10 @@ DEFINE_FREDENTRY_RAW(int80_emulation)
> * orig_ax, the int return value truncates it. This matches
> * the semantics of syscall_get_nr().
> */
> - nr = syscall_enter_from_user_mode(regs, nr);
> + nr = syscall_enter_from_user_mode_randomize_stack(regs, nr);
> +
> instrumentation_begin();
>
> - add_random_kstack_offset();
> do_syscall_32_irqs_on(regs, nr);
>
> instrumentation_end();
> @@ -268,15 +265,9 @@ static noinstr bool __do_fast_syscall_32
> int nr = syscall_32_enter(regs);
> int res;
>
> - /*
> - * This cannot use syscall_enter_from_user_mode() as it has to
> - * fetch EBP before invoking any of the syscall entry work
> - * functions.
> - */
> - enter_from_user_mode(regs);
> + enter_from_user_mode_randomize_stack(regs);
>
> instrumentation_begin();
> - add_random_kstack_offset();
> local_irq_enable();
> /* Fetch EBP from where the vDSO stashed it. */
> if (IS_ENABLED(CONFIG_X86_64)) {
> --- a/arch/x86/entry/syscall_64.c
> +++ b/arch/x86/entry/syscall_64.c
> @@ -86,10 +86,9 @@ static __always_inline bool do_syscall_x
> /* Returns true to return using SYSRET, or false to use IRET */
> __visible noinstr bool do_syscall_64(struct pt_regs *regs, int nr)
> {
> - nr = syscall_enter_from_user_mode(regs, nr);
> + nr = syscall_enter_from_user_mode_randomize_stack(regs, nr);
>
> instrumentation_begin();
> - add_random_kstack_offset();
>
> if (!do_syscall_x64(regs, nr) && !do_syscall_x32(regs, nr) && nr != -1) {
> /* Invalid system call, but still a system call. */
> --- a/arch/x86/include/asm/entry-common.h
> +++ b/arch/x86/include/asm/entry-common.h
> @@ -2,7 +2,6 @@
> #ifndef _ASM_X86_ENTRY_COMMON_H
> #define _ASM_X86_ENTRY_COMMON_H
>
> -#include <linux/randomize_kstack.h>
> #include <linux/user-return-notifier.h>
>
> #include <asm/nospec-branch.h>
>
Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
^ permalink raw reply
* Re: [patch 09/18] entry: Remove syscall_enter_from_user_mode()
From: Mukesh Kumar Chaurasiya @ 2026-07-09 18:49 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, Peter Zijlstra, Michael Ellerman, Shrikanth Hegde,
linuxppc-dev, Kees Cook, Huacai Chen, loongarch, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Sven Schnelle, linux-s390, x86,
Mark Rutland, Jinjie Ruan, Andy Lutomirski, Oleg Nesterov,
Richard Henderson, Russell King, Catalin Marinas, Guo Ren,
Geert Uytterhoeven, Thomas Bogendoerfer, Helge Deller,
Yoshinori Sato, Richard Weinberger, Chris Zankel,
linux-arm-kernel, linux-alpha, linux-csky, linux-m68k, linux-mips,
linux-parisc, linux-sh, linux-um, Arnd Bergmann, Vineet Gupta,
Will Deacon, Brian Cain, Michal Simek, Dinh Nguyen,
David S. Miller, Andreas Larsson, linux-snps-arc, linux-hexagon,
linux-openrisc, sparclinux, linux-arch, Michal Suchánek,
Jonathan Corbet, linux-doc
In-Reply-To: <20260707190254.132654198@kernel.org>
On Tue, Jul 07, 2026 at 09:06:32PM +0200, Thomas Gleixner wrote:
> All architecture use either:
>
> nr = enter_from_user_mode_randomize_stack(regs, nr);
>
> or
>
> enter_from_user_mode_randomize_stack(regs);
> nr = syscall_enter_from_user_mode_work(regs, nr);
>
> Remove the now unused function.
>
> Signed-off-by: Thomas Gleixner <tglx@kernel.org>
> ---
> Documentation/core-api/entry.rst | 17 +++++++++-------
> include/linux/entry-common.h | 40 +++------------------------------------
> include/linux/irq-entry-common.h | 6 ++---
> 3 files changed, 17 insertions(+), 46 deletions(-)
>
> --- a/Documentation/core-api/entry.rst
> +++ b/Documentation/core-api/entry.rst
> @@ -68,7 +68,7 @@ low-level C code must not be instrumente
> noinstr void syscall(struct pt_regs *regs, int nr)
> {
> arch_syscall_enter(regs);
> - nr = syscall_enter_from_user_mode(regs, nr);
> + nr = syscall_enter_from_user_mode_randomize_stack(regs, nr);
>
> instrumentation_begin();
> if (!invoke_syscall(regs, nr) && nr != -1)
> @@ -78,12 +78,14 @@ low-level C code must not be instrumente
> syscall_exit_to_user_mode(regs);
> }
>
> -syscall_enter_from_user_mode() first invokes enter_from_user_mode() which
> -establishes state in the following order:
> +syscall_enter_from_user_mode_randomize_stack() first invokes
> +enter_from_user_mode_randomize_stack() which establishes state in the
> +following order:
>
> * Lockdep
> * RCU / Context tracking
> * Tracing
> + * Apply stack randomization
>
> and then invokes the various entry work functions like ptrace, seccomp, audit,
> syscall tracing, etc. After all that is done, the instrumentable invoke_syscall
> @@ -99,10 +101,11 @@ that it invokes exit_to_user_mode() whic
> * RCU / Context tracking
> * Lockdep
>
> -syscall_enter_from_user_mode() and syscall_exit_to_user_mode() are also
> -available as fine grained subfunctions in cases where the architecture code
> -has to do extra work between the various steps. In such cases it has to
> -ensure that enter_from_user_mode() is called first on entry and
> +syscall_enter_from_user_mode_randomize_stack() and
> +syscall_exit_to_user_mode() are also available as fine grained subfunctions
> +in cases where the architecture code has to do extra work between the
> +various steps. In such cases it has to ensure that
> +enter_from_user_mode_randomize_stack() is called first on entry and
> exit_to_user_mode() is called last on exit.
>
> Do not nest syscalls. Nested syscalls will cause RCU and/or context tracking
> --- a/include/linux/entry-common.h
> +++ b/include/linux/entry-common.h
> @@ -19,7 +19,7 @@
> #endif
>
> /*
> - * SYSCALL_WORK flags handled in syscall_enter_from_user_mode()
> + * SYSCALL_WORK flags handled in syscall_enter_from_user_mode_work()
> */
> #define SYSCALL_WORK_ENTER (SYSCALL_WORK_SECCOMP | \
> SYSCALL_WORK_SYSCALL_TRACEPOINT | \
> @@ -205,42 +205,10 @@ do { \
> _ret; \
> })
>
> -/**
> - * syscall_enter_from_user_mode - Establish state and check and handle work
> - * before invoking a syscall
> - * @regs: Pointer to currents pt_regs
> - * @syscall: The syscall number
> - *
> - * Invoked from architecture specific syscall entry code with interrupts
> - * disabled. The calling code has to be non-instrumentable. When the
> - * function returns all state is correct, interrupts are enabled and the
> - * subsequent functions can be instrumented.
> - *
> - * This is the combination of enter_from_user_mode() and
> - * syscall_enter_from_user_mode_work() to be used when there is no
> - * architecture specific work to be done between the two.
> - *
> - * Returns: The original or a modified syscall number. See
> - * syscall_enter_from_user_mode_work() for further explanation.
> - */
> -static __always_inline long syscall_enter_from_user_mode(struct pt_regs *regs, long syscall)
> -{
> - long ret;
> -
> - enter_from_user_mode(regs);
> -
> - instrumentation_begin();
> - local_irq_enable();
> - ret = syscall_enter_from_user_mode_work(regs, syscall);
> - instrumentation_end();
> -
> - return ret;
> -}
> -
> /*
> - * If SYSCALL_EMU is set, then the only reason to report is when
> - * SINGLESTEP is set (i.e. PTRACE_SYSEMU_SINGLESTEP). This syscall
> - * instruction has been already reported in syscall_enter_from_user_mode().
> + * If SYSCALL_EMU is set, then the only reason to report is when SINGLESTEP is
> + * set (i.e. PTRACE_SYSEMU_SINGLESTEP). This syscall instruction has been
> + * already reported in syscall_enter_from_user_mode_work().
> */
> static __always_inline bool report_single_step(unsigned long work)
> {
> --- a/include/linux/irq-entry-common.h
> +++ b/include/linux/irq-entry-common.h
> @@ -49,9 +49,9 @@
> * Defaults to an empty implementation. Can be replaced by architecture
> * specific code.
> *
> - * Invoked from syscall_enter_from_user_mode() in the non-instrumentable
> - * section. Use __always_inline so the compiler cannot push it out of line
> - * and make it instrumentable.
> + * Invoked from enter_from_user_mode_syscall_and_randomize_stack() in the
> + * non-instrumentable section. Use __always_inline so the compiler cannot push
> + * it out of line and make it instrumentable.
> */
> static __always_inline void arch_enter_from_user_mode(struct pt_regs *regs);
>
>
Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
^ permalink raw reply
* Re: [patch 10/18] entry: Use syscall number instead of rereading it
From: Mukesh Kumar Chaurasiya @ 2026-07-09 18:50 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, Peter Zijlstra, Michael Ellerman, Shrikanth Hegde,
linuxppc-dev, Kees Cook, Huacai Chen, loongarch, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Sven Schnelle, linux-s390, x86,
Mark Rutland, Jinjie Ruan, Andy Lutomirski, Oleg Nesterov,
Richard Henderson, Russell King, Catalin Marinas, Guo Ren,
Geert Uytterhoeven, Thomas Bogendoerfer, Helge Deller,
Yoshinori Sato, Richard Weinberger, Chris Zankel,
linux-arm-kernel, linux-alpha, linux-csky, linux-m68k, linux-mips,
linux-parisc, linux-sh, linux-um, Arnd Bergmann, Vineet Gupta,
Will Deacon, Brian Cain, Michal Simek, Dinh Nguyen,
David S. Miller, Andreas Larsson, linux-snps-arc, linux-hexagon,
linux-openrisc, sparclinux, linux-arch, Michal Suchánek,
Jonathan Corbet, linux-doc
In-Reply-To: <20260707190254.181086755@kernel.org>
On Tue, Jul 07, 2026 at 09:06:36PM +0200, Thomas Gleixner wrote:
> rseq_syscall_enter_work() is invoked before the syscall number can be
> modified. So there is no point in rereading it from pt_regs.
>
> Signed-off-by: Thomas Gleixner <tglx@kernel.org>
> ---
> include/linux/entry-common.h | 9 +++++----
> 1 file changed, 5 insertions(+), 4 deletions(-)
>
> --- a/include/linux/entry-common.h
> +++ b/include/linux/entry-common.h
> @@ -70,9 +70,10 @@ static inline void syscall_enter_audit(s
> }
> }
>
> -static __always_inline long syscall_trace_enter(struct pt_regs *regs, unsigned long work)
> +static __always_inline long syscall_trace_enter(struct pt_regs *regs, unsigned long work,
> + long syscall)
> {
> - long syscall, ret = 0;
> + long ret = 0;
>
> /*
> * Handle Syscall User Dispatch. This must comes first, since
> @@ -90,7 +91,7 @@ static __always_inline long syscall_trac
> * through hrtimer_interrupt().
> */
> if (work & SYSCALL_WORK_SYSCALL_RSEQ_SLICE)
> - rseq_syscall_enter_work(syscall_get_nr(current, regs));
> + rseq_syscall_enter_work(syscall);
>
> /* Handle ptrace */
> if (work & (SYSCALL_WORK_SYSCALL_TRACE | SYSCALL_WORK_SYSCALL_EMU)) {
> @@ -145,7 +146,7 @@ static __always_inline long syscall_ente
> unsigned long work = READ_ONCE(current_thread_info()->syscall_work);
>
> if (work & SYSCALL_WORK_ENTER)
> - syscall = syscall_trace_enter(regs, work);
> + syscall = syscall_trace_enter(regs, work, syscall);
>
> return syscall;
> }
>
Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
^ permalink raw reply
* [PATCH v2 0/2] PCI: Add boot options to disable DPC and AER recovery
From: Yury Murashka @ 2026-07-09 18:54 UTC (permalink / raw)
To: bhelgaas
Cc: corbet, skhan, mahesh, oohall, linux-pci, linux-doc, linux-kernel,
linuxppc-dev, Yury Murashka
On large modular systems with a complex PCIe tree, the default kernel
AER recovery and DPC behavior could cause unexpected side effects.
Sometimes it would be nice to have the option to keep the system in an
unmodified state and be able to handle PCIe errors from userspace.
This series adds two new PCI kernel boot options:
pci=noaer_recovery - Disable AER error recovery while still logging
AER errors
pci=nodpc - Disable PCI Downstream Port Containment entirely
---
v1 -> v2:
- Patch 1: Move disable check from pcie_do_recovery() in err.c to
pci_aer_handle_error()/aer_recover_work_func() in aer.c for
proper AER-specific scoping. AER error status bits are now
preserved (not cleared) so userspace can inspect the error state
- Patch 2: Also skip DPC port service driver registration in
pcie_dpc_init() when disabled
- Rebase onto v7.2-rc2
v1: https://lore.kernel.org/linux-pci/20260602105558.1799563-1-yurypm@arista.com/
Yury Murashka (2):
PCI: Add pci=noaer_recovery kernel boot option
PCI: Add pci=nodpc kernel boot option
.../admin-guide/kernel-parameters.txt | 9 ++++++
drivers/pci/pci.c | 4 +++
drivers/pci/pci.h | 4 +++
drivers/pci/pcie/aer.c | 30 ++++++++++++-------
drivers/pci/pcie/dpc.c | 19 ++++++++++--
5 files changed, 53 insertions(+), 13 deletions(-)
base-commit: 0e35b9b6ec0ffcc5e23cbdec09f5c622ad532b53
--
2.51.0
^ permalink raw reply
* [PATCH v2 1/2] PCI: Add pci=noaer_recovery kernel boot option
From: Yury Murashka @ 2026-07-09 18:54 UTC (permalink / raw)
To: bhelgaas
Cc: corbet, skhan, mahesh, oohall, linux-pci, linux-doc, linux-kernel,
linuxppc-dev, Yury Murashka
In-Reply-To: <20260709185429.627968-1-yurypm@arista.com>
AER error recovery is part of the AER error handling subsystem in the
Linux kernel. On large modular systems with a complex PCIe tree, AER
recovery could cause unexpected behavior and side effects. Sometimes it
would be nice to have the option to keep the system in an unmodified
state and be able to handle PCIe errors from userspace.
Add pci=noaer_recovery kernel boot option to disable AER error recovery
when an uncorrectable error is reported. When this option is set, the
error is still logged but no recovery actions are taken. AER error
status bits are preserved so userspace can inspect the error state.
Signed-off-by: Yury Murashka <yurypm@arista.com>
---
.../admin-guide/kernel-parameters.txt | 6 ++++
drivers/pci/pci.c | 2 ++
drivers/pci/pci.h | 2 ++
drivers/pci/pcie/aer.c | 30 ++++++++++++-------
4 files changed, 30 insertions(+), 10 deletions(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index b5493a7f8f22..13c6b53bb9ee 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -5053,6 +5053,12 @@ Kernel parameters
noaer [PCIE] If the PCIEAER kernel config parameter is
enabled, this kernel boot option can be used to
disable the use of PCIE advanced error reporting.
+ noaer_recovery [PCIE] If the PCIEAER kernel config parameter is
+ enabled, this kernel boot option can be used to
+ disable AER error recovery when an uncorrectable
+ error is reported. AER error status bits are
+ preserved so userspace can inspect the error
+ state.
nodomains [PCI] Disable support for multiple PCI
root domains (aka PCI segments, in ACPI-speak).
nommconf [X86] Disable use of MMCONFIG for PCI
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index 77b17b13ee61..a215dd567d5d 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -6756,6 +6756,8 @@ static int __init pci_setup(char *str)
pcie_ats_disabled = true;
} else if (!strcmp(str, "noaer")) {
pci_no_aer();
+ } else if (!strcmp(str, "noaer_recovery")) {
+ pci_no_aer_recovery();
} else if (!strcmp(str, "earlydump")) {
pci_early_dump = true;
} else if (!strncmp(str, "realloc=", 8)) {
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index 4469e1a77f3c..c4a42bbc277b 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -1283,6 +1283,7 @@ static inline void of_pci_remove_host_bridge_node(struct pci_host_bridge *bridge
#ifdef CONFIG_PCIEAER
void pci_no_aer(void);
+void pci_no_aer_recovery(void);
void pci_aer_init(struct pci_dev *dev);
void pci_aer_exit(struct pci_dev *dev);
extern const struct attribute_group aer_stats_attr_group;
@@ -1294,6 +1295,7 @@ void pci_save_aer_state(struct pci_dev *dev);
void pci_restore_aer_state(struct pci_dev *dev);
#else
static inline void pci_no_aer(void) { }
+static inline void pci_no_aer_recovery(void) { }
static inline void pci_aer_init(struct pci_dev *d) { }
static inline void pci_aer_exit(struct pci_dev *d) { }
static inline void pci_aer_clear_fatal_status(struct pci_dev *dev) { }
diff --git a/drivers/pci/pcie/aer.c b/drivers/pci/pcie/aer.c
index c4fd9c0b2a54..fb79990ce8ff 100644
--- a/drivers/pci/pcie/aer.c
+++ b/drivers/pci/pcie/aer.c
@@ -128,6 +128,7 @@ struct aer_info {
PCI_ERR_ROOT_MULTI_UNCOR_RCV)
static bool pcie_aer_disable;
+static bool pcie_aer_recovery_disable;
static pci_ers_result_t aer_root_reset(struct pci_dev *dev);
void pci_no_aer(void)
@@ -135,6 +136,11 @@ void pci_no_aer(void)
pcie_aer_disable = true;
}
+void pci_no_aer_recovery(void)
+{
+ pcie_aer_recovery_disable = true;
+}
+
bool pci_aer_available(void)
{
return !pcie_aer_disable && pci_msi_enabled();
@@ -1187,10 +1193,12 @@ static void pci_aer_handle_error(struct pci_dev *dev, struct aer_err_info *info)
pdrv->err_handler->cor_error_detected(dev);
pcie_clear_device_status(dev);
}
- } else if (info->severity == AER_NONFATAL)
- pcie_do_recovery(dev, pci_channel_io_normal, aer_root_reset);
- else if (info->severity == AER_FATAL)
- pcie_do_recovery(dev, pci_channel_io_frozen, aer_root_reset);
+ } else if (!pcie_aer_recovery_disable) {
+ if (info->severity == AER_NONFATAL)
+ pcie_do_recovery(dev, pci_channel_io_normal, aer_root_reset);
+ else if (info->severity == AER_FATAL)
+ pcie_do_recovery(dev, pci_channel_io_frozen, aer_root_reset);
+ }
}
static void handle_error_source(struct pci_dev *dev, struct aer_err_info *info)
@@ -1242,12 +1250,14 @@ static void aer_recover_work_func(struct work_struct *work)
ghes_estatus_pool_region_free((unsigned long)entry.regs,
sizeof(struct aer_capability_regs));
- if (entry.severity == AER_NONFATAL)
- pcie_do_recovery(pdev, pci_channel_io_normal,
- aer_root_reset);
- else if (entry.severity == AER_FATAL)
- pcie_do_recovery(pdev, pci_channel_io_frozen,
- aer_root_reset);
+ if (!pcie_aer_recovery_disable) {
+ if (entry.severity == AER_NONFATAL)
+ pcie_do_recovery(pdev, pci_channel_io_normal,
+ aer_root_reset);
+ else if (entry.severity == AER_FATAL)
+ pcie_do_recovery(pdev, pci_channel_io_frozen,
+ aer_root_reset);
+ }
pci_dev_put(pdev);
}
}
--
2.51.0
^ permalink raw reply related
* [PATCH v2 2/2] PCI: Add pci=nodpc kernel boot option
From: Yury Murashka @ 2026-07-09 18:54 UTC (permalink / raw)
To: bhelgaas
Cc: corbet, skhan, mahesh, oohall, linux-pci, linux-doc, linux-kernel,
linuxppc-dev, Yury Murashka
In-Reply-To: <20260709185429.627968-1-yurypm@arista.com>
PCI DPC (Downstream Port Containment) support can be advertised by PCIe
devices, but it might not be fully supported in the firmware. On large
modular systems with a complex PCIe tree, enabling DPC could cause
unexpected behavior and side effects. Sometimes it would be nice to have
the option to keep the system in an unmodified state and be able to
handle PCIe errors from userspace.
Add pci=nodpc kernel boot option to disable PCI DPC. When this option
is set, DPC initialization, state save/restore, recovery, and driver
registration are all skipped.
Signed-off-by: Yury Murashka <yurypm@arista.com>
---
.../admin-guide/kernel-parameters.txt | 3 +++
drivers/pci/pci.c | 2 ++
drivers/pci/pci.h | 2 ++
drivers/pci/pcie/dpc.c | 19 ++++++++++++++++---
4 files changed, 23 insertions(+), 3 deletions(-)
diff --git a/Documentation/admin-guide/kernel-parameters.txt b/Documentation/admin-guide/kernel-parameters.txt
index 13c6b53bb9ee..ccc26849a1ae 100644
--- a/Documentation/admin-guide/kernel-parameters.txt
+++ b/Documentation/admin-guide/kernel-parameters.txt
@@ -5050,6 +5050,9 @@ Kernel parameters
through ports 0xC000-0xCFFF).
See http://wiki.osdev.org/PCI for more info
on the configuration access mechanisms.
+ nodpc [PCIE] If the PCIE_DPC kernel config parameter is
+ enabled, this kernel boot option can be used to
+ disable the use of PCIE DPC.
noaer [PCIE] If the PCIEAER kernel config parameter is
enabled, this kernel boot option can be used to
disable the use of PCIE advanced error reporting.
diff --git a/drivers/pci/pci.c b/drivers/pci/pci.c
index a215dd567d5d..6cd06a872ba0 100644
--- a/drivers/pci/pci.c
+++ b/drivers/pci/pci.c
@@ -6754,6 +6754,8 @@ static int __init pci_setup(char *str)
} else if (!strncmp(str, "noats", 5)) {
pr_info("PCIe: ATS is disabled\n");
pcie_ats_disabled = true;
+ } else if (!strcmp(str, "nodpc")) {
+ pci_no_dpc();
} else if (!strcmp(str, "noaer")) {
pci_no_aer();
} else if (!strcmp(str, "noaer_recovery")) {
diff --git a/drivers/pci/pci.h b/drivers/pci/pci.h
index c4a42bbc277b..146cd5df985a 100644
--- a/drivers/pci/pci.h
+++ b/drivers/pci/pci.h
@@ -880,6 +880,7 @@ struct rcec_ea {
#endif
#ifdef CONFIG_PCIE_DPC
+void pci_no_dpc(void);
void pci_save_dpc_state(struct pci_dev *dev);
void pci_restore_dpc_state(struct pci_dev *dev);
void pci_dpc_init(struct pci_dev *pdev);
@@ -888,6 +889,7 @@ pci_ers_result_t dpc_reset_link(struct pci_dev *pdev);
bool pci_dpc_recovered(struct pci_dev *pdev);
unsigned int dpc_tlp_log_len(struct pci_dev *dev);
#else
+static inline void pci_no_dpc(void) { }
static inline void pci_save_dpc_state(struct pci_dev *dev) { }
static inline void pci_restore_dpc_state(struct pci_dev *dev) { }
static inline void pci_dpc_init(struct pci_dev *pdev) { }
diff --git a/drivers/pci/pcie/dpc.c b/drivers/pci/pcie/dpc.c
index 2b779bd1d861..759d9f18812e 100644
--- a/drivers/pci/pcie/dpc.c
+++ b/drivers/pci/pcie/dpc.c
@@ -43,12 +43,19 @@ static const char * const rp_pio_error_string[] = {
"Memory Request Completion Timeout", /* Bit Position 18 */
};
+static bool pcie_dpc_disable;
+
+void pci_no_dpc(void)
+{
+ pcie_dpc_disable = true;
+}
+
void pci_save_dpc_state(struct pci_dev *dev)
{
struct pci_cap_saved_state *save_state;
u16 *cap;
- if (!pci_is_pcie(dev))
+ if (pcie_dpc_disable || !pci_is_pcie(dev))
return;
save_state = pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_DPC);
@@ -64,7 +71,7 @@ void pci_restore_dpc_state(struct pci_dev *dev)
struct pci_cap_saved_state *save_state;
u16 *cap;
- if (!pci_is_pcie(dev))
+ if (pcie_dpc_disable || !pci_is_pcie(dev))
return;
save_state = pci_find_saved_ext_cap(dev, PCI_EXT_CAP_ID_DPC);
@@ -104,7 +111,7 @@ bool pci_dpc_recovered(struct pci_dev *pdev)
{
struct pci_host_bridge *host;
- if (!pdev->dpc_cap)
+ if (pcie_dpc_disable || !pdev->dpc_cap)
return false;
/*
@@ -404,6 +411,9 @@ void pci_dpc_init(struct pci_dev *pdev)
{
u16 cap;
+ if (pcie_dpc_disable)
+ return;
+
pdev->dpc_cap = pci_find_ext_capability(pdev, PCI_EXT_CAP_ID_DPC);
if (!pdev->dpc_cap)
return;
@@ -532,5 +542,8 @@ static struct pcie_port_service_driver dpcdriver = {
int __init pcie_dpc_init(void)
{
+ if (pcie_dpc_disable)
+ return 0;
+
return pcie_port_service_register(&dpcdriver);
}
--
2.51.0
^ permalink raw reply related
* Re: [patch 11/18] seccomp, treewide: Rename and convert __secure_computing() to return boolean
From: Mukesh Kumar Chaurasiya @ 2026-07-09 19:10 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, Peter Zijlstra, Mark Rutland, Jinjie Ruan, Kees Cook,
Andy Lutomirski, Oleg Nesterov, Richard Henderson, Russell King,
Catalin Marinas, Guo Ren, Geert Uytterhoeven, Thomas Bogendoerfer,
Helge Deller, Yoshinori Sato, Richard Weinberger, Chris Zankel,
linux-arm-kernel, linux-alpha, linux-csky, linux-m68k, linux-mips,
linux-parisc, linux-sh, linux-um, Michael Ellerman,
Shrikanth Hegde, linuxppc-dev, Huacai Chen, loongarch,
Paul Walmsley, Palmer Dabbelt, linux-riscv, Sven Schnelle,
linux-s390, x86, Arnd Bergmann, Vineet Gupta, Will Deacon,
Brian Cain, Michal Simek, Dinh Nguyen, David S. Miller,
Andreas Larsson, linux-snps-arc, linux-hexagon, linux-openrisc,
sparclinux, linux-arch, Michal Suchánek, Jonathan Corbet,
linux-doc
In-Reply-To: <20260707190254.230735780@kernel.org>
On Tue, Jul 07, 2026 at 09:06:40PM +0200, Thomas Gleixner wrote:
> From: Jinjie Ruan <ruanjinjie@huawei.com>
>
> The return value of __secure_computing() currently uses 0 to indicate
> that a system call should be allowed, and -1 to indicate that it should
> be blocked/killed. This 0/-1 pattern is non-intuitive for a security
> check function and makes the control flow at the call sites less readable.
>
> Furthermore, any potential future changes to these return values would
> require a high-risk, error-prone audit of all its users across different
> architectures.
>
> Sanitize this logic by converting the return type of __secure_computing()
> to a proper boolean, where 'true' explicitly means 'allow' and 'false'
> means 'fail/deny'.
>
> Update all the two dozen or so call sites across the tree to align with
> this new boolean semantic. No functional changes are intended, as the
> callers still return -1 to the lower-level assembly entry code upon
> seccomp denial.
>
> Rename the function to __seccomp_permit_syscall() so that the purpose is
> entirely clear.
>
> [ tglx: Rename the function ]
>
> Suggested-by: Thomas Gleixner <tglx@kernel.org>
> Suggested-by: Mark Rutland <mark.rutland@arm.com>
> Signed-off-by: Jinjie Ruan <ruanjinjie@huawei.com>
> Signed-off-by: Thomas Gleixner <tglx@kernel.org>
> Cc: Kees Cook <kees@kernel.org>
> Cc: Andy Lutomirski <luto@kernel.org>
> Cc: Oleg Nesterov <oleg@redhat.com>
> Cc: Richard Henderson <richard.henderson@linaro.org>
> Cc: Russell King <linux@armlinux.org.uk>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Guo Ren <guoren@kernel.org>
> Cc: Geert Uytterhoeven <geert@linux-m68k.org>
> Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
> Cc: Helge Deller <deller@gmx.de>
> Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
> Cc: Richard Weinberger <richard@nod.at>
> Cc: Chris Zankel <chris@zankel.net>
> Cc: linux-arm-kernel@lists.infradead.org
> Cc: linux-alpha@vger.kernel.org
> Cc: linux-csky@vger.kernel.org
> Cc: linux-m68k@lists.linux-m68k.org
> Cc: linux-mips@vger.kernel.org
> Cc: linux-parisc@vger.kernel.org
> Cc: linux-sh@vger.kernel.org
> Cc: linux-um@lists.infradead.org
> ---
> arch/alpha/kernel/ptrace.c | 2 -
> arch/arm/kernel/ptrace.c | 2 -
> arch/arm64/kernel/ptrace.c | 2 -
> arch/csky/kernel/ptrace.c | 2 -
> arch/m68k/kernel/ptrace.c | 2 -
> arch/mips/kernel/ptrace.c | 2 -
> arch/parisc/kernel/ptrace.c | 2 -
> arch/sh/kernel/ptrace_32.c | 2 -
> arch/um/kernel/skas/syscall.c | 2 -
> arch/x86/entry/vsyscall/vsyscall_64.c | 14 ++++++-------
> arch/xtensa/kernel/ptrace.c | 3 --
> include/linux/entry-common.h | 9 +++-----
> include/linux/seccomp.h | 12 +++++------
> kernel/seccomp.c | 35 +++++++++++++++++-----------------
> 14 files changed, 45 insertions(+), 46 deletions(-)
> --- a/arch/alpha/kernel/ptrace.c
> +++ b/arch/alpha/kernel/ptrace.c
> @@ -387,7 +387,7 @@ asmlinkage unsigned long syscall_trace_e
> * If this fails, seccomp may already have set up the return value
> * (e.g. SECCOMP_RET_ERRNO / TRACE).
> */
> - if (secure_computing() == -1) {
> + if (!seccomp_permit_syscall()) {
> if (regs->r19 == 0 && regs->r0 == (unsigned long)-1)
> syscall_set_return_value(current, regs, -ENOSYS, 0);
> syscall_set_nr(current, regs, -1);
> --- a/arch/arm/kernel/ptrace.c
> +++ b/arch/arm/kernel/ptrace.c
> @@ -855,7 +855,7 @@ asmlinkage int syscall_trace_enter(struc
>
> /* Do seccomp after ptrace; syscall may have changed. */
> #ifdef CONFIG_HAVE_ARCH_SECCOMP_FILTER
> - if (secure_computing() == -1)
> + if (!seccomp_permit_syscall())
> return -1;
> #else
> /* XXX: remove this once OABI gets fixed */
> --- a/arch/arm64/kernel/ptrace.c
> +++ b/arch/arm64/kernel/ptrace.c
> @@ -2420,7 +2420,7 @@ int syscall_trace_enter(struct pt_regs *
> }
>
> /* Do the secure computing after ptrace; failures should be fast. */
> - if (secure_computing() == -1)
> + if (!seccomp_permit_syscall())
> return NO_SYSCALL;
>
> if (test_thread_flag(TIF_SYSCALL_TRACEPOINT))
> --- a/arch/csky/kernel/ptrace.c
> +++ b/arch/csky/kernel/ptrace.c
> @@ -323,7 +323,7 @@ asmlinkage int syscall_trace_enter(struc
> if (ptrace_report_syscall_entry(regs))
> return -1;
>
> - if (secure_computing() == -1)
> + if (!seccomp_permit_syscall())
> return -1;
>
> if (test_thread_flag(TIF_SYSCALL_TRACEPOINT))
> --- a/arch/m68k/kernel/ptrace.c
> +++ b/arch/m68k/kernel/ptrace.c
> @@ -281,7 +281,7 @@ asmlinkage int syscall_trace_enter(void)
> if (test_thread_flag(TIF_SYSCALL_TRACE))
> ret = ptrace_report_syscall_entry(task_pt_regs(current));
>
> - if (secure_computing() == -1)
> + if (!seccomp_permit_syscall())
> return -1;
>
> return ret;
> --- a/arch/mips/kernel/ptrace.c
> +++ b/arch/mips/kernel/ptrace.c
> @@ -1328,7 +1328,7 @@ asmlinkage long syscall_trace_enter(stru
> return -1;
> }
>
> - if (secure_computing())
> + if (!seccomp_permit_syscall())
> return -1;
>
> if (unlikely(test_thread_flag(TIF_SYSCALL_TRACEPOINT)))
> --- a/arch/parisc/kernel/ptrace.c
> +++ b/arch/parisc/kernel/ptrace.c
> @@ -351,7 +351,7 @@ long do_syscall_trace_enter(struct pt_re
> }
>
> /* Do the secure computing check after ptrace. */
> - if (secure_computing() == -1)
> + if (!seccomp_permit_syscall())
> return -1;
>
> #ifdef CONFIG_HAVE_SYSCALL_TRACEPOINTS
> --- a/arch/sh/kernel/ptrace_32.c
> +++ b/arch/sh/kernel/ptrace_32.c
> @@ -460,7 +460,7 @@ asmlinkage long do_syscall_trace_enter(s
> return -1;
> }
>
> - if (secure_computing() == -1)
> + if (!seccomp_permit_syscall())
> return -1;
>
> if (unlikely(test_thread_flag(TIF_SYSCALL_TRACEPOINT)))
> --- a/arch/um/kernel/skas/syscall.c
> +++ b/arch/um/kernel/skas/syscall.c
> @@ -27,7 +27,7 @@ void handle_syscall(struct uml_pt_regs *
> goto out;
>
> /* Do the seccomp check after ptrace; failures should be fast. */
> - if (secure_computing() == -1)
> + if (!seccomp_permit_syscall())
> goto out;
>
> syscall = UPT_SYSCALL_NR(r);
> --- a/arch/x86/entry/vsyscall/vsyscall_64.c
> +++ b/arch/x86/entry/vsyscall/vsyscall_64.c
> @@ -118,10 +118,10 @@ static bool write_ok_or_segv(unsigned lo
>
> static bool __emulate_vsyscall(struct pt_regs *regs, unsigned long address)
> {
> - unsigned long caller;
> - int vsyscall_nr, syscall_nr, tmp;
> + unsigned long caller, orig_dx;
> + int vsyscall_nr, syscall_nr;
> + bool skip;
> long ret;
> - unsigned long orig_dx;
>
> /* Confirm that the fault happened in 64-bit user mode */
> if (!user_64bit_mode(regs))
> @@ -197,16 +197,16 @@ static bool __emulate_vsyscall(struct pt
> */
> regs->orig_ax = syscall_nr;
> regs->ax = -ENOSYS;
> - tmp = secure_computing();
> - if ((!tmp && regs->orig_ax != syscall_nr) || regs->ip != address) {
> + skip = !seccomp_permit_syscall();
> + if ((!skip && regs->orig_ax != syscall_nr) || regs->ip != address) {
> warn_bad_vsyscall(KERN_DEBUG, regs,
> "seccomp tried to change syscall nr or ip");
> force_exit_sig(SIGSYS);
> return true;
> }
> regs->orig_ax = -1;
> - if (tmp)
> - goto do_ret; /* skip requested */
> + if (skip)
> + goto do_ret;
>
> /*
> * With a real vsyscall, page faults cause SIGSEGV.
> --- a/arch/xtensa/kernel/ptrace.c
> +++ b/arch/xtensa/kernel/ptrace.c
> @@ -553,8 +553,7 @@ int do_syscall_trace_enter(struct pt_reg
> return 0;
> }
>
> - if (regs->syscall == NO_SYSCALL ||
> - secure_computing() == -1) {
> + if (regs->syscall == NO_SYSCALL || !seccomp_permit_syscall()) {
> do_syscall_trace_leave(regs);
> return 0;
> }
> --- a/include/linux/entry-common.h
> +++ b/include/linux/entry-common.h
> @@ -102,9 +102,8 @@ static __always_inline long syscall_trac
>
> /* Do seccomp after ptrace, to catch any tracer changes. */
> if (work & SYSCALL_WORK_SECCOMP) {
> - ret = __secure_computing();
> - if (ret == -1L)
> - return ret;
> + if (!__seccomp_permit_syscall())
> + return -1L;
> }
>
> /* Either of the above might have changed the syscall number */
> @@ -115,7 +114,7 @@ static __always_inline long syscall_trac
>
> syscall_enter_audit(regs, syscall);
>
> - return ret ? : syscall;
> + return syscall;
> }
>
> /**
> @@ -138,7 +137,7 @@ static __always_inline long syscall_trac
> * It handles the following work items:
> *
> * 1) syscall_work flag dependent invocations of
> - * ptrace_report_syscall_entry(), __secure_computing(), trace_sys_enter()
> + * ptrace_report_syscall_entry(), __seccomp_permit_syscall(), trace_sys_enter()
> * 2) Invocation of audit_syscall_entry()
> */
> static __always_inline long syscall_enter_from_user_mode_work(struct pt_regs *regs, long syscall)
> --- a/include/linux/seccomp.h
> +++ b/include/linux/seccomp.h
> @@ -22,14 +22,14 @@
> #include <linux/atomic.h>
> #include <asm/seccomp.h>
>
> -extern int __secure_computing(void);
> +extern bool __seccomp_permit_syscall(void);
>
> #ifdef CONFIG_HAVE_ARCH_SECCOMP_FILTER
> -static inline int secure_computing(void)
> +static __always_inline bool seccomp_permit_syscall(void)
> {
> if (unlikely(test_syscall_work(SECCOMP)))
> - return __secure_computing();
> - return 0;
> + return __seccomp_permit_syscall();
> + return true;
> }
> #else
> extern void secure_computing_strict(int this_syscall);
> @@ -50,11 +50,11 @@ static inline int seccomp_mode(struct se
> struct seccomp_data;
>
> #ifdef CONFIG_HAVE_ARCH_SECCOMP_FILTER
> -static inline int secure_computing(void) { return 0; }
> +static inline bool seccomp_permit_syscall(void) { return true; }
> #else
> static inline void secure_computing_strict(int this_syscall) { return; }
> #endif
> -static inline int __secure_computing(void) { return 0; }
> +static inline bool __seccomp_permit_syscall(void) { return true; }
>
> static inline long prctl_get_seccomp(void)
> {
> --- a/kernel/seccomp.c
> +++ b/kernel/seccomp.c
> @@ -1100,12 +1100,13 @@ void secure_computing_strict(int this_sy
> else
> BUG();
> }
> -int __secure_computing(void)
> +
> +bool __seccomp_permit_syscall(void)
> {
> int this_syscall = syscall_get_nr(current, current_pt_regs());
>
> secure_computing_strict(this_syscall);
> - return 0;
> + return true;
> }
> #else
>
> @@ -1256,7 +1257,7 @@ static int seccomp_do_user_notification(
> return -1;
> }
>
> -static int __seccomp_filter(int this_syscall, const bool recheck_after_trace)
> +static bool __seccomp_filter(int this_syscall, const bool recheck_after_trace)
> {
> u32 filter_ret, action;
> struct seccomp_data sd;
> @@ -1294,7 +1295,7 @@ static int __seccomp_filter(int this_sys
> case SECCOMP_RET_TRACE:
> /* We've been put in this state by the ptracer already. */
> if (recheck_after_trace)
> - return 0;
> + return true;
>
> /* ENOSYS these calls if there is no tracer attached. */
> if (!ptrace_event_enabled(current, PTRACE_EVENT_SECCOMP)) {
> @@ -1330,19 +1331,19 @@ static int __seccomp_filter(int this_sys
> * a skip would have already been reported.
> */
> if (__seccomp_filter(this_syscall, true))
> - return -1;
> + return false;
>
> - return 0;
> + return true;
>
> case SECCOMP_RET_USER_NOTIF:
> if (seccomp_do_user_notification(this_syscall, match, &sd))
> goto skip;
>
> - return 0;
> + return true;
>
> case SECCOMP_RET_LOG:
> seccomp_log(this_syscall, 0, action, true);
> - return 0;
> + return true;
>
> case SECCOMP_RET_ALLOW:
> /*
> @@ -1350,7 +1351,7 @@ static int __seccomp_filter(int this_sys
> * this action since SECCOMP_RET_ALLOW is the starting
> * state in seccomp_run_filters().
> */
> - return 0;
> + return true;
>
> case SECCOMP_RET_KILL_THREAD:
> case SECCOMP_RET_KILL_PROCESS:
> @@ -1367,46 +1368,46 @@ static int __seccomp_filter(int this_sys
> } else {
> do_exit(SIGSYS);
> }
> - return -1; /* skip the syscall go directly to signal handling */
> + return false; /* skip the syscall go directly to signal handling */
> }
>
> unreachable();
>
> skip:
> seccomp_log(this_syscall, 0, action, match ? match->log : false);
> - return -1;
> + return false;
> }
> #else
> -static int __seccomp_filter(int this_syscall, const bool recheck_after_trace)
> +static bool __seccomp_filter(int this_syscall, const bool recheck_after_trace)
> {
> BUG();
>
> - return -1;
> + return false;
> }
> #endif
>
> -int __secure_computing(void)
> +bool __seccomp_permit_syscall(void)
> {
> int mode = current->seccomp.mode;
> int this_syscall;
>
> if (IS_ENABLED(CONFIG_CHECKPOINT_RESTORE) &&
> unlikely(current->ptrace & PT_SUSPEND_SECCOMP))
> - return 0;
> + return true;
>
> this_syscall = syscall_get_nr(current, current_pt_regs());
>
> switch (mode) {
> case SECCOMP_MODE_STRICT:
> __secure_computing_strict(this_syscall); /* may call do_exit */
> - return 0;
> + return true;
> case SECCOMP_MODE_FILTER:
> return __seccomp_filter(this_syscall, false);
> /* Surviving SECCOMP_RET_KILL_* must be proactively impossible. */
> case SECCOMP_MODE_DEAD:
> WARN_ON_ONCE(1);
> do_exit(SIGKILL);
> - return -1;
> + return false;
> default:
> BUG();
> }
>
Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
^ permalink raw reply
* Re: [patch 12/18] ptrace, treewide: Rename ptrace_report_syscall_entry() to ptrace_report_syscall_permit_entry()
From: Mukesh Kumar Chaurasiya @ 2026-07-09 19:22 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, Peter Zijlstra, Arnd Bergmann, Oleg Nesterov,
Richard Henderson, Vineet Gupta, Russell King, Catalin Marinas,
Will Deacon, Guo Ren, Brian Cain, Geert Uytterhoeven,
Michal Simek, Thomas Bogendoerfer, Dinh Nguyen, Helge Deller,
Yoshinori Sato, David S. Miller, Andreas Larsson, Chris Zankel,
linux-alpha, linux-snps-arc, linux-arm-kernel, linux-csky,
linux-hexagon, linux-m68k, linux-mips, linux-openrisc,
linux-parisc, linux-sh, sparclinux, linux-um, linux-arch,
Michael Ellerman, Shrikanth Hegde, linuxppc-dev, Kees Cook,
Huacai Chen, loongarch, Paul Walmsley, Palmer Dabbelt,
linux-riscv, Sven Schnelle, linux-s390, x86, Mark Rutland,
Jinjie Ruan, Andy Lutomirski, Richard Weinberger,
Michal Suchánek, Jonathan Corbet, linux-doc
In-Reply-To: <20260707190254.280015701@kernel.org>
On Tue, Jul 07, 2026 at 09:06:44PM +0200, Thomas Gleixner wrote:
> The return value of that function is boolean and tells the caller whether
> to permit the syscall processing or not.
>
> Rename the function so the purpose is clear and make the return type bool.
>
> Signed-off-by: Thomas Gleixner <tglx@kernel.org>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: Oleg Nesterov <oleg@redhat.com>
> Cc: Richard Henderson <richard.henderson@linaro.org>
> Cc: Vineet Gupta <vgupta@kernel.org>
> Cc: Russell King <linux@armlinux.org.uk>
> Cc: Catalin Marinas <catalin.marinas@arm.com>
> Cc: Will Deacon <will@kernel.org>
> Cc: Guo Ren <guoren@kernel.org>
> Cc: Brian Cain <bcain@kernel.org>
> Cc: Geert Uytterhoeven <geert@linux-m68k.org>
> Cc: Michal Simek <monstr@monstr.eu>
> Cc: Thomas Bogendoerfer <tsbogend@alpha.franken.de>
> Cc: Dinh Nguyen <dinguyen@kernel.org>
> Cc: Helge Deller <deller@gmx.de>
> Cc: Yoshinori Sato <ysato@users.sourceforge.jp>
> Cc: "David S. Miller" <davem@davemloft.net>
> Cc: Andreas Larsson <andreas@gaisler.com>
> Cc: Chris Zankel <chris@zankel.net>
> Cc: linux-alpha@vger.kernel.org
> Cc: linux-snps-arc@lists.infradead.org
> Cc: linux-arm-kernel@lists.infradead.org
> Cc: linux-csky@vger.kernel.org
> Cc: linux-hexagon@vger.kernel.org
> Cc: linux-m68k@lists.linux-m68k.org
> Cc: linux-mips@vger.kernel.org
> Cc: linux-openrisc@vger.kernel.org
> Cc: linux-parisc@vger.kernel.org
> Cc: linux-sh@vger.kernel.org
> Cc: sparclinux@vger.kernel.org
> Cc: linux-um@lists.infradead.org
> Cc: linux-arch@vger.kernel.org
> ---
> arch/alpha/kernel/ptrace.c | 2 +-
> arch/arc/kernel/ptrace.c | 2 +-
> arch/arm/kernel/ptrace.c | 2 +-
> arch/arm64/kernel/ptrace.c | 2 +-
> arch/csky/kernel/ptrace.c | 2 +-
> arch/hexagon/kernel/traps.c | 2 +-
> arch/m68k/kernel/ptrace.c | 2 +-
> arch/microblaze/kernel/ptrace.c | 2 +-
> arch/mips/kernel/ptrace.c | 2 +-
> arch/nios2/kernel/ptrace.c | 2 +-
> arch/openrisc/kernel/ptrace.c | 2 +-
> arch/parisc/kernel/ptrace.c | 10 ++++------
> arch/sh/kernel/ptrace_32.c | 2 +-
> arch/sparc/kernel/ptrace_32.c | 2 +-
> arch/sparc/kernel/ptrace_64.c | 2 +-
> arch/um/kernel/ptrace.c | 2 +-
> arch/xtensa/kernel/ptrace.c | 2 +-
> include/asm-generic/syscall.h | 4 ++--
> include/linux/entry-common.h | 25 ++++++++++++-------------
> include/linux/ptrace.h | 13 ++++++-------
> 20 files changed, 40 insertions(+), 44 deletions(-)
>
[...]
> --- a/arch/parisc/kernel/ptrace.c
> +++ b/arch/parisc/kernel/ptrace.c
> @@ -326,7 +326,7 @@ long compat_arch_ptrace(struct task_stru
> long do_syscall_trace_enter(struct pt_regs *regs)
> {
> if (test_thread_flag(TIF_SYSCALL_TRACE)) {
> - int rc = ptrace_report_syscall_entry(regs);
> + bool permit = ptrace_report_syscall_permit_entry(regs);
>
> /*
> * As tracesys_next does not set %r28 to -ENOSYS
> @@ -334,12 +334,10 @@ long do_syscall_trace_enter(struct pt_re
> */
> regs->gr[28] = -ENOSYS;
>
> - if (rc) {
> + if (!permit) {
> /*
> - * A nonzero return code from
> - * ptrace_report_syscall_entry() tells us
> - * to prevent the syscall execution. Skip
> - * the syscall call and the syscall restart handling.
> + * Skip the syscall call and the syscall restart
> + * handling.
Hey Thomas,
This comment `syscall call` sounds a bit weird.
Apart from this, Everything looks good.
Regards,
Mukesh
Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
^ permalink raw reply
* Re: [patch 14/18] entry: Make return type of syscall_trace_enter() bool
From: Mukesh Kumar Chaurasiya @ 2026-07-09 19:36 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, Peter Zijlstra, Michael Ellerman, Shrikanth Hegde,
linuxppc-dev, Kees Cook, Huacai Chen, loongarch, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Sven Schnelle, linux-s390, x86,
Mark Rutland, Jinjie Ruan, Andy Lutomirski, Oleg Nesterov,
Richard Henderson, Russell King, Catalin Marinas, Guo Ren,
Geert Uytterhoeven, Thomas Bogendoerfer, Helge Deller,
Yoshinori Sato, Richard Weinberger, Chris Zankel,
linux-arm-kernel, linux-alpha, linux-csky, linux-m68k, linux-mips,
linux-parisc, linux-sh, linux-um, Arnd Bergmann, Vineet Gupta,
Will Deacon, Brian Cain, Michal Simek, Dinh Nguyen,
David S. Miller, Andreas Larsson, linux-snps-arc, linux-hexagon,
linux-openrisc, sparclinux, linux-arch, Michal Suchánek,
Jonathan Corbet, linux-doc
In-Reply-To: <20260707190254.392010241@kernel.org>
On Tue, Jul 07, 2026 at 09:06:53PM +0200, Thomas Gleixner wrote:
> This prepares for changing the return types of
> syscall_enter_from_user_mode[_work]() to bool, which in turn separates the
> decision of invoking the syscall from the syscall number, which might have
> been changed in the call by ptrace, seccomp, tracing.
>
> Signed-off-by: Thomas Gleixner <tglx@kernel.org>
> ---
> include/linux/entry-common.h | 28 +++++++++++++++-------------
> 1 file changed, 15 insertions(+), 13 deletions(-)
>
> --- a/include/linux/entry-common.h
> +++ b/include/linux/entry-common.h
> @@ -71,8 +71,8 @@ static inline void syscall_enter_audit(s
> }
> }
>
> -static __always_inline long syscall_trace_enter(struct pt_regs *regs, unsigned long work,
> - long syscall)
> +static __always_inline bool syscall_trace_enter(struct pt_regs *regs, unsigned long work,
> + long *syscall)
> {
> /*
> * Handle Syscall User Dispatch. This must comes first, since
> @@ -81,7 +81,7 @@ static __always_inline long syscall_trac
> */
> if (work & SYSCALL_WORK_SYSCALL_USER_DISPATCH) {
> if (syscall_user_dispatch(regs))
> - return -1L;
> + return false;
> }
>
> /*
> @@ -90,32 +90,32 @@ static __always_inline long syscall_trac
> * through hrtimer_interrupt().
> */
> if (work & SYSCALL_WORK_SYSCALL_RSEQ_SLICE)
> - rseq_syscall_enter_work(syscall);
> + rseq_syscall_enter_work(*syscall);
>
> /* Handle ptrace */
> if (work & (SYSCALL_WORK_SYSCALL_TRACE | SYSCALL_WORK_SYSCALL_EMU)) {
> if (!arch_ptrace_report_syscall_permit_entry(regs) ||
> (work & SYSCALL_WORK_SYSCALL_EMU))
> - return -1L;
> + return false;
> }
>
> /* Do seccomp after ptrace, to catch any tracer changes. */
> if (work & SYSCALL_WORK_SECCOMP) {
> if (!__seccomp_permit_syscall())
> - return -1L;
> + return false;
> }
>
> /* Either of the above might have changed the syscall number */
> - syscall = syscall_get_nr(current, regs);
> + *syscall = syscall_get_nr(current, regs);
>
> if (unlikely(work & SYSCALL_WORK_SYSCALL_TRACEPOINT)) {
> - if (!trace_syscall_enter(regs, &syscall))
> - return -1L;
> + if (!trace_syscall_enter(regs, syscall))
> + return false;
> }
>
> - syscall_enter_audit(regs, syscall);
> + syscall_enter_audit(regs, *syscall);
>
> - return syscall;
> + return true;
> }
>
> /**
> @@ -145,8 +145,10 @@ static __always_inline long syscall_ente
> {
> unsigned long work = READ_ONCE(current_thread_info()->syscall_work);
>
> - if (work & SYSCALL_WORK_ENTER)
> - syscall = syscall_trace_enter(regs, work, syscall);
> + if (work & SYSCALL_WORK_ENTER) {
> + if (!syscall_trace_enter(regs, work, &syscall))
> + return -1L;
> + }
>
> return syscall;
> }
>
Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
^ permalink raw reply
* Re: [patch 15/18] x86/entry: Make syscall functions static
From: Mukesh Kumar Chaurasiya @ 2026-07-09 19:43 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, Peter Zijlstra, Michael Ellerman, Shrikanth Hegde,
linuxppc-dev, Kees Cook, Huacai Chen, loongarch, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Sven Schnelle, linux-s390, x86,
Mark Rutland, Jinjie Ruan, Andy Lutomirski, Oleg Nesterov,
Richard Henderson, Russell King, Catalin Marinas, Guo Ren,
Geert Uytterhoeven, Thomas Bogendoerfer, Helge Deller,
Yoshinori Sato, Richard Weinberger, Chris Zankel,
linux-arm-kernel, linux-alpha, linux-csky, linux-m68k, linux-mips,
linux-parisc, linux-sh, linux-um, Arnd Bergmann, Vineet Gupta,
Will Deacon, Brian Cain, Michal Simek, Dinh Nguyen,
David S. Miller, Andreas Larsson, linux-snps-arc, linux-hexagon,
linux-openrisc, sparclinux, linux-arch, Michal Suchánek,
Jonathan Corbet, linux-doc
In-Reply-To: <20260707190254.438361565@kernel.org>
On Tue, Jul 07, 2026 at 09:06:57PM +0200, Thomas Gleixner wrote:
> They are only used in the respective source files. No point in exposing
> them.
>
> Signed-off-by: Thomas Gleixner <tglx@kernel.org>
> ---
> arch/x86/entry/syscall_32.c | 2 +-
> arch/x86/entry/syscall_64.c | 10 ++++++----
> arch/x86/include/asm/syscall.h | 8 --------
> 3 files changed, 7 insertions(+), 13 deletions(-)
>
> --- a/arch/x86/entry/syscall_32.c
> +++ b/arch/x86/entry/syscall_32.c
> @@ -41,7 +41,7 @@ const sys_call_ptr_t sys_call_table[] =
> #endif
>
> #define __SYSCALL(nr, sym) case nr: return __ia32_##sym(regs);
> -long ia32_sys_call(const struct pt_regs *regs, unsigned int nr)
> +static noinline long ia32_sys_call(const struct pt_regs *regs, unsigned int nr)
> {
> switch (nr) {
> #include <asm/syscalls_32.h>
> --- a/arch/x86/entry/syscall_64.c
> +++ b/arch/x86/entry/syscall_64.c
> @@ -32,7 +32,7 @@ const sys_call_ptr_t sys_call_table[] =
> #undef __SYSCALL
>
> #define __SYSCALL(nr, sym) case nr: return __x64_##sym(regs);
> -long x64_sys_call(const struct pt_regs *regs, unsigned int nr)
> +static noinline long x64_sys_call(const struct pt_regs *regs, unsigned int nr)
> {
> switch (nr) {
> #include <asm/syscalls_64.h>
> @@ -40,15 +40,17 @@ long x64_sys_call(const struct pt_regs *
> }
> }
>
> -#ifdef CONFIG_X86_X32_ABI
> -long x32_sys_call(const struct pt_regs *regs, unsigned int nr)
> +static noinline long x32_sys_call(const struct pt_regs *regs, unsigned int nr)
> {
> +#ifdef CONFIG_X86_X32_ABI
> switch (nr) {
> #include <asm/syscalls_x32.h>
> default: return __x64_sys_ni_syscall(regs);
> }
> -}
> +#else
> + return -ENOSYS;
> #endif
> +}
>
> static __always_inline bool do_syscall_x64(struct pt_regs *regs, int nr)
> {
> --- a/arch/x86/include/asm/syscall.h
> +++ b/arch/x86/include/asm/syscall.h
> @@ -21,14 +21,6 @@ typedef long (*sys_call_ptr_t)(const str
> extern const sys_call_ptr_t sys_call_table[];
>
> /*
> - * These may not exist, but still put the prototypes in so we
> - * can use IS_ENABLED().
> - */
> -extern long ia32_sys_call(const struct pt_regs *, unsigned int nr);
> -extern long x32_sys_call(const struct pt_regs *, unsigned int nr);
> -extern long x64_sys_call(const struct pt_regs *, unsigned int nr);
> -
> -/*
> * Only the low 32 bits of orig_ax are meaningful, so we return int.
> * This importantly ignores the high bits on 64-bit, so comparisons
> * sign-extend the low 32 bits.
>
Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
^ permalink raw reply
* Re: [patch 18/18] entry, treewide: Make syscall_enter_from_user_mode[_work]() indicate syscall execution
From: Mukesh Kumar Chaurasiya @ 2026-07-09 19:49 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, Peter Zijlstra, Michal Suchánek, Jonathan Corbet,
Arnd Bergmann, Mark Rutland, Huacai Chen, Michael Ellerman,
Shrikanth Hegde, Paul Walmsley, Palmer Dabbelt, Sven Schnelle,
linux-doc, loongarch, linuxppc-dev, linux-riscv, linux-s390,
Kees Cook, x86, Jinjie Ruan, Andy Lutomirski, Oleg Nesterov,
Richard Henderson, Russell King, Catalin Marinas, Guo Ren,
Geert Uytterhoeven, Thomas Bogendoerfer, Helge Deller,
Yoshinori Sato, Richard Weinberger, Chris Zankel,
linux-arm-kernel, linux-alpha, linux-csky, linux-m68k, linux-mips,
linux-parisc, linux-sh, linux-um, Vineet Gupta, Will Deacon,
Brian Cain, Michal Simek, Dinh Nguyen, David S. Miller,
Andreas Larsson, linux-snps-arc, linux-hexagon, linux-openrisc,
sparclinux, linux-arch
In-Reply-To: <20260707190254.603111179@kernel.org>
On Tue, Jul 07, 2026 at 09:07:09PM +0200, Thomas Gleixner wrote:
> From: Michal Suchánek <msuchanek@suse.de>
>
> The return values of syscall_enter_from_user_mode[_work]() are
> non-intuitive. Both functions return the syscall number which should be
> invoked by the architecture specific syscall entry code. The returned
> number can be:
>
> - the unmodified syscall number which was handed in by the caller
>
> - a modified syscall number (ptrace, seccomp, trace/probe/bpf)
>
> That has an additional twist. If the return value is -1L then the caller is
> not allowed to modify the return value as that indicates that the modifying
> entity requests to abort the syscall and set the return value already. That
> can obviously not be differentiated from a syscall which handed in -1 as
> syscall number.
>
> The established way to deal with that is:
>
> set_return_value(regs, -ENOSYS);
> nr = syscall_enter_from_user_mode(regs, nr);
> if ((unsigned)nr < SYSCALLNR_MAX)
> handle_syscall(regs, nr);
> else if (nr != -1)
> set_return_value(regs, -ENOSYS);
>
> The latter is obviously redundant, but that's just a leftover of the
> historical evolution of this code. S390 has some special requirements here,
> which can be avoided when the return value is not ambiguous.
>
> Now that the functions which modify the syscall number and want to abort
> are converted to indicate that with a boolean return value, it's obvious to
> hand this through to the callers.
>
> Rework syscall_enter_from_user_mode[_work]) so they take a pointer to the
> syscall number and return a boolean, which indicates whether the syscall
> should be handled or not.
>
> That's not only more intuitive, it also results in slightly denser
> executable code on x86 at least, but perf results are neutral and within
> the noise.
>
> [ tglx: Adopted it to the changes in the generic entry code, fixed up the
> 32-bit fallout and rewrote change log ]
>
> Signed-off-by: Michal Suchánek <msuchanek@suse.de>
> Signed-off-by: Thomas Gleixner <tglx@kernel.org>
> Cc: Jonathan Corbet <corbet@lwn.net>
> Cc: Arnd Bergmann <arnd@arndb.de>
> Cc: Mark Rutland <mark.rutland@arm.com>
> Cc: Huacai Chen <chenhuacai@kernel.org>
> Cc: Michael Ellerman <mpe@ellerman.id.au>
> Cc: Shrikanth Hegde <sshegde@linux.ibm.com>
> Cc: Paul Walmsley <pjw@kernel.org>
> Cc: Palmer Dabbelt <palmer@dabbelt.com>
> Cc: Sven Schnelle <svens@linux.ibm.com>
> Cc: linux-doc@vger.kernel.org
> Cc: loongarch@lists.linux.dev
> Cc: linuxppc-dev@lists.ozlabs.org
> Cc: linux-riscv@lists.infradead.org
> Cc: linux-s390@vger.kernel.org
> ---
> Documentation/core-api/entry.rst | 18 +++++++++++-------
> arch/loongarch/kernel/syscall.c | 14 +++++++-------
> arch/powerpc/kernel/syscall.c | 3 ++-
> arch/riscv/kernel/traps.c | 11 +++++------
> arch/s390/kernel/syscall.c | 7 +++++--
> arch/x86/entry/syscall_32.c | 25 ++++++++++++-------------
> arch/x86/entry/syscall_64.c | 12 ++++++------
> include/linux/entry-common.h | 12 +++++-------
> 8 files changed, 53 insertions(+), 49 deletions(-)
>
>
[...]
> syscall_exit_to_user_mode(regs);
> --- a/arch/powerpc/kernel/syscall.c
> +++ b/arch/powerpc/kernel/syscall.c
> @@ -18,7 +18,8 @@ notrace long system_call_exception(struc
> long ret;
> syscall_fn f;
>
> - r0 = syscall_enter_from_user_mode_randomize_stack(regs, r0);
> + if (unlikely(!syscall_enter_from_user_mode_randomize_stack(regs, &r0))
Missing one closing ')'.
> + return syscall_get_error(current, regs);
>
> if (unlikely(r0 >= NR_syscalls)) {
> if (unlikely(trap_is_unsupported_scv(regs))) {
Apart from this.
Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
^ permalink raw reply
* Re: [patch 00/18] entry: Consolidate and rework syscall entry handling
From: Mukesh Kumar Chaurasiya @ 2026-07-09 20:15 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, Peter Zijlstra, Michael Ellerman, Shrikanth Hegde,
linuxppc-dev, Kees Cook, Huacai Chen, loongarch, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Sven Schnelle, linux-s390, x86,
Mark Rutland, Jinjie Ruan, Andy Lutomirski, Oleg Nesterov,
Richard Henderson, Russell King, Catalin Marinas, Guo Ren,
Geert Uytterhoeven, Thomas Bogendoerfer, Helge Deller,
Yoshinori Sato, Richard Weinberger, Chris Zankel,
linux-arm-kernel, linux-alpha, linux-csky, linux-m68k, linux-mips,
linux-parisc, linux-sh, linux-um, Arnd Bergmann, Vineet Gupta,
Will Deacon, Brian Cain, Michal Simek, Dinh Nguyen,
David S. Miller, Andreas Larsson, linux-snps-arc, linux-hexagon,
linux-openrisc, sparclinux, linux-arch, Michal Suchánek,
Jonathan Corbet, linux-doc
In-Reply-To: <20260707181957.433213175@kernel.org>
On Tue, Jul 07, 2026 at 09:05:53PM +0200, Thomas Gleixner wrote:
> Sorry for the long CC list, but this is a treewide change.
>
> Michal recently posted a RFC patch to separate the potential syscall number
> modifications in syscall_enter_user_mode_work() from the information
> whether the syscall should be processed and the return value modified:
>
> https://lore.kernel.org/lkml/CE1qW@kunlun.suse.cz
>
> The existing logic is:
>
> arch_syscall()
> regs->result = -ENOSYS;
>
> syscallnr = syscall_enter_from_user_mode(regs, syscall);
>
> if (syscallnr != -1L)
> regs->result = invoke_syscall(regs, syscall;
>
> syscall_enter_from_user_mode() invokes ptrace, seccomp and
> tracing/BPF/Probes. All of them can modify the syscall number.
>
> ptrace and seccomp explicitly set the syscall number to -1L to indicate
> that the syscall invocation needs to be skipped and the result has not to
> be modified as it might have been modified by ptrace or seccomp. The
> tracer/BPF/Probes mechanism can modify the syscall number as well and
> relies implicitly on the -1L logic.
>
> This can obviously not be differentiated from a syscall invocation where
> userspace provided -1 as syscall number.
>
> The general agreement of the discussion was that the current mechanism,
> while functionally correct is non-intuitive and something like Michals
> proposal would make that code clearer and easier to handle on the
> architecture side:
>
> arch_syscall()
> regs->result = -ENOSYS;
>
> if (syscall_enter_from_user_mode(regs, &syscall))
> regs->result = invoke_syscall(regs, syscall;
>
> That discussion made me look deeper into the related code and as usual
> there were a lot of other things to discover.
>
> 1) Stack randomization
>
> add_random_kstack_offset() can only be invoked after
> enter_from_user_mode() established proper state as it calls into
> instrumentable code.
>
> PowerPC got that wrong and the other architectures either invoke it
> after enter_from_user_mode() or after syscall_enter_from_user_mode().
>
> The latter is suboptimal as the randomization takes place after all
> the user mode entry work. Aside of that add_random_kstack_offset()
> uses get/put_cpu_var(), which makes it usable in preemptible code, but
> when invoked in the interrupt disabled region that's pointless
> overhead.
>
> 2) As discussed in the above thread just changing the function signature
> of syscall_enter_from_user_mode[_work]() so they take a pointer
> argument for the syscall and then return 0 on success is not really
> intuitive either. Aside of that this breaks the implicit assumption of
> the tracer when setting the syscall number to -1.
>
> 3) The x86 entry code has some historically accumulated oddities
>
> The following series addresses this by:
>
> 1) Providing new [syscall_]enter_from_user_mode() variants, which include
> stack randomization and utilize a new add_random_kstack_offset_irqsoff()
> variant, which avoids the get/put_cpu_var() overhead and converting all
> usage sites over
>
> 2) Picking up Jinjie's seccomp patch from:
>
> https://lore.kernel.org/lkml/20260629130616.642022-2-ruanjinjie@huawei.com
>
> and addressing the feedback (renaming the seccomp functions)
>
> 3) Making the ptrace and tracer related functions return a boolean value
> to indicate syscall permission
>
> 4) Addressing the x86 oddities
>
> 5) Converting the tree over to the new scheme
>
> With that all architectures using the generic syscall entry code follow the
> same scheme, apply stack randomization at the correct and earliest possible
> place and skip syscall processing depending on the boolean return value of
> syscall_enter_from_user_mode[_work]().
>
> There should be no functional changes, at least there are none intended.
>
> The resulting text size for the syscall entry code on x8664 is slightly
> smaller than before these changes.
>
> Testing syscall heavy workloads and micro benchmarks shows a small
> performance gain for the general rework, but the last patch, which changes
> the logic to be more understandable has no measurable impact in either
> direction.
>
> The series applies on Linus tree and is also available from git:
>
> git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git entry-rework-v1
>
> Thanks,
>
> tglx
> ---
> Documentation/core-api/entry.rst | 33 +++++---
> arch/alpha/kernel/ptrace.c | 4 -
> arch/arc/kernel/ptrace.c | 2
> arch/arm/kernel/ptrace.c | 4 -
> arch/arm64/kernel/ptrace.c | 4 -
> arch/csky/kernel/ptrace.c | 4 -
> arch/hexagon/kernel/traps.c | 2
> arch/loongarch/kernel/syscall.c | 17 +---
> arch/m68k/kernel/ptrace.c | 4 -
> arch/microblaze/kernel/ptrace.c | 2
> arch/mips/kernel/ptrace.c | 4 -
> arch/nios2/kernel/ptrace.c | 2
> arch/openrisc/kernel/ptrace.c | 2
> arch/parisc/kernel/ptrace.c | 12 +--
> arch/powerpc/kernel/syscall.c | 5 -
> arch/riscv/kernel/traps.c | 14 +--
> arch/s390/kernel/syscall.c | 11 +-
> arch/sh/kernel/ptrace_32.c | 4 -
> arch/sparc/kernel/ptrace_32.c | 2
> arch/sparc/kernel/ptrace_64.c | 2
> arch/um/kernel/ptrace.c | 2
> arch/um/kernel/skas/syscall.c | 2
> arch/x86/entry/syscall_32.c | 70 +++++++------------
> arch/x86/entry/syscall_64.c | 61 ++++++----------
> arch/x86/entry/vsyscall/vsyscall_64.c | 14 +--
> arch/x86/include/asm/entry-common.h | 1
> arch/x86/include/asm/syscall.h | 10 --
> arch/xtensa/kernel/ptrace.c | 5 -
> include/asm-generic/syscall.h | 4 -
> include/linux/entry-common.h | 125 ++++++++++++++++++++--------------
> include/linux/irq-entry-common.h | 6 -
> include/linux/ptrace.h | 13 +--
> include/linux/randomize_kstack.h | 19 +++++
> include/linux/seccomp.h | 12 +--
> kernel/entry/syscall-common.c | 7 +
> kernel/seccomp.c | 35 ++++-----
> 36 files changed, 264 insertions(+), 256 deletions(-)
>
>
>
> _______________________________________________
> linux-snps-arc mailing list
> linux-snps-arc@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-snps-arc
Boot tested this on P11 LPAR and P9 powernv system.
Tested-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
^ permalink raw reply
* Re: [patch 03/18] entry: Provide [syscall_]enter_from_user_mode_randomize_stack()
From: Mukesh Kumar Chaurasiya @ 2026-07-09 20:16 UTC (permalink / raw)
To: Thomas Gleixner
Cc: LKML, Peter Zijlstra, Michael Ellerman, Shrikanth Hegde,
linuxppc-dev, Kees Cook, Huacai Chen, loongarch, Paul Walmsley,
Palmer Dabbelt, linux-riscv, Sven Schnelle, linux-s390, x86,
Mark Rutland, Jinjie Ruan, Andy Lutomirski, Oleg Nesterov,
Richard Henderson, Russell King, Catalin Marinas, Guo Ren,
Geert Uytterhoeven, Thomas Bogendoerfer, Helge Deller,
Yoshinori Sato, Richard Weinberger, Chris Zankel,
linux-arm-kernel, linux-alpha, linux-csky, linux-m68k, linux-mips,
linux-parisc, linux-sh, linux-um, Arnd Bergmann, Vineet Gupta,
Will Deacon, Brian Cain, Michal Simek, Dinh Nguyen,
David S. Miller, Andreas Larsson, linux-snps-arc, linux-hexagon,
linux-openrisc, sparclinux, linux-arch, Michal Suchánek,
Jonathan Corbet, linux-doc
In-Reply-To: <20260707190253.816918647@kernel.org>
On Tue, Jul 07, 2026 at 09:06:07PM +0200, Thomas Gleixner wrote:
> Randomizing the syscall stack can only happen after state is established
> via enter_from_user_mode() or syscall_enter_from_user_mode(). The earlier
> it happens the better.
>
> Provide two new macros to consolidate that:
>
> - enter_from_user_mode_randomize_stack()
> enter_from_user_mode();
> add_random_kstack_offset_irqsoff();
>
> - syscall_enter_from_user_mode_randomize_stack()
> enter_from_user_mode_randomize_stack();
> syscall_enter_from_user_mode_work();
>
> to reduce boiler plate code.
>
> Those are macros and not inline functions as the latter would limit the
> stack randomization scope to the inline function itself.
>
> Signed-off-by: Thomas Gleixner <tglx@kernel.org>
> ---
> include/linux/entry-common.h | 56 +++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 56 insertions(+)
>
> --- a/include/linux/entry-common.h
> +++ b/include/linux/entry-common.h
> @@ -6,6 +6,7 @@
> #include <linux/irq-entry-common.h>
> #include <linux/livepatch.h>
> #include <linux/ptrace.h>
> +#include <linux/randomize_kstack.h>
> #include <linux/resume_user_mode.h>
> #include <linux/seccomp.h>
> #include <linux/sched.h>
> @@ -150,6 +151,61 @@ static __always_inline long syscall_ente
> }
>
> /**
> + * enter_from_user_mode_randomize_stack - Establish state and add stack randomization
> + * before invoking syscall_enter_from_user_mode_work()
> + * @regs: Pointer to currents pt_regs
> + *
> + * Invoked from architecture specific syscall entry code with interrupts
> + * disabled. The calling code has to be non-instrumentable. When the function
> + * returns all state is correct, interrupts are still disabled and the
> + * subsequent functions can be instrumented.
> + *
> + * Implemented as a macro so that the stack randomization is effective
> + * throughout the function in which it is invoked. An inline would only make it
> + * effective in the scope of the inline function.
> + */
> +#define enter_from_user_mode_randomize_stack(regs) \
> +do { \
> + enter_from_user_mode(regs); \
> + instrumentation_begin(); \
> + add_random_kstack_offset_irqsoff(); \
> + instrumentation_end(); \
> +} while (0)
> +
> +/**
> + * syscall_enter_from_user_mode_randomize_stack - Establish state and check and handle work
> + * before invoking a syscall
> + * @regs: Pointer to currents pt_regs
> + * @syscall: The syscall number
> + *
> + * Invoked from architecture specific syscall entry code with interrupts
> + * disabled. The calling code has to be non-instrumentable. When the
> + * function returns all state is correct, interrupts are enabled and the
> + * subsequent functions can be instrumented.
> + *
> + * This is the combination of enter_from_user_mode_randomize_stack() and
> + * syscall_enter_from_user_mode_work() to be used when there is no
> + * architecture specific work to be done between the two.
> + *
> + * Returns: The original or a modified syscall number. See
> + * syscall_enter_from_user_mode_work() for further explanation.
> + *
> + * Implemented as a macro to make stack randomization effective in the calling
> + * scope.
> + */
> +#define syscall_enter_from_user_mode_randomize_stack(regs, syscall) \
> +({ \
> + enter_from_user_mode_randomize_stack(regs); \
> + \
> + instrumentation_begin(); \
> + local_irq_enable(); \
> + long _ret = syscall_enter_from_user_mode_work(regs, syscall); \
> + instrumentation_end(); \
> + \
> + _ret; \
> +})
> +
> +/**
> * syscall_enter_from_user_mode - Establish state and check and handle work
> * before invoking a syscall
> * @regs: Pointer to currents pt_regs
>
Reviewed-by: Mukesh Kumar Chaurasiya (IBM) <mkchauras@gmail.com>
>
> _______________________________________________
> linux-snps-arc mailing list
> linux-snps-arc@lists.infradead.org
> http://lists.infradead.org/mailman/listinfo/linux-snps-arc
^ permalink raw reply
* [PATCH 7.1.y 0/6] cBPF JIT spray hardening
From: Pawan Gupta @ 2026-07-09 22:22 UTC (permalink / raw)
To: stable, Greg Kroah-Hartman, Sasha Levin
Cc: bpf, linux-arm-kernel, loongarch, linuxppc-dev, linux-riscv, x86,
Alexei Starovoitov, Daniel Borkmann, Dave Hansen
Hi,
These backports harden BPF JIT against spectre-v2 class of attacks. Without
a predictor flush, execution of new BPF program may use stale prediction
left behind by the freed one.
To avoid this, issue an IBPB flush on all CPUs on JIT program allocation.
The flush is conditional to spectre-v2 mitigation applied.
Patch 1-2: Adds the predictor flush hook and enables it on x86 via IBPB.
bpf: Support for hardening against JIT spraying
x86/bugs: Enable IBPB flush on BPF JIT allocation
Patch 3-6: Narrow the flush to only unprivileged JIT allocations
to avoid redundant flushes. Also adds pack-selection changes
that minimizes flushes.
bpf: Restrict JIT predictor flush to cBPF
bpf: Skip redundant IBPB in pack allocator
bpf: Prefer packs that won't trigger an IBPB flush on allocation
bpf: Prefer dirty packs for eBPF allocations
Note: Patches were only build tested on x86 (they applied smoothly without
any conflict).
Thanks,
Pawan
---
Pawan Gupta (6):
bpf: Support for hardening against JIT spraying
x86/bugs: Enable IBPB flush on BPF JIT allocation
bpf: Restrict JIT predictor flush to cBPF
bpf: Skip redundant IBPB in pack allocator
bpf: Prefer packs that won't trigger an IBPB flush on allocation
bpf: Prefer dirty packs for eBPF allocations
arch/arm64/net/bpf_jit_comp.c | 4 +--
arch/loongarch/net/bpf_jit.c | 5 +--
arch/powerpc/net/bpf_jit_comp.c | 4 +--
arch/riscv/net/bpf_jit_comp64.c | 2 +-
arch/riscv/net/bpf_jit_core.c | 3 +-
arch/x86/include/asm/nospec-branch.h | 4 +++
arch/x86/kernel/cpu/bugs.c | 50 +++++++++++++++++++++++---
arch/x86/net/bpf_jit_comp.c | 5 +--
include/linux/filter.h | 15 ++++++--
kernel/bpf/core.c | 68 ++++++++++++++++++++++++++++++++----
kernel/bpf/dispatcher.c | 2 +-
11 files changed, 138 insertions(+), 24 deletions(-)
---
base-commit: 199c9959d3a9b53f346c221757fc7ac507fbac50
change-id: 20260709-cbpf-jit-spray-hardening-7-1-y-b33b43f197e8
Best regards,
--
Thanks,
Pawan
^ permalink raw reply
* [PATCH 7.1.y 1/6] bpf: Support for hardening against JIT spraying
From: Pawan Gupta @ 2026-07-09 22:23 UTC (permalink / raw)
To: stable, Greg Kroah-Hartman, Sasha Levin
Cc: bpf, linux-arm-kernel, loongarch, linuxppc-dev, linux-riscv, x86,
Alexei Starovoitov, Daniel Borkmann
In-Reply-To: <20260709-cbpf-jit-spray-hardening-7-1-y-v1-0-5ac5a2d6797f@linux.intel.com>
commit 96cce16e26dd02a8678f1e87f88a4b5cdb63b995 upstream.
The BPF JIT allocator packs many small programs into larger executable
allocations and reuses space within those allocations as programs are
loaded and freed. When fresh code is written into space that a previous
program occupied, an indirect jump into the new program can reuse a branch
prediction left behind by the old one.
Flush the indirect branch predictors before reusing JIT memory so that
indirect jumps into a newly written program don't reuse predictions from an
old program that occupied the same space.
Introduce bpf_arch_pred_flush_enabled static key and bpf_arch_pred_flush
static call for flushing the branch predictors on JIT memory reuse.
Architectures that need a flush, can update it to a predictor flush
function. By default, its a NOP and does not emit any CALL.
Allocations larger than a pack are not covered by this flush. That is safe
because cBPF programs (the unprivileged attack surface) are bounded well
below a pack size. Issue a warning if this assumption is ever violated
while the flush is active.
Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
---
include/linux/filter.h | 10 ++++++++++
kernel/bpf/core.c | 19 +++++++++++++++++++
2 files changed, 29 insertions(+)
diff --git a/include/linux/filter.h b/include/linux/filter.h
index 88a241aac36a..2a7e6cbbbe1d 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -21,6 +21,7 @@
#include <linux/if_vlan.h>
#include <linux/vmalloc.h>
#include <linux/sockptr.h>
+#include <linux/static_call.h>
#include <linux/u64_stats_sync.h>
#include <net/sch_generic.h>
@@ -1291,6 +1292,15 @@ extern long bpf_jit_limit_max;
typedef void (*bpf_jit_fill_hole_t)(void *area, unsigned int size);
+/*
+ * Flush the indirect branch predictors before reusing JIT memory, so that
+ * indirect jumps into a newly written program don't reuse predictions left
+ * behind by an old program that occupied the same space.
+ */
+void bpf_arch_pred_flush(void);
+DECLARE_STATIC_CALL(bpf_arch_pred_flush, bpf_arch_pred_flush);
+DECLARE_STATIC_KEY_FALSE(bpf_pred_flush_enabled);
+
void bpf_jit_fill_hole_with_zero(void *area, unsigned int size);
struct bpf_binary_header *
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 6aa2a8b24030..f49b9b23f95e 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -20,6 +20,7 @@
#include <uapi/linux/btf.h>
#include <linux/filter.h>
#include <linux/skbuff.h>
+#include <linux/static_call.h>
#include <linux/vmalloc.h>
#include <linux/prandom.h>
#include <linux/bpf.h>
@@ -883,6 +884,15 @@ void bpf_jit_fill_hole_with_zero(void *area, unsigned int size)
memset(area, 0, size);
}
+DEFINE_STATIC_CALL_NULL(bpf_arch_pred_flush, bpf_arch_pred_flush);
+
+/*
+ * Enabled once bpf_arch_pred_flush points at a real flush routine. Lets the
+ * pack allocator test "is a predictor flush wired up at all" with a cheap
+ * static branch instead of repeatedly querying the static call target.
+ */
+DEFINE_STATIC_KEY_FALSE(bpf_pred_flush_enabled);
+
#define BPF_PROG_SIZE_TO_NBITS(size) (round_up(size, BPF_PROG_CHUNK_SIZE) / BPF_PROG_CHUNK_SIZE)
static DEFINE_MUTEX(pack_mutex);
@@ -941,6 +951,14 @@ void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns)
mutex_lock(&pack_mutex);
if (size > BPF_PROG_PACK_SIZE) {
+ /*
+ * Allocations larger than a pack get their own pages, and
+ * predictors are not flushed for such allocation. This is only
+ * safe because cBPF programs (the unprivileged attack surface)
+ * are bounded well below a pack size.
+ */
+ if (static_branch_unlikely(&bpf_pred_flush_enabled))
+ pr_warn_once("BPF: Predictors not flushed for allocations greater than BPF_PROG_PACK_SIZE\n");
size = round_up(size, PAGE_SIZE);
ptr = bpf_jit_alloc_exec(size);
if (ptr) {
@@ -971,6 +989,7 @@ void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns)
pos = 0;
found_free_area:
+ static_call_cond(bpf_arch_pred_flush)();
bitmap_set(pack->bitmap, pos, nbits);
ptr = (void *)(pack->ptr) + (pos << BPF_PROG_CHUNK_SHIFT);
--
2.43.0
^ permalink raw reply related
* [PATCH 7.1.y 2/6] x86/bugs: Enable IBPB flush on BPF JIT allocation
From: Pawan Gupta @ 2026-07-09 22:23 UTC (permalink / raw)
To: stable, Greg Kroah-Hartman, Sasha Levin
Cc: bpf, linux-arm-kernel, loongarch, linuxppc-dev, linux-riscv, x86,
Alexei Starovoitov, Daniel Borkmann, Dave Hansen
In-Reply-To: <20260709-cbpf-jit-spray-hardening-7-1-y-v1-0-5ac5a2d6797f@linux.intel.com>
commit a3af84b0fa00ead01fcd0e28b5d773ff25990a0d upstream.
Enable hardening against JIT spraying when Spectre-v2 mitigations are in
use. Specifically, issue an IBPB flush on BPF JIT memory reuse. Skip
enabling the IBPB flush if the BPF dispatcher is already using a retpoline
sequence.
This hardening applies only when BPF-JIT is in use. Guard the enabling
under CONFIG_BPF_JIT so that bugs.c still builds with CONFIG_BPF_JIT=n.
Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Acked-by: Dave Hansen <dave.hansen@linux.intel.com>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
---
arch/x86/include/asm/nospec-branch.h | 4 +++
arch/x86/kernel/cpu/bugs.c | 50 ++++++++++++++++++++++++++++++++----
2 files changed, 49 insertions(+), 5 deletions(-)
diff --git a/arch/x86/include/asm/nospec-branch.h b/arch/x86/include/asm/nospec-branch.h
index 4f4b5e8a1574..b68892e6d58c 100644
--- a/arch/x86/include/asm/nospec-branch.h
+++ b/arch/x86/include/asm/nospec-branch.h
@@ -388,6 +388,10 @@ extern void srso_alias_return_thunk(void);
extern void entry_untrain_ret(void);
extern void write_ibpb(void);
+#ifdef CONFIG_BPF_JIT
+extern void bpf_arch_ibpb(void);
+#endif
+
#ifdef CONFIG_X86_64
extern void clear_bhb_loop(void);
#endif
diff --git a/arch/x86/kernel/cpu/bugs.c b/arch/x86/kernel/cpu/bugs.c
index 83f51cab0b1e..d9af230c0512 100644
--- a/arch/x86/kernel/cpu/bugs.c
+++ b/arch/x86/kernel/cpu/bugs.c
@@ -16,6 +16,7 @@
#include <linux/sched/smt.h>
#include <linux/pgtable.h>
#include <linux/bpf.h>
+#include <linux/filter.h>
#include <linux/kvm_types.h>
#include <asm/spec-ctrl.h>
@@ -1651,8 +1652,21 @@ static inline const char *spectre_v2_module_string(void)
{
return spectre_v2_bad_module ? " - vulnerable module loaded" : "";
}
+
+/*
+ * The "retpoline sequence" is the "call;mov;ret" sequence that
+ * replaces normal indirect branch instructions. Differentiate
+ * *the* retpoline sequence from the LFENCE-prefixed indirect
+ * branches that simply use the retpoline infrastructure.
+ */
+static inline bool retpoline_seq_enabled(void)
+{
+ return boot_cpu_has(X86_FEATURE_RETPOLINE) && !boot_cpu_has(X86_FEATURE_RETPOLINE_LFENCE);
+}
+
#else
static inline const char *spectre_v2_module_string(void) { return ""; }
+static inline bool retpoline_seq_enabled(void) { return false; }
#endif
#define SPECTRE_V2_LFENCE_MSG "WARNING: LFENCE mitigation is not recommended for this CPU, data leaks possible!\n"
@@ -2095,8 +2109,7 @@ static void __init bhi_apply_mitigation(void)
return;
/* Retpoline mitigates against BHI unless the CPU has RRSBA behavior */
- if (boot_cpu_has(X86_FEATURE_RETPOLINE) &&
- !boot_cpu_has(X86_FEATURE_RETPOLINE_LFENCE)) {
+ if (retpoline_seq_enabled()) {
spec_ctrl_disable_kernel_rrsba();
if (rrsba_disabled)
return;
@@ -2238,6 +2251,27 @@ static void __init spectre_v2_update_mitigation(void)
pr_info("%s\n", spectre_v2_strings[spectre_v2_enabled]);
}
+#ifdef CONFIG_BPF_JIT
+static void __bpf_arch_ibpb(void *unused)
+{
+ write_ibpb();
+}
+
+void bpf_arch_ibpb(void)
+{
+ on_each_cpu(__bpf_arch_ibpb, NULL, 1);
+}
+
+static bool __init cpu_wants_ibpb_bpf(void)
+{
+ /* A genuine retpoline already neutralizes ring0 indirect predictions */
+ if (retpoline_seq_enabled())
+ return false;
+
+ return boot_cpu_has(X86_FEATURE_IBPB);
+}
+#endif
+
static void __init spectre_v2_apply_mitigation(void)
{
if (spectre_v2_enabled == SPECTRE_V2_EIBRS && unprivileged_ebpf_enabled())
@@ -2314,6 +2348,14 @@ static void __init spectre_v2_apply_mitigation(void)
setup_force_cpu_cap(X86_FEATURE_USE_IBRS_FW);
pr_info("Enabling Restricted Speculation for firmware calls\n");
}
+
+#ifdef CONFIG_BPF_JIT
+ if (cpu_wants_ibpb_bpf()) {
+ static_call_update(bpf_arch_pred_flush, bpf_arch_ibpb);
+ static_branch_enable(&bpf_pred_flush_enabled);
+ pr_info("Enabling IBPB for BPF\n");
+ }
+#endif
}
static void update_stibp_msr(void * __unused)
@@ -3490,9 +3532,7 @@ static const char *spectre_bhi_state(void)
return "; BHI: BHI_DIS_S";
else if (boot_cpu_has(X86_FEATURE_CLEAR_BHB_LOOP))
return "; BHI: SW loop, KVM: SW loop";
- else if (boot_cpu_has(X86_FEATURE_RETPOLINE) &&
- !boot_cpu_has(X86_FEATURE_RETPOLINE_LFENCE) &&
- rrsba_disabled)
+ else if (retpoline_seq_enabled() && rrsba_disabled)
return "; BHI: Retpoline";
else if (boot_cpu_has(X86_FEATURE_CLEAR_BHB_VMEXIT))
return "; BHI: Vulnerable, KVM: SW loop";
--
2.43.0
^ permalink raw reply related
* [PATCH 7.1.y 3/6] bpf: Restrict JIT predictor flush to cBPF
From: Pawan Gupta @ 2026-07-09 22:23 UTC (permalink / raw)
To: stable, Greg Kroah-Hartman, Sasha Levin
Cc: bpf, linux-arm-kernel, loongarch, linuxppc-dev, linux-riscv, x86,
Alexei Starovoitov, Daniel Borkmann
In-Reply-To: <20260709-cbpf-jit-spray-hardening-7-1-y-v1-0-5ac5a2d6797f@linux.intel.com>
commit 0bb99f2cfaae6822d734d69722de30af823efdf3 upstream.
Currently predictor flush on memory reuse is done for all BPF JIT
allocations, but only cBPF programs can be loaded by an unprivileged user.
eBPF is privileged by default, and flushing predictors for all CPUs on
every eBPF reuse penalizes the common case for no security benefit.
eBPF allocations can be frequent on busy systems, only flush predictors
for cBPF programs. Trampoline and dispatcher allocations also skip the
flush as they are eBPF-only.
Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
---
arch/arm64/net/bpf_jit_comp.c | 4 ++--
arch/loongarch/net/bpf_jit.c | 5 +++--
arch/powerpc/net/bpf_jit_comp.c | 4 ++--
arch/riscv/net/bpf_jit_comp64.c | 2 +-
arch/riscv/net/bpf_jit_core.c | 3 ++-
arch/x86/net/bpf_jit_comp.c | 5 +++--
include/linux/filter.h | 5 +++--
kernel/bpf/core.c | 13 ++++++++-----
kernel/bpf/dispatcher.c | 2 +-
9 files changed, 25 insertions(+), 18 deletions(-)
diff --git a/arch/arm64/net/bpf_jit_comp.c b/arch/arm64/net/bpf_jit_comp.c
index 0816c40fc7af..9491c987e50c 100644
--- a/arch/arm64/net/bpf_jit_comp.c
+++ b/arch/arm64/net/bpf_jit_comp.c
@@ -2094,7 +2094,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
image_size = extable_offset + extable_size;
ro_header = bpf_jit_binary_pack_alloc(image_size, &ro_image_ptr,
sizeof(u64), &header, &image_ptr,
- jit_fill_hole);
+ jit_fill_hole, was_classic);
if (!ro_header)
goto out_off;
@@ -2782,7 +2782,7 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
void *arch_alloc_bpf_trampoline(unsigned int size)
{
- return bpf_prog_pack_alloc(size, jit_fill_hole);
+ return bpf_prog_pack_alloc(size, jit_fill_hole, false);
}
void arch_free_bpf_trampoline(void *image, unsigned int size)
diff --git a/arch/loongarch/net/bpf_jit.c b/arch/loongarch/net/bpf_jit.c
index 24913dc7f4e8..e14a8aa47fc8 100644
--- a/arch/loongarch/net/bpf_jit.c
+++ b/arch/loongarch/net/bpf_jit.c
@@ -1762,7 +1762,7 @@ static int invoke_bpf(struct jit_ctx *ctx, struct bpf_tramp_links *tl,
void *arch_alloc_bpf_trampoline(unsigned int size)
{
- return bpf_prog_pack_alloc(size, jit_fill_hole);
+ return bpf_prog_pack_alloc(size, jit_fill_hole, false);
}
void arch_free_bpf_trampoline(void *image, unsigned int size)
@@ -2228,7 +2228,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
image_size = prog_size + extable_size;
/* Now we know the size of the structure to make */
ro_header = bpf_jit_binary_pack_alloc(image_size, &ro_image_ptr, sizeof(u32),
- &header, &image_ptr, jit_fill_hole);
+ &header, &image_ptr, jit_fill_hole,
+ bpf_prog_was_classic(prog));
if (!ro_header)
goto out_offset;
diff --git a/arch/powerpc/net/bpf_jit_comp.c b/arch/powerpc/net/bpf_jit_comp.c
index 53ab97ad6074..73ff8a64bb7f 100644
--- a/arch/powerpc/net/bpf_jit_comp.c
+++ b/arch/powerpc/net/bpf_jit_comp.c
@@ -295,7 +295,7 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
alloclen = proglen + FUNCTION_DESCR_SIZE + fixup_len + extable_len;
fhdr = bpf_jit_binary_pack_alloc(alloclen, &fimage, 4, &hdr, &image,
- bpf_jit_fill_ill_insns);
+ bpf_jit_fill_ill_insns, bpf_prog_was_classic(fp));
if (!fhdr)
goto out_err;
@@ -583,7 +583,7 @@ bool bpf_jit_inlines_helper_call(s32 imm)
void *arch_alloc_bpf_trampoline(unsigned int size)
{
- return bpf_prog_pack_alloc(size, bpf_jit_fill_ill_insns);
+ return bpf_prog_pack_alloc(size, bpf_jit_fill_ill_insns, false);
}
void arch_free_bpf_trampoline(void *image, unsigned int size)
diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c
index 2f1109dbf105..404a98a0fc90 100644
--- a/arch/riscv/net/bpf_jit_comp64.c
+++ b/arch/riscv/net/bpf_jit_comp64.c
@@ -1321,7 +1321,7 @@ int arch_bpf_trampoline_size(const struct btf_func_model *m, u32 flags,
void *arch_alloc_bpf_trampoline(unsigned int size)
{
- return bpf_prog_pack_alloc(size, bpf_fill_ill_insns);
+ return bpf_prog_pack_alloc(size, bpf_fill_ill_insns, false);
}
void arch_free_bpf_trampoline(void *image, unsigned int size)
diff --git a/arch/riscv/net/bpf_jit_core.c b/arch/riscv/net/bpf_jit_core.c
index 4365d07aaf54..ce3bd3762e08 100644
--- a/arch/riscv/net/bpf_jit_core.c
+++ b/arch/riscv/net/bpf_jit_core.c
@@ -109,7 +109,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
bpf_jit_binary_pack_alloc(prog_size + extable_size,
&jit_data->ro_image, sizeof(u32),
&jit_data->header, &jit_data->image,
- bpf_fill_ill_insns);
+ bpf_fill_ill_insns,
+ bpf_prog_was_classic(prog));
if (!jit_data->ro_header)
goto out_offset;
diff --git a/arch/x86/net/bpf_jit_comp.c b/arch/x86/net/bpf_jit_comp.c
index ea9e707e8abf..ef8b112fd7b6 100644
--- a/arch/x86/net/bpf_jit_comp.c
+++ b/arch/x86/net/bpf_jit_comp.c
@@ -3520,7 +3520,7 @@ static int __arch_prepare_bpf_trampoline(struct bpf_tramp_image *im, void *rw_im
void *arch_alloc_bpf_trampoline(unsigned int size)
{
- return bpf_prog_pack_alloc(size, jit_fill_hole);
+ return bpf_prog_pack_alloc(size, jit_fill_hole, false);
}
void arch_free_bpf_trampoline(void *image, unsigned int size)
@@ -3831,7 +3831,8 @@ struct bpf_prog *bpf_int_jit_compile(struct bpf_verifier_env *env, struct bpf_pr
/* allocate module memory for x86 insns and extable */
header = bpf_jit_binary_pack_alloc(roundup(proglen, align) + extable_size,
&image, align, &rw_header, &rw_image,
- jit_fill_hole);
+ jit_fill_hole,
+ bpf_prog_was_classic(prog));
if (!header)
goto out_addrs;
prog->aux->extable = (void *) image + roundup(proglen, align);
diff --git a/include/linux/filter.h b/include/linux/filter.h
index 2a7e6cbbbe1d..ea654453e43a 100644
--- a/include/linux/filter.h
+++ b/include/linux/filter.h
@@ -1315,7 +1315,7 @@ void bpf_jit_free(struct bpf_prog *fp);
struct bpf_binary_header *
bpf_jit_binary_pack_hdr(const struct bpf_prog *fp);
-void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns);
+void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns, bool was_classic);
void bpf_prog_pack_free(void *ptr, u32 size);
static inline bool bpf_prog_kallsyms_verify_off(const struct bpf_prog *fp)
@@ -1329,7 +1329,8 @@ bpf_jit_binary_pack_alloc(unsigned int proglen, u8 **ro_image,
unsigned int alignment,
struct bpf_binary_header **rw_hdr,
u8 **rw_image,
- bpf_jit_fill_hole_t bpf_fill_ill_insns);
+ bpf_jit_fill_hole_t bpf_fill_ill_insns,
+ bool was_classic);
int bpf_jit_binary_pack_finalize(struct bpf_binary_header *ro_header,
struct bpf_binary_header *rw_header);
void bpf_jit_binary_pack_free(struct bpf_binary_header *ro_header,
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index f49b9b23f95e..1b7e74e63bd4 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -942,7 +942,7 @@ static struct bpf_prog_pack *alloc_new_pack(bpf_jit_fill_hole_t bpf_fill_ill_ins
return NULL;
}
-void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns)
+void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns, bool was_classic)
{
unsigned int nbits = BPF_PROG_SIZE_TO_NBITS(size);
struct bpf_prog_pack *pack;
@@ -957,7 +957,7 @@ void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns)
* safe because cBPF programs (the unprivileged attack surface)
* are bounded well below a pack size.
*/
- if (static_branch_unlikely(&bpf_pred_flush_enabled))
+ if (was_classic && static_branch_unlikely(&bpf_pred_flush_enabled))
pr_warn_once("BPF: Predictors not flushed for allocations greater than BPF_PROG_PACK_SIZE\n");
size = round_up(size, PAGE_SIZE);
ptr = bpf_jit_alloc_exec(size);
@@ -989,7 +989,9 @@ void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns)
pos = 0;
found_free_area:
- static_call_cond(bpf_arch_pred_flush)();
+ /* Flush only for cBPF as it may contain a crafted gadget */
+ if (static_branch_unlikely(&bpf_pred_flush_enabled) && was_classic)
+ static_call_cond(bpf_arch_pred_flush)();
bitmap_set(pack->bitmap, pos, nbits);
ptr = (void *)(pack->ptr) + (pos << BPF_PROG_CHUNK_SHIFT);
@@ -1149,7 +1151,8 @@ bpf_jit_binary_pack_alloc(unsigned int proglen, u8 **image_ptr,
unsigned int alignment,
struct bpf_binary_header **rw_header,
u8 **rw_image,
- bpf_jit_fill_hole_t bpf_fill_ill_insns)
+ bpf_jit_fill_hole_t bpf_fill_ill_insns,
+ bool was_classic)
{
struct bpf_binary_header *ro_header;
u32 size, hole, start;
@@ -1162,7 +1165,7 @@ bpf_jit_binary_pack_alloc(unsigned int proglen, u8 **image_ptr,
if (bpf_jit_charge_modmem(size))
return NULL;
- ro_header = bpf_prog_pack_alloc(size, bpf_fill_ill_insns);
+ ro_header = bpf_prog_pack_alloc(size, bpf_fill_ill_insns, was_classic);
if (!ro_header) {
bpf_jit_uncharge_modmem(size);
return NULL;
diff --git a/kernel/bpf/dispatcher.c b/kernel/bpf/dispatcher.c
index b77db7413f8c..ea2d60dc1fee 100644
--- a/kernel/bpf/dispatcher.c
+++ b/kernel/bpf/dispatcher.c
@@ -145,7 +145,7 @@ void bpf_dispatcher_change_prog(struct bpf_dispatcher *d, struct bpf_prog *from,
mutex_lock(&d->mutex);
if (!d->image) {
- d->image = bpf_prog_pack_alloc(PAGE_SIZE, bpf_jit_fill_hole_with_zero);
+ d->image = bpf_prog_pack_alloc(PAGE_SIZE, bpf_jit_fill_hole_with_zero, false);
if (!d->image)
goto out;
d->rw_image = bpf_jit_alloc_exec(PAGE_SIZE);
--
2.43.0
^ permalink raw reply related
* [PATCH 7.1.y 4/6] bpf: Skip redundant IBPB in pack allocator
From: Pawan Gupta @ 2026-07-09 22:23 UTC (permalink / raw)
To: stable, Greg Kroah-Hartman, Sasha Levin
Cc: bpf, linux-arm-kernel, loongarch, linuxppc-dev, linux-riscv, x86,
Alexei Starovoitov, Daniel Borkmann
In-Reply-To: <20260709-cbpf-jit-spray-hardening-7-1-y-v1-0-5ac5a2d6797f@linux.intel.com>
commit a23c1c5396a91680703360d1ee28a44657c503c4 upstream.
bpf_prog_pack_alloc() issues IBPB on all CPUs on every cBPF allocation,
even when reusing chunks from an existing pack where no new memory was
touched since the last IBPB.
Since IBPB on all CPUs is heavy, Dave Hansen suggested to track allocation
since last IBPB, and only issue IBPB at reuse for the chunks that have not
seen an IBPB since they were last freed.
Track per-pack whether an IBPB is needed via arch_flush_needed. Set it when
allocating a chunk, reset on IBPB flush. On reuse, conditionally issue the
flush. Since IBPB invalidates all BTB entries, clear the flag on all packs
after flushing.
Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
---
kernel/bpf/core.c | 15 ++++++++++++++-
1 file changed, 14 insertions(+), 1 deletion(-)
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 1b7e74e63bd4..1bf4b8dffe91 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -876,6 +876,7 @@ int bpf_jit_add_poke_descriptor(struct bpf_prog *prog,
struct bpf_prog_pack {
struct list_head list;
void *ptr;
+ bool arch_flush_needed;
unsigned long bitmap[];
};
@@ -928,6 +929,8 @@ static struct bpf_prog_pack *alloc_new_pack(bpf_jit_fill_hole_t bpf_fill_ill_ins
bpf_fill_ill_insns(pack->ptr, BPF_PROG_PACK_SIZE);
bitmap_zero(pack->bitmap, BPF_PROG_PACK_SIZE / BPF_PROG_CHUNK_SIZE);
+ if (static_branch_unlikely(&bpf_pred_flush_enabled))
+ pack->arch_flush_needed = true;
set_vm_flush_reset_perms(pack->ptr);
err = set_memory_rox((unsigned long)pack->ptr,
BPF_PROG_PACK_SIZE / PAGE_SIZE);
@@ -990,8 +993,15 @@ void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns, bool
found_free_area:
/* Flush only for cBPF as it may contain a crafted gadget */
- if (static_branch_unlikely(&bpf_pred_flush_enabled) && was_classic)
+ if (static_branch_unlikely(&bpf_pred_flush_enabled) &&
+ pack->arch_flush_needed &&
+ was_classic) {
+ struct bpf_prog_pack *p;
+
static_call_cond(bpf_arch_pred_flush)();
+ list_for_each_entry(p, &pack_list, list)
+ p->arch_flush_needed = false;
+ }
bitmap_set(pack->bitmap, pos, nbits);
ptr = (void *)(pack->ptr) + (pos << BPF_PROG_CHUNK_SHIFT);
@@ -1029,6 +1039,9 @@ void bpf_prog_pack_free(void *ptr, u32 size)
"bpf_prog_pack bug: missing bpf_arch_text_invalidate?\n");
bitmap_clear(pack->bitmap, pos, nbits);
+
+ if (static_branch_unlikely(&bpf_pred_flush_enabled))
+ pack->arch_flush_needed = true;
if (bitmap_find_next_zero_area(pack->bitmap, BPF_PROG_CHUNK_COUNT, 0,
BPF_PROG_CHUNK_COUNT, 0) == 0) {
list_del(&pack->list);
--
2.43.0
^ permalink raw reply related
* [PATCH 7.1.y 5/6] bpf: Prefer packs that won't trigger an IBPB flush on allocation
From: Pawan Gupta @ 2026-07-09 22:24 UTC (permalink / raw)
To: stable, Greg Kroah-Hartman, Sasha Levin
Cc: bpf, linux-arm-kernel, loongarch, linuxppc-dev, linux-riscv, x86,
Alexei Starovoitov, Daniel Borkmann
In-Reply-To: <20260709-cbpf-jit-spray-hardening-7-1-y-v1-0-5ac5a2d6797f@linux.intel.com>
commit a9b1f19a6a673ba06820898d0f1ad02883ea1639 upstream.
Currently BPF pack allocator picks the chunks from the first available
pack. While this is okay, it naturally leads to more frequent flushes
when there are multiple packs in the system that weren't used since the
last flush.
As an optimization prefer allocating the new programs from packs that
are unused since last flush. When all packs are dirty, allocation forces
a flush and marks all packs clean.
Below are some future optimizations ideas:
1. Currently, the "dirty" tracking is only done at the pack-level.
Flush frequency can further be reduced with chunk-level tracking.
This requires a new bitmap per-pack to track the dirty state.
2. IBPB flush is done on all CPUs, even if only a single CPU ran the
BPF program. On a system with hundreds of CPUs this could be a
major bottleneck forcing hundreds of IPIs to deliver the flush.
The solution is to track the CPUs where a BPF program ran, and
issue IBPB only on those CPUs.
3. Avoid IBPB when flush is already done at other sources (e.g.
context switch).
Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
---
kernel/bpf/core.c | 27 ++++++++++++++++++++++++---
1 file changed, 24 insertions(+), 3 deletions(-)
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 1bf4b8dffe91..bba4acd61d41 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -948,8 +948,8 @@ static struct bpf_prog_pack *alloc_new_pack(bpf_jit_fill_hole_t bpf_fill_ill_ins
void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns, bool was_classic)
{
unsigned int nbits = BPF_PROG_SIZE_TO_NBITS(size);
- struct bpf_prog_pack *pack;
- unsigned long pos;
+ struct bpf_prog_pack *pack, *fallback_pack = NULL;
+ unsigned long pos, fallback_pos = 0;
void *ptr = NULL;
mutex_lock(&pack_mutex);
@@ -981,8 +981,29 @@ void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns, bool
list_for_each_entry(pack, &pack_list, list) {
pos = bitmap_find_next_zero_area(pack->bitmap, BPF_PROG_CHUNK_COUNT, 0,
nbits, 0);
- if (pos < BPF_PROG_CHUNK_COUNT)
+ if (pos >= BPF_PROG_CHUNK_COUNT)
+ continue;
+ /* Flush not enabled, use any pack */
+ if (!static_branch_unlikely(&bpf_pred_flush_enabled))
goto found_free_area;
+ /*
+ * cBPF reuse of a dirty pack triggers a flush, so prefer a
+ * clean pack for cBPF. eBPF never flushes, so pick the first
+ * free pack, dirty or clean.
+ */
+ if (!was_classic || !pack->arch_flush_needed)
+ goto found_free_area;
+ if (!fallback_pack) {
+ fallback_pack = pack;
+ fallback_pos = pos;
+ }
+ }
+
+ /* No preferred pack found */
+ if (fallback_pack) {
+ pack = fallback_pack;
+ pos = fallback_pos;
+ goto found_free_area;
}
pack = alloc_new_pack(bpf_fill_ill_insns);
--
2.43.0
^ permalink raw reply related
* [PATCH 7.1.y 6/6] bpf: Prefer dirty packs for eBPF allocations
From: Pawan Gupta @ 2026-07-09 22:24 UTC (permalink / raw)
To: stable, Greg Kroah-Hartman, Sasha Levin
Cc: bpf, linux-arm-kernel, loongarch, linuxppc-dev, linux-riscv, x86,
Alexei Starovoitov, Daniel Borkmann
In-Reply-To: <20260709-cbpf-jit-spray-hardening-7-1-y-v1-0-5ac5a2d6797f@linux.intel.com>
commit b72e29e0f7ee329d89f86db8700c8ea99b4a370a upstream.
The pack allocator only flushes predictors when reusing a dirty pack for
cBPF, eBPF allocations never trigger a flush. Currently, eBPF picks the
first free pack, which could be a clean pack. As an optimization, leaving
a clean pack for cBPF can avoid flushes.
Prefer dirty packs for eBPF and keep clean packs free for cBPF. This
mirrors the existing cBPF preference for clean packs: each program kind
prefers the pack that avoids an extra flush, and falls back to the other
kind only when no preferred pack has room. eBPF reuse of a dirty pack is
harmless since eBPF being privileged does not flush.
Signed-off-by: Pawan Gupta <pawan.kumar.gupta@linux.intel.com>
Acked-by: Daniel Borkmann <daniel@iogearbox.net>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
---
kernel/bpf/core.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index bba4acd61d41..de61e1894452 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -988,10 +988,10 @@ void *bpf_prog_pack_alloc(u32 size, bpf_jit_fill_hole_t bpf_fill_ill_insns, bool
goto found_free_area;
/*
* cBPF reuse of a dirty pack triggers a flush, so prefer a
- * clean pack for cBPF. eBPF never flushes, so pick the first
- * free pack, dirty or clean.
+ * clean pack for cBPF. eBPF never flushes, so steer it to a
+ * dirty pack and keep clean packs free for cBPF.
*/
- if (!was_classic || !pack->arch_flush_needed)
+ if (was_classic ^ pack->arch_flush_needed)
goto found_free_area;
if (!fallback_pack) {
fallback_pack = pack;
--
2.43.0
^ permalink raw reply related
page: next (older) | prev (newer) | latest
- recent:[subjects (threaded)|topics (new)|topics (active)]
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox