[patch V5 00/31] rseq: Optimize exit to user space

[Thomas Gleixner <tglx@linutronix.de>]

This is a follow up on the V4 series, which can be found here:

   https://lore.kernel.org/all/20250908212737.353775467@linutronix.de

The V2 posting contains a detailed list of the addressed problems:

   https://lore.kernel.org/20250916163004.674341701@linutronix.de

TLDR:
    - A significant amount of pointless RSEQ operations on exit to user
      space, which have been reported by people as measurable impact after
      glibc switched to use RSEQ

    - Suboptimal hotpath handling both in the scheduler and on exit to user
      space.

This series addresses these issues by:

  1) Limiting the RSEQ work to the actual conditions where it is
     required. The full benefit is only available for architectures using
     the generic entry infrastructure. All others get at least the basic
     improvements.

  2) Re-implementing the whole user space handling based on proper data
     structures and by actually looking at the impact it creates in the
     fast path.

  3) Moving the actual handling of RSEQ out to the latest point in the exit
     path, where possible. This is fully inlined into the fast path to keep
     the impact confined.

Changes vs. V4:

  - Add a comment to the KVM retrigger logic - Sean

  - Reduce the fast path decision to event::sched_switch - Mathieu

  - Initialize IDs on registration, keep them on fork and lift the first
    exit restriction in the debug code - Mathieu

  - Update comments and fix typos - Mathieu

  - Adapted it to the uaccess changes

  - Dropped the already upstream parts and prerequisites

As for the previous version these patches have a dependency on the uaccess
scope series:

   https://lore.kernel.org/20251022102427.400699796@linutronix.de

which is available at:

    git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git uaccess/scoped

For your convenience the combination of both is available from git:

    git://git.kernel.org/pub/scm/linux/kernel/git/tglx/devel.git rseq/perf

Thanks,

	tglx
---
Thomas Gleixner (31):
      rseq: Avoid pointless evaluation in __rseq_notify_resume()
      rseq: Condense the inline stubs
      rseq: Move algorithm comment to top
      rseq: Remove the ksig argument from rseq_handle_notify_resume()
      rseq: Simplify registration
      rseq: Simplify the event notification
      rseq, virt: Retrigger RSEQ after vcpu_run()
      rseq: Avoid CPU/MM CID updates when no event pending
      rseq: Introduce struct rseq_data
      entry: Cleanup header
      entry: Remove syscall_enter_from_user_mode_prepare()
      entry: Inline irqentry_enter/exit_from/to_user_mode()
      sched: Move MM CID related functions to sched.h
      rseq: Cache CPU ID and MM CID values
      rseq: Record interrupt from user space
      rseq: Provide tracepoint wrappers for inline code
      rseq: Expose lightweight statistics in debugfs
      rseq: Provide static branch for runtime debugging
      rseq: Provide and use rseq_update_user_cs()
      rseq: Replace the original debug implementation
      rseq: Make exit debugging static branch based
      rseq: Use static branch for syscall exit debug when GENERIC_IRQ_ENTRY=y
      rseq: Provide and use rseq_set_ids()
      rseq: Separate the signal delivery path
      rseq: Rework the TIF_NOTIFY handler
      rseq: Optimize event setting
      rseq: Implement fast path for exit to user
      rseq: Switch to fast path processing on exit to user
      entry: Split up exit_to_user_mode_prepare()
      rseq: Split up rseq_exit_to_user_mode()
      rseq: Switch to TIF_RSEQ if supported

 Documentation/admin-guide/kernel-parameters.txt |    4 
 arch/x86/entry/syscall_32.c                     |    3 
 drivers/hv/mshv_root_main.c                     |    3 
 fs/binfmt_elf.c                                 |    2 
 fs/exec.c                                       |    2 
 include/asm-generic/thread_info_tif.h           |    3 
 include/linux/entry-common.h                    |   38 -
 include/linux/irq-entry-common.h                |   68 ++
 include/linux/mm.h                              |   25 
 include/linux/resume_user_mode.h                |    2 
 include/linux/rseq.h                            |  228 +++++---
 include/linux/rseq_entry.h                      |  592 +++++++++++++++++++++
 include/linux/rseq_types.h                      |   93 +++
 include/linux/sched.h                           |   48 +
 include/linux/thread_info.h                     |    5 
 include/trace/events/rseq.h                     |    4 
 include/uapi/linux/rseq.h                       |   21 
 init/Kconfig                                    |   28 -
 kernel/entry/common.c                           |   39 -
 kernel/entry/syscall-common.c                   |    8 
 kernel/ptrace.c                                 |    6 
 kernel/rseq.c                                   |  654 ++++++++++--------------
 kernel/sched/core.c                             |   10 
 kernel/sched/membarrier.c                       |    8 
 kernel/sched/sched.h                            |    5 
 virt/kvm/kvm_main.c                             |    7 
 26 files changed, 1301 insertions(+), 605 deletions(-)