[PATCH bpf-next v14 0/5] emit ENDBR/BTI instructions for indirect

[Xu Kuohai <xukuohai@huaweicloud.com>]

From: Xu Kuohai <xukuohai@hotmail.com>

On architectures with CFI protection enabled that require landing pad
instructions at indirect jump targets, such as x86 with CET/IBT enabled
and arm64 with BTI enabled, kernel panics when an indirect jump lands on
a target without landing pad. Therefore, the JIT must emit landing pad
instructions for indirect jump targets.

The verifier already recognizes which instructions are indirect jump
targets during the verification phase. So we can store this information
in env->insn_aux_data and pass it to the JIT as new parameter, allowing
the JIT to consult env->insn_aux_data to determine which instructions are
indirect jump targets.

During JIT, constants blinding is performed. It rewrites the private copy
of instructions for the JITed program, but it does not adjust the global
env->insn_aux_data array. As a result, after constants blinding, the
instruction indexes used by JIT may no longer match the indexes in
env->insn_aux_data, so the JIT can not use env->insn_aux_data directly.

To avoid this mismatch, and given that all existing arch-specific JITs
already implement constants blinding with largely duplicated code, move
constants blinding from JIT to generic code.

v14:
- Rebase
- Fix comment style
- Fix incorrect variable and function name used in commit message

v13: https://lore.kernel.org/bpf/20260411133847.1042658-1-xukuohai@huaweicloud.com
- Use vmalloc to allocate memory for insn_aux_data copies to match with vfree
- Do not free the copied memory of insn_aux_data when restoring from failure
- Code cleanup

v12: https://lore.kernel.org/bpf/20260403132811.753894-1-xukuohai@huaweicloud.com
- Restore env->insn_aux_data on JIT failure
- Fix incorrect error code sign (-EFAULT vs EFAULT)
- Fix incorrect prog used in the restore path

v11: https://lore.kernel.org/bpf/20260403090915.473493-1-xukuohai@huaweicloud.com
- Restore env->subprog_info after jit_subprogs() fails
- Clear prog->jit_requested and prog->blinding_requested on failure
- Use the actual env->insn_aux_data size in clear_insn_aux_data() on failure 

v10: https://lore.kernel.org/bpf/20260324122052.342751-1-xukuohai@huaweicloud.com
- Fix the incorrect call_imm restore in jit_subprogs 
- Define a dummy void version of bpf_jit_prog_release_other and
  bpf_patch_insn_data when the corresponding config is not set
- Remove the unnecessary #ifdef in x86_64 JIT (Leon Hwang)

v9: https://lore.kernel.org/bpf/20260312170255.3427799-1-xukuohai@huaweicloud.com
- Make constant blinding available for classic bpf (Eduard)
- Clear prog->bpf_func, prog->jited ... on the error path of extra pass (Eduard)
- Fix spelling errors and remove unused parameter (Anton Protopopov)

v8: https://lore.kernel.org/bpf/20260309140044.2652538-1-xukuohai@huaweicloud.com
- Define void bpf_jit_blind_constants() function when CONFIG_BPF_JIT is not set 
- Move indirect_target fixup for insn patching from bpf_jit_blind_constants()
  to adjust_insn_aux_data()

v7: https://lore.kernel.org/bpf/20260307103949.2340104-1-xukuohai@huaweicloud.com
- Move constants blinding logic back to bpf/core.c
- Compute ip address before switch statement in x86 JIT
- Clear JIT state from error path on arm64 and loongarch 

v6: https://lore.kernel.org/bpf/20260306102329.2056216-1-xukuohai@huaweicloud.com
- Move constants blinding from JIT to verifier
- Move call to bpf_prog_select_runtime from bpf_prog_load to verifier

v5: https://lore.kernel.org/bpf/20260302102726.1126019-1-xukuohai@huaweicloud.com
- Switch to pass env to JIT directly to get rid of copying private insn_aux_data for
  each prog

v4: https://lore.kernel.org/all/20260114093914.2403982-1-xukuohai@huaweicloud.com
- Switch to the approach proposed by Eduard, using insn_aux_data to identify indirect
  jump targets, and emit ENDBR on x86

v3: https://lore.kernel.org/bpf/20251227081033.240336-1-xukuohai@huaweicloud.com
- Get rid of unnecessary enum definition (Yonghong Song, Anton Protopopov)

v2: https://lore.kernel.org/bpf/20251223085447.139301-1-xukuohai@huaweicloud.com
- Exclude instruction arrays not used for indirect jumps (Anton Protopopov)

v1: https://lore.kernel.org/bpf/20251127140318.3944249-1-xukuohai@huaweicloud.com

Xu Kuohai (5):
  bpf: Move constants blinding out of arch-specific JITs
  bpf: Pass bpf_verifier_env to JIT
  bpf: Add helper to detect indirect jump targets
  bpf, x86: Emit ENDBR for indirect jump targets
  bpf, arm64: Emit BTI for indirect jump target

 arch/arc/net/bpf_jit_core.c      |  41 +++-----
 arch/arm/net/bpf_jit_32.c        |  43 ++-------
 arch/arm64/net/bpf_jit_comp.c    |  87 ++++++-----------
 arch/loongarch/net/bpf_jit.c     |  61 ++++--------
 arch/mips/net/bpf_jit_comp.c     |  22 +----
 arch/parisc/net/bpf_jit_core.c   |  75 ++++++---------
 arch/powerpc/net/bpf_jit_comp.c  |  70 +++++---------
 arch/riscv/net/bpf_jit_core.c    |  63 +++++-------
 arch/s390/net/bpf_jit_comp.c     |  61 ++++--------
 arch/sparc/net/bpf_jit_comp_64.c |  63 +++++-------
 arch/x86/net/bpf_jit_comp.c      |  73 +++++---------
 arch/x86/net/bpf_jit_comp32.c    |  35 +------
 include/linux/bpf.h              |   2 +
 include/linux/bpf_verifier.h     |   9 +-
 include/linux/filter.h           |  50 +++++++++-
 kernel/bpf/core.c                | 138 ++++++++++++++++++--------
 kernel/bpf/fixups.c              | 160 ++++++++++++++++++++++++++-----
 kernel/bpf/syscall.c             |   4 -
 kernel/bpf/verifier.c            |  21 ++--
 19 files changed, 526 insertions(+), 552 deletions(-)

-- 
2.47.3