[PATCH bpf-next v1] bpf: Fix unaligned interpreter panic on JIT fallback path

[PATCH bpf-next v1] bpf: Fix unaligned interpreter panic on JIT fallback path

[Tiezhu Yang]

When an architecture implements bpf_jit_inlines_helper_call(), such
as LoongArch, ARM64, and RISC-V, the BPF verifier skips rewriting
the helper call offset (insn->imm) during the bpf_do_misc_fixups()
phase if the helper is expected to be inlined by the JIT compiler.
As a result, insn->imm remains as the raw helper enum ID.

However, if JIT is disabled at runtime (net.core.bpf_jit_enable=0)
or if the JIT compilation later dynamically fails (e.g., due to OOM
during bpf_jit_alloc_exec()), the core BPF subsystem falls back to
the BPF interpreter.

When the fallback interpreter executes (__bpf_call_base + insn->imm)
with the unpatched raw helper ID, it jumps into an unaligned invalid
address space, triggering a fatal instruction alignment fault (ADEF)
or illegal memory access kernel panic.

This issue impacts all architectures that support helper inlining.
Introduce a late fixup phase in __bpf_prog_select_runtime() to fix
it. When the JIT compilation is skipped or fails (!fp->jited), just
traverse the BPF instructions and rewrite the call offsets for the
inlined helpers right before transferring control to the interpreter,
ensuring a safe fallback path.

1. Test case (test_panic.c):

  #include <linux/bpf.h>
  #include <bpf/bpf_helpers.h>

  SEC("kprobe/sys_getpid")
  int test_panic(void *ctx)
  {
  	struct task_struct *task;

  	task = (struct task_struct *)bpf_get_current_task();
  	if (task)
  		bpf_printk("Task address: %p\n", task);

  	return 0;
  }

  char LICENSE[] SEC("license") = "GPL";

2. Reproduction steps on LoongArch:

  $ clang -target bpf -O2 -g -c test_panic.c -o test_panic.o
  $ sudo sysctl -w net.core.bpf_jit_enable=0
  $ sudo bpftool prog load test_panic.o /sys/fs/bpf/test_panic autoattach
  $ sudo cat /sys/kernel/debug/tracing/trace_pipe

3. Panic information on LoongArch:

  Kernel ade access[#1]:
  ...
      ra: 9000000000486e50 ___bpf_prog_run+0x1370/0x36b0
     ERA: 9000000000485383 __bpf_prog_ret0_warn+0x13/0x20
  ...
   ESTAT: 00080000 [ADEF] (IS= ECode=8 EsubCode=0)

Fixes: 2ddec2c80b44 ("riscv, bpf: inline bpf_get_smp_processor_id()")
Signed-off-by: Tiezhu Yang <yangtiezhu@loongson.cn>
---
 kernel/bpf/core.c | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)

diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
index 649cce41e13f..de0c46b495b4 100644
--- a/kernel/bpf/core.c
+++ b/kernel/bpf/core.c
@@ -2608,6 +2608,30 @@ static struct bpf_prog *bpf_prog_jit_compile(struct bpf_verifier_env *env, struc
 	return prog;
 }
 
+/*
+ * Rewrite the helper call offset for inlined helpers when fallback to
+ * the interpreter happens due to JIT compilation failure or JIT disabled.
+ */
+static void bpf_fixup_fallback_inline_helpers(struct bpf_verifier_env *env, struct bpf_prog *fp)
+{
+	struct bpf_insn *insn = fp->insnsi;
+	const struct bpf_func_proto *fn;
+	int i;
+
+	if (!env || !env->ops->get_func_proto)
+		return;
+
+	for (i = 0; i < fp->len; i++, insn++) {
+		if (insn->code == (BPF_JMP | BPF_CALL) && insn->src_reg == 0) {
+			if (bpf_jit_inlines_helper_call(insn->imm)) {
+				fn = env->ops->get_func_proto(insn->imm, fp);
+				if (fn && fn->func)
+					insn->imm = fn->func - __bpf_call_base;
+			}
+		}
+	}
+}
+
 struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct bpf_prog *fp,
 					   int *err)
 {
@@ -2639,6 +2663,15 @@ struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct
 
 		fp = bpf_prog_jit_compile(env, fp);
 		bpf_prog_jit_attempt_done(fp);
+
+		/*
+		 * If JIT compilation failed or is disabled (!fp->jited), we are
+		 * about to fall back to the interpreter path. Fix up the call
+		 * offsets to prevent unaligned memory access panic.
+		 */
+		if (!fp->jited)
+			bpf_fixup_fallback_inline_helpers(env, fp);
+
 		if (!fp->jited && jit_needed) {
 			*err = -ENOTSUPP;
 			return fp;
-- 
2.42.0

Re: [PATCH bpf-next v1] bpf: Fix unaligned interpreter panic on JIT fallback path

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Type confusion between BPF helper enum IDs and address offsets in `bpf_fixup_fallback_inline_helpers()` leads to potential silent instruction corruption.
--

commit e06144ee47e54fe6934472bff4024530f26b7737
Author: Tiezhu Yang <yangtiezhu@loongson.cn>

bpf: Fix unaligned interpreter panic on JIT fallback path

This commit introduces a fallback fixup phase to rewrite the call offsets
for inlined helpers when JIT compilation fails or is skipped. This prevents
unaligned memory access panics when falling back to the interpreter.

> diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
> index 649cce41e13f3..de0c46b495b40 100644
> --- a/kernel/bpf/core.c
> +++ b/kernel/bpf/core.c
[ ... ]
> @@ -2608,6 +2608,30 @@ static struct bpf_prog *bpf_prog_jit_compile(struct bpf_verifier_env *env, struc
>  	return prog;
>  }
>  
> +/*
> + * Rewrite the helper call offset for inlined helpers when fallback to
> + * the interpreter happens due to JIT compilation failure or JIT disabled.
> + */
> +static void bpf_fixup_fallback_inline_helpers(struct bpf_verifier_env *env, struct bpf_prog *fp)
> +{
> +	struct bpf_insn *insn = fp->insnsi;
> +	const struct bpf_func_proto *fn;
> +	int i;
> +
> +	if (!env || !env->ops->get_func_proto)
> +		return;
> +
> +	for (i = 0; i < fp->len; i++, insn++) {
> +		if (insn->code == (BPF_JMP | BPF_CALL) && insn->src_reg == 0) {
> +			if (bpf_jit_inlines_helper_call(insn->imm)) {

[Severity: High]
Could this cause a type confusion where a patched address offset is
misinterpreted as a helper enum ID?

Since bpf_do_misc_fixups() already patches insn->imm with a 32-bit address
offset (fn->func - __bpf_call_base) for helpers that are NOT expected to be
inlined by the JIT, insn->imm will already be an offset when reaching this
loop for non-inlined helpers.

If the distance between a non-inlined helper and __bpf_call_base happens to
match the enum ID of an inlined helper (for example,
BPF_FUNC_get_smp_processor_id = 8), wouldn't bpf_jit_inlines_helper_call()
falsely return true?

> +				fn = env->ops->get_func_proto(insn->imm, fp);
> +				if (fn && fn->func)
> +					insn->imm = fn->func - __bpf_call_base;

If the type confusion triggers, it seems this line would overwrite the
original, correct offset with the offset of an entirely different helper
function, potentially causing the interpreter to execute the wrong helper.

> +			}
> +		}
> +	}
> +}

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260611070009.26257-1-yangtiezhu@loongson.cn?part=1

Re: [PATCH bpf-next v1] bpf: Fix unaligned interpreter panic on JIT fallback path

[Leon Hwang]

On 11/6/26 15:00, Tiezhu Yang wrote:
[...]
>  
> +/*
> + * Rewrite the helper call offset for inlined helpers when fallback to
> + * the interpreter happens due to JIT compilation failure or JIT disabled.
> + */
> +static void bpf_fixup_fallback_inline_helpers(struct bpf_verifier_env *env, struct bpf_prog *fp)
> +{
> +	struct bpf_insn *insn = fp->insnsi;
> +	const struct bpf_func_proto *fn;
> +	int i;
> +
> +	if (!env || !env->ops->get_func_proto)
> +		return;
> +
> +	for (i = 0; i < fp->len; i++, insn++) {
> +		if (insn->code == (BPF_JMP | BPF_CALL) && insn->src_reg == 0) {
> +			if (bpf_jit_inlines_helper_call(insn->imm)) {
> +				fn = env->ops->get_func_proto(insn->imm, fp);
> +				if (fn && fn->func)
> +					insn->imm = fn->func - __bpf_call_base;

Might have pointer-to-integer warning?

> +			}
> +		}
> +	}
> +}
> +
>  struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct bpf_prog *fp,
>  					   int *err)
>  {
> @@ -2639,6 +2663,15 @@ struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct
>  
>  		fp = bpf_prog_jit_compile(env, fp);
>  		bpf_prog_jit_attempt_done(fp);
> +
> +		/*
> +		 * If JIT compilation failed or is disabled (!fp->jited), we are
> +		 * about to fall back to the interpreter path. Fix up the call
> +		 * offsets to prevent unaligned memory access panic.
> +		 */
> +		if (!fp->jited)
> +			bpf_fixup_fallback_inline_helpers(env, fp);
> +

Better to move it after (!fp->jited && jit_needed)?

Thanks,
Leon

>  		if (!fp->jited && jit_needed) {
>  			*err = -ENOTSUPP;
>  			return fp;

Re: [PATCH bpf-next v1] bpf: Fix unaligned interpreter panic on JIT fallback path

[Tiezhu Yang]

On 2026/6/11 下午3:18, sashiko-bot@kernel.org wrote:
> Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
> - [High] Type confusion between BPF helper enum IDs and address offsets in `bpf_fixup_fallback_inline_helpers()` leads to potential silent instruction corruption.
> --
> 
> commit e06144ee47e54fe6934472bff4024530f26b7737
> Author: Tiezhu Yang <yangtiezhu@loongson.cn>
> 
> bpf: Fix unaligned interpreter panic on JIT fallback path
> 
> This commit introduces a fallback fixup phase to rewrite the call offsets
> for inlined helpers when JIT compilation fails or is skipped. This prevents
> unaligned memory access panics when falling back to the interpreter.
> 
>> diff --git a/kernel/bpf/core.c b/kernel/bpf/core.c
>> index 649cce41e13f3..de0c46b495b40 100644
>> --- a/kernel/bpf/core.c
>> +++ b/kernel/bpf/core.c
> [ ... ]
>> @@ -2608,6 +2608,30 @@ static struct bpf_prog *bpf_prog_jit_compile(struct bpf_verifier_env *env, struc
>>   	return prog;
>>   }
>>   
>> +/*
>> + * Rewrite the helper call offset for inlined helpers when fallback to
>> + * the interpreter happens due to JIT compilation failure or JIT disabled.
>> + */
>> +static void bpf_fixup_fallback_inline_helpers(struct bpf_verifier_env *env, struct bpf_prog *fp)
>> +{
>> +	struct bpf_insn *insn = fp->insnsi;
>> +	const struct bpf_func_proto *fn;
>> +	int i;
>> +
>> +	if (!env || !env->ops->get_func_proto)
>> +		return;
>> +
>> +	for (i = 0; i < fp->len; i++, insn++) {
>> +		if (insn->code == (BPF_JMP | BPF_CALL) && insn->src_reg == 0) {
>> +			if (bpf_jit_inlines_helper_call(insn->imm)) {
> 
> [Severity: High]
> Could this cause a type confusion where a patched address offset is
> misinterpreted as a helper enum ID?
> 
> Since bpf_do_misc_fixups() already patches insn->imm with a 32-bit address
> offset (fn->func - __bpf_call_base) for helpers that are NOT expected to be
> inlined by the JIT, insn->imm will already be an offset when reaching this
> loop for non-inlined helpers.
> 
> If the distance between a non-inlined helper and __bpf_call_base happens to
> match the enum ID of an inlined helper (for example,
> BPF_FUNC_get_smp_processor_id = 8), wouldn't bpf_jit_inlines_helper_call()
> falsely return true?
> 
>> +				fn = env->ops->get_func_proto(insn->imm, fp);
>> +				if (fn && fn->func)
>> +					insn->imm = fn->func - __bpf_call_base;
> 
> If the type confusion triggers, it seems this line would overwrite the
> original, correct offset with the offset of an entirely different helper
> function, potentially causing the interpreter to execute the wrong helper.

To address the type confusion risk between raw helper enum IDs and
already-patched massive 32-bit address offsets (since regular helper
conversions leave src_reg at 0), I will apply a strict bounds check
against __BPF_FUNC_MAX_ID to safely isolate and filter out previously
fixed-up helper calls. Also, I have replaced fp with env->prog for
better type safety when calling get_func_proto().

Here is the updated function:

static void bpf_fixup_fallback_inline_helpers(struct bpf_verifier_env 
*env, struct bpf_prog *fp)
{
         struct bpf_insn *insn = fp->insnsi;
         const struct bpf_func_proto *fn;
         int i;

         if (!env || !env->ops->get_func_proto)
                 return;

         for (i = 0; i < fp->len; i++, insn++) {
                 if (insn->code == (BPF_JMP | BPF_CALL) && insn->src_reg 
== 0) {
                         /* Filter out already-patched address offsets. */
                         if (insn->imm >= __BPF_FUNC_MAX_ID)
                                 continue;

                         if (bpf_jit_inlines_helper_call(insn->imm)) {
                                 fn = 
env->ops->get_func_proto(insn->imm, env->prog);
                                 if (fn && fn->func)
                                         insn->imm = fn->func - 
__bpf_call_base;
                         }
                 }
         }
}

Thanks,
Tiezhu

Re: [PATCH bpf-next v1] bpf: Fix unaligned interpreter panic on JIT fallback path

[Tiezhu Yang]

On 2026/6/11 下午3:37, Leon Hwang wrote:
> On 11/6/26 15:00, Tiezhu Yang wrote:
> [...]
>>   
>> +/*
>> + * Rewrite the helper call offset for inlined helpers when fallback to
>> + * the interpreter happens due to JIT compilation failure or JIT disabled.
>> + */
>> +static void bpf_fixup_fallback_inline_helpers(struct bpf_verifier_env *env, struct bpf_prog *fp)
>> +{
>> +	struct bpf_insn *insn = fp->insnsi;
>> +	const struct bpf_func_proto *fn;
>> +	int i;
>> +
>> +	if (!env || !env->ops->get_func_proto)
>> +		return;
>> +
>> +	for (i = 0; i < fp->len; i++, insn++) {
>> +		if (insn->code == (BPF_JMP | BPF_CALL) && insn->src_reg == 0) {
>> +			if (bpf_jit_inlines_helper_call(insn->imm)) {
>> +				fn = env->ops->get_func_proto(insn->imm, fp);
>> +				if (fn && fn->func)
>> +					insn->imm = fn->func - __bpf_call_base;
> 
> Might have pointer-to-integer warning?

Thanks for the review!

I followed the existing implementation patterns in bpf_do_misc_fixups(),
where it explicitly assigns the delta directly without an explicit cast:

   insn->imm = fn->func - __bpf_call_base;

Keeping it this way ensures maximum consistency with the native code
style in the BPF subsystem, and it does not trigger pointer-to-integer
conversion warnings during my compilation tests.

>> +			}
>> +		}
>> +	}
>> +}
>> +
>>   struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct bpf_prog *fp,
>>   					   int *err)
>>   {
>> @@ -2639,6 +2663,15 @@ struct bpf_prog *__bpf_prog_select_runtime(struct bpf_verifier_env *env, struct
>>   
>>   		fp = bpf_prog_jit_compile(env, fp);
>>   		bpf_prog_jit_attempt_done(fp);
>> +
>> +		/*
>> +		 * If JIT compilation failed or is disabled (!fp->jited), we are
>> +		 * about to fall back to the interpreter path. Fix up the call
>> +		 * offsets to prevent unaligned memory access panic.
>> +		 */
>> +		if (!fp->jited)
>> +			bpf_fixup_fallback_inline_helpers(env, fp);
>> +
> 
> Better to move it after (!fp->jited && jit_needed)?

This is a very sharp catch!

Moving it after (!fp->jited && jit_needed) avoids redundant instruction
traversal on mandatory JIT failure paths. I will update the placement
in the next vision.

Thanks,
Tiezhu