[PATCH v3] bpf, riscv: fix extable handling for arena load_acquire

[PATCH v3] bpf, riscv: fix extable handling for arena load_acquire

[Feng Jiang]

emit_atomic_ld_st() returns 1 to have build_body() skip the zext after
a sub-word load_acquire. The caller does "ret = ret ?:
add_exception_handler(...)", which skips add_exception_handler() on any
non-zero ret, so the extable entry is missing and a faulting
PROBE_ATOMIC load_acquire oopses.

REG_DONT_CLEAR_MARKER leaves rd stale on fault, and the verifier still
thinks the load overwrote it, so a program can leak it through a map.

Check ret >= 0 before calling add_exception_handler(), and pass rd for
LOAD_ACQ so the fault zeroes rd like a PROBE_MEM load. Return ret
unchanged for the zext skip.

Fixes: fb7cefabae81 ("riscv, bpf: Add support arena atomics for RV64")
Signed-off-by: Feng Jiang <jiangfeng@kylinos.cn>
---
Changes in v3:
- Pass rd to add_exception_handler() for LOAD_ACQ so the destination is
  zeroed on fault instead of REG_DONT_CLEAR_MARKER (Björn Töpel).
- Link to v2: https://lore.kernel.org/r/20260716-bpf-riscv-fix-extable-v2-1-421f75812533@kylinos.cn

Changes in v2:
- Drop extra variable as suggested by Pu Lehui.
- Link to v1: https://lore.kernel.org/r/20260716-bpf-riscv-fix-extable-v1-1-af023cc17af4@kylinos.cn
---
 arch/riscv/net/bpf_jit_comp64.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c
index f9d5347ba966..e1519ab4356c 100644
--- a/arch/riscv/net/bpf_jit_comp64.c
+++ b/arch/riscv/net/bpf_jit_comp64.c
@@ -1986,7 +1986,12 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx,
 		else
 			ret = emit_atomic_rmw(rd, rs, insn, ctx);
 
-		ret = ret ?: add_exception_handler(insn, REG_DONT_CLEAR_MARKER, ctx);
+		/* ret can be 1 (skip-zext); extable entry still needs to be added */
+		if (ret >= 0)
+			ret = add_exception_handler(insn,
+				insn->imm == BPF_LOAD_ACQ ? rd : REG_DONT_CLEAR_MARKER,
+				ctx) ?: ret;
+
 		if (ret)
 			return ret;
 		break;

---
base-commit: a13c140cc289c0b7b3770bce5b3ad42ab35074aa
change-id: 20260716-bpf-riscv-fix-extable-399947ca778d

Best regards,
-- 
Feng Jiang <jiangfeng@kylinos.cn>


_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

[PATCH v3] bpf, riscv: fix extable handling for arena load_acquire

[Feng Jiang]

emit_atomic_ld_st() returns 1 to have build_body() skip the zext after
a sub-word load_acquire. The caller does "ret = ret ?:
add_exception_handler(...)", which skips add_exception_handler() on any
non-zero ret, so the extable entry is missing and a faulting
PROBE_ATOMIC load_acquire oopses.

REG_DONT_CLEAR_MARKER leaves rd stale on fault, and the verifier still
thinks the load overwrote it, so a program can leak it through a map.

Check ret >= 0 before calling add_exception_handler(), and pass rd for
LOAD_ACQ so the fault zeroes rd like a PROBE_MEM load. Return ret
unchanged for the zext skip.

Fixes: fb7cefabae81 ("riscv, bpf: Add support arena atomics for RV64")
Signed-off-by: Feng Jiang <jiangfeng@kylinos.cn>
---
Changes in v3:
- Pass rd to add_exception_handler() for LOAD_ACQ so the destination is
  zeroed on fault instead of REG_DONT_CLEAR_MARKER (Björn Töpel).
- Link to v2: https://lore.kernel.org/r/20260716-bpf-riscv-fix-extable-v2-1-421f75812533@kylinos.cn

Changes in v2:
- Drop extra variable as suggested by Pu Lehui.
- Link to v1: https://lore.kernel.org/r/20260716-bpf-riscv-fix-extable-v1-1-af023cc17af4@kylinos.cn
---
 arch/riscv/net/bpf_jit_comp64.c | 7 ++++++-
 1 file changed, 6 insertions(+), 1 deletion(-)

diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c
index f9d5347ba966..e1519ab4356c 100644
--- a/arch/riscv/net/bpf_jit_comp64.c
+++ b/arch/riscv/net/bpf_jit_comp64.c
@@ -1986,7 +1986,12 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx,
 		else
 			ret = emit_atomic_rmw(rd, rs, insn, ctx);
 
-		ret = ret ?: add_exception_handler(insn, REG_DONT_CLEAR_MARKER, ctx);
+		/* ret can be 1 (skip-zext); extable entry still needs to be added */
+		if (ret >= 0)
+			ret = add_exception_handler(insn,
+				insn->imm == BPF_LOAD_ACQ ? rd : REG_DONT_CLEAR_MARKER,
+				ctx) ?: ret;
+
 		if (ret)
 			return ret;
 		break;

---
base-commit: a13c140cc289c0b7b3770bce5b3ad42ab35074aa
change-id: 20260716-bpf-riscv-fix-extable-399947ca778d

Best regards,
-- 
Feng Jiang <jiangfeng@kylinos.cn>

Re: [PATCH v3] bpf, riscv: fix extable handling for arena load_acquire

[sashiko-bot]

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:

Pre-existing issues:
- [High] When generating a `BPF_CMPXCHG` instruction without the `ZACAS` extension, `emit_cmpxchg()` emits a standard `lr`/`sc` loop but fails to set `ctx->ex_insn_off` and `ctx->ex_jmp_off`. This causes `add_exception_handler()` to silently skip adding an exception table entry, leading to an unhandled kernel page fault (Oops) if the atomic operation accesses invalid arena memory.
--

--- Patch [1]: [PATCH v3] bpf, riscv: fix extable handling for arena load_acquire ---
commit fa077ce2664d4c4babfa668d0c19369f9d827836
Author: Feng Jiang <jiangfeng@kylinos.cn>

bpf, riscv: fix extable handling for arena load_acquire

This patch modifies the BPF RISC-V JIT to ensure that an exception table
entry is correctly added for PROBE_ATOMIC load_acquire instructions. It also
ensures the destination register is zeroed on fault instead of being left
stale.

> diff --git a/arch/riscv/net/bpf_jit_comp64.c b/arch/riscv/net/bpf_jit_comp64.c
> index f9d5347ba966b..e1519ab4356c1 100644
> --- a/arch/riscv/net/bpf_jit_comp64.c
> +++ b/arch/riscv/net/bpf_jit_comp64.c
> @@ -1986,7 +1986,12 @@ int bpf_jit_emit_insn(const struct bpf_insn *insn, struct rv_jit_context *ctx,
>  		else
>  			ret = emit_atomic_rmw(rd, rs, insn, ctx);
>  
> -		ret = ret ?: add_exception_handler(insn, REG_DONT_CLEAR_MARKER, ctx);
> +		/* ret can be 1 (skip-zext); extable entry still needs to be added */
> +		if (ret >= 0)
> +			ret = add_exception_handler(insn,
> +				insn->imm == BPF_LOAD_ACQ ? rd : REG_DONT_CLEAR_MARKER,
> +				ctx) ?: ret;
> +

[Severity: High]
This isn't a bug introduced by this patch, but does emit_cmpxchg() fail to
set ctx->ex_insn_off and ctx->ex_jmp_off on machines without the ZACAS
extension?

Looking at emit_atomic_rmw(), it calls emit_cmpxchg() for BPF_CMPXCHG.
If emit_cmpxchg() falls back to a standard lr/sc loop:

	emit(is64 ? rv_lr_d(r0, 0, rd, 0, 0) :
	     rv_lr_w(r0, 0, rd, 0, 0), ctx);

Because ctx->ex_insn_off is left at 0, won't add_exception_handler() simply
return early without adding an exception table entry?

This seems like it could cause an unhandled kernel page fault (Oops) if a
PROBE_ATOMIC cmpxchg accesses invalid arena memory.

>  		if (ret)
>  			return ret;
>  		break;

-- 
Sashiko AI review · https://sashiko.dev/#/patchset/20260720-bpf-riscv-fix-extable-v3-1-e9a603882309@kylinos.cn?part=1

Re: [PATCH v3] bpf, riscv: fix extable handling for arena load_acquire

[Björn Töpel]

Feng Jiang <jiangfeng@kylinos.cn> writes:

> emit_atomic_ld_st() returns 1 to have build_body() skip the zext after
> a sub-word load_acquire. The caller does "ret = ret ?:
> add_exception_handler(...)", which skips add_exception_handler() on any
> non-zero ret, so the extable entry is missing and a faulting
> PROBE_ATOMIC load_acquire oopses.
>
> REG_DONT_CLEAR_MARKER leaves rd stale on fault, and the verifier still
> thinks the load overwrote it, so a program can leak it through a map.
>
> Check ret >= 0 before calling add_exception_handler(), and pass rd for
> LOAD_ACQ so the fault zeroes rd like a PROBE_MEM load. Return ret
> unchanged for the zext skip.
>
> Fixes: fb7cefabae81 ("riscv, bpf: Add support arena atomics for RV64")
> Signed-off-by: Feng Jiang <jiangfeng@kylinos.cn>

Thank you for spinning a new version!

Reviewed-by: Björn Töpel <bjorn@kernel.org>
Acked-by: Björn Töpel <bjorn@kernel.org>

Re: [PATCH v3] bpf, riscv: fix extable handling for arena load_acquire

[Björn Töpel]

Feng Jiang <jiangfeng@kylinos.cn> writes:

> emit_atomic_ld_st() returns 1 to have build_body() skip the zext after
> a sub-word load_acquire. The caller does "ret = ret ?:
> add_exception_handler(...)", which skips add_exception_handler() on any
> non-zero ret, so the extable entry is missing and a faulting
> PROBE_ATOMIC load_acquire oopses.
>
> REG_DONT_CLEAR_MARKER leaves rd stale on fault, and the verifier still
> thinks the load overwrote it, so a program can leak it through a map.
>
> Check ret >= 0 before calling add_exception_handler(), and pass rd for
> LOAD_ACQ so the fault zeroes rd like a PROBE_MEM load. Return ret
> unchanged for the zext skip.
>
> Fixes: fb7cefabae81 ("riscv, bpf: Add support arena atomics for RV64")
> Signed-off-by: Feng Jiang <jiangfeng@kylinos.cn>

Thank you for spinning a new version!

Reviewed-by: Björn Töpel <bjorn@kernel.org>
Acked-by: Björn Töpel <bjorn@kernel.org>

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3] bpf, riscv: fix extable handling for arena load_acquire

[Kumar Kartikeya Dwivedi]

On Tue Jul 21, 2026 at 8:17 PM CEST, Björn Töpel wrote:
> Feng Jiang <jiangfeng@kylinos.cn> writes:
>
>> emit_atomic_ld_st() returns 1 to have build_body() skip the zext after
>> a sub-word load_acquire. The caller does "ret = ret ?:
>> add_exception_handler(...)", which skips add_exception_handler() on any
>> non-zero ret, so the extable entry is missing and a faulting
>> PROBE_ATOMIC load_acquire oopses.
>>
>> REG_DONT_CLEAR_MARKER leaves rd stale on fault, and the verifier still
>> thinks the load overwrote it, so a program can leak it through a map.
>>
>> Check ret >= 0 before calling add_exception_handler(), and pass rd for
>> LOAD_ACQ so the fault zeroes rd like a PROBE_MEM load. Return ret
>> unchanged for the zext skip.
>>
>> Fixes: fb7cefabae81 ("riscv, bpf: Add support arena atomics for RV64")
>> Signed-off-by: Feng Jiang <jiangfeng@kylinos.cn>
>
> Thank you for spinning a new version!
>
> Reviewed-by: Björn Töpel <bjorn@kernel.org>
> Acked-by: Björn Töpel <bjorn@kernel.org>

There was an identical v4 resend. I applied the acks there and pushed.
Thanks.

_______________________________________________
linux-riscv mailing list
linux-riscv@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-riscv

Re: [PATCH v3] bpf, riscv: fix extable handling for arena load_acquire

[Kumar Kartikeya Dwivedi]

On Tue Jul 21, 2026 at 8:17 PM CEST, Björn Töpel wrote:
> Feng Jiang <jiangfeng@kylinos.cn> writes:
>
>> emit_atomic_ld_st() returns 1 to have build_body() skip the zext after
>> a sub-word load_acquire. The caller does "ret = ret ?:
>> add_exception_handler(...)", which skips add_exception_handler() on any
>> non-zero ret, so the extable entry is missing and a faulting
>> PROBE_ATOMIC load_acquire oopses.
>>
>> REG_DONT_CLEAR_MARKER leaves rd stale on fault, and the verifier still
>> thinks the load overwrote it, so a program can leak it through a map.
>>
>> Check ret >= 0 before calling add_exception_handler(), and pass rd for
>> LOAD_ACQ so the fault zeroes rd like a PROBE_MEM load. Return ret
>> unchanged for the zext skip.
>>
>> Fixes: fb7cefabae81 ("riscv, bpf: Add support arena atomics for RV64")
>> Signed-off-by: Feng Jiang <jiangfeng@kylinos.cn>
>
> Thank you for spinning a new version!
>
> Reviewed-by: Björn Töpel <bjorn@kernel.org>
> Acked-by: Björn Töpel <bjorn@kernel.org>

There was an identical v4 resend. I applied the acks there and pushed.
Thanks.