[PATCH bpf-next 2/4] bpf: Mark pending zero extension of arena ptrs before pruning a state

[Daniel Borkmann <daniel@iogearbox.net>]

bpf_mark_live_subregs_zext() marks the definitions of the registers live
at a pruning point so that a subreg_def whose only 64-bit read lies beyond
the prune is not left unmarked, but only SCALAR_VALUE currently.

PTR_TO_ARENA is the one other type that carries a real subreg_def and can
be read as a full 64-bit value: the addr_space_cast to arena (cast_kern)
records the cast insn in subreg_def, and the later 64-bit use (a load,
store or ALU64 with the pointer as source) is what marks it. The arena
access computes its address as pointer + arena_vm_start and trusts the
pointer's upper half to be zero, e.g. an indexed 'llgc %dst,off(%src,%arena)'
on s390 or a 'src + arena_vm_start' add on riscv64 and x86-64.

On x86-64 and riscv64 that upper half is cleared regardless of the mark:
x86-64 zero extends natively, and the riscv64 JIT emits its own zextw for
the cast. On a pure bpf_jit_needs_zext() architecture such as s390 the
cast_kern emits nothing and relies solely on the mark driven BPF_ZEXT_REG.
Thus, also mark the definitions of live PTR_TO_ARENA registers.

Fixes: 107e16979905 ("bpf: disable and remove registers chain based liveness")
Reported-by: STAR Labs SG <info@starlabs.sg>
Signed-off-by: Daniel Borkmann <daniel@iogearbox.net>
---
 kernel/bpf/verifier.c | 14 +++++++++-----
 1 file changed, 9 insertions(+), 5 deletions(-)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index e62b350b37af..4de9b464a9fc 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -3172,11 +3172,14 @@ static void mark_insn_zext(struct bpf_verifier_env *env,
  * which is only ever read as a sub-register also gets its definition marked,
  * at the cost of a zero extension that is not needed.
  *
- * Only scalars are considered since a live_regs_before bit does not imply that
- * the register holds a readable value: the caller saved regs of a frame below
+ * Scalars and arena pointers are considered, the two types that carry a
+ * subreg_def and can still be read as a full 64-bit value past this point.
+ * The arena case matters on a pure bpf_jit_needs_zext() architecture. Other
+ * types are skipped since a live_regs_before bit does not imply that the
+ * register holds a readable value: the caller saved regs of a frame below
  * the current one are clobbered to NOT_INIT at the call while keeping the
- * subreg_def of the call insn. Such a definition must not be marked, the call
- * insn has no destination register to zero extend.
+ * subreg_def of the call insn. Such a definition must not be marked, the
+ * call insn has no destination register to zero extend.
  */
 void bpf_mark_live_subregs_zext(struct bpf_verifier_env *env,
 				struct bpf_verifier_state *vstate)
@@ -3192,7 +3195,8 @@ void bpf_mark_live_subregs_zext(struct bpf_verifier_env *env,
 		for (j = 0; j < BPF_REG_FP; j++) {
 			if (!(live_regs & BIT(j)))
 				continue;
-			if (func->regs[j].type != SCALAR_VALUE)
+			if (func->regs[j].type != SCALAR_VALUE &&
+			    func->regs[j].type != PTR_TO_ARENA)
 				continue;
 			mark_insn_zext(env, &func->regs[j]);
 		}
-- 
2.43.0