[PATCH bpf-next v8 10/22] bpf: Do not use a private stack for a program that can unwind

[Yonghong Song <yonghong.song@linux.dev>]

A private stack keeps its frame pointer in %r9 on x86-64, and the JIT
brackets every call in push_r9/pop_r9. An unwind skips the pop: a frame
resumed at its landing pad addresses its stack through a stale pointer,
and one sent to its epilogue pops its callee-saved registers one slot off.

So no private stack for a program that can unwind, on every architecture
rather than just that one. A table is not the only condition: a
bpf_unwind() with no record over it sends its callers to their epilogues
just the same. In check_max_stack_depth(), force NO_PRIV_STACK so the JIT
does not use a private stack.

Signed-off-by: Yonghong Song <yonghong.song@linux.dev>
---
 kernel/bpf/verifier.c | 11 +++++++++++
 1 file changed, 11 insertions(+)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index f3ed68960d70..488ceb9dae1b 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -5769,6 +5769,17 @@ static int check_max_stack_depth(struct bpf_verifier_env *env)
 		}
 	}
 
+	/*
+	 * A private stack keeps its frame pointer in %r9 on x86-64, restored
+	 * by a pop after the call that an unwind skips. A frame resumed at a
+	 * pad then addresses its stack through a stale pointer, and a frame
+	 * sent to its epilogue instead pops its callee-saved registers one
+	 * slot off. Refuse a private stack for any program that can unwind,
+	 * on every arch for now.
+	 */
+	if (env->cleanup_info_cnt || bpf_prog_may_unwind(env))
+		priv_stack_mode = NO_PRIV_STACK;
+
 	if (priv_stack_mode == PRIV_STACK_UNKNOWN)
 		priv_stack_mode = bpf_enable_priv_stack(env->prog);
 
-- 
2.53.0-Meta