[PATCH bpf] bpf: check cpu_possible() in per-cpu helpers

["bo cao" <treebacker@foxmail.com>]

Two BPF helpers take a target CPU id as a raw u32 (arg_type
ARG_ANYTHING, so the verifier does not constrain it) and only reject
cpu >= nr_cpu_ids before the value is used as a per_cpu_ptr() index.

bpf_map_lookup_percpu_elem() dispatches to the per-cpu map
implementations, and all three of them (percpu array, percpu hash,
percpu LRU hash) check nothing beyond that upper bound:

	if (cpu >= nr_cpu_ids)
		return NULL;
	...
	return per_cpu_ptr(..., cpu);

bpf_per_cpu_ptr() has the same weak check before returning
per_cpu_ptr(ptr, cpu) for a kernel percpu variable.

Per-cpu memory is only allocated for possible CPUs, and
__per_cpu_offset[] is only initialized for them (on arm64,
setup_per_cpu_areas() walks for_each_possible_cpu).  On a system
with a sparse possible mask - e.g. an arm64 guest with a device-tree
CPU hole, possible mask 0,2-3 with nr_cpu_ids = 4 - a call with the
hole CPU id passes the weak check, and per_cpu_ptr() computes the
returned pointer from the uninitialized __per_cpu_offset[hole].

For bpf_map_lookup_percpu_elem() the helper then hands the BPF
program a pointer outside any valid per-cpu chunk of the map: reads
leak the stale bytes through the map value, and writes corrupt
whatever the offset happens to land on.  For bpf_per_cpu_ptr() the
leak target is kernel memory instead of map values (the return type
is marked MEM_RDONLY, so reads only).

Reproduced on an arm64 guest with possible mask 0,2-3 (Ubuntu
7.3.0-9-generic): a socket filter program passes the pointer
returned by bpf_map_lookup_percpu_elem(map, key, 1) on as the value
argument of bpf_map_update_elem(), and the copy inside
array_map_update_elem() faults on it:

  pc : __pi_memcpy_generic+0x3c/0x230
  lr : array_map_update_elem+0x148/0x2a8
  Call trace:
   __pi_memcpy_generic+0x3c/0x230 (P)
   bpf_prog_835a4ba2ea6755f0_poc+0x90/0xc0
   bpf_test_run+0x2d0/0x460
   bpf_prog_test_run_skb+0x4ac/0x920
   __sys_bpf+0x614/0xbe8
   __arm64_sys_bpf+0x38/0x78

The syscall side of the same problem (BPF_F_CPU map operation flags)
was fixed by ed54bf564ac5 ("bpf: Fix BPF_F_CPU validation for sparse
CPU IDs"), which added cpu_possible() validation to
bpf_map_check_op_flags().  The helper paths do not go through that
function, so the same class of hole CPU id needs to be rejected
there as well.

Fix the map helper centrally in its dispatch so all map
implementations are covered, and fix bpf_per_cpu_ptr() in place.

Fixes: 07343110b293 ("bpf: add bpf_map_lookup_percpu_elem for percpu map")
Fixes: eaa6bcb71ef6 ("bpf: Introduce bpf_per_cpu_ptr()")
Cc: stable@vger.kernel.org
Signed-off-by: bo cao <treebacker@foxmail.com>
---
 kernel/bpf/helpers.c | 6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)

diff --git a/kernel/bpf/helpers.c b/kernel/bpf/helpers.c
index 648abe1ba63a..f9f2bd665038 100644
--- a/kernel/bpf/helpers.c
+++ b/kernel/bpf/helpers.c
@@ -133,6 +133,10 @@ const struct bpf_func_proto bpf_map_peek_elem_proto = {
 BPF_CALL_3(bpf_map_lookup_percpu_elem, struct bpf_map *, map, void *, key, u32, cpu)
 {
 	WARN_ON_ONCE(!bpf_rcu_lock_held());
+
+	if (cpu >= nr_cpu_ids || !cpu_possible(cpu))
+		return 0;
+
 	return (unsigned long) map->ops->map_lookup_percpu_elem(map, key, cpu);
 }
 
@@ -753,7 +757,7 @@ const struct bpf_func_proto bpf_copy_from_user_task_proto = {
 
 BPF_CALL_2(bpf_per_cpu_ptr, const void *, ptr, u32, cpu)
 {
-	if (cpu >= nr_cpu_ids)
+	if (cpu >= nr_cpu_ids || !cpu_possible(cpu))
 		return (unsigned long)NULL;
 
 	return (unsigned long)per_cpu_ptr((const void __percpu *)(const uintptr_t)ptr, cpu);
-- 
2.39.5 (Apple Git-154)