[RFC PATCH bpf-next v1 06/16] bpf: Support special fields in typed arena objects

[Kumar Kartikeya Dwivedi <memxor@gmail.com>]

Let bpf_kptr_xchg() take a pointer to a referenced or percpu kptr field of a
typed arena object, so that objects in arena memory can own references to
kernel objects and to program-allocated objects. The exchange is a single
atomic word swap on a native address, so it needs nothing from the arena:
the field's record comes from the struct's BTF as it does for an allocated
object, the destination may carry the field's offset, and the value is
matched against the field's type as before. Direct loads and stores of the
field stay rejected, as they are for allocated objects; the exchange is the
only access. A reference left in an object is dropped when its chunk is
released or the map is destroyed, including one stored into a dummy object
of the scratch chunk by a program that used a pointer nobody allocated.

Kptrs are the only special fields a typed arena object may hold, and the
reason is how its memory behaves. A chunk is released after a grace period
that covers the invocations that started before the release was requested,
not the ones that start after it and cast into the range, and a fault in
the middle of a kfunc swaps the page under the object for the scratch
chunk, so the rest of the kfunc runs on the dummy object that every
unallocated slot shares. A field survives that only if each operation on it
is one atomic instruction, so that it lands on the real page or on the
dummy but is never split, and if the kernel keeps no pointer into the
object, so that nothing dangles once the memory is gone or reused. A kptr
is exactly that: the exchange is the only operation, and the referenced
object is held through the pointer value alone.

Timers, workqueues, task work and RCU heads are not. Their kfuncs take a
lock in the field and touch it several times, the async callback object
records the object's address for the callback, and an RCU head is linked
into the RCU callback list in place. A release or a fault in the middle
leaves a lock taken on a real page and dropped on the dummy, a callback
running on memory that now holds another object, or the dummy's RCU head
queued twice. Locks, lists, rbtrees and refcounts stay refused as well; the
memory is native, and a program builds those from typed pointer fields and
atomics without kernel help.

The way to give a typed object a timer, a workqueue, task work or an RCU
head is to allocate an object holding them with bpf_obj_new() and keep it
in a kptr field of the typed object. Allocated memory is owned by the
allocator and freed only by bpf_obj_drop() after its fields are cancelled,
so nothing swaps it or reuses it under a kfunc or a callback. That needs
those fields to be allowed in allocated objects, which a later patch adds.

Signed-off-by: Kumar Kartikeya Dwivedi <memxor@gmail.com>
---
 kernel/bpf/verifier.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 75697e52a2df..f854d8419fff 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -386,7 +386,7 @@ static struct btf_record *reg_btf_record(const struct bpf_reg_state *reg)
 
 	if (reg->type == PTR_TO_MAP_VALUE) {
 		rec = reg->map_ptr->record;
-	} else if (type_is_ptr_alloc_obj(reg->type)) {
+	} else if (type_is_ptr_alloc_obj(reg->type) || type_is_typed_arena_obj(reg->type)) {
 		meta = btf_find_struct_meta(reg->btf, reg->btf_id);
 		if (meta)
 			rec = meta->record;
@@ -8063,7 +8063,7 @@ static int process_kptr_func(struct bpf_verifier_env *env, int regno,
 	struct btf_record *rec;
 	u32 kptr_off;
 
-	if (type_is_ptr_alloc_obj(reg->type)) {
+	if (type_is_ptr_alloc_obj(reg->type) || type_is_typed_arena_obj(reg->type)) {
 		rec = reg_btf_record(reg);
 	} else { /* PTR_TO_MAP_VALUE */
 		map_ptr = reg->map_ptr;
@@ -8834,6 +8834,7 @@ static const struct bpf_reg_types kptr_xchg_dest_types = {
 		PTR_TO_BTF_ID | MEM_ALLOC,
 		PTR_TO_BTF_ID | MEM_ALLOC | NON_OWN_REF,
 		PTR_TO_BTF_ID | MEM_ALLOC | NON_OWN_REF | MEM_RCU,
+		PTR_TO_BTF_ID | MEM_ARENA,
 	}
 };
 static const struct bpf_reg_types dynptr_types = {
@@ -9154,6 +9155,7 @@ static int check_func_arg_reg_off(struct bpf_verifier_env *env,
 	case PTR_TO_BTF_ID | MEM_RCU:
 	case PTR_TO_BTF_ID | MEM_ALLOC | NON_OWN_REF:
 	case PTR_TO_BTF_ID | MEM_ALLOC | NON_OWN_REF | MEM_RCU:
+	case PTR_TO_BTF_ID | MEM_ARENA:
 		/* When referenced PTR_TO_BTF_ID is passed to release function,
 		 * its fixed offset must be 0. bpf_refcount_acquire() returns the
 		 * pointer it was given while incrementing the refcount at the
-- 
2.53.0