* [PATCH bpf-next v6 1/2] bpf: Track borrowed iterator results by default
2026-09-27 12:04 [PATCH bpf-next v6 0/2] bpf: Track borrowed iterator result lifetimes Xu Yunxiang
@ 2026-09-27 12:04 ` Xu Yunxiang
2026-10-21 0:05 ` Alexei Starovoitov
2026-09-27 12:04 ` [PATCH bpf-next v6 2/2] selftests/bpf: Test borrowed iterator result lifetimes Xu Yunxiang
2026-10-14 3:07 ` [PATCH bpf-next v6 0/2] bpf: Track " Xu Yunxiang
2 siblings, 1 reply; 6+ messages in thread
From: Xu Yunxiang @ 2026-09-27 12:04 UTC (permalink / raw)
To: bpf
Cc: ast, daniel, andrii, eddyz87, memxor, ameryhung, yonghong.song,
puranjay
An open-coded iterator can return a struct pointer borrowed from its
current element. The verifier has not tied such results to iterator
state since the task_vma iterator was introduced, so a saved result
can still be used after destroy. Commit bee9ef4a40a27 ("bpf: switch
task_vma iterator from mmap_lock to per-VMA locks") also made next
release the protection of the prior element. A later snapshot
conversion made vm_file explicitly borrowed from that element.
Associate non-RCU, non-acquired struct results with the iterator
reference by default. Invalidate dependents before next forks its
NULL and non-NULL paths, while retaining the iterator itself. Let
destroy propagate child-reference errors. Add KF_ITER_RET_INDEPENDENT
for results that outlive the current element, preserving existing
RCU and acquired-reference rules.
Propagate the source lifetime through trusted field loads and dynptr
constructors so their descendants remain in the invalidation graph.
Reuse the reference-release traversal for next without releasing the
iterator's own reference.
Fixes: 4ac454682158 ("bpf: Introduce task_vma open-coded iterator kfuncs")
Suggested-by: Amery Hung <ameryhung@gmail.com>
Assisted-by: LLM
Signed-off-by: Xu Yunxiang <xyx2021@mail.ustc.edu.cn>
---
Documentation/bpf/bpf_iterators.rst | 10 +++
include/linux/btf.h | 1 +
kernel/bpf/btf.c | 13 +++-
kernel/bpf/verifier.c | 103 +++++++++++++++++++++++-----
4 files changed, 108 insertions(+), 19 deletions(-)
diff --git a/Documentation/bpf/bpf_iterators.rst b/Documentation/bpf/bpf_iterators.rst
index 189e3ec1c6c8e..72435e32ffa16 100644
--- a/Documentation/bpf/bpf_iterators.rst
+++ b/Documentation/bpf/bpf_iterators.rst
@@ -86,6 +86,16 @@ Constructor/next/destructor implementation contract is as follows:
resources and marks stack space used by `struct bpf_iter_<type>` as usable
for something else. Destructor is marked with KF_ITER_DESTROY flag.
+The verifier treats a non-RCU, non-acquired struct result as borrowed from
+the iterator's current element. It invalidates the previous result and its
+tracked derived pointers on the next call, including when it returns NULL,
+and on destruction of the iterator. Resources borrowing that lifetime must
+be released first. A next method whose struct result has an independent
+lifetime must use ``KF_ITER_RET_INDEPENDENT``. This flag requires
+``KF_ITER_NEXT`` and a struct pointer return type, and cannot be combined
+with ``KF_ACQUIRE``. RCU-protected and acquired results retain their own
+lifetime rules.
+
Any open-coded BPF iterator implementation has to implement at least these
three methods. It is enforced that for any given type of iterator only
applicable constructor/destructor/next are callable. I.e., verifier ensures
diff --git a/include/linux/btf.h b/include/linux/btf.h
index 4b63bb91550a1..3c2f92e517849 100644
--- a/include/linux/btf.h
+++ b/include/linux/btf.h
@@ -81,6 +81,7 @@
#define KF_IMPLICIT_ARGS (1 << 16) /* kfunc has implicit arguments supplied by the verifier */
#define KF_SPINLOCK_SAFE (1 << 17) /* kfunc is allowed inside bpf_spin_lock-ed region */
#define KF_PERFMON (1 << 18) /* kfunc requires CAP_PERFMON */
+#define KF_ITER_RET_INDEPENDENT (1 << 19) /* next result outlives the current element */
/*
* Tag marking a kernel function as a kfunc. This is meant to minimize the
diff --git a/kernel/bpf/btf.c b/kernel/bpf/btf.c
index 9bcfefdfb7346..df7545e16995c 100644
--- a/kernel/bpf/btf.c
+++ b/kernel/bpf/btf.c
@@ -9245,6 +9245,10 @@ static int btf_check_iter_kfuncs(struct btf *btf, const char *func_name,
if (!flags || (flags & (flags - 1)))
return -EINVAL;
+ if ((func_flags & KF_ITER_RET_INDEPENDENT) &&
+ (!(flags & KF_ITER_NEXT) || (func_flags & KF_ACQUIRE)))
+ return -EINVAL;
+
/* any BPF iter kfunc should have `struct bpf_iter_<type> *` first arg */
nr_args = btf_type_vlen(func);
if (nr_args < 1)
@@ -9285,6 +9289,12 @@ static int btf_check_iter_kfuncs(struct btf *btf, const char *func_name,
t = btf_type_skip_modifiers(btf, func->type, NULL);
if (!t || !btf_type_is_ptr(t))
return -EINVAL;
+ /* The opt-out applies only to non-acquired struct results. */
+ if (func_flags & KF_ITER_RET_INDEPENDENT) {
+ t = btf_type_skip_modifiers(btf, t->type, NULL);
+ if (!t || !__btf_type_is_struct(t))
+ return -EINVAL;
+ }
}
if (flags & KF_ITER_DESTROY) {
@@ -9351,7 +9361,8 @@ static int btf_check_kfunc_protos(struct btf *btf, u32 func_id, u32 func_flags)
if (!func || !btf_type_is_func_proto(func))
return -EINVAL;
- if (func_flags & (KF_ITER_NEW | KF_ITER_NEXT | KF_ITER_DESTROY)) {
+ if (func_flags & (KF_ITER_NEW | KF_ITER_NEXT | KF_ITER_DESTROY |
+ KF_ITER_RET_INDEPENDENT)) {
err = btf_check_iter_kfuncs(btf, func_name, func, func_flags);
if (err)
return err;
diff --git a/kernel/bpf/verifier.c b/kernel/bpf/verifier.c
index 03dbc0e003986..4c8aa607cb86f 100644
--- a/kernel/bpf/verifier.c
+++ b/kernel/bpf/verifier.c
@@ -209,6 +209,10 @@ static int acquire_reference(struct bpf_verifier_env *env, int insn_idx, int par
static int __release_reference_nomark(struct bpf_verifier_state *state, int id);
static int release_reference_nomark(struct bpf_verifier_env *env, int id);
static int release_reference(struct bpf_verifier_env *env, int id);
+static int invalidate_reference_dependents(struct bpf_verifier_env *env, int id);
+static int check_reference_children_leak(struct bpf_verifier_env *env, int parent_id);
+static u32 reg_lifetime_id(struct bpf_verifier_env *env,
+ const struct bpf_reg_state *reg);
static void invalidate_non_owning_refs(struct bpf_verifier_env *env);
static void invalidate_rcu_protected_refs(struct bpf_verifier_env *env);
static bool in_rbtree_lock_required_cb(struct bpf_verifier_env *env);
@@ -765,8 +769,8 @@ static int mark_stack_slots_dynptr(struct bpf_verifier_env *env, struct bpf_reg_
if (err)
return err;
- /* Track parent's id if the parent is a referenced object */
- parent_id = ref_obj->id;
+ /* All non-clone constructors take their backing object in R1. */
+ parent_id = reg_lifetime_id(env, &cur_regs(env)[BPF_REG_1]);
if (dynptr_type_referenced(type)) {
int id;
@@ -1041,7 +1045,7 @@ static int unmark_stack_slots_iter(struct bpf_verifier_env *env,
struct bpf_reg_state *reg, int nr_slots)
{
struct bpf_func_state *state = bpf_func(env, reg);
- int spi, i, j;
+ int spi, i, j, err;
spi = iter_get_spi(env, reg, nr_slots);
if (spi < 0)
@@ -1051,8 +1055,11 @@ static int unmark_stack_slots_iter(struct bpf_verifier_env *env,
struct bpf_stack_state *slot = bpf_stack_slot(state, spi - i);
struct bpf_reg_state *st = &slot->spilled_ptr;
- if (i == 0)
- WARN_ON_ONCE(release_reference(env, st->id));
+ if (i == 0) {
+ err = release_reference(env, st->id);
+ if (err)
+ return err;
+ }
bpf_mark_reg_not_init(env, st);
@@ -1594,6 +1601,12 @@ static bool reg_is_referenced(struct bpf_verifier_env *env, const struct bpf_reg
return find_reference_state(env->cur_state, reg->id);
}
+static u32 reg_lifetime_id(struct bpf_verifier_env *env,
+ const struct bpf_reg_state *reg)
+{
+ return reg_is_referenced(env, reg) ? reg->id : reg->parent_id;
+}
+
static int release_lock_state(struct bpf_verifier_env *env, int type, int id, void *ptr)
{
struct bpf_verifier_state *state = env->cur_state;
@@ -6490,9 +6503,14 @@ static int check_ptr_to_btf_access(struct bpf_verifier_env *env,
}
if (atype == BPF_READ && value_regno >= 0) {
+ u32 parent_id = reg_lifetime_id(env, reg);
+
ret = mark_btf_ld_reg(env, regs, value_regno, ret, reg->btf, btf_id, flag);
if (ret < 0)
return ret;
+ if ((regs[value_regno].type & PTR_TRUSTED) &&
+ !(regs[value_regno].type & (MEM_RCU | MEM_PERCPU | MEM_USER)))
+ regs[value_regno].parent_id = parent_id;
}
return 0;
@@ -8251,6 +8269,21 @@ static bool is_iter_destroy_kfunc(struct bpf_call_arg_meta *meta)
return meta->kfunc_flags & KF_ITER_DESTROY;
}
+static bool iter_next_borrows_struct(struct bpf_call_arg_meta *meta)
+{
+ const struct btf_type *t;
+
+ if (!(meta->kfunc_flags & KF_ITER_NEXT) ||
+ (meta->kfunc_flags & (KF_ITER_RET_INDEPENDENT | KF_ACQUIRE)))
+ return false;
+
+ t = btf_type_skip_modifiers(meta->btf, meta->func_proto->type, NULL);
+ if (!t || !btf_type_is_ptr(t))
+ return false;
+ t = btf_type_skip_modifiers(meta->btf, t->type, NULL);
+ return t && __btf_type_is_struct(t);
+}
+
static bool is_kfunc_arg_iter(struct bpf_call_arg_meta *meta, int arg_idx,
const struct btf_param *arg)
{
@@ -8361,6 +8394,14 @@ static int process_iter_arg(struct bpf_verifier_env *env, struct bpf_reg_state *
meta->iter.frameno = reg->frameno;
update_ref_obj(&meta->ref_obj, &bpf_stack_slot(state, spi)->spilled_ptr);
+ if (iter_next_borrows_struct(meta) &&
+ !is_kfunc_rcu_protected(meta) &&
+ !(bpf_stack_slot(state, spi)->spilled_ptr.type & MEM_RCU)) {
+ err = invalidate_reference_dependents(env, meta->ref_obj.id);
+ if (err)
+ return err;
+ }
+
if (is_iter_destroy_kfunc(meta)) {
err = unmark_stack_slots_iter(env, reg, nr_slots);
if (err)
@@ -10578,8 +10619,25 @@ static int idstack_pop(struct bpf_idmap *idmap)
return idmap->map[--idmap->cnt].old;
}
-/* Release id and objects derived from it iteratively in a DFS manner */
-static int release_reference(struct bpf_verifier_env *env, int id)
+static int check_reference_children_leak(struct bpf_verifier_env *env, int parent_id)
+{
+ struct bpf_verifier_state *state = env->cur_state;
+ int i;
+
+ for (i = 0; i < state->acquired_refs; i++) {
+ if (state->refs[i].type != REF_TYPE_PTR ||
+ state->refs[i].parent_id != parent_id)
+ continue;
+ verbose(env, "Leaking reference id=%d alloc_insn=%d. Release it first.\n",
+ state->refs[i].id, state->refs[i].insn_idx);
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
+/* Invalidate id and its descendants, optionally retaining the root reference. */
+static int __release_reference(struct bpf_verifier_env *env, int id, bool release_root)
{
u32 mask = (1 << STACK_SPILL) | (1 << STACK_DYNPTR);
struct bpf_verifier_state *vstate = env->cur_state;
@@ -10587,14 +10645,15 @@ static int release_reference(struct bpf_verifier_env *env, int id)
struct bpf_stack_state *stack;
struct bpf_func_state *state;
struct bpf_reg_state *reg;
- int i, err;
+ int root_id = id;
+ int err;
idstack->cnt = 0;
err = idstack_push(idstack, id);
if (err)
return err;
- if (find_reference_state(vstate, id)) {
+ if (release_root && find_reference_state(vstate, id)) {
err = release_reference_nomark(env, id);
WARN_ON_ONCE(err);
}
@@ -10604,19 +10663,15 @@ static int release_reference(struct bpf_verifier_env *env, int id)
* Child references are inaccessible after parent is released,
* any child references that exist at this point are a leak.
*/
- for (i = 0; i < vstate->acquired_refs; i++) {
- if (vstate->refs[i].type != REF_TYPE_PTR)
- continue;
- if (vstate->refs[i].parent_id != id)
- continue;
- verbose(env, "Leaking reference id=%d alloc_insn=%d. Release it first.\n",
- vstate->refs[i].id, vstate->refs[i].insn_idx);
- return -EINVAL;
- }
+ err = check_reference_children_leak(env, id);
+ if (err)
+ return err;
bpf_for_each_reg_in_vstate_mask(vstate, state, reg, stack, mask, ({
if (reg->id != id && reg->parent_id != id)
continue;
+ if (!release_root && id == root_id && reg->parent_id != id)
+ continue;
/* Free objects derived from the current object */
if (reg->parent_id == id) {
@@ -10650,6 +10705,16 @@ static int release_reference(struct bpf_verifier_env *env, int id)
return 0;
}
+static int release_reference(struct bpf_verifier_env *env, int id)
+{
+ return __release_reference(env, id, true);
+}
+
+static int invalidate_reference_dependents(struct bpf_verifier_env *env, int id)
+{
+ return __release_reference(env, id, false);
+}
+
static void invalidate_non_owning_refs(struct bpf_verifier_env *env)
{
struct bpf_func_state *unused;
@@ -15153,6 +15218,8 @@ static int check_kfunc_call(struct bpf_verifier_env *env, struct bpf_insn *insn,
regs[BPF_REG_0].btf = desc_btf;
regs[BPF_REG_0].type = type;
regs[BPF_REG_0].btf_id = ptr_type_id;
+ if (iter_next_borrows_struct(&meta) && !(type & MEM_RCU))
+ regs[BPF_REG_0].parent_id = meta.ref_obj.id;
}
if (is_kfunc_ret_null(&meta)) {
--
2.43.0
^ permalink raw reply related [flat|nested] 6+ messages in thread* [PATCH bpf-next v6 2/2] selftests/bpf: Test borrowed iterator result lifetimes
2026-09-27 12:04 [PATCH bpf-next v6 0/2] bpf: Track borrowed iterator result lifetimes Xu Yunxiang
2026-09-27 12:04 ` [PATCH bpf-next v6 1/2] bpf: Track borrowed iterator results by default Xu Yunxiang
@ 2026-09-27 12:04 ` Xu Yunxiang
2026-09-27 20:28 ` bot+bpf-ci
2026-10-14 3:07 ` [PATCH bpf-next v6 0/2] bpf: Track " Xu Yunxiang
2 siblings, 1 reply; 6+ messages in thread
From: Xu Yunxiang @ 2026-09-27 12:04 UTC (permalink / raw)
To: bpf
Cc: ast, daniel, andrii, eddyz87, memxor, ameryhung, yonghong.song,
puranjay
Exercise borrowed VMA results before and after iterator advancement
and destruction, including NULL next, trusted fields, and a FILE
dynptr that must be discarded before advancing. Keep positive controls
for using the current element and discarding the dynptr first.
Use a test-module iterator returning current to verify the explicit
independent-lifetime exception. Retain controls for RCU-protected
results. The annotated programs check verifier loading and diagnostics;
they do not request runtime execution of the added programs.
Assisted-by: LLM
Signed-off-by: Xu Yunxiang <xyx2021@mail.ustc.edu.cn>
---
.../selftests/bpf/progs/iters_testmod.c | 370 ++++++++++++++++++
.../selftests/bpf/test_kmods/bpf_testmod.c | 25 ++
.../selftests/bpf/test_kmods/bpf_testmod.h | 4 +
.../bpf/test_kmods/bpf_testmod_kfunc.h | 5 +
4 files changed, 404 insertions(+)
diff --git a/tools/testing/selftests/bpf/progs/iters_testmod.c b/tools/testing/selftests/bpf/progs/iters_testmod.c
index f65cc9766633e..18b99f1684dfa 100644
--- a/tools/testing/selftests/bpf/progs/iters_testmod.c
+++ b/tools/testing/selftests/bpf/progs/iters_testmod.c
@@ -8,6 +8,44 @@
char _license[] SEC("license") = "GPL";
+struct bpf_iter_testmod_current {
+ u64 __opaque;
+};
+
+SEC("raw_tp/sys_enter")
+__success
+int iter_independent_lifetime_after_next(const void *ctx)
+{
+ struct bpf_iter_testmod_current it;
+ struct task_struct *task;
+
+ bpf_iter_testmod_current_new(&it);
+ task = bpf_iter_testmod_current_next(&it);
+ if (!task)
+ goto out;
+
+ bpf_iter_testmod_current_next(&it);
+ bpf_kfunc_trusted_task_test(task);
+out:
+ bpf_iter_testmod_current_destroy(&it);
+ return 0;
+}
+
+SEC("raw_tp/sys_enter")
+__success
+int iter_independent_lifetime_after_destroy(const void *ctx)
+{
+ struct bpf_iter_testmod_current it;
+ struct task_struct *task;
+
+ bpf_iter_testmod_current_new(&it);
+ task = bpf_iter_testmod_current_next(&it);
+ bpf_iter_testmod_current_destroy(&it);
+ if (task)
+ bpf_kfunc_trusted_task_test(task);
+ return 0;
+}
+
SEC("raw_tp/sys_enter")
__success
int iter_next_trusted(const void *ctx)
@@ -28,6 +66,298 @@ int iter_next_trusted(const void *ctx)
return 0;
}
+SEC("raw_tp/sys_enter")
+__failure __msg("invalid mem access 'scalar'")
+int iter_next_trusted_after_destroy(const void *ctx)
+{
+ struct task_struct *cur_task = bpf_get_current_task_btf();
+ struct bpf_iter_task_vma vma_it;
+ struct vm_area_struct *vma_ptr;
+
+ bpf_iter_task_vma_new(&vma_it, cur_task, 0);
+
+ vma_ptr = bpf_iter_task_vma_next(&vma_it);
+ if (!vma_ptr)
+ goto out;
+
+ bpf_iter_task_vma_destroy(&vma_it);
+ return vma_ptr->vm_start;
+out:
+ bpf_iter_task_vma_destroy(&vma_it);
+ return 0;
+}
+
+SEC("raw_tp/sys_enter")
+__failure __msg("invalid mem access 'scalar'")
+int iter_next_trusted_after_next(const void *ctx)
+{
+ struct task_struct *cur_task = bpf_get_current_task_btf();
+ struct bpf_iter_task_vma vma_it;
+ struct vm_area_struct *vma_ptr, *next_vma_ptr;
+ u64 vm_start;
+
+ bpf_iter_task_vma_new(&vma_it, cur_task, 0);
+
+ vma_ptr = bpf_iter_task_vma_next(&vma_it);
+ if (!vma_ptr)
+ goto out;
+
+ next_vma_ptr = bpf_iter_task_vma_next(&vma_it);
+ if (!next_vma_ptr)
+ goto out;
+
+ vm_start = vma_ptr->vm_start;
+ bpf_iter_task_vma_destroy(&vma_it);
+ return vm_start;
+out:
+ bpf_iter_task_vma_destroy(&vma_it);
+ return 0;
+}
+
+SEC("raw_tp/sys_enter")
+__failure __msg("invalid mem access 'scalar'")
+int iter_next_trusted_after_next_null(const void *ctx)
+{
+ struct task_struct *cur_task = bpf_get_current_task_btf();
+ struct bpf_iter_task_vma vma_it;
+ struct vm_area_struct *vma_ptr, *next_vma_ptr;
+ u64 vm_start;
+
+ bpf_iter_task_vma_new(&vma_it, cur_task, 0);
+
+ vma_ptr = bpf_iter_task_vma_next(&vma_it);
+ if (!vma_ptr)
+ goto out;
+
+ next_vma_ptr = bpf_iter_task_vma_next(&vma_it);
+ if (next_vma_ptr)
+ goto out;
+
+ vm_start = vma_ptr->vm_start;
+ bpf_iter_task_vma_destroy(&vma_it);
+ return vm_start;
+out:
+ bpf_iter_task_vma_destroy(&vma_it);
+ return 0;
+}
+
+SEC("raw_tp/sys_enter")
+__failure __msg("invalid mem access 'scalar'")
+int iter_next_trusted_field_after_destroy(const void *ctx)
+{
+ struct task_struct *cur_task = bpf_get_current_task_btf();
+ struct bpf_iter_task_vma vma_it;
+ struct vm_area_struct *vma_ptr;
+ struct file *file_ptr;
+
+ bpf_iter_task_vma_new(&vma_it, cur_task, 0);
+
+ vma_ptr = bpf_iter_task_vma_next(&vma_it);
+ if (!vma_ptr)
+ goto out;
+
+ file_ptr = vma_ptr->vm_file;
+ bpf_iter_task_vma_destroy(&vma_it);
+ if (file_ptr)
+ return file_ptr->f_mode;
+ return 0;
+out:
+ bpf_iter_task_vma_destroy(&vma_it);
+ return 0;
+}
+
+SEC("raw_tp/sys_enter")
+__failure __msg("invalid mem access 'scalar'")
+int iter_next_trusted_field_after_next(const void *ctx)
+{
+ struct task_struct *cur_task = bpf_get_current_task_btf();
+ struct bpf_iter_task_vma vma_it;
+ struct vm_area_struct *vma_ptr, *next_vma_ptr;
+ struct file *file_ptr;
+ u32 mode;
+
+ bpf_iter_task_vma_new(&vma_it, cur_task, 0);
+
+ vma_ptr = bpf_iter_task_vma_next(&vma_it);
+ if (!vma_ptr)
+ goto out;
+
+ file_ptr = vma_ptr->vm_file;
+ if (!file_ptr)
+ goto out;
+
+ next_vma_ptr = bpf_iter_task_vma_next(&vma_it);
+ if (!next_vma_ptr)
+ goto out;
+
+ mode = file_ptr->f_mode;
+ bpf_iter_task_vma_destroy(&vma_it);
+ return mode;
+out:
+ bpf_iter_task_vma_destroy(&vma_it);
+ return 0;
+}
+
+SEC("raw_tp/sys_enter")
+__failure __msg("Leaking reference id=")
+int iter_next_file_dynptr_after_next(const void *ctx)
+{
+ struct task_struct *cur_task = bpf_get_current_task_btf();
+ struct bpf_iter_task_vma vma_it;
+ struct vm_area_struct *vma_ptr;
+ struct bpf_dynptr dynptr;
+ struct file *file_ptr;
+
+ bpf_iter_task_vma_new(&vma_it, cur_task, 0);
+
+ vma_ptr = bpf_iter_task_vma_next(&vma_it);
+ if (!vma_ptr)
+ goto out;
+
+ file_ptr = vma_ptr->vm_file;
+ if (!file_ptr)
+ goto out;
+
+ bpf_dynptr_from_file(file_ptr, 0, &dynptr);
+ bpf_iter_task_vma_next(&vma_it);
+ bpf_dynptr_file_discard(&dynptr);
+out:
+ bpf_iter_task_vma_destroy(&vma_it);
+ return 0;
+}
+
+SEC("raw_tp/sys_enter")
+__failure __msg("Leaking reference id=")
+int iter_next_file_dynptr_after_destroy(const void *ctx)
+{
+ struct task_struct *cur_task = bpf_get_current_task_btf();
+ struct bpf_iter_task_vma vma_it;
+ struct vm_area_struct *vma_ptr;
+ struct bpf_dynptr dynptr;
+ struct file *file_ptr;
+
+ bpf_iter_task_vma_new(&vma_it, cur_task, 0);
+
+ vma_ptr = bpf_iter_task_vma_next(&vma_it);
+ if (!vma_ptr)
+ goto out;
+
+ file_ptr = vma_ptr->vm_file;
+ if (!file_ptr)
+ goto out;
+
+ bpf_dynptr_from_file(file_ptr, 0, &dynptr);
+ bpf_iter_task_vma_destroy(&vma_it);
+ bpf_dynptr_file_discard(&dynptr);
+ return 0;
+out:
+ bpf_iter_task_vma_destroy(&vma_it);
+ return 0;
+}
+
+SEC("raw_tp/sys_enter")
+__success
+int iter_next_file_dynptr_discard_before_advance(const void *ctx)
+{
+ struct task_struct *cur_task = bpf_get_current_task_btf();
+ struct bpf_iter_task_vma vma_it;
+ struct vm_area_struct *vma_ptr;
+ struct bpf_dynptr dynptr;
+ struct file *file_ptr;
+
+ bpf_iter_task_vma_new(&vma_it, cur_task, 0);
+
+ vma_ptr = bpf_iter_task_vma_next(&vma_it);
+ if (!vma_ptr)
+ goto out;
+
+ file_ptr = vma_ptr->vm_file;
+ if (!file_ptr)
+ goto out;
+
+ bpf_dynptr_from_file(file_ptr, 0, &dynptr);
+ bpf_dynptr_file_discard(&dynptr);
+ bpf_iter_task_vma_next(&vma_it);
+out:
+ bpf_iter_task_vma_destroy(&vma_it);
+ return 0;
+}
+
+SEC("raw_tp/sys_enter")
+__success
+int iter_next_trusted_current_after_next(const void *ctx)
+{
+ struct task_struct *cur_task = bpf_get_current_task_btf();
+ struct bpf_iter_task_vma vma_it;
+ struct vm_area_struct *vma_ptr;
+
+ bpf_iter_task_vma_new(&vma_it, cur_task, 0);
+
+ vma_ptr = bpf_iter_task_vma_next(&vma_it);
+ if (!vma_ptr)
+ goto out;
+
+ vma_ptr = bpf_iter_task_vma_next(&vma_it);
+ if (vma_ptr)
+ bpf_kfunc_trusted_vma_test(vma_ptr);
+out:
+ bpf_iter_task_vma_destroy(&vma_it);
+ return 0;
+}
+
+SEC("raw_tp/sys_enter")
+__success
+int iter_next_trusted_rcu_field_after_destroy(const void *ctx)
+{
+ struct task_struct *cur_task = bpf_get_current_task_btf();
+ struct bpf_iter_task_vma vma_it;
+ struct vm_area_struct *vma_ptr;
+ struct mm_struct *mm_ptr;
+ struct file *file_ptr = NULL;
+
+ bpf_iter_task_vma_new(&vma_it, cur_task, 0);
+
+ vma_ptr = bpf_iter_task_vma_next(&vma_it);
+ if (vma_ptr) {
+ mm_ptr = vma_ptr->vm_mm;
+ /* exe_file has RCU protection independent of the iterator. */
+ if (mm_ptr)
+ file_ptr = mm_ptr->exe_file;
+ }
+
+ bpf_iter_task_vma_destroy(&vma_it);
+ if (file_ptr)
+ return file_ptr->f_mode;
+ return 0;
+}
+
+SEC("raw_tp/sys_enter")
+__success
+int iter_next_trusted_rcu_field_after_next(const void *ctx)
+{
+ struct task_struct *cur_task = bpf_get_current_task_btf();
+ struct bpf_iter_task_vma vma_it;
+ struct vm_area_struct *vma_ptr;
+ struct mm_struct *mm_ptr;
+ struct file *file_ptr = NULL;
+ u32 mode = 0;
+
+ bpf_iter_task_vma_new(&vma_it, cur_task, 0);
+
+ vma_ptr = bpf_iter_task_vma_next(&vma_it);
+ if (vma_ptr) {
+ mm_ptr = vma_ptr->vm_mm;
+ if (mm_ptr)
+ file_ptr = mm_ptr->exe_file;
+ }
+
+ bpf_iter_task_vma_next(&vma_it);
+ if (file_ptr)
+ mode = file_ptr->f_mode;
+ bpf_iter_task_vma_destroy(&vma_it);
+ return mode;
+}
+
SEC("raw_tp/sys_enter")
__failure __msg("Possibly NULL pointer passed to trusted R1")
int iter_next_trusted_or_null(const void *ctx)
@@ -66,6 +396,46 @@ int iter_next_rcu(const void *ctx)
return 0;
}
+SEC("raw_tp/sys_enter")
+__success
+int iter_next_rcu_after_destroy(const void *ctx)
+{
+ struct task_struct *cur_task = bpf_get_current_task_btf();
+ struct bpf_iter_task task_it;
+ struct task_struct *task_ptr;
+
+ bpf_iter_task_new(&task_it, cur_task, 0);
+
+ task_ptr = bpf_iter_task_next(&task_it);
+ bpf_iter_task_destroy(&task_it);
+ if (!task_ptr)
+ return 0;
+
+ bpf_kfunc_rcu_task_test(task_ptr);
+ return 0;
+}
+
+SEC("raw_tp/sys_enter")
+__success
+int iter_next_rcu_after_next(const void *ctx)
+{
+ struct task_struct *cur_task = bpf_get_current_task_btf();
+ struct bpf_iter_task task_it;
+ struct task_struct *task_ptr;
+
+ bpf_iter_task_new(&task_it, cur_task, 0);
+
+ task_ptr = bpf_iter_task_next(&task_it);
+ if (!task_ptr)
+ goto out;
+
+ bpf_iter_task_next(&task_it);
+ bpf_kfunc_rcu_task_test(task_ptr);
+out:
+ bpf_iter_task_destroy(&task_it);
+ return 0;
+}
+
SEC("raw_tp/sys_enter")
__failure __msg("Possibly NULL pointer passed to trusted R1")
int iter_next_rcu_or_null(const void *ctx)
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
index 93847ca6293b4..6e50c1f7f13aa 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.c
@@ -244,6 +244,27 @@ __bpf_kfunc void bpf_iter_testmod_seq_destroy(struct bpf_iter_testmod_seq *it)
it->cnt = 0;
}
+__bpf_kfunc int bpf_iter_testmod_current_new(struct bpf_iter_testmod_current *it)
+{
+ it->remaining = 2;
+ return 0;
+}
+
+__bpf_kfunc struct task_struct *
+bpf_iter_testmod_current_next(struct bpf_iter_testmod_current *it)
+{
+ if (!it->remaining)
+ return NULL;
+ it->remaining--;
+ /* current remains alive independently of the iterator. */
+ return current;
+}
+
+__bpf_kfunc void bpf_iter_testmod_current_destroy(struct bpf_iter_testmod_current *it)
+{
+ it->remaining = 0;
+}
+
__bpf_kfunc void bpf_kfunc_common_test(void)
{
}
@@ -893,6 +914,10 @@ BTF_ID_FLAGS(func, bpf_iter_testmod_seq_new, KF_ITER_NEW)
BTF_ID_FLAGS(func, bpf_iter_testmod_seq_next, KF_ITER_NEXT | KF_RET_NULL)
BTF_ID_FLAGS(func, bpf_iter_testmod_seq_destroy, KF_ITER_DESTROY)
BTF_ID_FLAGS(func, bpf_iter_testmod_seq_value)
+BTF_ID_FLAGS(func, bpf_iter_testmod_current_new, KF_ITER_NEW)
+BTF_ID_FLAGS(func, bpf_iter_testmod_current_next,
+ KF_ITER_NEXT | KF_RET_NULL | KF_ITER_RET_INDEPENDENT)
+BTF_ID_FLAGS(func, bpf_iter_testmod_current_destroy, KF_ITER_DESTROY)
BTF_ID_FLAGS(func, bpf_kfunc_common_test)
BTF_ID_FLAGS(func, bpf_kfunc_arena_arg_test)
BTF_ID_FLAGS(func, bpf_kfunc_arena_cap_test)
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h
index 2b12f81bf7487..94a228e807582 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod.h
@@ -31,6 +31,10 @@ struct bpf_iter_testmod_seq {
int cnt;
};
+struct bpf_iter_testmod_current {
+ u64 remaining;
+};
+
struct bpf_testmod_ops {
int (*test_1)(void);
void (*test_2)(int a, int b);
diff --git a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
index 67c02a421d133..13be4b6012cb9 100644
--- a/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
+++ b/tools/testing/selftests/bpf/test_kmods/bpf_testmod_kfunc.h
@@ -33,6 +33,11 @@ struct prog_test_ref_kfunc {
#endif
struct bpf_iter_testmod_seq;
+struct bpf_iter_testmod_current;
+
+int bpf_iter_testmod_current_new(struct bpf_iter_testmod_current *it) __ksym;
+struct task_struct *bpf_iter_testmod_current_next(struct bpf_iter_testmod_current *it) __ksym;
+void bpf_iter_testmod_current_destroy(struct bpf_iter_testmod_current *it) __ksym;
struct prog_test_pass1 {
int x0;
--
2.43.0
^ permalink raw reply related [flat|nested] 6+ messages in thread