[PATCH bpf-next 0/6] workqueue: introduce BPF iterators for workqueues, worker pools and pending work

[PATCH bpf-next 0/6] workqueue: introduce BPF iterators for workqueues, worker pools and pending work

[Imran Khan]

Nearly every subsystem defers work to workqueues, and their state can
already be observed on a live system, just not at a granularity that is
convenient to consume. sysrq dumps every workqueue and pool to the kernel
log in a fixed format, with no way to select or aggregate; the WQ_SYSFS
interface only covers workqueues that ask for it and exposes attributes
rather than runtime state; and the drgn scripts under tools/workqueue/ need
a debuginfo-equipped userspace and read the state from the outside, without
the locks that protect it.

This series adds BPF iterators for workqueues, worker pools and pending
work items, so that this state can be walked from inside the kernel, under
the right locking, with filtering and aggregation done in place and only
the interesting part copied to userspace.

For the workqueue and worker_pool targets both forms are provided: the
open-coded form (bpf_iter_*_{new,next,destroy} kfuncs), usable from any BPF
program and thus combinable with other tracing and with maps, and the
seq_file form, pinnable to bpffs and readable via bpftool iter or read(),
for standalone dump-on-demand tools.

Patches:

 1-2: workqueue iterator. Walks the global workqueues list. The list is
      RCU-protected for reads and workqueue_struct is freed via call_rcu(),
      so the open-coded iterator is KF_RCU_PROTECTED and the caller must
      hold bpf_rcu_read_lock() across new()..destroy(). The seq form
      supplies the RCU section itself, per read chunk, and re-derives the
      position from *pos on each start so nothing is dereferenced across
      the gap between chunks.

 3-4: worker_pool iterator. Walks worker_pool_idr in ascending pool-id
      order, mirroring the pool enumeration done by wq_dump.py. The idr
      walk stays in kernel C (idr_get_next()), so no BPF-side idr support
      is needed. Same RCU rules and same seq scheme as above.

   5: pending work iterator ("workqueue_pending_work"), seq_file only.
      This one is deliberately not open-coded: pool->worklist is protected
      by pool->lock, and struct work_struct has neither a refcount nor
      RCU-freeing, so a pending work item cannot be kept alive for a
      suspended iterator. Instead a bounded snapshot of a pool's pending
      works (pool id, work address, work function) is copied out while
      pool->lock is held, and the BPF program then runs over that stable
      snapshot with no lock held. Pools are visited in worker_pool_idr
      order, which is a stable resume key across read() chunks; a pool with
      more than WQ_PENDING_SNAP_MAX pending works is truncated, best-effort,
      like the kernel's own worklist dump.

   6: selftests covering both forms of the workqueue and worker_pool
      iterators, the pending-work iterator, and a verifier test that the
      RCU-protected open-coded iterators are rejected outside an RCU
      critical section.

All new code sits behind CONFIG_BPF_SYSCALL. Tested with
"test_progs -t wq_iter" and result has been pasted below:
....
+ [ -x /etc/rcS.d/S50-startup ]
+ /etc/rcS.d/S50-startup
./test_progs -t wq_iter
[    1.105718] bpf_testmod: loading out-of-tree module taints kernel.
[    1.106438] bpf_testmod: module verification failed: signature and/or required key missing - tainting kernel
719/1   wq_iter/open_coded:OK
719/2   wq_iter/seq:OK
719/3   wq_iter/workqueue_iter_no_rcu:OK
719/4   wq_iter/worker_pool_iter_no_rcu:OK
719     wq_iter:OK
Summary: 1/4 PASSED, 0 SKIPPED, 0 FAILED
....

A PR for this changeset has been created at [1].

Thanks,
Imran

Imran Khan (6):
  workqueue: introduce a BPF open-coded iterator for traversing
    workqueues list
  workqueue: introduce a seq_file form for the workqueue iterator
  workqueue: introduce open-coded BPF iterator for worker pools
  workqueue: introduce seq_file form of the worker_pool iterator
  workqueue: introduce BPF iterator for pending work items
  selftests/bpf: add tests for the workqueue BPF iterators

 kernel/workqueue.c                            | 556 ++++++++++++++++++
 .../testing/selftests/bpf/bpf_experimental.h  |  10 +
 .../selftests/bpf/prog_tests/wq_iter.c        |  80 +++
 tools/testing/selftests/bpf/progs/wq_iter.c   |  95 +++
 .../selftests/bpf/progs/wq_iter_fail.c        |  37 ++
 5 files changed, 778 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/wq_iter.c
 create mode 100644 tools/testing/selftests/bpf/progs/wq_iter.c
 create mode 100644 tools/testing/selftests/bpf/progs/wq_iter_fail.c


base-commit: 87267b89459813cb50ab5377e076b639c07b4491

[1]: https://github.com/kernel-patches/bpf/pull/12981
-- 
2.43.0

[PATCH bpf-next 1/6] workqueue: introduce a BPF open-coded iterator for traversing workqueues list

[Imran Khan]

Add bpf_iter_workqueue_{new,next,destroy} kfuncs so BPF programs can walk
every workqueue on the global workqueues list.

The workqueues list is RCU-protected for reads and each workqueue_struct
is freed via call_rcu(), so the iterator is KF_RCU_PROTECTED i.e. the
caller must hold bpf_rcu_read_lock() across new()..destroy(), which keeps
the list and each returned wq alive between next() calls.

The kfuncs live in kernel/workqueue.c because struct workqueue_struct's
list anchor and the workqueues root are local to this file.

Signed-off-by: Imran Khan <imran.f.khan@oracle.com>
---
 kernel/workqueue.c | 103 +++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 103 insertions(+)

diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 78068ae8f28a..c4ec4812db64 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -56,6 +56,7 @@
 #include <linux/kvm_para.h>
 #include <linux/delay.h>
 #include <linux/irq_work.h>
+#include <linux/btf_ids.h>
 
 #include "workqueue_internal.h"
 
@@ -8473,3 +8474,105 @@ static int __init workqueue_unbound_cpus_setup(char *str)
 	return 1;
 }
 __setup("workqueue.unbound_cpus=", workqueue_unbound_cpus_setup);
+
+#ifdef CONFIG_BPF_SYSCALL
+
+/*
+ * BPF open-coded iterator over all workqueues.
+ *
+ * Walks the global @workqueues list, letting BPF programs read live workqueue
+ * state.
+ *
+ * The caller must hold bpf_rcu_read_lock() across new()->next()->destroy().
+ */
+
+/* Sentinel for "new() done, next() not yet called" vs "iteration ended". */
+#define BPF_WQ_ITER_START ((void *)1L)
+
+struct bpf_iter_workqueue {
+	__u64 __opaque[1];
+} __aligned(8);
+
+struct bpf_iter_workqueue_kern {
+	struct workqueue_struct *pos;
+} __aligned(8);
+
+__bpf_kfunc_start_defs();
+
+/**
+ * bpf_iter_workqueue_new() - Initialize a workqueue iterator
+ * @it: The new bpf_iter_workqueue to initialize
+ *
+ * Iterates every workqueue on the global @workqueues list. Must be used inside
+ * a bpf_rcu_read_lock() region (KF_RCU_PROTECTED).
+ */
+__bpf_kfunc int bpf_iter_workqueue_new(struct bpf_iter_workqueue *it)
+{
+	struct bpf_iter_workqueue_kern *kit = (void *)it;
+
+	BUILD_BUG_ON(sizeof(struct bpf_iter_workqueue_kern) >
+		     sizeof(struct bpf_iter_workqueue));
+	BUILD_BUG_ON(__alignof__(struct bpf_iter_workqueue_kern) !=
+		     __alignof__(struct bpf_iter_workqueue));
+
+	kit->pos = BPF_WQ_ITER_START;
+	return 0;
+}
+
+/**
+ * bpf_iter_workqueue_next() - Get the next workqueue
+ * @it: The bpf_iter_workqueue
+ *
+ * Returns a pointer to the next struct workqueue_struct, or NULL when done.
+ */
+__bpf_kfunc struct workqueue_struct *
+bpf_iter_workqueue_next(struct bpf_iter_workqueue *it)
+{
+	struct bpf_iter_workqueue_kern *kit = (void *)it;
+
+	if (!kit->pos)			/* already reached the end */
+		return NULL;
+
+	if (kit->pos == BPF_WQ_ITER_START)
+		kit->pos = list_first_or_null_rcu(&workqueues,
+						  struct workqueue_struct, list);
+	else
+		kit->pos = list_next_or_null_rcu(&workqueues, &kit->pos->list,
+						 struct workqueue_struct, list);
+	return kit->pos;
+}
+
+/**
+ * bpf_iter_workqueue_destroy() - Destroy a workqueue iterator
+ * @it: The bpf_iter_workqueue to destroy
+ */
+__bpf_kfunc void bpf_iter_workqueue_destroy(struct bpf_iter_workqueue *it)
+{
+	/* No references taken; RCU is held by the caller. */
+}
+
+__bpf_kfunc_end_defs();
+
+BTF_KFUNCS_START(workqueue_iter_kfunc_ids)
+BTF_ID_FLAGS(func, bpf_iter_workqueue_new, KF_ITER_NEW | KF_RCU_PROTECTED)
+BTF_ID_FLAGS(func, bpf_iter_workqueue_next, KF_ITER_NEXT | KF_RET_NULL)
+BTF_ID_FLAGS(func, bpf_iter_workqueue_destroy, KF_ITER_DESTROY)
+BTF_KFUNCS_END(workqueue_iter_kfunc_ids)
+
+static const struct btf_kfunc_id_set workqueue_iter_kfunc_set = {
+	.owner	= THIS_MODULE,
+	.set	= &workqueue_iter_kfunc_ids,
+};
+
+static int __init bpf_workqueue_iter_init(void)
+{
+	int ret;
+
+	ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING,
+					&workqueue_iter_kfunc_set);
+	return ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SYSCALL,
+						&workqueue_iter_kfunc_set);
+}
+late_initcall(bpf_workqueue_iter_init);
+
+#endif /* CONFIG_BPF_SYSCALL */
-- 
2.43.0

[PATCH bpf-next 2/6] workqueue: introduce a seq_file form for the workqueue iterator

[Imran Khan]

Add a 'workqueue' BPF iterator target (bpftool-iter / pinnable) that
reuses the open-coded bpf_iter_workqueue_next().

Since the open-coded next() is KF_RCU_PROTECTED, the seq path supplies the
RCU section: seq_start() takes rcu_read_lock() and holds it across the read
chunk, so the returned workq and the cursor stay alive while the program
runs and next() steps; position is re-derived from *pos on each start, so
the cursor is never dereferenced across the RCU gap between chunks.

Signed-off-by: Imran Khan <imran.f.khan@oracle.com>
---
 kernel/workqueue.c | 119 ++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 117 insertions(+), 2 deletions(-)

diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index c4ec4812db64..bb109d851d1c 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -57,6 +57,7 @@
 #include <linux/delay.h>
 #include <linux/irq_work.h>
 #include <linux/btf_ids.h>
+#include <linux/bpf.h>
 
 #include "workqueue_internal.h"
 
@@ -8564,14 +8565,128 @@ static const struct btf_kfunc_id_set workqueue_iter_kfunc_set = {
 	.set	= &workqueue_iter_kfunc_ids,
 };
 
+/*
+ * seq_file form of the workqueue iterator. It reuses the open-coded
+ * bpf_iter_workqueue_next().
+ *
+ * Since the open-coded next() is KF_RCU_PROTECTED, the seq path supplies
+ * the RCU section itself.
+ * seq_start() takes rcu_read_lock() and holds it across the whole read chunk
+ * (start..stop). Position is re-derived from *pos on each start, so nothing
+ * is dereferenced across the rcu gap between chunks. The attached BPF program
+ * must be non-sleepable.
+ */
+union workqueue_iter_priv {
+	struct bpf_iter_workqueue it;
+	struct bpf_iter_workqueue_kern kit;
+};
+
+struct bpf_iter__workqueue {
+	__bpf_md_ptr(struct bpf_iter_meta *, meta);
+	__bpf_md_ptr(struct workqueue_struct *, wq);
+};
+
+static void *workqueue_iter_seq_start(struct seq_file *seq, loff_t *pos)
+{
+	union workqueue_iter_priv *p = seq->private;
+	struct workqueue_struct *wq;
+	loff_t cnt = 0;
+
+	rcu_read_lock();	/* held until seq_stop() */
+	list_for_each_entry_rcu(wq, &workqueues, list) {
+		if (cnt == *pos) {
+			p->kit.pos = wq;
+			return wq;
+		}
+		cnt++;
+	}
+	p->kit.pos = NULL;
+	return NULL;
+}
+
+static void *workqueue_iter_seq_next(struct seq_file *seq, void *v, loff_t *pos)
+{
+	union workqueue_iter_priv *p = seq->private;
+
+	++*pos;
+	return bpf_iter_workqueue_next(&p->it);
+}
+
+static int workqueue_iter_seq_show(struct seq_file *seq, void *v)
+{
+	struct bpf_iter__workqueue ctx;
+	struct bpf_iter_meta meta;
+	struct bpf_prog *prog;
+
+	meta.seq = seq;
+	prog = bpf_iter_get_info(&meta, false);
+	if (!prog)
+		return 0;
+	ctx.meta = &meta;
+	ctx.wq = v;
+	return bpf_iter_run_prog(prog, &ctx);
+}
+
+static void workqueue_iter_seq_stop(struct seq_file *seq, void *v)
+{
+	struct bpf_iter__workqueue ctx;
+	struct bpf_iter_meta meta;
+	struct bpf_prog *prog;
+
+	if (!v) {
+		meta.seq = seq;
+		prog = bpf_iter_get_info(&meta, true);
+		if (prog) {
+			ctx.meta = &meta;
+			ctx.wq = NULL;
+			bpf_iter_run_prog(prog, &ctx);
+		}
+	}
+	rcu_read_unlock();	/* paired with seq_start() */
+}
+
+static const struct seq_operations workqueue_iter_seq_ops = {
+	.start	= workqueue_iter_seq_start,
+	.next	= workqueue_iter_seq_next,
+	.stop	= workqueue_iter_seq_stop,
+	.show	= workqueue_iter_seq_show,
+};
+
+DEFINE_BPF_ITER_FUNC(workqueue, struct bpf_iter_meta *meta,
+		     struct workqueue_struct *wq)
+
+static const struct bpf_iter_seq_info workqueue_iter_seq_info = {
+	.seq_ops	= &workqueue_iter_seq_ops,
+	.seq_priv_size	= sizeof(union workqueue_iter_priv),
+};
+
+BTF_ID_LIST_SINGLE(workqueue_btf_id, struct, workqueue_struct)
+
+static struct bpf_iter_reg workqueue_iter_reg_info = {
+	.target			= "workqueue",
+	.ctx_arg_info_size	= 1,
+	.ctx_arg_info		= {
+		{ offsetof(struct bpf_iter__workqueue, wq),
+		  PTR_TO_BTF_ID_OR_NULL | PTR_TRUSTED },
+	},
+	.seq_info		= &workqueue_iter_seq_info,
+};
+
 static int __init bpf_workqueue_iter_init(void)
 {
 	int ret;
 
 	ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_TRACING,
 					&workqueue_iter_kfunc_set);
-	return ret ?: register_btf_kfunc_id_set(BPF_PROG_TYPE_SYSCALL,
-						&workqueue_iter_kfunc_set);
+	if (ret)
+		return ret;
+	ret = register_btf_kfunc_id_set(BPF_PROG_TYPE_SYSCALL,
+					&workqueue_iter_kfunc_set);
+	if (ret)
+		return ret;
+
+	workqueue_iter_reg_info.ctx_arg_info[0].btf_id = workqueue_btf_id[0];
+	return bpf_iter_reg_target(&workqueue_iter_reg_info);
 }
 late_initcall(bpf_workqueue_iter_init);
 
-- 
2.43.0

[PATCH bpf-next 3/6] workqueue: introduce open-coded BPF iterator for worker pools

[Imran Khan]

Add bpf_iter_worker_pool_{new,next,destroy} kfuncs so BPF programs can
walk every worker_pool in the system via worker_pool_idr, in ascending
pool-id order.
This mirrors the pool enumeration in the drgn tools/workqueue/wq_dump.py
script.

The idr walk (idr_get_next) runs as kernel C, so no BPF-side idr support
is needed. worker_pool_idr is RCU-readable and each worker_pool is freed
via call_rcu(), so the iterator is KF_RCU_PROTECTED, like the workqueue
iterator: the caller holds bpf_rcu_read_lock() across new()..destroy().

Signed-off-by: Imran Khan <imran.f.khan@oracle.com>
---
 kernel/workqueue.c | 60 ++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 60 insertions(+)

diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index bb109d851d1c..039b2d0c8669 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -8498,6 +8498,14 @@ struct bpf_iter_workqueue_kern {
 	struct workqueue_struct *pos;
 } __aligned(8);
 
+struct bpf_iter_worker_pool {
+	__u64 __opaque[1];
+} __aligned(8);
+
+struct bpf_iter_worker_pool_kern {
+	int id;		/* next worker_pool_idr id to search from */
+} __aligned(8);
+
 __bpf_kfunc_start_defs();
 
 /**
@@ -8552,12 +8560,64 @@ __bpf_kfunc void bpf_iter_workqueue_destroy(struct bpf_iter_workqueue *it)
 	/* No references taken; RCU is held by the caller. */
 }
 
+/**
+ * bpf_iter_worker_pool_new() - Initialize a worker_pool iterator
+ * @it: The new bpf_iter_worker_pool to initialize
+ *
+ * Iterates every worker_pool in the system (worker_pool_idr) in ascending
+ * pool-id order.
+ * The caller should hold RCU read lock.
+ */
+__bpf_kfunc int bpf_iter_worker_pool_new(struct bpf_iter_worker_pool *it)
+{
+	struct bpf_iter_worker_pool_kern *kit = (void *)it;
+
+	BUILD_BUG_ON(sizeof(struct bpf_iter_worker_pool_kern) >
+		     sizeof(struct bpf_iter_worker_pool));
+	BUILD_BUG_ON(__alignof__(struct bpf_iter_worker_pool_kern) !=
+		     __alignof__(struct bpf_iter_worker_pool));
+
+	kit->id = 0;
+	return 0;
+}
+
+/**
+ * bpf_iter_worker_pool_next() - Get the next worker_pool
+ * @it: The bpf_iter_worker_pool
+ *
+ * Returns a pointer to the next struct worker_pool, or NULL when done.
+ */
+__bpf_kfunc struct worker_pool *
+bpf_iter_worker_pool_next(struct bpf_iter_worker_pool *it)
+{
+	struct bpf_iter_worker_pool_kern *kit = (void *)it;
+	struct worker_pool *pool;
+
+	/* idr_get_next() is RCU-safe; see the for_each_pool() lock legend. */
+	pool = idr_get_next(&worker_pool_idr, &kit->id);
+	if (pool)
+		kit->id++;	/* advance past this pool for the next call */
+	return pool;
+}
+
+/**
+ * bpf_iter_worker_pool_destroy() - Destroy a worker_pool iterator
+ * @it: The bpf_iter_worker_pool to destroy
+ */
+__bpf_kfunc void bpf_iter_worker_pool_destroy(struct bpf_iter_worker_pool *it)
+{
+	/* Nothing to be done; RCU is held by the caller. */
+}
+
 __bpf_kfunc_end_defs();
 
 BTF_KFUNCS_START(workqueue_iter_kfunc_ids)
 BTF_ID_FLAGS(func, bpf_iter_workqueue_new, KF_ITER_NEW | KF_RCU_PROTECTED)
 BTF_ID_FLAGS(func, bpf_iter_workqueue_next, KF_ITER_NEXT | KF_RET_NULL)
 BTF_ID_FLAGS(func, bpf_iter_workqueue_destroy, KF_ITER_DESTROY)
+BTF_ID_FLAGS(func, bpf_iter_worker_pool_new, KF_ITER_NEW | KF_RCU_PROTECTED)
+BTF_ID_FLAGS(func, bpf_iter_worker_pool_next, KF_ITER_NEXT | KF_RET_NULL)
+BTF_ID_FLAGS(func, bpf_iter_worker_pool_destroy, KF_ITER_DESTROY)
 BTF_KFUNCS_END(workqueue_iter_kfunc_ids)
 
 static const struct btf_kfunc_id_set workqueue_iter_kfunc_set = {
-- 
2.43.0

[PATCH bpf-next 4/6] workqueue: introduce seq_file form of the worker_pool iterator

[Imran Khan]

Add a 'worker_pool' BPF iterator target, backed by the open-coded
bpf_iter_worker_pool_next() via a shared union, using the same
RCU-per-chunk scheme as the workqueue seq iterator.

Signed-off-by: Imran Khan <imran.f.khan@oracle.com>
---
 kernel/workqueue.c | 108 ++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 107 insertions(+), 1 deletion(-)

diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 039b2d0c8669..1c9f4bf77f10 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -8732,6 +8732,107 @@ static struct bpf_iter_reg workqueue_iter_reg_info = {
 	.seq_info		= &workqueue_iter_seq_info,
 };
 
+/*
+ * seq_file form of the worker_pool iterator; same RCU-per-chunk scheme as the
+ * workqueue one, reusing the open-coded bpf_iter_worker_pool_next().
+ */
+union worker_pool_iter_priv {
+	struct bpf_iter_worker_pool it;
+	struct bpf_iter_worker_pool_kern kit;
+};
+
+struct bpf_iter__worker_pool {
+	__bpf_md_ptr(struct bpf_iter_meta *, meta);
+	__bpf_md_ptr(struct worker_pool *, pool);
+};
+
+static void *worker_pool_iter_seq_start(struct seq_file *seq, loff_t *pos)
+{
+	union worker_pool_iter_priv *p = seq->private;
+	struct worker_pool *pool;
+	loff_t cnt = 0;
+	int id = 0;
+
+	rcu_read_lock();
+	while ((pool = idr_get_next(&worker_pool_idr, &id))) {
+		if (cnt == *pos) {
+			p->kit.id = id + 1;
+			return pool;
+		}
+		cnt++;
+		id++;
+	}
+	return NULL;
+}
+
+static void *worker_pool_iter_seq_next(struct seq_file *seq, void *v, loff_t *pos)
+{
+	union worker_pool_iter_priv *p = seq->private;
+
+	++*pos;
+	return bpf_iter_worker_pool_next(&p->it);
+}
+
+static int worker_pool_iter_seq_show(struct seq_file *seq, void *v)
+{
+	struct bpf_iter__worker_pool ctx;
+	struct bpf_iter_meta meta;
+	struct bpf_prog *prog;
+
+	meta.seq = seq;
+	prog = bpf_iter_get_info(&meta, false);
+	if (!prog)
+		return 0;
+	ctx.meta = &meta;
+	ctx.pool = v;
+	return bpf_iter_run_prog(prog, &ctx);
+}
+
+static void worker_pool_iter_seq_stop(struct seq_file *seq, void *v)
+{
+	struct bpf_iter__worker_pool ctx;
+	struct bpf_iter_meta meta;
+	struct bpf_prog *prog;
+
+	if (!v) {
+		meta.seq = seq;
+		prog = bpf_iter_get_info(&meta, true);
+		if (prog) {
+			ctx.meta = &meta;
+			ctx.pool = NULL;
+			bpf_iter_run_prog(prog, &ctx);
+		}
+	}
+	rcu_read_unlock();
+}
+
+static const struct seq_operations worker_pool_iter_seq_ops = {
+	.start	= worker_pool_iter_seq_start,
+	.next	= worker_pool_iter_seq_next,
+	.stop	= worker_pool_iter_seq_stop,
+	.show	= worker_pool_iter_seq_show,
+};
+
+DEFINE_BPF_ITER_FUNC(worker_pool, struct bpf_iter_meta *meta,
+		     struct worker_pool *pool)
+
+static const struct bpf_iter_seq_info worker_pool_iter_seq_info = {
+	.seq_ops	= &worker_pool_iter_seq_ops,
+	.seq_priv_size	= sizeof(union worker_pool_iter_priv),
+};
+
+BTF_ID_LIST_SINGLE(worker_pool_btf_id, struct, worker_pool)
+
+static struct bpf_iter_reg worker_pool_iter_reg_info = {
+	.target			= "worker_pool",
+	.ctx_arg_info_size	= 1,
+	.ctx_arg_info		= {
+		{ offsetof(struct bpf_iter__worker_pool, pool),
+		  PTR_TO_BTF_ID_OR_NULL | PTR_TRUSTED },
+	},
+	.seq_info		= &worker_pool_iter_seq_info,
+};
+
 static int __init bpf_workqueue_iter_init(void)
 {
 	int ret;
@@ -8746,7 +8847,12 @@ static int __init bpf_workqueue_iter_init(void)
 		return ret;
 
 	workqueue_iter_reg_info.ctx_arg_info[0].btf_id = workqueue_btf_id[0];
-	return bpf_iter_reg_target(&workqueue_iter_reg_info);
+	ret = bpf_iter_reg_target(&workqueue_iter_reg_info);
+	if (ret)
+		return ret;
+
+	worker_pool_iter_reg_info.ctx_arg_info[0].btf_id = worker_pool_btf_id[0];
+	return bpf_iter_reg_target(&worker_pool_iter_reg_info);
 }
 late_initcall(bpf_workqueue_iter_init);
 
-- 
2.43.0

[PATCH bpf-next 5/6] workqueue: introduce BPF iterator for pending work items

[Imran Khan]

Add a seq_file BPF iterator ("workqueue_pending_work") that walks the
pending work items of every worker_pool. Unlike the workqueue and
worker_pool iterators this is deliberately not open-coded because
pool->worklist is protected by pool->lock and struct work_struct has
neither a refcount nor RCU-freeing, so a pending work cannot be kept alive
for a suspended open-coded iterator.

Instead, for each pool a bounded snapshot of its pending works (pool id,
work address, work function) is copied out while pool->lock is held and
the BPF program runs over the stable snapshot with no lock held.
worker_pool(s) are visited in worker_pool_idr order.

Signed-off-by: Imran Khan <imran.f.khan@oracle.com>
---
 kernel/workqueue.c | 174 ++++++++++++++++++++++++++++++++++++++++++++-
 1 file changed, 173 insertions(+), 1 deletion(-)

diff --git a/kernel/workqueue.c b/kernel/workqueue.c
index 1c9f4bf77f10..6e879490c811 100644
--- a/kernel/workqueue.c
+++ b/kernel/workqueue.c
@@ -8833,6 +8833,173 @@ static struct bpf_iter_reg worker_pool_iter_reg_info = {
 	.seq_info		= &worker_pool_iter_seq_info,
 };
 
+/*
+ * seq-file BPF iterator over the pending work items of every worker_pool.
+ *
+ * pool->worklist is protected by pool->lock and work_struct has neither a
+ * refcount nor RCU-freeing, so a pending work cannot be handed to a suspended
+ * (open-coded) iterator safely.
+ * Instead, for each pool a bounded snapshot of its pending works is copied out
+ * while pool->lock is held, mirroring the printk_deferred section of
+ * show_one_worker_pool() and then the lock is dropped.
+ * The BPF program then runs over the stable snapshot with no lock held.
+ *
+ * Pools are visited in worker_pool_idr order (a stable resume key); each pool's
+ * snapshot lives in the persistent seq_private, so the walk resumes correctly
+ * across read() chunks. A pool with more than WQ_PENDING_SNAP_MAX pending works
+ * is truncated (best-effort, like the kernel's own worklist dump).
+ */
+#define WQ_PENDING_SNAP_MAX 128
+
+/*
+ * One projected pending work item, as seen by the BPF program. Addresses only
+ * -- the live work_struct is not exposed (it may be freed after the lock).
+ */
+struct wq_pending_work_info {
+	__u64 pool_id;
+	__u64 work;
+	__u64 func;
+};
+
+struct wq_pending_iter_priv {
+	int          next_pool;	/* worker_pool_idr cursor for the next fill */
+	unsigned int idx;	/* position within snap[]                   */
+	unsigned int count;	/* valid entries in snap[]                  */
+	struct wq_pending_work_info snap[WQ_PENDING_SNAP_MAX];
+};
+
+struct bpf_iter__workqueue_pending_work {
+	__bpf_md_ptr(struct bpf_iter_meta *, meta);
+	__bpf_md_ptr(struct wq_pending_work_info *, info);
+};
+
+/*
+ * Snapshot the next non-empty pool's pending works into priv->snap[], advancing
+ * priv->next_pool past it. Returns true if a pool was captured, false at end.
+ * pool->lock is held only for the field copy; RCU keeps each pool alive.
+ */
+static bool wq_pending_fill(struct wq_pending_iter_priv *priv)
+{
+	struct worker_pool *pool;
+	struct work_struct *work;
+	int id;
+
+	rcu_read_lock();
+	for (id = priv->next_pool; (pool = idr_get_next(&worker_pool_idr, &id)); id++) {
+		unsigned int n = 0;
+
+		raw_spin_lock_irq(&pool->lock);
+		list_for_each_entry(work, &pool->worklist, entry) {
+			if (n >= WQ_PENDING_SNAP_MAX)
+				break;
+			priv->snap[n].pool_id = pool->id;
+			priv->snap[n].work = (__u64)(unsigned long)work;
+			priv->snap[n].func = (__u64)(unsigned long)work->func;
+			n++;
+		}
+		raw_spin_unlock_irq(&pool->lock);
+
+		if (n) {
+			priv->count = n;
+			priv->idx = 0;
+			priv->next_pool = id + 1;
+			rcu_read_unlock();
+			return true;
+		}
+	}
+	rcu_read_unlock();
+	return false;
+}
+
+static void *wq_pending_seq_start(struct seq_file *seq, loff_t *pos)
+{
+	struct wq_pending_iter_priv *priv = seq->private;
+
+	if (*pos == 0) {
+		priv->next_pool = 0;
+		priv->idx = 0;
+		priv->count = 0;
+	}
+	while (priv->idx >= priv->count) {
+		if (!wq_pending_fill(priv))
+			return NULL;
+	}
+	return &priv->snap[priv->idx];
+}
+
+static void *wq_pending_seq_next(struct seq_file *seq, void *v, loff_t *pos)
+{
+	struct wq_pending_iter_priv *priv = seq->private;
+
+	++*pos;
+	priv->idx++;
+	while (priv->idx >= priv->count) {
+		if (!wq_pending_fill(priv))
+			return NULL;
+	}
+	return &priv->snap[priv->idx];
+}
+
+static int wq_pending_seq_show(struct seq_file *seq, void *v)
+{
+	struct bpf_iter__workqueue_pending_work ctx;
+	struct bpf_iter_meta meta;
+	struct bpf_prog *prog;
+
+	meta.seq = seq;
+	prog = bpf_iter_get_info(&meta, false);
+	if (!prog)
+		return 0;
+	ctx.meta = &meta;
+	ctx.info = v;
+	return bpf_iter_run_prog(prog, &ctx);
+}
+
+static void wq_pending_seq_stop(struct seq_file *seq, void *v)
+{
+	struct bpf_iter__workqueue_pending_work ctx;
+	struct bpf_iter_meta meta;
+	struct bpf_prog *prog;
+
+	if (v)
+		return;
+	meta.seq = seq;
+	prog = bpf_iter_get_info(&meta, true);
+	if (prog) {
+		ctx.meta = &meta;
+		ctx.info = NULL;
+		bpf_iter_run_prog(prog, &ctx);
+	}
+}
+
+static const struct seq_operations wq_pending_seq_ops = {
+	.start	= wq_pending_seq_start,
+	.next	= wq_pending_seq_next,
+	.stop	= wq_pending_seq_stop,
+	.show	= wq_pending_seq_show,
+};
+
+DEFINE_BPF_ITER_FUNC(workqueue_pending_work, struct bpf_iter_meta *meta,
+		     struct wq_pending_work_info *info)
+
+static const struct bpf_iter_seq_info wq_pending_seq_info = {
+	.seq_ops	= &wq_pending_seq_ops,
+	.seq_priv_size	= sizeof(struct wq_pending_iter_priv),
+};
+
+BTF_ID_LIST_SINGLE(wq_pending_work_info_btf_id, struct, wq_pending_work_info)
+
+static struct bpf_iter_reg wq_pending_reg_info = {
+	.target			= "workqueue_pending_work",
+	.feature		= BPF_ITER_RESCHED,
+	.ctx_arg_info_size	= 1,
+	.ctx_arg_info		= {
+		{ offsetof(struct bpf_iter__workqueue_pending_work, info),
+		  PTR_TO_BTF_ID_OR_NULL },
+	},
+	.seq_info		= &wq_pending_seq_info,
+};
+
 static int __init bpf_workqueue_iter_init(void)
 {
 	int ret;
@@ -8852,7 +9019,12 @@ static int __init bpf_workqueue_iter_init(void)
 		return ret;
 
 	worker_pool_iter_reg_info.ctx_arg_info[0].btf_id = worker_pool_btf_id[0];
-	return bpf_iter_reg_target(&worker_pool_iter_reg_info);
+	ret = bpf_iter_reg_target(&worker_pool_iter_reg_info);
+	if (ret)
+		return ret;
+
+	wq_pending_reg_info.ctx_arg_info[0].btf_id = wq_pending_work_info_btf_id[0];
+	return bpf_iter_reg_target(&wq_pending_reg_info);
 }
 late_initcall(bpf_workqueue_iter_init);
 
-- 
2.43.0

[PATCH bpf-next 6/6] selftests/bpf: add tests for the workqueue BPF iterators

[Imran Khan]

Cover both the open-coded and seq_file forms of the workqueue and
worker_pool iterators, plus the pending-work seq iterator:

 - count_workqueues / count_worker_pools drive the open-coded iterators
   (inside bpf_rcu_read_lock()); userspace checks the counts are non-zero.
 - dump_workqueues / dump_worker_pools attach the seq forms and drain
   them, exercising the RCU-per-chunk seq path that reuses the open-coded
   next().
 - dump_pending attaches the workqueue_pending_work seq iterator and
   drains it, exercising the per-pool snapshot path.
 - wq_iter_fail.c checks the verifier rejects the RCU-protected
   open-coded iterators when used outside an RCU critical section.

Also declare the open-coded iterator kfuncs in bpf_experimental.h.

Signed-off-by: Imran Khan <imran.f.khan@oracle.com>
---
 .../testing/selftests/bpf/bpf_experimental.h  | 10 ++
 .../selftests/bpf/prog_tests/wq_iter.c        | 80 ++++++++++++++++
 tools/testing/selftests/bpf/progs/wq_iter.c   | 95 +++++++++++++++++++
 .../selftests/bpf/progs/wq_iter_fail.c        | 37 ++++++++
 4 files changed, 222 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/wq_iter.c
 create mode 100644 tools/testing/selftests/bpf/progs/wq_iter.c
 create mode 100644 tools/testing/selftests/bpf/progs/wq_iter_fail.c

diff --git a/tools/testing/selftests/bpf/bpf_experimental.h b/tools/testing/selftests/bpf/bpf_experimental.h
index 67ff7882299e..6da93c486650 100644
--- a/tools/testing/selftests/bpf/bpf_experimental.h
+++ b/tools/testing/selftests/bpf/bpf_experimental.h
@@ -361,6 +361,16 @@ extern int bpf_iter_dmabuf_new(struct bpf_iter_dmabuf *it) __weak __ksym;
 extern struct dma_buf *bpf_iter_dmabuf_next(struct bpf_iter_dmabuf *it) __weak __ksym;
 extern void bpf_iter_dmabuf_destroy(struct bpf_iter_dmabuf *it) __weak __ksym;
 
+struct bpf_iter_workqueue;
+extern int bpf_iter_workqueue_new(struct bpf_iter_workqueue *it) __weak __ksym;
+extern struct workqueue_struct *bpf_iter_workqueue_next(struct bpf_iter_workqueue *it) __weak __ksym;
+extern void bpf_iter_workqueue_destroy(struct bpf_iter_workqueue *it) __weak __ksym;
+
+struct bpf_iter_worker_pool;
+extern int bpf_iter_worker_pool_new(struct bpf_iter_worker_pool *it) __weak __ksym;
+extern struct worker_pool *bpf_iter_worker_pool_next(struct bpf_iter_worker_pool *it) __weak __ksym;
+extern void bpf_iter_worker_pool_destroy(struct bpf_iter_worker_pool *it) __weak __ksym;
+
 extern int bpf_cgroup_read_xattr(struct cgroup *cgroup, const char *name__str,
 				 struct bpf_dynptr *value_p) __weak __ksym;
 
diff --git a/tools/testing/selftests/bpf/prog_tests/wq_iter.c b/tools/testing/selftests/bpf/prog_tests/wq_iter.c
new file mode 100644
index 000000000000..73383c263bee
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/wq_iter.c
@@ -0,0 +1,80 @@
+// SPDX-License-Identifier: GPL-2.0
+#include <test_progs.h>
+#include "wq_iter.skel.h"
+#include "wq_iter_fail.skel.h"
+
+static void subtest_open_coded(struct wq_iter *skel)
+{
+	LIBBPF_OPTS(bpf_test_run_opts, opts);
+	int err;
+
+	err = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.count_workqueues),
+				     &opts);
+	if (!ASSERT_OK(err, "run count_workqueues"))
+		return;
+	/* There are always several system workqueues (events, ...). */
+	ASSERT_GT(skel->bss->nr_workqueues, 0, "nr_workqueues");
+
+	err = bpf_prog_test_run_opts(bpf_program__fd(skel->progs.count_worker_pools),
+				     &opts);
+	if (!ASSERT_OK(err, "run count_worker_pools"))
+		return;
+	/* At least the per-CPU normal/highpri pools exist. */
+	ASSERT_GT(skel->bss->nr_worker_pools, 0, "nr_worker_pools");
+	ASSERT_GT(skel->bss->nr_percpu_pools, 0, "nr_percpu_pools");
+}
+
+static void drain_iter(struct bpf_program *prog, const char *name)
+{
+	struct bpf_link *link;
+	char buf[512];
+	int iter_fd;
+	ssize_t n;
+
+	link = bpf_program__attach_iter(prog, NULL);
+	if (!ASSERT_OK_PTR(link, name))
+		return;
+	iter_fd = bpf_iter_create(bpf_link__fd(link));
+	if (!ASSERT_GE(iter_fd, 0, "iter_create"))
+		goto out;
+	while ((n = read(iter_fd, buf, sizeof(buf))) > 0)
+		;
+	ASSERT_GE(n, 0, "read iter");
+	close(iter_fd);
+out:
+	bpf_link__destroy(link);
+}
+
+static void subtest_seq(struct wq_iter *skel)
+{
+	/* seq forms are backed by the same open-coded next(). */
+	drain_iter(skel->progs.dump_workqueues, "attach workqueue iter");
+	ASSERT_GT(skel->bss->nr_wq_seq, 0, "nr_wq_seq");
+
+	drain_iter(skel->progs.dump_worker_pools, "attach worker_pool iter");
+	ASSERT_GT(skel->bss->nr_pool_seq, 0, "nr_pool_seq");
+
+	/*
+	 * worklists are usually empty on an idle system; this drives the whole
+	 * per-pool snapshot path and verifies it drains cleanly.
+	 */
+	drain_iter(skel->progs.dump_pending, "attach pending_work iter");
+}
+
+void test_wq_iter(void)
+{
+	struct wq_iter *skel;
+
+	skel = wq_iter__open_and_load();
+	if (!ASSERT_OK_PTR(skel, "wq_iter__open_and_load"))
+		return;
+
+	if (test__start_subtest("open_coded"))
+		subtest_open_coded(skel);
+	if (test__start_subtest("seq"))
+		subtest_seq(skel);
+
+	wq_iter__destroy(skel);
+
+	RUN_TESTS(wq_iter_fail);
+}
diff --git a/tools/testing/selftests/bpf/progs/wq_iter.c b/tools/testing/selftests/bpf/progs/wq_iter.c
new file mode 100644
index 000000000000..15f081bc634b
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/wq_iter.c
@@ -0,0 +1,95 @@
+// SPDX-License-Identifier: GPL-2.0
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include "bpf_experimental.h"
+
+char _license[] SEC("license") = "GPL";
+
+void bpf_rcu_read_lock(void) __ksym;
+void bpf_rcu_read_unlock(void) __ksym;
+
+/* Results, checked by userspace. */
+int nr_workqueues;
+int nr_worker_pools;
+int nr_percpu_pools;
+int nr_wq_seq;
+int nr_pool_seq;
+int nr_pending;
+
+/* --- open-coded iterators (KF_RCU_PROTECTED) --- */
+
+SEC("syscall")
+int count_workqueues(const void *ctx)
+{
+	struct workqueue_struct *wq;
+	int n = 0;
+
+	bpf_rcu_read_lock();
+	bpf_for_each(workqueue, wq)
+		n++;
+	bpf_rcu_read_unlock();
+
+	nr_workqueues = n;
+	return 0;
+}
+
+SEC("syscall")
+int count_worker_pools(const void *ctx)
+{
+	struct worker_pool *pool;
+	int n = 0, percpu = 0;
+
+	bpf_rcu_read_lock();
+	bpf_for_each(worker_pool, pool) {
+		n++;
+		if (pool->cpu >= 0)		/* per-CPU pool */
+			percpu++;
+	}
+	bpf_rcu_read_unlock();
+
+	nr_worker_pools = n;
+	nr_percpu_pools = percpu;
+	return 0;
+}
+
+/* --- seq_file iterators --- */
+
+SEC("iter/workqueue")
+int dump_workqueues(struct bpf_iter__workqueue *ctx)
+{
+	struct seq_file *seq = ctx->meta->seq;
+	struct workqueue_struct *wq = ctx->wq;
+
+	if (!wq)
+		return 0;
+	nr_wq_seq++;
+	BPF_SEQ_PRINTF(seq, "%s\n", wq->name);
+	return 0;
+}
+
+SEC("iter/worker_pool")
+int dump_worker_pools(struct bpf_iter__worker_pool *ctx)
+{
+	struct worker_pool *pool = ctx->pool;
+
+	if (!pool)
+		return 0;
+	nr_pool_seq++;
+	return 0;
+}
+
+SEC("iter/workqueue_pending_work")
+int dump_pending(struct bpf_iter__workqueue_pending_work *ctx)
+{
+	struct seq_file *seq = ctx->meta->seq;
+	struct wq_pending_work_info *info = ctx->info;
+
+	if (!info)			/* final call */
+		return 0;
+
+	nr_pending++;
+	BPF_SEQ_PRINTF(seq, "pool %llu work 0x%llx func 0x%llx\n",
+		       info->pool_id, info->work, info->func);
+	return 0;
+}
diff --git a/tools/testing/selftests/bpf/progs/wq_iter_fail.c b/tools/testing/selftests/bpf/progs/wq_iter_fail.c
new file mode 100644
index 000000000000..fb2dcf9847e2
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/wq_iter_fail.c
@@ -0,0 +1,37 @@
+// SPDX-License-Identifier: GPL-2.0
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+#include "bpf_misc.h"
+#include "bpf_experimental.h"
+
+char _license[] SEC("license") = "GPL";
+
+/*
+ * The workqueue and worker_pool open-coded iterators are KF_RCU_PROTECTED:
+ * using them outside a bpf_rcu_read_lock() region must be rejected.
+ */
+
+SEC("?syscall")
+__failure __msg("kernel func bpf_iter_workqueue_new requires RCU critical section protection")
+int workqueue_iter_no_rcu(const void *ctx)
+{
+	struct workqueue_struct *wq;
+	int n = 0;
+
+	bpf_for_each(workqueue, wq)
+		n++;
+	return n;
+}
+
+SEC("?syscall")
+__failure __msg("kernel func bpf_iter_worker_pool_new requires RCU critical section protection")
+int worker_pool_iter_no_rcu(const void *ctx)
+{
+	struct worker_pool *pool;
+	int n = 0;
+
+	bpf_for_each(worker_pool, pool)
+		n++;
+	return n;
+}
-- 
2.43.0

Re: [PATCH bpf-next 0/6] workqueue: introduce BPF iterators for workqueues, worker pools and pending work

[Tejun Heo]

Hello,

On Tue, Jul 28, 2026 at 02:55:19PM +0800, Imran Khan wrote:
> Nearly every subsystem defers work to workqueues, and their state can
> already be observed on a live system, just not at a granularity that is
> convenient to consume. sysrq dumps every workqueue and pool to the kernel
> log in a fixed format, with no way to select or aggregate; the WQ_SYSFS
> interface only covers workqueues that ask for it and exposes attributes
> rather than runtime state; and the drgn scripts under tools/workqueue/ need
> a debuginfo-equipped userspace and read the state from the outside, without
> the locks that protect it.
> 
> This series adds BPF iterators for workqueues, worker pools and pending
> work items, so that this state can be walked from inside the kernel, under
> the right locking, with filtering and aggregation done in place and only
> the interesting part copied to userspace.

I'm not necessarily against it but what are the use cases here? If for
debugging, isn't drgn + hooking into tracepoints mostly enough? Can you give
concrete examples where bpf iterators are essential?

Thanks.

-- 
tejun

Re: [PATCH bpf-next 0/6] workqueue: introduce BPF iterators for workqueues, worker pools and pending work

[imran.f.khan]

On 2/8/2026 10:59 am, Tejun Heo wrote:
Hello Tejun,
Thanks for taking a look into this.

> Hello,
> 
> On Tue, Jul 28, 2026 at 02:55:19PM +0800, Imran Khan wrote:
>> Nearly every subsystem defers work to workqueues, and their state can
>> already be observed on a live system, just not at a granularity that is
>> convenient to consume. sysrq dumps every workqueue and pool to the kernel
>> log in a fixed format, with no way to select or aggregate; the WQ_SYSFS
>> interface only covers workqueues that ask for it and exposes attributes
>> rather than runtime state; and the drgn scripts under tools/workqueue/ need
>> a debuginfo-equipped userspace and read the state from the outside, without
>> the locks that protect it.
>>
>> This series adds BPF iterators for workqueues, worker pools and pending
>> work items, so that this state can be walked from inside the kernel, under
>> the right locking, with filtering and aggregation done in place and only
>> the interesting part copied to userspace.
> 
> I'm not necessarily against it but what are the use cases here? If for
> debugging, isn't drgn + hooking into tracepoints mostly enough? Can you give
> concrete examples where bpf iterators are essential?
> 

The main use case is being able to peek at workqueue state right when a problem
(where checking workqueue state makes sense) is detected.
One very common case for us is getting RDS TX timeouts because of the the stuck work items.
At the moment I don't have a system where I can both use the patched kernel and
reproduce the Tx timeout issue, so I have tried to explain the same using block
layer and dm-device as affected subsystem.

The bpf program shown below measures block_bio_queue -> block_bio_complete on the dm
device:

#include "vmlinux.h"
#include <bpf/bpf_helpers.h>
#include <bpf/bpf_tracing.h>
#include <bpf/bpf_core_read.h>

char _license[] SEC("license") = "GPL";

#define MIN_LAT_MS      5
#define HZ_HINT         1000

void bpf_rcu_read_lock(void) __ksym;
void bpf_rcu_read_unlock(void) __ksym;

extern int bpf_iter_workqueue_new(struct bpf_iter_workqueue *it) __weak __ksym;
extern struct workqueue_struct *bpf_iter_workqueue_next(struct bpf_iter_workqueue *it) __weak __ksym;
extern void bpf_iter_workqueue_destroy(struct bpf_iter_workqueue *it) __weak __ksym;

extern int bpf_iter_worker_pool_new(struct bpf_iter_worker_pool *it) __weak __ksym;
extern struct worker_pool *bpf_iter_worker_pool_next(struct bpf_iter_worker_pool *it) __weak __ksym;
extern void bpf_iter_worker_pool_destroy(struct bpf_iter_worker_pool *it) __weak __ksym;

SEC("iter/workqueue_pending_work")
int dump_pending(struct bpf_iter__workqueue_pending_work *ctx)
{
        struct seq_file *seq = ctx->meta->seq;
        struct wq_pending_work_info *info = ctx->info;

        if (!info)
                return 0;

        BPF_SEQ_PRINTF(seq, "pool=%-3llu work=0x%llx func=%pS\n",
                       info->pool_id, info->work, (void *)info->func);
        return 0;
}

struct {
        __uint(type, BPF_MAP_TYPE_HASH);
        __uint(max_entries, 65536);
        __type(key, __u64);
        __type(value, __u64);
} inflight SEC(".maps");

static __always_inline int is_dm_bio(struct bio *bio)
{
        struct gendisk *disk;
        char name[8] = {};

        disk = BPF_CORE_READ(bio, bi_bdev, bd_disk);
        if (!disk)
                return 0;

        bpf_probe_read_kernel_str(name, sizeof(name), disk->disk_name);
        return name[0] == 'd' && name[1] == 'm' && name[2] == '-';
}

__u64 pending_report_us;
__u32 nr_slow;
__u32 nr_reports;

static __always_inline void dump_wq_state(__u64 lat_us)
{
        struct bpf_iter_worker_pool pit;
        struct bpf_iter_workqueue wit;
        struct workqueue_struct *wq;
        struct worker_pool *pool;
        __u64 now_j = bpf_jiffies64();
        __u32 this_cpu = bpf_get_smp_processor_id();

        bpf_printk("STALL: dm-crypt bio took %llu us; sampling workqueue state now (cpu%d)",
                   lat_us, this_cpu);

        bpf_rcu_read_lock();

        bpf_iter_worker_pool_new(&pit);
        while ((pool = bpf_iter_worker_pool_next(&pit))) {
                __u64 stale_ms = (now_j - pool->last_progress_ts) * 1000 / HZ_HINT;
                int has_work = pool->worklist.next != &pool->worklist;
                int is_this_cpu = pool->cpu == (int)this_cpu;

                if (stale_ms > 60000)
                        continue;
                if (!has_work && !is_this_cpu)
                        continue;

                bpf_printk("  %s pool %d cpu %d: no progress for %llu ms, work_waiting=%d, running=%d idle=%d/%d",
                           is_this_cpu ? "->" : "  ",
                           pool->id, pool->cpu, stale_ms, has_work,
                           pool->nr_running, pool->nr_idle, pool->nr_workers);
        }
        bpf_iter_worker_pool_destroy(&pit);

        bpf_iter_workqueue_new(&wit);
        while ((wq = bpf_iter_workqueue_next(&wit))) {
                char name[10];

                bpf_probe_read_kernel_str(name, sizeof(name), wq->name);
                if (name[0] != 'k' || name[1] != 'c' || name[2] != 'r')
                        continue;

                bpf_printk("  wq %s: max_active=%d flags=0x%x",
                           name, wq->max_active, wq->flags);
        }
        bpf_iter_workqueue_destroy(&wit);

        bpf_rcu_read_unlock();
}

SEC("tp_btf/block_bio_queue")
int BPF_PROG(on_bio_queue, struct bio *bio)
{
        __u64 key = (__u64)(long)bio;
        __u64 now = bpf_ktime_get_ns();
        __u64 lat_us;

        if (!is_dm_bio(bio))
                return 0;

        bpf_map_update_elem(&inflight, &key, &now, BPF_ANY);

        lat_us = pending_report_us;
        if (lat_us) {
                pending_report_us = 0;
                nr_reports++;
                dump_wq_state(lat_us);
        }
        return 0;
}

SEC("tp_btf/block_bio_complete")
int BPF_PROG(on_bio_complete, struct request_queue *q, struct bio *bio)
{
        __u64 key = (__u64)(long)bio;
        __u64 *tsp, lat;

        tsp = bpf_map_lookup_elem(&inflight, &key);
        if (!tsp)
                return 0;

        lat = bpf_ktime_get_ns() - *tsp;
        bpf_map_delete_elem(&inflight, &key);

        if (lat < (__u64)MIN_LAT_MS * 1000000ULL)
                return 0;

        nr_slow++;
        pending_report_us = lat / 1000;
        return 0;
}


The above bpf programs can be attached using following 2 commands:

bpftool prog loadall sample.bpf.o /sys/fs/bpf/dm autoattach
bpftool iter pin sample.bpf.o /sys/fs/bpf/dm_pending
(sample.bpf.c is the name of file containing above code)

When a bio exceeds the threshold it walks the open-coded
worker_pool and workqueue iterators and prints what it finds:

  fio-2409 [003] 1246.736278: STALL: dm-crypt bio took 950434 us; sampling workqueue state now (cpu3)
  fio-2409 [003] 1246.736282:    pool 10 cpu 2: no progress for 935 ms, work_waiting=1, running=0 idle=3/3
  fio-2409 [003] 1246.736285:    pool 22 cpu 5: no progress for 934 ms, work_waiting=1, running=0 idle=3/3
  fio-2409 [003] 1246.736299:    wq kcryptd_i: max_active=1 flags=0x148
  fio-2409 [003] 1246.736300:    wq kcryptd-2: max_active=1 flags=0x168

In order to reproduce the issue easily I had RT threads hogging the CPUs and
thus preventing timely run of workers.

Another limitation with drgn and traces is that often the production systems
don't have drgn and/or debuginfo installed and sometimes the systems are itself
in such a bad shape that running drgn becomes challenging.
For such cases as well, a quick look into the bpffs to find pending works
(like shown below) helps:

cat /sys/fs/bpf/dm_pending
  pool=2   work=0xffff9987afc27c48 func=delayed_vfree_work+0x0/0x50
  pool=6   work=0xffff9987afc67560 func=vmstat_update+0x0/0x50
  pool=14  work=0xffff998484b39e20 func=kcryptd_crypt+0x0/0x310 [dm_crypt]
  pool=22  work=0xffff9987afd67560 func=vmstat_update+0x0/0x50
  pool=26  work=0xffff9987afda7560 func=vmstat_update+0x0/0x50
  pool=30  work=0xffff9987afde7560 func=vmstat_update+0x0/0x50

If the issue happens randomly in short windows of few secs, collecting
the traces for long intervals and looking for data of that short window is
not easy.

These are the use cases/limitation I had in mind. Could you please let
me know your thoughts?

Thanks,
Imran


> Thanks.
> 

Re: [PATCH bpf-next 0/6] workqueue: introduce BPF iterators for workqueues, worker pools and pending work

[Tejun Heo]

Hello,

On Thu, Aug 06, 2026 at 04:41:01PM +0800, imran.f.khan@oracle.com wrote:
...
> When a bio exceeds the threshold it walks the open-coded
> worker_pool and workqueue iterators and prints what it finds:
> 
>   fio-2409 [003] 1246.736278: STALL: dm-crypt bio took 950434 us; sampling workqueue state now (cpu3)
>   fio-2409 [003] 1246.736282:    pool 10 cpu 2: no progress for 935 ms, work_waiting=1, running=0 idle=3/3
>   fio-2409 [003] 1246.736285:    pool 22 cpu 5: no progress for 934 ms, work_waiting=1, running=0 idle=3/3
>   fio-2409 [003] 1246.736299:    wq kcryptd_i: max_active=1 flags=0x148
>   fio-2409 [003] 1246.736300:    wq kcryptd-2: max_active=1 flags=0x168
> 
> In order to reproduce the issue easily I had RT threads hogging the CPUs and
> thus preventing timely run of workers.
> 
> Another limitation with drgn and traces is that often the production systems
> don't have drgn and/or debuginfo installed and sometimes the systems are itself
> in such a bad shape that running drgn becomes challenging.
> For such cases as well, a quick look into the bpffs to find pending works
> (like shown below) helps:
> 
> cat /sys/fs/bpf/dm_pending
>   pool=2   work=0xffff9987afc27c48 func=delayed_vfree_work+0x0/0x50
>   pool=6   work=0xffff9987afc67560 func=vmstat_update+0x0/0x50
>   pool=14  work=0xffff998484b39e20 func=kcryptd_crypt+0x0/0x310 [dm_crypt]
>   pool=22  work=0xffff9987afd67560 func=vmstat_update+0x0/0x50
>   pool=26  work=0xffff9987afda7560 func=vmstat_update+0x0/0x50
>   pool=30  work=0xffff9987afde7560 func=vmstat_update+0x0/0x50
> 
> If the issue happens randomly in short windows of few secs, collecting
> the traces for long intervals and looking for data of that short window is
> not easy.
> 
> These are the use cases/limitation I had in mind. Could you please let
> me know your thoughts?

I feel a bit conflicted because this is something only useful for debugging
and doing this subsystem-by-subsystem would mean hard coding data structure
iterators and accessors into every subsystem, when there already is a
generic, albeit with different trade-offs, way to access about the same data
through debug info and vmcore (ie. drgn).

Even in the example you gave, if you had a standing drgn script running and
trigger it on timeout threshold, it'd be able to produce the same data
that's needed. It's more cumbersome but it's also more generic and flexible.
If you really want to do it in BPF, it's not *that* difficult to write an
adhoc iterator with bpf_core_read() and friends either.

So, my concern mostly is that the use cases seem to restricted. There's no
"production" usefulness where e.g. performance or overhead matters which I
think makes the full-on iterators somewhat over-engineered.

Thanks.

-- 
tejun