Re: [PATCH bpf-next 0/6] workqueue: introduce BPF iterators for workqueues, worker pools and pending work

[Tejun Heo <tj@kernel.org>]

Hello,

On Thu, Aug 06, 2026 at 04:41:01PM +0800, imran.f.khan@oracle.com wrote:
...
> When a bio exceeds the threshold it walks the open-coded
> worker_pool and workqueue iterators and prints what it finds:
> 
>   fio-2409 [003] 1246.736278: STALL: dm-crypt bio took 950434 us; sampling workqueue state now (cpu3)
>   fio-2409 [003] 1246.736282:    pool 10 cpu 2: no progress for 935 ms, work_waiting=1, running=0 idle=3/3
>   fio-2409 [003] 1246.736285:    pool 22 cpu 5: no progress for 934 ms, work_waiting=1, running=0 idle=3/3
>   fio-2409 [003] 1246.736299:    wq kcryptd_i: max_active=1 flags=0x148
>   fio-2409 [003] 1246.736300:    wq kcryptd-2: max_active=1 flags=0x168
> 
> In order to reproduce the issue easily I had RT threads hogging the CPUs and
> thus preventing timely run of workers.
> 
> Another limitation with drgn and traces is that often the production systems
> don't have drgn and/or debuginfo installed and sometimes the systems are itself
> in such a bad shape that running drgn becomes challenging.
> For such cases as well, a quick look into the bpffs to find pending works
> (like shown below) helps:
> 
> cat /sys/fs/bpf/dm_pending
>   pool=2   work=0xffff9987afc27c48 func=delayed_vfree_work+0x0/0x50
>   pool=6   work=0xffff9987afc67560 func=vmstat_update+0x0/0x50
>   pool=14  work=0xffff998484b39e20 func=kcryptd_crypt+0x0/0x310 [dm_crypt]
>   pool=22  work=0xffff9987afd67560 func=vmstat_update+0x0/0x50
>   pool=26  work=0xffff9987afda7560 func=vmstat_update+0x0/0x50
>   pool=30  work=0xffff9987afde7560 func=vmstat_update+0x0/0x50
> 
> If the issue happens randomly in short windows of few secs, collecting
> the traces for long intervals and looking for data of that short window is
> not easy.
> 
> These are the use cases/limitation I had in mind. Could you please let
> me know your thoughts?

I feel a bit conflicted because this is something only useful for debugging
and doing this subsystem-by-subsystem would mean hard coding data structure
iterators and accessors into every subsystem, when there already is a
generic, albeit with different trade-offs, way to access about the same data
through debug info and vmcore (ie. drgn).

Even in the example you gave, if you had a standing drgn script running and
trigger it on timeout threshold, it'd be able to produce the same data
that's needed. It's more cumbersome but it's also more generic and flexible.
If you really want to do it in BPF, it's not *that* difficult to write an
adhoc iterator with bpf_core_read() and friends either.

So, my concern mostly is that the use cases seem to restricted. There's no
"production" usefulness where e.g. performance or overhead matters which I
think makes the full-on iterators somewhat over-engineered.

Thanks.

-- 
tejun