* [RFC PATCH v14 0/4] blk-iocost: BPF struct_ops cost model
@ 2026-11-18 13:10 Tao Cui
2026-11-18 13:10 ` [RFC PATCH v14 1/4] blk-iocost: add BPF struct_ops cost model support Tao Cui
` (4 more replies)
0 siblings, 5 replies; 9+ messages in thread
From: Tao Cui @ 2026-11-18 13:10 UTC (permalink / raw)
To: tj, josef, axboe, ast, alexei.starovoitov
Cc: cgroups, linux-block, linux-kernel, bpf, andrii, eddyz87, daniel,
linux-kselftest, cui.tao, cuitao, ameryhung
From: Tao Cui <cuitao@kylinos.cn>
This series lets a BPF program take over the cost model of one
device. The BPF model provides calc_cost() for charged IOs plus two
transfer coefficients for the completion-time sizing, io.cost.model
selects between the attached model and the builtin one, and enabling
and disabling the controller stays with io.cost.qos.
The builtin model is a linear formula derived from a handful of device
parameters, and the right formula is device specific. A BPF model can
be adapted to the device and updated without rebuilding the kernel.
The series is based on next-20261117.
Changes since v13:
- p1: stop updating attached and attached_link under ioc->lock;
rq_qos_mutex already orders all their accessors (review from tj)
- p1: name the calc_cost flags enum iocost_calc_flags and rename
ioc_model()/model to ioc_bpf_model()/bpf_model so that the
builtin model isn't called a NULL model (review from tj)
- p1: document why iocost holds its own model reference, the
field locking and the merge flag semantics, and reword the
commit message to describe guarantees instead of code paths
(review from tj)
- p1: clear the autop stepping timestamps when a BPF model is
attached; without this, the timestamps from before the attach
survive until a model=linear switchback and can trigger an
immediate profile step on that stale pre-attach evidence
(report from sashiko)
- p1: align the BPF pricing scope with the builtin model:
dataless bios with no flush state and ops other than reads,
writes and zone appends cost nothing, completion-time sizing
prices reads, writes and zone appends, and a dead owning map no
longer reports -EBUSY (report from sashiko)
- p3: probe the iocost_ioc_tick tracepoint, unwind the private
mount namespace on setup failure, report fopen errors faithfully
and fix an inverted posix_memalign() assertion that made the
streams test skip its whole IO block (report from sashiko)
- p2/p4: document the iocost_ioc_tick fields in the trace header
and extend the io.cost.model docs with the dev supply and the
-ENODEV attach conditions (report from sashiko)
Tested on next-20261117 (qemu x86_64): the selftests pass, and the 2x
example model doubles the charged cost over the builtin on a
rotational device (2.00x measured), returning to 1.00x after
"model=linear".
Link: https://lore.kernel.org/r/20261106122356.3224638-1-cui.tao@linux.dev # v13
Tao Cui (4):
blk-iocost: add BPF struct_ops cost model support
blk-iocost: add iocost_ioc_tick tracepoint for per-period device
summary
selftests/bpf: add iocost cost model test
docs: cgroup-v2: document the iocost BPF cost model attachment
Documentation/admin-guide/cgroup-v2.rst | 55 +-
MAINTAINERS | 1 +
block/Kconfig | 4 +
block/blk-iocost.c | 360 ++++++-
include/trace/events/iocost.h | 57 ++
tools/testing/selftests/bpf/config | 2 +
.../selftests/bpf/prog_tests/iocost_model.c | 947 ++++++++++++++++++
.../selftests/bpf/progs/iocost_model.c | 188 ++++
tools/testing/selftests/bpf/progs/iocost_ms.c | 195 ++++
9 files changed, 1797 insertions(+), 12 deletions(-)
create mode 100644 tools/testing/selftests/bpf/prog_tests/iocost_model.c
create mode 100644 tools/testing/selftests/bpf/progs/iocost_model.c
create mode 100644 tools/testing/selftests/bpf/progs/iocost_ms.c
^ permalink raw reply [flat|nested] 9+ messages in thread
* [RFC PATCH v14 1/4] blk-iocost: add BPF struct_ops cost model support
2026-11-18 13:10 [RFC PATCH v14 0/4] blk-iocost: BPF struct_ops cost model Tao Cui
@ 2026-11-18 13:10 ` Tao Cui
2026-11-18 13:10 ` [RFC PATCH v14 2/4] blk-iocost: add iocost_ioc_tick tracepoint for per-period device summary Tao Cui
` (3 subsequent siblings)
4 siblings, 0 replies; 9+ messages in thread
From: Tao Cui @ 2026-11-18 13:10 UTC (permalink / raw)
To: tj, josef, axboe, ast, alexei.starovoitov
Cc: cgroups, linux-block, linux-kernel, bpf, andrii, eddyz87, daniel,
linux-kselftest, cui.tao, cuitao, ameryhung
From: Tao Cui <cuitao@kylinos.cn>
iocost prices IO with a linear model derived from a handful of
device parameters. The real cost of an IO depends on device
internals the formula cannot capture and the right formula is
device specific. This lets a BPF program take over pricing for
a device so that the model can be adapted to it without
rebuilding the kernel.
Attaching an iocost_model_ops struct_ops binds the model to one
device: each device has at most one attached model, and
attaching switches the device's pricing to it while io.cost.qos
keeps controlling whether the device is throttled at all. The
model remains the pricing authority across io.cost.model
writes: "model=linear" and "model=bpf" only select between the
attached model and the builtin one. The model steps down when
the struct_ops link is deleted or the device is removed, at
which point the device reverts to the builtin model. Attaching
does not enable the controller.
Attaching also guarantees that the model cannot go away while it
is pricing: iocost holds its own reference on the model, so
neither deleting the link nor removing the device in the middle
of a removal race can free a model whose prices are still being
applied, and the model is released only after in-flight
calc_cost() calls have finished. Models without calc_cost or
with zero or out-of-range transfer coefficients are rejected
when attached, and the transfer coefficients drive the
completion-time request sizing, so latency tracking and vrate
adjustment follow the model's pricing.
The kernel keeps no per-cgroup state for the model; models
which need per-cgroup data keep it in their own maps, created
and freed with the cgroups. Without CONFIG_BLK_CGROUP_IOCOST_BPF
the struct_ops is not built and the builtin model is the only
choice.
Suggested-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
Acked-by: Alexei Starovoitov <ast@kernel.org>
---
block/Kconfig | 4 +
block/blk-iocost.c | 342 ++++++++++++++++++++++++++++++++++++++++++++-
2 files changed, 342 insertions(+), 4 deletions(-)
diff --git a/block/Kconfig b/block/Kconfig
index 70e4a66d941f..26badf54a477 100644
--- a/block/Kconfig
+++ b/block/Kconfig
@@ -161,6 +161,10 @@ config BLK_CGROUP_IOCOST
distributes IO capacity between different groups based on
their share of the overall weight distribution.
+config BLK_CGROUP_IOCOST_BPF
+ def_bool y
+ depends on BLK_CGROUP_IOCOST && BPF_SYSCALL && BPF_JIT && DEBUG_INFO_BTF
+
config BLK_CGROUP_IOPRIO
bool "Cgroup I/O controller for assigning an I/O priority class"
depends on BLK_CGROUP
diff --git a/block/blk-iocost.c b/block/blk-iocost.c
index 0ffe34bd95b7..8398a406f7d3 100644
--- a/block/blk-iocost.c
+++ b/block/blk-iocost.c
@@ -34,7 +34,7 @@
* device time. If a given IO is estimated to cost 10ms, the device should
* be able to process ~100 of those IOs in a second.
*
- * Currently, there's only one builtin cost model - linear. Each IO is
+ * The builtin cost model is linear. Each IO is
* classified as sequential or random and given a base cost accordingly.
* On top of that, a size cost proportional to the length of the IO is
* added. While simple, this model captures the operational
@@ -179,8 +179,12 @@
#include <linux/time64.h>
#include <linux/parser.h>
#include <linux/sched/signal.h>
+#include <linux/bpf.h>
+#include <linux/bpf_verifier.h>
+#include <linux/btf.h>
#include <asm/local.h>
#include <asm/local64.h>
+#include "blk.h"
#include "blk-rq-qos.h"
#include "blk-stat.h"
#include "blk-wbt.h"
@@ -405,6 +409,39 @@ struct ioc_pcpu_stat {
u64 last_rq_wait_ns;
};
+/*
+ * Flags for iocost_model_ops.calc_cost(), named after the callback
+ * they're passed to so that BPF programs can look them up through BTF.
+ */
+enum iocost_calc_flags {
+ /*
+ * The bio is being merged into an IO which was already charged
+ * its base cost. The model should price only the incremental
+ * cost: the builtin model skips the seek component and charges
+ * the page cost in this case.
+ */
+ IOCOST_CALC_MERGE = 1 << 0,
+};
+
+/* bpf cost model of one device */
+struct iocost_model_ops {
+ /* whole disk, in the new_encode_dev() encoding */
+ dev_t dev;
+
+ /* used by the completion path instead of LCOEF_[RW]PAGE */
+ u64 read_vtime_per_page;
+ u64 write_vtime_per_page;
+
+ /*
+ * Called from the submission and merge paths for every charged
+ * bio (reads, writes, zone appends and flush-flagged bios)
+ * under rcu_read_lock(), possibly with a scheduler lock held
+ * on the merge path; must not sleep. Returns the cost of the
+ * bio in vtime units, clamped to VTIME_PER_SEC by the caller.
+ */
+ u64 (*calc_cost)(struct bio *bio, u64 calc_flags);
+};
+
/* per device */
struct ioc {
struct rq_qos rqos;
@@ -448,6 +485,28 @@ struct ioc {
int autop_idx;
bool user_qos_params:1;
bool user_cost_model:1;
+
+ /*
+ * Holds a reference on the struct_ops map, NULL if none is
+ * attached. The reference is what keeps the map alive when the
+ * link is destroyed after del_gendisk() has unhashed the bdev
+ * but before rq_qos_exit(); .unreg can't look up the disk in
+ * that window and returns without detaching, and the link
+ * drops its reference; iocost's own survives until
+ * ioc_rqos_exit() detaches the model.
+ *
+ * The attached and attached_link fields are protected by
+ * rq_qos_mutex.
+ *
+ * The bpf_model field is written under rq_qos_mutex and
+ * ioc->lock, and read with either RCU or ioc->lock held; the
+ * timer and the io.cost.model show paths read it under
+ * ioc->lock without rq_qos_mutex.
+ */
+ const struct iocost_model_ops *attached;
+ struct bpf_link *attached_link;
+ /* either NULL (linear) or ->attached */
+ const struct iocost_model_ops __rcu *bpf_model;
};
struct iocg_pcpu_stat {
@@ -672,6 +731,13 @@ static struct ioc *q_to_ioc(struct request_queue *q)
return rqos_to_ioc(rq_qos_id(q, RQ_QOS_COST));
}
+static const struct iocost_model_ops *ioc_bpf_model(struct ioc *ioc)
+{
+ if (!IS_ENABLED(CONFIG_BLK_CGROUP_IOCOST_BPF))
+ return NULL;
+ return rcu_dereference_check(ioc->bpf_model, lockdep_is_held(&ioc->lock));
+}
+
static const char __maybe_unused *ioc_name(struct ioc *ioc)
{
struct gendisk *disk = ioc->rqos.disk;
@@ -857,7 +923,7 @@ static int ioc_autop_idx(struct ioc *ioc, struct gendisk *disk)
return AUTOP_SSD_DFL;
/* if user is overriding anything, maintain what was there */
- if (ioc->user_qos_params || ioc->user_cost_model)
+ if (ioc->user_qos_params || ioc->user_cost_model || ioc_bpf_model(ioc))
return idx;
/* step up/down based on the vrate */
@@ -2657,6 +2723,41 @@ static u64 calc_vtime_cost(struct bio *bio, struct ioc_gq *iocg, bool is_merge)
{
u64 cost;
+ if (IS_ENABLED(CONFIG_BLK_CGROUP_IOCOST_BPF)) {
+ const struct iocost_model_ops *model;
+
+ guard(rcu)();
+ model = ioc_bpf_model(iocg->ioc);
+ if (model) {
+ bool is_flush = (bio->bi_opf & REQ_PREFLUSH) ||
+ ((bio->bi_opf & REQ_FUA) &&
+ !bdev_fua(bio->bi_bdev));
+
+ /*
+ * Keep the pricing scope the same as the builtin
+ * model: bios with no data and no flush state, and
+ * ops other than reads, writes and zone appends,
+ * cost nothing. Flushes are passed through so that
+ * the model can price them from the bio flags.
+ */
+ switch (bio_op(bio)) {
+ case REQ_OP_READ:
+ case REQ_OP_WRITE:
+ case REQ_OP_ZONE_APPEND:
+ if (!bio->bi_iter.bi_size && !is_flush)
+ return 0;
+ break;
+ default:
+ if (!is_flush)
+ return 0;
+ break;
+ }
+
+ cost = model->calc_cost(bio, is_merge ? IOCOST_CALC_MERGE : 0);
+ return min(cost, VTIME_PER_SEC);
+ }
+ }
+
calc_vtime_cost_builtin(bio, iocg, is_merge, &cost);
return cost;
}
@@ -2683,6 +2784,27 @@ static u64 calc_size_vtime_cost(struct request *rq, struct ioc *ioc)
{
u64 cost;
+ if (IS_ENABLED(CONFIG_BLK_CGROUP_IOCOST_BPF)) {
+ const struct iocost_model_ops *model;
+ unsigned int pages;
+
+ guard(rcu)();
+ model = ioc_bpf_model(ioc);
+ if (model) {
+ /* same op scope as the builtin model */
+ pages = blk_rq_stats_sectors(rq) >> IOC_SECT_TO_PAGE_SHIFT;
+ switch (req_op(rq)) {
+ case REQ_OP_READ:
+ return pages * model->read_vtime_per_page;
+ case REQ_OP_ZONE_APPEND:
+ case REQ_OP_WRITE:
+ return pages * model->write_vtime_per_page;
+ default:
+ return 0;
+ }
+ }
+ }
+
calc_size_vtime_cost_builtin(rq, ioc, &cost);
return cost;
}
@@ -2980,10 +3102,33 @@ static void ioc_rqos_queue_depth_changed(struct rq_qos *rqos)
spin_unlock_irq(&ioc->lock);
}
+/* Called by .unreg and by device removal, whichever is first. */
+static void ioc_detach_model(struct ioc *ioc)
+{
+#ifdef CONFIG_BLK_CGROUP_IOCOST_BPF
+ const struct iocost_model_ops *ops = ioc->attached;
+
+ lockdep_assert_held(&ioc->rqos.disk->queue->rq_qos_mutex);
+
+ if (!ops)
+ return;
+
+ spin_lock_irq(&ioc->lock);
+ rcu_assign_pointer(ioc->bpf_model, NULL);
+ spin_unlock_irq(&ioc->lock);
+ ioc->attached = NULL;
+ ioc->attached_link = NULL;
+
+ /* calc_cost() can still run. The map is freed after an RCU gp. */
+ bpf_struct_ops_put(ops);
+#endif
+}
+
static void ioc_rqos_exit(struct rq_qos *rqos)
{
struct ioc *ioc = rqos_to_ioc(rqos);
+ ioc_detach_model(ioc);
blkcg_deactivate_policy(rqos->disk, &blkcg_policy_iocost);
spin_lock_irq(&ioc->lock);
@@ -3535,11 +3680,12 @@ static u64 ioc_cost_model_prfill(struct seq_file *sf,
return 0;
spin_lock_irq(&ioc->lock);
- seq_printf(sf, "%s ctrl=%s model=linear "
+ seq_printf(sf, "%s ctrl=%s model=%s "
"rbps=%llu rseqiops=%llu rrandiops=%llu "
"wbps=%llu wseqiops=%llu wrandiops=%llu "
"flushiops=%llu\n",
dname, ioc->user_cost_model ? "user" : "auto",
+ ioc_bpf_model(ioc) ? "bpf" : "linear",
u[I_LCOEF_RBPS], u[I_LCOEF_RSEQIOPS], u[I_LCOEF_RRANDIOPS],
u[I_LCOEF_WBPS], u[I_LCOEF_WSEQIOPS], u[I_LCOEF_WRANDIOPS],
u[I_LCOEF_FLUSHIOPS]);
@@ -3576,6 +3722,7 @@ static const match_table_t i_lcoef_tokens = {
static ssize_t ioc_cost_model_write(struct kernfs_open_file *of, char *input,
size_t nbytes, loff_t off)
{
+ const struct iocost_model_ops *model;
struct blkg_conf_ctx ctx;
struct request_queue *q;
unsigned int memflags;
@@ -3612,6 +3759,7 @@ static ssize_t ioc_cost_model_write(struct kernfs_open_file *of, char *input,
spin_lock_irq(&ioc->lock);
memcpy(u, ioc->params.i_lcoefs, sizeof(u));
user = ioc->user_cost_model;
+ model = ioc_bpf_model(ioc);
ret = -EINVAL;
@@ -3636,7 +3784,11 @@ static ssize_t ioc_cost_model_write(struct kernfs_open_file *of, char *input,
continue;
case COST_MODEL:
match_strlcpy(buf, &args[0], sizeof(buf));
- if (strcmp(buf, "linear"))
+ if (!strcmp(buf, "linear"))
+ model = NULL;
+ else if (!strcmp(buf, "bpf") && ioc->attached)
+ model = ioc->attached;
+ else
goto unlock;
continue;
}
@@ -3656,6 +3808,7 @@ static ssize_t ioc_cost_model_write(struct kernfs_open_file *of, char *input,
} else {
ioc->user_cost_model = false;
}
+ rcu_assign_pointer(ioc->bpf_model, model);
ioc_refresh_params(ioc, true);
ret = 0;
@@ -3703,6 +3856,187 @@ static struct blkcg_policy blkcg_policy_iocost = {
.pd_stat_fn = ioc_pd_stat,
};
+#ifdef CONFIG_BLK_CGROUP_IOCOST_BPF
+static int ioc_bpf_reg(void *kdata, struct bpf_link *link)
+{
+ struct iocost_model_ops *ops = kdata;
+ struct block_device *bdev;
+ struct request_queue *q;
+ unsigned int memflags;
+ struct ioc *ioc;
+ int ret;
+
+ bdev = blkdev_get_no_open(new_decode_dev(ops->dev), false);
+ if (!bdev)
+ return -ENODEV;
+
+ q = bdev_get_queue(bdev);
+ if (bdev_is_partition(bdev) || !queue_is_mq(q)) {
+ blkdev_put_no_open(bdev);
+ return -EOPNOTSUPP;
+ }
+
+ /* the order of ioc_qos_write() */
+ memflags = blk_mq_freeze_queue(q);
+ mutex_lock(&q->rq_qos_mutex);
+
+ ret = -ENODEV;
+ if (!disk_live(bdev->bd_disk))
+ goto out;
+
+ ioc = q_to_ioc(q);
+ if (!ioc) {
+ ret = blk_iocost_init(bdev->bd_disk);
+ if (ret)
+ goto out;
+ ioc = q_to_ioc(q);
+ }
+
+ if (ioc->attached) {
+ ret = -EBUSY;
+ goto out;
+ }
+ /* defensive; the owning map cannot be gone for a live link */
+ if (!bpf_struct_ops_get(ops)) {
+ ret = -ENODEV;
+ goto out;
+ }
+
+ ioc->attached = ops;
+ ioc->attached_link = link;
+
+ spin_lock_irq(&ioc->lock);
+ rcu_assign_pointer(ioc->bpf_model, ops);
+ /*
+ * Stepping is paused while a BPF model is selected. Drop any
+ * pending builtin step candidate so it cannot fire against the
+ * builtin model after a switchback.
+ */
+ ioc->autop_too_fast_at = 0;
+ ioc->autop_too_slow_at = 0;
+ /*
+ * Settle the autop state now that stepping is suspended; the
+ * stale pre-attach profile would otherwise linger until the
+ * next unrelated refresh event.
+ */
+ ioc_refresh_params(ioc, true);
+ spin_unlock_irq(&ioc->lock);
+
+ ret = 0;
+out:
+ mutex_unlock(&q->rq_qos_mutex);
+ blk_mq_unfreeze_queue(q, memflags);
+ blkdev_put_no_open(bdev);
+ return ret;
+}
+
+static void ioc_bpf_unreg(void *kdata, struct bpf_link *link)
+{
+ struct iocost_model_ops *ops = kdata;
+ struct block_device *bdev;
+ struct request_queue *q;
+ struct ioc *ioc;
+
+ /* del_gendisk() is in progress or done. ioc_rqos_exit() detaches. */
+ bdev = blkdev_get_no_open(new_decode_dev(ops->dev), false);
+ if (!bdev)
+ return;
+
+ q = bdev_get_queue(bdev);
+ mutex_lock(&q->rq_qos_mutex);
+ ioc = q_to_ioc(q);
+ /* the dev_t can belong to another disk and another model by now */
+ if (ioc && ioc->attached == ops && ioc->attached_link == link)
+ ioc_detach_model(ioc);
+ mutex_unlock(&q->rq_qos_mutex);
+ blkdev_put_no_open(bdev);
+}
+
+static int ioc_bpf_init(struct btf *btf)
+{
+ return 0;
+}
+
+static int ioc_bpf_check_member(const struct btf_type *t,
+ const struct btf_member *member,
+ const struct bpf_prog *prog)
+{
+ return prog->sleepable ? -EINVAL : 0;
+}
+
+static int ioc_bpf_init_member(const struct btf_type *t,
+ const struct btf_member *member,
+ void *kdata, const void *udata)
+{
+ const struct iocost_model_ops *uops = udata;
+ struct iocost_model_ops *ops = kdata;
+
+ switch (__btf_member_bit_offset(t, member) / 8) {
+ case offsetof(struct iocost_model_ops, dev):
+ ops->dev = uops->dev;
+ return 1;
+ case offsetof(struct iocost_model_ops, read_vtime_per_page):
+ ops->read_vtime_per_page = uops->read_vtime_per_page;
+ return 1;
+ case offsetof(struct iocost_model_ops, write_vtime_per_page):
+ ops->write_vtime_per_page = uops->write_vtime_per_page;
+ return 1;
+ }
+ return 0;
+}
+
+static int ioc_bpf_validate(void *kdata)
+{
+ struct iocost_model_ops *ops = kdata;
+
+ /*
+ * Bound the transfer coefficients so that coefficient * pages
+ * cannot wrap u64 for any possible request size.
+ */
+ if (ops->read_vtime_per_page > (VTIME_PER_SEC >> IOC_SECT_TO_PAGE_SHIFT) ||
+ ops->write_vtime_per_page > (VTIME_PER_SEC >> IOC_SECT_TO_PAGE_SHIFT))
+ return -EINVAL;
+
+ if (!ops->dev || !ops->calc_cost ||
+ !ops->read_vtime_per_page || !ops->write_vtime_per_page)
+ return -EINVAL;
+ return 0;
+}
+
+static u64 ioc_bpf_calc_cost_stub(struct bio *bio, u64 model_flags)
+{
+ return 0;
+}
+
+static struct iocost_model_ops ioc_bpf_stubs = {
+ .calc_cost = ioc_bpf_calc_cost_stub,
+};
+
+static const struct bpf_verifier_ops ioc_bpf_verifier_ops = {
+ .get_func_proto = bpf_base_func_proto,
+ .is_valid_access = bpf_tracing_btf_ctx_access,
+};
+
+static struct bpf_struct_ops ioc_bpf_struct_ops = {
+ .verifier_ops = &ioc_bpf_verifier_ops,
+ .init = ioc_bpf_init,
+ .check_member = ioc_bpf_check_member,
+ .init_member = ioc_bpf_init_member,
+ .validate = ioc_bpf_validate,
+ .reg = ioc_bpf_reg,
+ .unreg = ioc_bpf_unreg,
+ .cfi_stubs = &ioc_bpf_stubs,
+ .name = "iocost_model_ops",
+ .owner = THIS_MODULE,
+};
+
+static int __init ioc_bpf_struct_ops_init(void)
+{
+ return register_bpf_struct_ops(&ioc_bpf_struct_ops, iocost_model_ops);
+}
+late_initcall(ioc_bpf_struct_ops_init);
+#endif
+
static int __init ioc_init(void)
{
return blkcg_policy_register(&blkcg_policy_iocost);
--
2.53.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [RFC PATCH v14 2/4] blk-iocost: add iocost_ioc_tick tracepoint for per-period device summary
2026-11-18 13:10 [RFC PATCH v14 0/4] blk-iocost: BPF struct_ops cost model Tao Cui
2026-11-18 13:10 ` [RFC PATCH v14 1/4] blk-iocost: add BPF struct_ops cost model support Tao Cui
@ 2026-11-18 13:10 ` Tao Cui
2026-11-18 13:49 ` bot+bpf-ci
2026-11-18 13:10 ` [RFC PATCH v14 3/4] selftests/bpf: add iocost cost model test Tao Cui
` (2 subsequent siblings)
4 siblings, 1 reply; 9+ messages in thread
From: Tao Cui @ 2026-11-18 13:10 UTC (permalink / raw)
To: tj, josef, axboe, ast, alexei.starovoitov
Cc: cgroups, linux-block, linux-kernel, bpf, andrii, eddyz87, daniel,
linux-kselftest, cui.tao, cuitao, ameryhung
From: Tao Cui <cuitao@kylinos.cn>
Add iocost_ioc_tick, a tracepoint emitted once per period with a
summary of the controller's overall state.
Observing this state today requires drgn scripts to dig into ioc
internals. Exposing the per-period summary through a tracepoint lets
the common monitoring paths use standard tools (perf, bpftrace)
instead, and gives them an interface that does not shift with the
internal data layout. The emission points and the exact event fields
are documented in include/trace/events/iocost.h.
Suggested-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
Acked-by: Tejun Heo <tj@kernel.org>
---
MAINTAINERS | 1 +
block/blk-iocost.c | 18 +++++++++++
include/trace/events/iocost.h | 57 +++++++++++++++++++++++++++++++++++
3 files changed, 76 insertions(+)
diff --git a/MAINTAINERS b/MAINTAINERS
index 7d4f4312a2c2..cb9837e3bda7 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -6910,6 +6910,7 @@ F: block/blk-iocost.c
F: block/blk-iolatency.c
F: block/blk-throttle.c
F: include/linux/blk-cgroup.h
+F: include/trace/events/iocost.h
CONTROL GROUP - CPUSET
M: Waiman Long <longman@redhat.com>
diff --git a/block/blk-iocost.c b/block/blk-iocost.c
index 8398a406f7d3..607f886803b9 100644
--- a/block/blk-iocost.c
+++ b/block/blk-iocost.c
@@ -2378,6 +2378,7 @@ static void ioc_timer_fn(struct timer_list *timer)
struct ioc_now now;
LIST_HEAD(surpluses);
int nr_debtors, nr_shortages = 0, nr_lagging = 0;
+ int nr_active = 0;
u64 usage_us_sum = 0;
u32 ppm_rthr;
u32 ppm_wthr;
@@ -2575,6 +2576,23 @@ static void ioc_timer_fn(struct timer_list *timer)
ioc->busy_level = clamp(ioc->busy_level, -1000, 1000);
+ /*
+ * busy_level, cur_period, nr_active and usage_us_sum are complete
+ * here. usage is wall time over wall time, summed over iocgs, and
+ * is not capped at 100. vrate and period_us reflect the current
+ * controller state, which an io.cost.qos write may have changed
+ * mid-period. Emit before the refresh and the period transition.
+ */
+ if (trace_iocost_ioc_tick_enabled()) {
+ struct ioc_gq *active_iocg;
+
+ list_for_each_entry(active_iocg, &ioc->active_iocgs,
+ active_list)
+ nr_active++;
+
+ trace_iocost_ioc_tick(ioc, &now, nr_active, usage_us_sum);
+ }
+
ioc_adjust_base_vrate(ioc, rq_wait_pct, nr_lagging, nr_shortages,
prev_busy_level, missed_ppm);
diff --git a/include/trace/events/iocost.h b/include/trace/events/iocost.h
index eb711343ce4f..31df5ed5ee8c 100644
--- a/include/trace/events/iocost.h
+++ b/include/trace/events/iocost.h
@@ -181,6 +181,63 @@ TRACE_EVENT(iocost_ioc_vrate_adj,
)
);
+/*
+ * Periodic per-device summary, emitted once per period before the vrate
+ * adjustment and the period transition in ioc_timer_fn(). Unlike the
+ * state-change events above, this fires every period the controller is
+ * running, including steady states, and one final time before the
+ * controller goes idle, so the transition to dormancy is directly
+ * visible. It carries the overall controller state so basic monitoring
+ * doesn't require drgn.
+ *
+ * vrate is the current vrate in vtime units, where one IO time second
+ * is VTIME_PER_SEC (2^37) vtime and cost.vrate in io.stat reports the
+ * same value as a percentage of that base.
+ *
+ * nr_active is the number of iocgs on the active list when the event
+ * fires, gathered only when the tracepoint is enabled.
+ * usage_pct is the active iocgs' raw usage deltas summed over the period,
+ * as a percentage of the period wall time, the input to debt forgiveness
+ * decisions, before any per-iocg in-flight usage correction (donation
+ * decisions use the per-iocg usage after that correction). It can
+ * exceed 100% when concurrent service-time estimates exceed wall time.
+ */
+TRACE_EVENT(iocost_ioc_tick,
+
+ TP_PROTO(struct ioc *ioc, const struct ioc_now *now,
+ int nr_active, u64 usage_us_sum),
+
+ TP_ARGS(ioc, now, nr_active, usage_us_sum),
+
+ TP_STRUCT__entry (
+ __string(devname, ioc_name(ioc))
+ __field(u64, cur_period)
+ __field(u32, period_us)
+ __field(u64, vrate)
+ __field(int, busy_level)
+ __field(int, nr_active)
+ __field(u64, usage_pct)
+ ),
+
+ TP_fast_assign(
+ __assign_str(devname);
+ __entry->cur_period = atomic64_read(&ioc->cur_period);
+ __entry->period_us = ioc->period_us;
+ __entry->vrate = ioc->vtime_base_rate;
+ __entry->busy_level = ioc->busy_level;
+ __entry->nr_active = nr_active;
+ __entry->usage_pct = now->now > ioc->period_at ?
+ div64_u64(usage_us_sum * 100,
+ now->now - ioc->period_at) : 0;
+ ),
+
+ TP_printk("[%s] period=%llu:%uus vrate=%llu busy=%d active=%d usage=%llu%%",
+ __get_str(devname), __entry->cur_period, __entry->period_us,
+ __entry->vrate, __entry->busy_level, __entry->nr_active,
+ __entry->usage_pct
+ )
+);
+
TRACE_EVENT(iocost_iocg_forgive_debt,
TP_PROTO(struct ioc_gq *iocg, const char *path, struct ioc_now *now,
--
2.53.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [RFC PATCH v14 3/4] selftests/bpf: add iocost cost model test
2026-11-18 13:10 [RFC PATCH v14 0/4] blk-iocost: BPF struct_ops cost model Tao Cui
2026-11-18 13:10 ` [RFC PATCH v14 1/4] blk-iocost: add BPF struct_ops cost model support Tao Cui
2026-11-18 13:10 ` [RFC PATCH v14 2/4] blk-iocost: add iocost_ioc_tick tracepoint for per-period device summary Tao Cui
@ 2026-11-18 13:10 ` Tao Cui
2026-11-18 13:41 ` sashiko-bot
2026-11-18 13:49 ` bot+bpf-ci
2026-11-18 13:10 ` [RFC PATCH v14 4/4] docs: cgroup-v2: document the iocost BPF cost model attachment Tao Cui
2026-11-18 13:17 ` [RFC PATCH v14 0/4] blk-iocost: BPF struct_ops cost model Tao Cui
4 siblings, 2 replies; 9+ messages in thread
From: Tao Cui @ 2026-11-18 13:10 UTC (permalink / raw)
To: tj, josef, axboe, ast, alexei.starovoitov
Cc: cgroups, linux-block, linux-kernel, bpf, andrii, eddyz87, daniel,
linux-kselftest, cui.tao, cuitao, ameryhung
From: Tao Cui <cuitao@kylinos.cn>
Add two example iocost cost models and tests which attach them to a
null_blk instance the tests create through configfs, so everything
runs without external setup.
The first model implements the builtin linear HDD formula at double
cost. The second implements multi-stream sequentiality detection: it
keeps a per-cgroup table of stream slots instead of the single builtin
cursor, so concurrent sequential streams in one cgroup are priced
sequentially. Both set the target device through the struct_ops
member before load, and the maps are declared ".struct_ops.link" so
that a test dying in between does not leave the model attached.
The tests verify the model=bpf readback while attached, that ctrl=bpf
is rejected, that a second model on the same device fails with -EBUSY,
that model=linear and model=bpf switch between the attached and
builtin models without detaching, and that detaching restores the
builtin model. With the controller enabled through io.cost.qos, they
also verify that IOs are priced through calc_cost() and that the
iocost_ioc_tick tracepoint is seen from BPF. They also verify that
device removal under the attached model ejects it, with a recreated
device attachable again.
Suggested-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
---
tools/testing/selftests/bpf/config | 2 +
.../selftests/bpf/prog_tests/iocost_model.c | 947 ++++++++++++++++++
.../selftests/bpf/progs/iocost_model.c | 188 ++++
tools/testing/selftests/bpf/progs/iocost_ms.c | 195 ++++
4 files changed, 1332 insertions(+)
create mode 100644 tools/testing/selftests/bpf/prog_tests/iocost_model.c
create mode 100644 tools/testing/selftests/bpf/progs/iocost_model.c
create mode 100644 tools/testing/selftests/bpf/progs/iocost_ms.c
diff --git a/tools/testing/selftests/bpf/config b/tools/testing/selftests/bpf/config
index 30b744844ef5..4d84ec6904d0 100644
--- a/tools/testing/selftests/bpf/config
+++ b/tools/testing/selftests/bpf/config
@@ -142,3 +142,5 @@ CONFIG_SMC_HS_CTRL_BPF=y
CONFIG_DIBS=y
CONFIG_DIBS_LO=y
CONFIG_PM_WAKELOCKS=y
+CONFIG_BLK_CGROUP_IOCOST=y
+CONFIG_BLK_DEV_NULL_BLK=y
diff --git a/tools/testing/selftests/bpf/prog_tests/iocost_model.c b/tools/testing/selftests/bpf/prog_tests/iocost_model.c
new file mode 100644
index 000000000000..0fbaeb7c5b7e
--- /dev/null
+++ b/tools/testing/selftests/bpf/prog_tests/iocost_model.c
@@ -0,0 +1,947 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2026 KylinSoft Corporation. */
+
+#define _GNU_SOURCE
+#include <test_progs.h>
+#include <fcntl.h>
+#include <malloc.h>
+#include <sched.h>
+#include <sys/statfs.h>
+#include <linux/magic.h>
+#include <sys/mount.h>
+#include <sys/stat.h>
+#include <sys/sysmacros.h>
+#include <unistd.h>
+#include "iocost_model.skel.h"
+#include "iocost_ms.skel.h"
+#include "cgroup_helpers.h"
+
+/*
+ * Write val to the file at path (configfs or cgroupfs), returning the
+ * errno of the write negated (0 on success).
+ */
+static int write_file(const char *path, const char *val)
+{
+ FILE *fp;
+ int err = 0;
+
+ fp = fopen(path, "w");
+ if (!fp)
+ return -errno;
+ if (fwrite(val, 1, strlen(val), fp) != strlen(val))
+ err = ferror(fp) ? (errno ? -errno : -EIO) : -EIO;
+ if (fclose(fp) && !err)
+ err = -errno;
+ return err;
+}
+
+/*
+ * Read the first token of the file at path, for the dev attribute.
+ */
+static int read_file_token(const char *path, char *tok, size_t tok_sz)
+{
+ char fmt[16];
+ FILE *fp;
+
+ fp = fopen(path, "r");
+ if (!fp)
+ return -errno;
+ snprintf(fmt, sizeof(fmt), "%%%zus", tok_sz - 1);
+ if (fscanf(fp, fmt, tok) != 1) {
+ fclose(fp);
+ return -EIO;
+ }
+ fclose(fp);
+ return 0;
+}
+
+/*
+ * Create a null_blk instance through configfs and copy its maj:min
+ * into @dev. The device appears only once power is written, and is
+ * removed by rmdir of the instance directory; the maj:min comes from
+ * the sysfs directory of the created disk, which is named after the
+ * instance.
+ */
+static int nullb_create(const char *name, char *dev, size_t dev_sz)
+{
+ char path[128];
+ char p2[128];
+ int err;
+
+ snprintf(path, sizeof(path), "/sys/kernel/config/nullb/%s", name);
+ if (mkdir(path, 0755)) {
+ if (errno != EEXIST)
+ return -errno;
+ /*
+ * a leftover instance from an earlier crashed run may
+ * still be powered: tear it down before reusing the
+ * name
+ */
+ snprintf(p2, sizeof(p2),
+ "/sys/kernel/config/nullb/%s/power", name);
+ write_file(p2, "0");
+ if (rmdir(path) && errno != ENOENT)
+ return -errno;
+ if (mkdir(path, 0755) && errno != EEXIST)
+ return -errno;
+ }
+
+ snprintf(path, sizeof(path), "/sys/kernel/config/nullb/%s/size", name);
+ err = write_file(path, "128");
+ if (err)
+ goto err_rmdir;
+
+ snprintf(path, sizeof(path), "/sys/kernel/config/nullb/%s/power", name);
+ err = write_file(path, "1");
+ if (err)
+ goto err_rmdir;
+
+ snprintf(path, sizeof(path), "/sys/block/%s/dev", name);
+ err = read_file_token(path, dev, dev_sz);
+ if (err) {
+ /* do not leave a powered instance behind */
+ snprintf(path, sizeof(path),
+ "/sys/kernel/config/nullb/%s/power", name);
+ write_file(path, "0");
+ snprintf(path, sizeof(path),
+ "/sys/kernel/config/nullb/%s", name);
+ rmdir(path);
+ }
+ return err;
+
+ /*
+ * roll the half-created instance back: power it off in case the
+ * failure left it on, then remove the directory
+ */
+err_rmdir:
+ snprintf(p2, sizeof(p2), "/sys/kernel/config/nullb/%s/power", name);
+ write_file(p2, "0");
+ snprintf(path, sizeof(path), "/sys/kernel/config/nullb/%s", name);
+ rmdir(path);
+ return err;
+}
+
+static int nullb_remove(const char *name)
+{
+ char path[128];
+
+ /* a powered instance cannot be removed */
+ snprintf(path, sizeof(path), "/sys/kernel/config/nullb/%s/power", name);
+ write_file(path, "0");
+ snprintf(path, sizeof(path), "/sys/kernel/config/nullb/%s", name);
+ if (rmdir(path))
+ return -errno;
+ return 0;
+}
+
+#define IOCOST_NULLB "iocost_model_test"
+#define IOCOST_NULLB2 "iocost_model_test2"
+
+/*
+ * register_bpf_struct_ops() emits the "bpf_struct_ops_iocost_model_ops"
+ * wrapper type into /sys/kernel/btf/vmlinux only when the struct_ops
+ * registration is compiled in. The bare "iocost_model_ops" struct is
+ * no such signal: blk-iocost.c defines it outside the config guard and
+ * its name lands in the BTF whenever the file is built, even if the
+ * registration itself is compiled out. Probing the wrapper name is
+ * what makes this definitive.
+ *
+ * Returns 1 if supported, 0 if definitively not, -1 on probe errors
+ * (not cached).
+ */
+static int iocost_bpf_supported(void)
+{
+ static const char probe[] = "bpf_struct_ops_iocost_model_ops";
+ static int cached = -1;
+ struct stat st;
+ char *buf = NULL;
+ ssize_t got, total = 0;
+ bool on;
+ int fd;
+
+ if (cached >= 0)
+ return cached;
+
+ fd = open("/sys/kernel/btf/vmlinux", O_RDONLY);
+ if (fd < 0)
+ return -1;
+
+ if (fstat(fd, &st) < 0 || !st.st_size) {
+ close(fd);
+ return -1;
+ }
+
+ buf = malloc(st.st_size);
+ if (!buf) {
+ close(fd);
+ return -1;
+ }
+
+ /* reads may come up short; keep going until EOF */
+ while (total < st.st_size) {
+ got = read(fd, buf + total, st.st_size - total);
+ if (got <= 0)
+ break;
+ total += got;
+ }
+ close(fd);
+
+ /* short reads and allocation failures are transient: don't cache */
+ if (total != st.st_size) {
+ free(buf);
+ return -1;
+ }
+
+ on = memmem(buf, total, probe, sizeof(probe) - 1) != NULL;
+ free(buf);
+ cached = on;
+ return on;
+}
+
+
+/* open the nullb device node, creating it manually when udev hasn't */
+static int open_nullb_dev(const char *name, int flags)
+{
+ char path[32], syspath[64], devtok[32];
+ unsigned int maj, min;
+ dev_t dev;
+ struct stat st;
+ int fd;
+
+ snprintf(syspath, sizeof(syspath), "/sys/block/%s/dev", name);
+ if (read_file_token(syspath, devtok, sizeof(devtok)) ||
+ sscanf(devtok, "%u:%u", &maj, &min) != 2)
+ return -1;
+ dev = makedev(maj, min);
+
+ snprintf(path, sizeof(path), "/dev/%s", name);
+ fd = open(path, flags);
+ if (fd >= 0 && fstat(fd, &st) == 0 && S_ISBLK(st.st_mode) &&
+ st.st_rdev == dev)
+ return fd;
+ if (fd >= 0) {
+ /* a stale node from a previous run, drop and recreate */
+ close(fd);
+ fd = -1;
+ }
+ if (unlink(path) && errno != ENOENT)
+ return -1;
+ if (mknod(path, S_IFBLK | 0600, dev))
+ return -1;
+ fd = open(path, flags);
+ if (fd < 0)
+ unlink(path);
+ return fd;
+}
+
+/*
+ * True if the named controller is available on the cgroup2 hierarchy
+ * mounted at /sys/fs/cgroup, that is, it shows up in the root
+ * cgroup.controllers list. Tokens are compared whole so that "io"
+ * does not match inside another controller's name.
+ */
+static bool cgroupfs_has_controller(const char *name)
+{
+ char line[512], *tok;
+ FILE *fp;
+ bool found = false;
+
+ fp = fopen("/sys/fs/cgroup/cgroup.controllers", "r");
+ if (!fp)
+ return false;
+ while (!found && fgets(line, sizeof(line), fp)) {
+ for (tok = strtok(line, " \t\n"); tok;
+ tok = strtok(NULL, " \t\n")) {
+ if (!strcmp(tok, name)) {
+ found = true;
+ break;
+ }
+ }
+ }
+ fclose(fp);
+ return found;
+}
+
+/*
+ * The tests run against their own null_blk instance created through
+ * configfs, so they need root, configfs and the null_blk driver. The
+ * BPF cost model further needs cgroup v2 at /sys/fs/cgroup and
+ * CONFIG_BLK_CGROUP_IOCOST_BPF in the running kernel; both are
+ * environment properties, so their absence is a skip.
+ */
+static bool iocost_test_env(void)
+{
+ static bool ready;
+ struct statfs sf;
+ int orig_ns = -1, supported;
+ bool ok = false;
+
+ if (geteuid() != 0) {
+ printf("%s:SKIP:needs root\n", __func__);
+ test__skip();
+ return false;
+ }
+
+ if (statfs("/sys/fs/cgroup", &sf) ||
+ sf.f_type != CGROUP2_SUPER_MAGIC) {
+ printf("%s:SKIP:needs cgroup v2 at /sys/fs/cgroup\n",
+ __func__);
+ test__skip();
+ return false;
+ }
+
+ /*
+ * The io controller can be missing from cgroup.controllers even
+ * with cgroup2 mounted: cgroup_disable=io leaves it out, and so
+ * does a cgroup v1 hierarchy claiming it. The io.* files the
+ * tests write do not exist then, so skip with a clear reason
+ * instead of failing on the first io.weight write.
+ */
+ if (!cgroupfs_has_controller("io")) {
+ printf("%s:SKIP:needs the io controller in cgroup.controllers\n",
+ __func__);
+ test__skip();
+ return false;
+ }
+
+ supported = iocost_bpf_supported();
+ if (supported < 0) {
+ printf("%s:SKIP:cannot probe /sys/kernel/btf/vmlinux: %s\n",
+ __func__, strerror(errno));
+ test__skip();
+ return false;
+ }
+ if (!supported) {
+ printf("%s:SKIP:needs CONFIG_BLK_CGROUP_IOCOST_BPF\n",
+ __func__);
+ test__skip();
+ return false;
+ }
+
+ /* the tracepoint arrives with the iocost BPF support (p2) */
+ if (access("/sys/kernel/tracing/events/iocost/iocost_ioc_tick", F_OK) &&
+ access("/sys/kernel/debug/tracing/events/iocost/iocost_ioc_tick",
+ F_OK)) {
+ printf("%s:SKIP:needs the iocost_ioc_tick tracepoint\n",
+ __func__);
+ test__skip();
+ return false;
+ }
+
+ /*
+ * The private mount namespace and the configfs mount are
+ * process-wide: set them up once so serial tests do not stack
+ * namespaces and pile configfs mounts on top of each other.
+ */
+ if (ready)
+ return true;
+
+ /*
+ * Isolate the configfs and cgroup mounts from the host. A
+ * failed setup unwinds the namespace so that a later retry does
+ * not stack private namespaces.
+ */
+ orig_ns = open("/proc/self/ns/mnt", O_RDONLY);
+ if (unshare(CLONE_NEWNS) ||
+ mount(NULL, "/", NULL, MS_REC | MS_PRIVATE, NULL)) {
+ printf("%s:SKIP:cannot make mounts private: %s\n",
+ __func__, strerror(errno));
+ test__skip();
+ goto out;
+ }
+ mkdir("/sys/kernel/config", 0755);
+ (void)mount("none", "/sys/kernel/config", "configfs", 0, NULL);
+ SYS_NOFAIL("modprobe null_blk nr_devices=0");
+ if (access("/sys/kernel/config/nullb", W_OK)) {
+ printf("%s:SKIP:needs null_blk with configfs support\n",
+ __func__);
+ test__skip();
+ goto out;
+ }
+
+ ready = true;
+ ok = true;
+
+out:
+ if (orig_ns >= 0) {
+ if (!ok)
+ setns(orig_ns, CLONE_NEWNS);
+ close(orig_ns);
+ }
+ return ok;
+}
+
+/*
+ * Read back the io.cost.model line of dev and copy the model= value into
+ * @model. Returns 0 on success.
+ */
+static int readback_model(const char *dev, char *model, size_t model_sz)
+{
+ char line[256], word[256], *m, *end;
+ FILE *fp;
+ int found = 0;
+
+ fp = fopen("/sys/fs/cgroup/io.cost.model", "r");
+ if (!fp)
+ return -1;
+ while (fgets(line, sizeof(line), fp)) {
+ if (sscanf(line, "%255s", word) == 1 && !strcmp(word, dev)) {
+ found = 1;
+ break;
+ }
+ }
+ fclose(fp);
+ if (!found)
+ return -1;
+
+ m = strstr(line, "model=");
+ if (!m)
+ return -1;
+ m += strlen("model=");
+ end = m;
+ while (*end && *end != ' ')
+ end++;
+ snprintf(model, model_sz, "%.*s", (int)(end - m), m);
+ return 0;
+}
+
+/*
+ * Read the cost.usage accumulator of dev from the cgroup's io.stat. Returns 0
+ * on success, -ENODEV when the device has no line yet (no charge recorded).
+ */
+static int read_cost_usage(const char *cgroup, const char *dev,
+ unsigned long long *usage)
+{
+ char buf[4096], *save = NULL, *linep, *tok, *word;
+ int found = 0, ret;
+
+ ret = read_cgroup_file(cgroup, "io.stat", buf, sizeof(buf));
+ if (ret)
+ return ret;
+ for (linep = strtok_r(buf, "\n", &save); linep;
+ linep = strtok_r(NULL, "\n", &save)) {
+ char *wsave = NULL;
+
+ tok = strtok_r(linep, " \t", &wsave);
+ if (!tok || strcmp(tok, dev))
+ continue;
+ found = 1;
+ while ((word = strtok_r(NULL, " \t", &wsave))) {
+ if (!strncmp(word, "cost.usage=", 11)) {
+ *usage = strtoull(word + 11, NULL, 0);
+ return 0;
+ }
+ }
+ }
+ return found ? -ENOENT : -ENODEV;
+}
+
+/*
+ * Write a line to io.cost.model for dev and return the errno of the write
+ * negated (0 on success), as ASSERT_ERR() expects.
+ */
+static int write_cost_model(const char *dev, const char *what)
+{
+ char buf[128];
+ FILE *fp;
+ int err = 0;
+
+ snprintf(buf, sizeof(buf), "%s %s", dev, what);
+ fp = fopen("/sys/fs/cgroup/io.cost.model", "w");
+ if (!fp)
+ return errno ? -errno : -EACCES;
+ if (fwrite(buf, 1, strlen(buf), fp) != strlen(buf))
+ err = ferror(fp) ? (errno ? -errno : -EIO) : -EIO;
+ if (fclose(fp) && !err)
+ err = -errno;
+ return err;
+}
+
+/*
+ * Attach the example model to the null_blk instance the test creates: the dev
+ * member is written through the struct_ops map's initial value before load,
+ * and attaching the struct_ops attaches the model to the device. Detaching the
+ * struct_ops restores the builtin model.
+ *
+ * Requires root, cgroup v2 and null_blk with configfs support.
+ */
+void serial_test_iocost_model(void)
+{
+ struct iocost_model *skel, *second, *bad;
+ unsigned int maj, min;
+ char buf[4096], qbuf[160], dev2[32];
+ int devfd, i, err, cg;
+ char model[32], dev[32];
+ unsigned long long u1 = 0, u2 = 0;
+
+ if (!iocost_test_env())
+ return;
+ if (!ASSERT_OK(nullb_create(IOCOST_NULLB, dev, sizeof(dev)),
+ "nullb_create"))
+ return;
+ if (sscanf(dev, "%u:%u", &maj, &min) != 2) {
+ ASSERT_TRUE(false, "dev_parse");
+ nullb_remove(IOCOST_NULLB);
+ return;
+ }
+
+ skel = iocost_model__open();
+ if (!ASSERT_OK_PTR(skel, "skel_open")) {
+ nullb_remove(IOCOST_NULLB);
+ return;
+ }
+
+ /*
+ * a model with a zero coefficient must fail to attach instead of
+ * silently zeroing the completion sizing (validate rejects it on
+ * the map update the attach performs; the link form of the
+ * struct_ops only reaches it there)
+ */
+ bad = iocost_model__open();
+ if (!ASSERT_OK_PTR(bad, "bad_open")) {
+ iocost_model__destroy(skel);
+ nullb_remove(IOCOST_NULLB);
+ return;
+ }
+ bad->struct_ops.iocost_2x->dev = (__u32)makedev(maj, min);
+ bad->struct_ops.iocost_2x->write_vtime_per_page = 0;
+ if (ASSERT_OK(iocost_model__load(bad), "bad_load"))
+ ASSERT_EQ(iocost_model__attach(bad), -EINVAL,
+ "attach_rejects_zero_coeff");
+ iocost_model__destroy(bad);
+
+ /* same for the read-side transfer coefficient */
+ bad = iocost_model__open();
+ if (!ASSERT_OK_PTR(bad, "bad_read_open")) {
+ iocost_model__destroy(skel);
+ nullb_remove(IOCOST_NULLB);
+ return;
+ }
+ bad->struct_ops.iocost_2x->dev = (__u32)makedev(maj, min);
+ bad->struct_ops.iocost_2x->read_vtime_per_page = 0;
+ if (ASSERT_OK(iocost_model__load(bad), "bad_read_load"))
+ ASSERT_EQ(iocost_model__attach(bad), -EINVAL,
+ "attach_rejects_zero_read");
+ iocost_model__destroy(bad);
+
+ skel->struct_ops.iocost_2x->dev = (__u32)makedev(maj, min);
+ /*
+ * cgroup_setup_and_join() creates the priced-IO cgroup with the
+ * available controllers enabled, so the cgroup has
+ * io.weight/io.stat and the iocg activates.
+ * cleanup_cgroup_environment() removes it from every exit path
+ * below. io.cost.qos and io.cost.model are root-only cgroupfs
+ * files: they are written at the hierarchy root and reverted on
+ * exit.
+ */
+ cg = cgroup_setup_and_join("/iocost_pairing");
+ if (!ASSERT_GE(cg, 0, "setup_cg")) {
+ iocost_model__destroy(skel);
+ nullb_remove(IOCOST_NULLB);
+ return;
+ }
+ close(cg);
+ devfd = open_nullb_dev(IOCOST_NULLB, O_RDONLY);
+ if (!ASSERT_OK(devfd < 0 ? -errno : 0, "open_dev")) {
+ cleanup_cgroup_environment();
+ iocost_model__destroy(skel);
+ nullb_remove(IOCOST_NULLB);
+ return;
+ }
+ /* drop stale page cache so that the warm reads issue real bios */
+ (void)posix_fadvise(devfd, 0, 0, POSIX_FADV_DONTNEED);
+ for (i = 0; i < 32; i++)
+ (void)read(devfd, buf, sizeof(buf));
+ close(devfd);
+
+ err = iocost_model__load(skel);
+ if (!ASSERT_OK(err, "skel_load")) {
+ iocost_model__destroy(skel);
+ cleanup_cgroup_environment();
+ nullb_remove(IOCOST_NULLB);
+ return;
+ }
+
+ err = iocost_model__attach(skel);
+ if (ASSERT_OK(err, "attach")) {
+ /*
+ * attached: the read path reports model=bpf until the
+ * struct_ops is detached; ctrl keeps describing the
+ * builtin coefficients
+ */
+ err = readback_model(dev, model, sizeof(model));
+ if (ASSERT_OK(err, "readback"))
+ ASSERT_STREQ(model, "bpf", "model_bpf");
+
+ /* a second model on the same device fails with -EBUSY */
+ second = iocost_model__open();
+ if (ASSERT_OK_PTR(second, "second_open")) {
+ second->struct_ops.iocost_2x->dev =
+ (__u32)makedev(maj, min);
+ err = iocost_model__load(second);
+ if (ASSERT_OK(err, "second_load")) {
+ struct bpf_link *l2;
+
+ /*
+ * the kernel rejects attaching a second
+ * model to the device with EBUSY
+ */
+ l2 = bpf_map__attach_struct_ops(
+ second->maps.iocost_2x);
+ if (!ASSERT_ERR_PTR(l2, "second_ebusy"))
+ bpf_link__destroy(l2);
+ else
+ ASSERT_EQ(libbpf_get_error(l2), -EBUSY,
+ "second_ebusy_errno");
+ }
+ iocost_model__destroy(second);
+ }
+
+ /*
+ * pricing coverage: root cgroup IOs bypass the model, so
+ * move into the pairing cgroup, enable the controller
+ * and verify IOs actually go through calc_cost() and get
+ * charged
+ */
+ ASSERT_OK(join_cgroup("/iocost_pairing"), "join_pairing");
+ snprintf(qbuf, sizeof(qbuf), "%s 1000", dev);
+ ASSERT_OK(write_cgroup_file("/iocost_pairing", "io.weight",
+ qbuf), "weight_pairing");
+ snprintf(qbuf, sizeof(qbuf),
+ "%s enable=1 ctrl=user min=100.00 max=100.00", dev);
+ ASSERT_OK(write_file("/sys/fs/cgroup/io.cost.qos", qbuf),
+ "qos_enable");
+
+ devfd = open_nullb_dev(IOCOST_NULLB, O_RDONLY);
+ if (!ASSERT_OK(devfd < 0 ? -errno : 0, "open_dev_priced")) {
+ iocost_model__destroy(skel);
+ snprintf(qbuf, sizeof(qbuf), "%s enable=0", dev);
+ (void)write_file("/sys/fs/cgroup/io.cost.qos", qbuf);
+ nullb_remove(IOCOST_NULLB);
+ ASSERT_OK(join_root_cgroup(), "join_root_fail");
+ cleanup_cgroup_environment();
+ return;
+ }
+ ASSERT_OK(read_cost_usage("/iocost_pairing",
+ dev, &u1), "cost_usage_pre");
+ /* drop stale page cache so that the reads issue real bios */
+ (void)posix_fadvise(devfd, 0, 0, POSIX_FADV_DONTNEED);
+ for (i = 0; i < 500; i++)
+ (void)read(devfd, buf, sizeof(buf));
+ close(devfd);
+ ASSERT_GT(skel->bss->calc_costs, 0, "model_priced");
+ /*
+ * the usage stat is flushed by the iocost timer, give
+ * the current period a chance to end before reading
+ */
+ sleep(3);
+ ASSERT_OK(read_cost_usage("/iocost_pairing",
+ dev, &u2), "cost_usage_post");
+ ASSERT_GT(u2, u1, "cost_charged");
+ ASSERT_GT(skel->bss->ioc_ticks, 0, "tick_seen");
+
+ /*
+ * a second device is independent: enabling it shows
+ * the builtin model, not the one attached to the
+ * first
+ */
+ if (ASSERT_OK(nullb_create(IOCOST_NULLB2, dev2,
+ sizeof(dev2)), "nullb_create2")) {
+ snprintf(qbuf, sizeof(qbuf),
+ "%s enable=1 ctrl=auto", dev2);
+ ASSERT_OK(write_file(
+ "/sys/fs/cgroup/io.cost.qos", qbuf),
+ "qos_nullb2");
+ err = readback_model(dev2, model, sizeof(model));
+ if (ASSERT_OK(err, "readback_nullb2"))
+ ASSERT_STREQ(model, "linear",
+ "model_linear_nullb2");
+ snprintf(qbuf, sizeof(qbuf), "%s enable=0", dev2);
+ ASSERT_OK(write_file(
+ "/sys/fs/cgroup/io.cost.qos", qbuf),
+ "qos_nullb2_off");
+ }
+ nullb_remove(IOCOST_NULLB2);
+
+ /*
+ * attachment and model selection are independent:
+ * ctrl=bpf is never accepted, model=linear switches
+ * back to the builtin without detaching, model=bpf
+ * switches to the attached model again
+ */
+ err = write_cost_model(dev, "ctrl=bpf");
+ ASSERT_ERR(err, "ctrl_bpf_rejected");
+
+ err = write_cost_model(dev, "model=linear");
+ ASSERT_OK(err, "model_linear_write");
+ err = readback_model(dev, model, sizeof(model));
+ if (ASSERT_OK(err, "readback_linear"))
+ ASSERT_STREQ(model, "linear", "model_linear");
+
+ /* still attached: the switch must not free the slot */
+ second = iocost_model__open();
+ if (ASSERT_OK_PTR(second, "second_open2")) {
+ second->struct_ops.iocost_2x->dev =
+ (__u32)makedev(maj, min);
+ err = iocost_model__load(second);
+ if (ASSERT_OK(err, "second_load2")) {
+ struct bpf_link *l2;
+
+ l2 = bpf_map__attach_struct_ops(
+ second->maps.iocost_2x);
+ if (!ASSERT_ERR_PTR(l2, "second_ebusy2"))
+ bpf_link__destroy(l2);
+ else
+ ASSERT_EQ(libbpf_get_error(l2), -EBUSY,
+ "second_ebusy2_errno");
+ }
+ iocost_model__destroy(second);
+ }
+
+ /* still attached: switching back to the model works */
+ err = write_cost_model(dev, "model=bpf");
+ ASSERT_OK(err, "model_bpf_write");
+ err = readback_model(dev, model, sizeof(model));
+ if (ASSERT_OK(err, "readback_bpf"))
+ ASSERT_STREQ(model, "bpf", "model_bpf");
+
+ bpf_link__destroy(skel->links.iocost_2x);
+ skel->links.iocost_2x = NULL;
+
+ /* after detach, model=bpf fails: nothing is attached */
+ err = write_cost_model(dev, "model=bpf");
+ ASSERT_ERR(err, "model_bpf_after_detach");
+
+ err = readback_model(dev, model, sizeof(model));
+ if (ASSERT_OK(err, "readback_after_detach"))
+ ASSERT_STREQ(model, "linear", "model_linear");
+
+ }
+
+ iocost_model__destroy(skel);
+ snprintf(qbuf, sizeof(qbuf), "%s enable=0", dev);
+ (void)write_file("/sys/fs/cgroup/io.cost.qos", qbuf);
+ nullb_remove(IOCOST_NULLB);
+ ASSERT_OK(join_root_cgroup(), "procs_back");
+ cleanup_cgroup_environment();
+}
+/*
+ * Same check for the multi-stream example model. Only one model can be
+ * attached to a device at a time; each test creates its own device and
+ * attaches and detaches, so they are serial.
+ */
+void serial_test_iocost_model_streams(void)
+{
+ struct iocost_ms *skel;
+ unsigned int maj, min;
+ int devfd, i, err, cg;
+ char model[32], dev[32], qbuf[160];
+
+ if (!iocost_test_env())
+ return;
+ if (!ASSERT_OK(nullb_create(IOCOST_NULLB, dev, sizeof(dev)),
+ "nullb_create"))
+ return;
+ if (sscanf(dev, "%u:%u", &maj, &min) != 2) {
+ ASSERT_TRUE(false, "dev_parse");
+ nullb_remove(IOCOST_NULLB);
+ return;
+ }
+
+ skel = iocost_ms__open();
+ if (!ASSERT_OK_PTR(skel, "skel_open")) {
+ nullb_remove(IOCOST_NULLB);
+ return;
+ }
+
+ skel->struct_ops.iocost_ms->dev = (__u32)makedev(maj, min);
+
+ err = iocost_ms__load(skel);
+ if (!ASSERT_OK(err, "skel_load")) {
+ iocost_ms__destroy(skel);
+ nullb_remove(IOCOST_NULLB);
+ return;
+ }
+
+ err = iocost_ms__attach(skel);
+ if (ASSERT_OK(err, "attach")) {
+ err = readback_model(dev, model, sizeof(model));
+ if (ASSERT_OK(err, "readback"))
+ ASSERT_STREQ(model, "bpf", "model_bpf");
+
+ /*
+ * drive IO through the model: enable the controller, price
+ * from a cgroup, and check the model actually charged them
+ */
+ cg = cgroup_setup_and_join("/iocost_ms");
+ if (!ASSERT_GE(cg, 0, "setup_cg_ms")) {
+ iocost_ms__destroy(skel);
+ nullb_remove(IOCOST_NULLB);
+ cleanup_cgroup_environment();
+ return;
+ }
+ close(cg);
+ snprintf(qbuf, sizeof(qbuf), "%s 1000", dev);
+ ASSERT_OK(write_cgroup_file("/iocost_ms", "io.weight", qbuf),
+ "weight_ms");
+ snprintf(qbuf, sizeof(qbuf),
+ "%s enable=1 ctrl=user min=100.00 max=100.00", dev);
+ ASSERT_OK(write_file("/sys/fs/cgroup/io.cost.qos", qbuf),
+ "qos_enable");
+ devfd = open_nullb_dev(IOCOST_NULLB, O_RDONLY | O_DIRECT);
+ if (ASSERT_OK(devfd < 0 ? -errno : 0, "open_dev_ms")) {
+ void *iobuf = NULL;
+ unsigned long long u1 = 0, u2 = 0;
+
+ /*
+ * O_DIRECT so every read issues real bios: cached
+ * reads would never reach iocost_ms_calc_cost().
+ */
+ if (!ASSERT_OK(posix_memalign(&iobuf, 4096, 4096),
+ "memalign_iobuf")) {
+ printf("%s:SKIP:cannot allocate the aligned IO buffer\n",
+ __func__);
+ test__skip();
+ } else {
+ ASSERT_OK(read_cost_usage("/iocost_ms",
+ dev, &u1), "cost_usage_pre");
+ for (i = 0; i < 500; i++)
+ (void)read(devfd, iobuf, 4096);
+ /*
+ * Two interleaved sequential streams: exercise
+ * the stream table, not just a single cursor.
+ */
+ for (i = 0; i < 200; i++) {
+ (void)pread(devfd, iobuf, 4096,
+ (i + 1) * 4096);
+ (void)pread(devfd, iobuf, 4096,
+ (1 << 20) +
+ (i + 1) * 4096);
+ }
+ free(iobuf);
+ sleep(3);
+ ASSERT_OK(read_cost_usage("/iocost_ms",
+ dev, &u2), "cost_usage_post");
+ ASSERT_GT(u2, u1, "ms_cost_charged");
+ ASSERT_GT(skel->bss->calc_costs, 0,
+ "ms_model_priced");
+ }
+ close(devfd);
+ }
+ ASSERT_OK(join_root_cgroup(), "procs_back");
+ snprintf(qbuf, sizeof(qbuf), "%s enable=0", dev);
+ (void)write_file("/sys/fs/cgroup/io.cost.qos", qbuf);
+ bpf_link__destroy(skel->links.iocost_ms);
+ skel->links.iocost_ms = NULL;
+
+ err = readback_model(dev, model, sizeof(model));
+ if (ASSERT_OK(err, "readback_after_detach"))
+ ASSERT_STREQ(model, "linear", "model_linear");
+
+ cleanup_cgroup_environment();
+ }
+
+ iocost_ms__destroy(skel);
+ nullb_remove(IOCOST_NULLB);
+}
+
+/*
+ * Removal of the device while a model is attached: the test builds
+ * its own null_blk instance through configfs like the other tests,
+ * attaches to it, then removes the device out from under the model.
+ * The ejection must drop the device from the cgroup interface,
+ * down, and a recreated device must be attachable again.
+ */
+void serial_test_iocost_model_removal(void)
+{
+ struct iocost_model *skel, *skel2;
+ unsigned int maj, min;
+ char dev[32], model[32];
+ int err;
+
+ if (!iocost_test_env())
+ return;
+
+ skel = iocost_model__open();
+ if (!ASSERT_OK_PTR(skel, "skel_open"))
+ return;
+
+ err = nullb_create("iocost_removal", dev, sizeof(dev));
+ if (!ASSERT_OK(err, "nullb_create"))
+ goto out;
+ if (sscanf(dev, "%u:%u", &maj, &min) != 2) {
+ ASSERT_TRUE(false, "dev_parse");
+ goto out;
+ }
+
+ skel->struct_ops.iocost_2x->dev = (__u32)makedev(maj, min);
+ err = iocost_model__load(skel);
+ if (!ASSERT_OK(err, "skel_load"))
+ goto out;
+
+ err = iocost_model__attach(skel);
+ if (!ASSERT_OK(err, "attach"))
+ goto out;
+
+ err = readback_model(dev, model, sizeof(model));
+ if (ASSERT_OK(err, "readback"))
+ ASSERT_STREQ(model, "bpf", "model_bpf");
+
+ /* remove the device out from under the attached model */
+ if (!ASSERT_OK(nullb_remove("iocost_removal"), "nullb_remove"))
+ goto detach;
+
+ /* the ejection drops the device from the cgroup files */
+ err = readback_model(dev, model, sizeof(model));
+ ASSERT_ERR(err, "readback_after_removal");
+
+ err = write_cost_model(dev, "model=bpf");
+ ASSERT_ERR(err, "model_bpf_after_removal");
+
+detach:
+ /*
+ * detaching after the device is gone must be a clean no-op;
+ * this exercises the eject-then-detach ordering in the kernel
+ */
+ bpf_link__destroy(skel->links.iocost_2x);
+ skel->links.iocost_2x = NULL;
+
+ /* the same attach and detach against a recreated device */
+ skel2 = iocost_model__open();
+ if (!ASSERT_OK_PTR(skel2, "skel2_open"))
+ goto out;
+
+ err = nullb_create("iocost_removal", dev, sizeof(dev));
+ if (!ASSERT_OK(err, "nullb_create_again"))
+ goto out2;
+ if (sscanf(dev, "%u:%u", &maj, &min) != 2) {
+ ASSERT_TRUE(false, "dev_parse_again");
+ goto out2;
+ }
+
+ skel2->struct_ops.iocost_2x->dev = (__u32)makedev(maj, min);
+ err = iocost_model__load(skel2);
+ if (!ASSERT_OK(err, "skel2_load"))
+ goto out2;
+
+ err = iocost_model__attach(skel2);
+ if (!ASSERT_OK(err, "attach_again"))
+ goto out2;
+
+ err = readback_model(dev, model, sizeof(model));
+ if (ASSERT_OK(err, "readback_again"))
+ ASSERT_STREQ(model, "bpf", "model_bpf_again");
+
+ bpf_link__destroy(skel2->links.iocost_2x);
+ skel2->links.iocost_2x = NULL;
+
+ err = readback_model(dev, model, sizeof(model));
+ if (ASSERT_OK(err, "readback_after_detach_again"))
+ ASSERT_STREQ(model, "linear", "model_linear_again");
+
+out2:
+ iocost_model__destroy(skel2);
+out:
+ nullb_remove("iocost_removal");
+ iocost_model__destroy(skel);
+}
diff --git a/tools/testing/selftests/bpf/progs/iocost_model.c b/tools/testing/selftests/bpf/progs/iocost_model.c
new file mode 100644
index 000000000000..3f071d871f2b
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/iocost_model.c
@@ -0,0 +1,188 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2026 KylinSoft Corporation. */
+/*
+ * Example iocost cost model: the builtin linear HDD formula with all costs
+ * doubled, for one device given by the dev member of the struct_ops.
+ *
+ * The constants mirror what calc_lcoefs() derives from the AUTOP_HDD defaults
+ * (rbps=174019176 rseqiops=41708 rrandiops=370, w-side analog) in vtime units
+ * where 1s == 2^37. On a rotational device still on ctrl=auto, a device with
+ * this model attached charges twice the builtin model under the same
+ * read/write workload (the builtin prices flushes as zero and adds a fixed
+ * flush term to data-bearing preflushes and to FUA writes on devices without
+ * native FUA support, which this model omits; this model prices flushes as
+ * one-page writes), so the doubled cost is a direct check
+ * that accounting goes through the BPF path. On a non-rotational device, or
+ * one with user-pinned coefficients, the ratio to the builtin model is
+ * arbitrary.
+ *
+ * The cursor handling follows the builtin: a zero cursor means "no previous
+ * IO", and the cursor is advanced for the bios the builtin prices
+ * (READ/WRITE/ZONE_APPEND with a non-zero size), so flushes and discards leave
+ * it alone and long merged streams do not drift past the 16MB seek threshold.
+ * One known deviation, mispricing only: the builtin does not move its cursor
+ * for an IO whose cgroup fails to activate; this model does not see the
+ * activation result.
+ *
+ * The model implements the full linear formula itself, including flushes:
+ * there is no fallback to the builtin model, which prices a dataless
+ * WRITE|REQ_PREFLUSH as zero unless flushiops is configured; this model prices
+ * it as a one-page sequential write. What a flush actually costs depends on
+ * the IOs before it, which a static formula cannot express; pricing it as a
+ * small constant is a crude penalty for unnecessarily frequent flushes, and a
+ * stateful model would be the way to do better. Discards and other unpriced
+ * operations pass through at zero cost, like the builtin.
+ */
+
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+
+/*
+ * VTIME_PER_SEC, IOC_PAGE_SIZE/SHIFT, IOC_SECT_TO_PAGE_SHIFT,
+ * LCOEF_RANDIO_PAGES (the 16MB seek threshold) and IOCOST_CALC_MERGE
+ * come from vmlinux.h (BTF constants; the enum64 ones need pahole
+ * >= 1.23 to be emitted)
+ */
+#define IOCOST_REQ_OP_MASK 0xff /* REQ_OP_MASK, not in BTF */
+
+/*
+ * DIV64_U64_ROUND_UP / DIV_ROUND_UP_ULL equivalents, folded at
+ * compile time
+ */
+#define RU(x, y) ((x) / (y) + (((x) % (y)) ? 1 : 0))
+
+#define RBPS 174019176ULL
+#define RSEQIOPS 41708ULL
+#define RRANDIOPS 370ULL
+#define WBPS 178075866ULL
+#define WSEQIOPS 42705ULL
+#define WRANDIOPS 378ULL
+
+#define RPAGE (RU(VTIME_PER_SEC, RU(RBPS, IOC_PAGE_SIZE)))
+#define RSEQIO (RU(VTIME_PER_SEC, RSEQIOPS) - RPAGE)
+#define RRANDIO (RU(VTIME_PER_SEC, RRANDIOPS) - RPAGE)
+#define WPAGE (RU(VTIME_PER_SEC, RU(WBPS, IOC_PAGE_SIZE)))
+#define WSEQIO (RU(VTIME_PER_SEC, WSEQIOPS) - WPAGE)
+#define WRANDIO (RU(VTIME_PER_SEC, WRANDIOPS) - WPAGE)
+
+/*
+ * per-cgroup cursor storage: keyed by the cgroup and freed with it,
+ * so a cgroup that comes back starts with no cursor. A struct_ops
+ * map attaches to at most one device (a second attach to the same
+ * device fails with -EBUSY), and this example uses one map, so
+ * per-cgroup keys are per (cgroup, device) in effect
+ */
+struct {
+ __uint(type, BPF_MAP_TYPE_CGRP_STORAGE);
+ __uint(map_flags, BPF_F_NO_PREALLOC);
+ __type(key, int);
+ __type(value, __u64);
+} cursor_store SEC(".maps");
+
+/*
+ * incremented by calc_cost(): the test asserts pricing went
+ * through the model while it is attached and the controller is
+ * enabled
+ */
+__u64 calc_costs;
+
+/* counts iocost_ioc_tick emissions observed from BPF side */
+__u64 ioc_ticks;
+
+SEC("struct_ops")
+u64 BPF_PROG(iocost_2x_calc_cost, struct bio *bio, u64 calc_flags)
+{
+ u64 opf = bio->bi_opf, nbytes = bio->bi_iter.bi_size;
+ u64 sector = bio->bi_iter.bi_sector;
+ struct blkcg *blkcg = bio->bi_blkg->blkcg;
+ u64 pages, seek_pages = 0, base, coef_page, randio, cost;
+ __u64 *cursor, cur;
+
+ calc_costs++;
+
+ /* builtin truncates: max(sectors >> IOC_SECT_TO_PAGE_SHIFT, 1) */
+ pages = nbytes >> IOC_PAGE_SHIFT;
+ if (!pages)
+ pages = 1;
+
+ if ((opf & IOCOST_REQ_OP_MASK) == REQ_OP_READ) {
+ base = RSEQIO; coef_page = RPAGE; randio = RRANDIO;
+ } else if ((opf & IOCOST_REQ_OP_MASK) == REQ_OP_WRITE ||
+ (opf & IOCOST_REQ_OP_MASK) == REQ_OP_ZONE_APPEND) {
+ base = WSEQIO; coef_page = WPAGE; randio = WRANDIO;
+ } else {
+ /*
+ * the builtin prices other operations at zero and
+ * leaves them unthrottled; pass them through the same
+ * way instead of inventing costs
+ */
+ return 0;
+ }
+
+ /* mirror the builtin cursor semantics described above */
+ cursor = bpf_cgrp_storage_get(&cursor_store,
+ blkcg->css.cgroup, NULL,
+ BPF_LOCAL_STORAGE_GET_F_CREATE);
+ if (!cursor) {
+ /*
+ * the cursor table is unavailable (storage lookup
+ * failure): fall back to unconditional sequential
+ * pricing without the randio determination; a known
+ * deviation, mispricing only and unreachable short of
+ * a storage allocation failure
+ */
+ if (calc_flags & IOCOST_CALC_MERGE)
+ base = 0;
+ return 2 * (base + pages * coef_page);
+ }
+ cur = *cursor;
+ /*
+ * a size-zero IO never moves the cursor, so it must not be
+ * judged against it either; the builtin skips it entirely,
+ * this model prices it as a one-page sequential write
+ */
+ if (cur && nbytes) {
+ seek_pages = sector > cur ? sector - cur
+ : cur - sector;
+ seek_pages >>= IOC_SECT_TO_PAGE_SHIFT;
+ if (seek_pages > LCOEF_RANDIO_PAGES)
+ base = randio;
+ }
+ /*
+ * the builtin moves the cursor unconditionally on submit, but
+ * on a merge only when the merged request ends at the cursor,
+ * a test on the request extent that both merge directions can
+ * satisfy; this model advances only when the merged bio itself
+ * starts right at the cursor, so the two can disagree on a
+ * front merge, a known deviation, mispricing only. a zero
+ * cursor means no history so it never matches
+ */
+ if (nbytes &&
+ (!(calc_flags & IOCOST_CALC_MERGE) || (cur && cur == sector)))
+ *cursor = sector + (nbytes >> 9);
+
+ if (calc_flags & IOCOST_CALC_MERGE)
+ base = 0;
+
+ cost = 2 * (base + pages * coef_page);
+ return cost;
+}
+
+/* consumes the new iocost_ioc_tick tracepoint from BPF side */
+SEC("tp_btf/iocost_ioc_tick")
+__u64 BPF_PROG(iocost_2x_tick_count, struct ioc *ioc,
+ const struct ioc_now *now, int nr_active, u64 usage_us_sum)
+{
+ ioc_ticks++;
+ return 0;
+}
+
+SEC(".struct_ops.link") /* detach on close */
+struct iocost_model_ops iocost_2x = {
+ .read_vtime_per_page = 2 * RPAGE,
+ .write_vtime_per_page = 2 * WPAGE,
+ .calc_cost = (void *)iocost_2x_calc_cost,
+};
+
+char LICENSE[] SEC("license") = "GPL";
diff --git a/tools/testing/selftests/bpf/progs/iocost_ms.c b/tools/testing/selftests/bpf/progs/iocost_ms.c
new file mode 100644
index 000000000000..e9e6b787d296
--- /dev/null
+++ b/tools/testing/selftests/bpf/progs/iocost_ms.c
@@ -0,0 +1,195 @@
+// SPDX-License-Identifier: GPL-2.0
+/* Copyright (C) 2026 KylinSoft Corporation. */
+/*
+ * Example multi-stream sequentiality detection cost model.
+ *
+ * The builtin model keeps a single cursor per cgroup, so two interleaved
+ * sequential readers in one cgroup are all priced random (measured 89x
+ * overcharge, 12.9x throughput collapse), while random IO inside a hot window
+ * smaller than the 16MB seek threshold is priced sequential (measured 107x
+ * undercharge). This model replaces the single cursor with a per-cgroup table
+ * of stream slots: an IO is sequential iff its sector matches the expected
+ * next sector of any tracked stream. Interleaved streams keep their own slots,
+ * and windowed random IO rarely matches a moving expectation.
+ *
+ * Stream state lives in a CGRP_STORAGE map, so it is created and freed with
+ * the cgroup. The model implements the builtin linear formula itself; unlike
+ * the builtin it has no flush term, so preflushed IOs are priced as plain
+ * writes (mispricing only, the test never issues flushes). A
+ * struct_ops map attaches to at most one device (a second attach
+ * to the same device fails with -EBUSY), and this example uses one
+ * map, so the per-cgroup keys are per (cgroup, device) in effect.
+ * Two more known deviations:
+ * a flush-flagged size-zero WRITE is priced as a one-page
+ * sequential write
+ * (the builtin prices it zero at flushiops=0), and discards do not
+ * advance the stream expectation; the builtin cursor does not move
+ * for them either.
+ */
+#include "vmlinux.h"
+#include <bpf/bpf_helpers.h>
+#include <bpf/bpf_tracing.h>
+
+/*
+ * VTIME_PER_SEC, IOC_PAGE_SIZE/SHIFT, IOC_SECT_TO_PAGE_SHIFT and
+ * IOCOST_CALC_MERGE comes from vmlinux.h (BTF enum constants)
+ */
+#define IOCOST_REQ_OP_MASK 0xff /* REQ_OP_MASK, not in BTF */
+
+/*
+ * DIV64_U64_ROUND_UP / DIV_ROUND_UP_ULL equivalents, folded at
+ * compile time
+ */
+#define RU(x, y) ((x) / (y) + (((x) % (y)) ? 1 : 0))
+
+#define RBPS 174019176ULL
+#define RSEQIOPS 41708ULL
+#define RRANDIOPS 370ULL
+#define WBPS 178075866ULL
+#define WSEQIOPS 42705ULL
+#define WRANDIOPS 378ULL
+
+#define RPAGE (RU(VTIME_PER_SEC, RU(RBPS, IOC_PAGE_SIZE)))
+#define RSEQIO (RU(VTIME_PER_SEC, RSEQIOPS) - RPAGE)
+#define RRANDIO (RU(VTIME_PER_SEC, RRANDIOPS) - RPAGE)
+#define WPAGE (RU(VTIME_PER_SEC, RU(WBPS, IOC_PAGE_SIZE)))
+#define WSEQIO (RU(VTIME_PER_SEC, WSEQIOPS) - WPAGE)
+#define WRANDIO (RU(VTIME_PER_SEC, WRANDIOPS) - WPAGE)
+
+#define NSLOTS 4
+
+struct streams {
+ __u64 expected[NSLOTS]; /* next expected sector, per stream */
+ __u64 stamp[NSLOTS]; /* LRU stamp, 0 = empty */
+};
+
+/*
+ * per-cgroup stream table: keyed by the cgroup, freed with it
+ */
+struct {
+ __uint(type, BPF_MAP_TYPE_CGRP_STORAGE);
+ __uint(map_flags, BPF_F_NO_PREALLOC);
+ __type(key, int);
+ __type(value, struct streams);
+} stream_tab SEC(".maps");
+
+/*
+ * incremented by calc_cost(): the streams test asserts pricing went
+ * through the model while it is attached
+ */
+__u64 calc_costs;
+
+SEC("struct_ops")
+u64 BPF_PROG(iocost_ms_calc_cost, struct bio *bio, u64 calc_flags)
+{
+ u64 opf = bio->bi_opf, nbytes = bio->bi_iter.bi_size;
+ u64 sector = bio->bi_iter.bi_sector;
+ struct blkcg *blkcg = bio->bi_blkg->blkcg;
+ struct streams *s;
+ u64 pages, base, coef_page, randio, advance, now;
+ u32 i, victim = 0, found = 0xFFFFFFFF;
+
+ calc_costs++;
+
+ if ((opf & IOCOST_REQ_OP_MASK) == REQ_OP_READ) {
+ base = RSEQIO; coef_page = RPAGE; randio = RRANDIO;
+ } else if ((opf & IOCOST_REQ_OP_MASK) == REQ_OP_WRITE ||
+ (opf & IOCOST_REQ_OP_MASK) == REQ_OP_ZONE_APPEND) {
+ base = WSEQIO; coef_page = WPAGE; randio = WRANDIO;
+ } else {
+ /*
+ * the builtin prices other operations at zero and
+ * leaves them unthrottled; pass them through the same
+ * way instead of inventing costs
+ */
+ return 0;
+ }
+ advance = nbytes >> 9; /* sectors, truncated like the builtin */
+
+ /*
+ * only sized bios participate in stream tracking; the early
+ * return below skips the merge discount, which only applies
+ * to tracked data bios
+ */
+ if (!nbytes) {
+ pages = nbytes >> IOC_PAGE_SHIFT;
+ if (!pages)
+ pages = 1;
+ return base + pages * coef_page;
+ }
+
+ s = bpf_cgrp_storage_get(&stream_tab, blkcg->css.cgroup, NULL,
+ BPF_LOCAL_STORAGE_GET_F_CREATE);
+ if (!s) {
+ /*
+ * no storage: price per page, truncating like the
+ * builtin; merges skip the base like the main path
+ */
+ if (calc_flags & IOCOST_CALC_MERGE)
+ base = 0;
+ pages = nbytes >> IOC_PAGE_SHIFT;
+ if (!pages)
+ pages = 1;
+ return base + pages * coef_page;
+ }
+
+ /*
+ * Slot access is lockless, mirroring the builtin single-cursor
+ * update in ioc_rqos_throttle(): concurrent CPUs submitting for
+ * the same cgroup can race on slot updates; mispricing is
+ * bounded and acceptable for an example model.
+ */
+ now = bpf_ktime_get_ns();
+ for (i = 0; i < NSLOTS; i++) {
+ if (s->expected[i] == sector && s->stamp[i]) {
+ found = i;
+ break;
+ }
+ }
+ if (found != 0xFFFFFFFF) {
+ /* sequential: keep the seq base from the op branch */
+ s->expected[found] = sector + advance;
+ s->stamp[found] = now;
+ } else {
+ /*
+ * a merge which matches no slot is not a new stream:
+ * replacing a slot could evict a live stream, so merged
+ * bios are priced without touching the table (and skip
+ * the base cost below)
+ */
+ if (!(calc_flags & IOCOST_CALC_MERGE)) {
+ base = randio;
+ for (i = 1; i < NSLOTS; i++) {
+ if (s->stamp[i] < s->stamp[victim])
+ victim = i;
+ }
+ s->expected[victim] = sector + advance;
+ s->stamp[victim] = now;
+ }
+ }
+
+ /* builtin truncates: max(sectors >> IOC_SECT_TO_PAGE_SHIFT, 1) */
+ pages = nbytes >> IOC_PAGE_SHIFT;
+ if (!pages)
+ pages = 1;
+ if (calc_flags & IOCOST_CALC_MERGE) {
+ /*
+ * merged bios skip the base cost but still advance
+ * the stream position above, so a merge at the
+ * expected sector does not make the following new IO
+ * look random
+ */
+ base = 0;
+ }
+
+ return base + pages * coef_page;
+}
+
+SEC(".struct_ops.link") /* detach on close */
+struct iocost_model_ops iocost_ms = {
+ .read_vtime_per_page = RPAGE,
+ .write_vtime_per_page = WPAGE,
+ .calc_cost = (void *)iocost_ms_calc_cost,
+};
+
+char LICENSE[] SEC("license") = "GPL";
--
2.53.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [RFC PATCH v14 4/4] docs: cgroup-v2: document the iocost BPF cost model attachment
2026-11-18 13:10 [RFC PATCH v14 0/4] blk-iocost: BPF struct_ops cost model Tao Cui
` (2 preceding siblings ...)
2026-11-18 13:10 ` [RFC PATCH v14 3/4] selftests/bpf: add iocost cost model test Tao Cui
@ 2026-11-18 13:10 ` Tao Cui
2026-11-18 13:17 ` [RFC PATCH v14 0/4] blk-iocost: BPF struct_ops cost model Tao Cui
4 siblings, 0 replies; 9+ messages in thread
From: Tao Cui @ 2026-11-18 13:10 UTC (permalink / raw)
To: tj, josef, axboe, ast, alexei.starovoitov
Cc: cgroups, linux-block, linux-kernel, bpf, andrii, eddyz87, daniel,
linux-kselftest, cui.tao, cuitao, ameryhung
From: Tao Cui <cuitao@kylinos.cn>
Document the BPF cost model attachment in the io.cost.model section
of the cgroup v2 documentation: attaching an iocost_model_ops
struct_ops to a device by its major:minor, the model=bpf readback
while attached (ctrl keeps describing the coefficients), the builtin
model coming back on detach, and ctrl writes never selecting a model.
Suggested-by: Tejun Heo <tj@kernel.org>
Signed-off-by: Tao Cui <cuitao@kylinos.cn>
---
Documentation/admin-guide/cgroup-v2.rst | 55 +++++++++++++++++++++----
1 file changed, 47 insertions(+), 8 deletions(-)
diff --git a/Documentation/admin-guide/cgroup-v2.rst b/Documentation/admin-guide/cgroup-v2.rst
index 9e3c67b02ae2..a3c402f03ae9 100644
--- a/Documentation/admin-guide/cgroup-v2.rst
+++ b/Documentation/admin-guide/cgroup-v2.rst
@@ -2056,8 +2056,9 @@ IO Interface Files
currently implements "io.weight" proportional control. Lines
are keyed by $MAJ:$MIN device numbers and not ordered. The
line for a given device is populated on the first write for
- the device on "io.cost.qos" or "io.cost.model". The following
- nested keys are defined.
+ the device on "io.cost.qos" or "io.cost.model", or when a BPF
+ cost model is attached to it. The following nested keys are
+ defined.
====== =====================================
enable Weight-based control enable
@@ -2111,13 +2112,50 @@ IO Interface Files
implements "io.weight" proportional control. Lines are keyed
by $MAJ:$MIN device numbers and not ordered. The line for a
given device is populated on the first write for the device on
- "io.cost.qos" or "io.cost.model". The following nested keys
- are defined.
+ "io.cost.qos" or "io.cost.model", or when a BPF cost model is
+ attached to it. The following nested keys are defined.
- ===== ================================
+ ===== ==========================================
ctrl "auto" or "user"
- model The cost model in use - "linear"
- ===== ================================
+ model The cost model in use - "linear" or "bpf"
+ ===== ==========================================
+
+ When BPF cost model support is available, a BPF cost model
+ can be bound to a device by attaching an
+ "iocost_model_ops" struct_ops carrying the whole disk's
+ major:minor as encoded by makedev(3) in its "dev" member (a
+ partition's major:minor is rejected). The loader supplies
+ "dev" when the map is created and it cannot be changed
+ afterward. Attaching creates the controller if needed, like an
+ io.cost.model write does, switches the device's pricing to the
+ model and populates the device's line in this file and in
+ "io.cost.qos" if it did not exist yet. Attaching fails with
+ -EBUSY if the device already has a model attached, with
+ -EINVAL if the model fails its attach-time validation, with
+ -EOPNOTSUPP if dev names a partition or a device without a
+ multiqueue queue, and with -ENODEV if dev does not name an
+ existing device or if the device has gone away. Updating the
+ link to a different model is not supported and fails with
+ -EINVAL or -EOPNOTSUPP; switching models requires detaching
+ first. The
+ attached model prices IOs only while the controller is enabled
+ through "io.cost.qos"; attaching does not enable it. IOs issued
+ directly by the root cgroup bypass the model's per-IO pricing;
+ the completion-time sizing of their requests still uses the
+ attached model's transfer coefficients.
+
+ While a model is attached, "model" selects between it and the
+ builtin model: "model=bpf" switches to the attached model and
+ "model=linear" switches back to the builtin model, and neither
+ detaches the struct_ops.
+
+ Detaching removes the model, after which "model=bpf"
+ fails and the device prices with the builtin model again.
+ Device removal ejects the attached model as well, while the
+ struct_ops link remains until userspace destroys it. "ctrl"
+ accepts only "auto" and "user" and never selects a model. It
+ keeps describing the builtin coefficients, which take effect
+ again when switched back.
When "ctrl" is "auto", the kernel may change all parameters
dynamically. When "ctrl" is set to "user" or any other
@@ -2125,7 +2163,8 @@ IO Interface Files
automatic changes are disabled.
When "model" is "linear", the following model parameters are
- defined.
+ defined. They remain readable and writable while "model" is
+ "bpf".
============= ========================================
[r|w]bps The maximum sequential IO throughput
--
2.53.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [RFC PATCH v14 0/4] blk-iocost: BPF struct_ops cost model
2026-11-18 13:10 [RFC PATCH v14 0/4] blk-iocost: BPF struct_ops cost model Tao Cui
` (3 preceding siblings ...)
2026-11-18 13:10 ` [RFC PATCH v14 4/4] docs: cgroup-v2: document the iocost BPF cost model attachment Tao Cui
@ 2026-11-18 13:17 ` Tao Cui
4 siblings, 0 replies; 9+ messages in thread
From: Tao Cui @ 2026-11-18 13:17 UTC (permalink / raw)
To: tj, josef, axboe, ast, alexei.starovoitov
Cc: cui.tao, cgroups, linux-block, linux-kernel, bpf, andrii, eddyz87,
daniel, linux-kselftest, cuitao, ameryhung
Hello, Alexei,
在 2026/11/18 21:10, Tao Cui 写道:
> This series lets a BPF program take over the cost model of one
> device. The BPF model provides calc_cost() for charged IOs plus two
> transfer coefficients for the completion-time sizing, io.cost.model
> selects between the attached model and the builtin one, and enabling
> and disabling the controller stays with io.cost.qos.
>
> The builtin model is a linear formula derived from a handful of device
> parameters, and the right formula is device specific. A BPF model can
> be adapted to the device and updated without rebuilding the kernel.
>
> The series is based on next-20261117.
p1 changed quite a bit since the version you acked, mostly from
tj's review and Sashiko's testing reports, so here's a quick
summary.
The flag enum and the model accessors are renamed. The merge flag
is now enum iocost_calc_flags with IOCOST_CALC_MERGE, calc_cost()
takes a calc_flags parameter, and ioc_model()/model become
ioc_bpf_model()/bpf_model so the builtin model is no longer
represented as a NULL model. The named enum also makes the flags
discoverable through BTF.
calc_cost() now has the same pricing scope as the builtin model:
bios with no data and no flush state and ops other than reads,
writes and zone appends cost nothing, flushes are passed through
so the model can price them from the bio flags, and
completion-time sizing now follows the model's transfer
coefficients.
On attach iocost holds its own reference on the struct_ops map,
so a model cannot go away while it is pricing. A dead owning map
reports -ENODEV instead of -EBUSY now, and the autop stepping
timestamps are cleared at attach so a stale pre-attach profile
cannot fire a step at the model=linear switchback.
The attached and attached_link updates also moved out of
ioc->lock: rq_qos_mutex already orders all their accessors, and
ioc->lock is only needed for publishing bpf_model to the timer
and show paths.
I'm still carrying your Acked-by. Since p1 changed quite a bit,
I'm happy to drop it until you've had a chance to take another
look if you'd prefer.
Thanks,
Tao
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [RFC PATCH v14 3/4] selftests/bpf: add iocost cost model test
2026-11-18 13:10 ` [RFC PATCH v14 3/4] selftests/bpf: add iocost cost model test Tao Cui
@ 2026-11-18 13:41 ` sashiko-bot
2026-11-18 13:49 ` bot+bpf-ci
1 sibling, 0 replies; 9+ messages in thread
From: sashiko-bot @ 2026-11-18 13:41 UTC (permalink / raw)
To: Tao Cui; +Cc: bpf, tj, hannes, mkoutny
Thank you for your contribution! Sashiko AI review found 2 potential
issues to consider:
- [Medium] selftests/bpf: iocost_test_env() hardcodes /sys/fs/cgroup
without mounting
- [Low] selftests/bpf: open_nullb_dev() callers check -errno instead of
fd
--
commit 7c8a11566574f6e4224eb8a8a97b22da0b3eb991
Author: Tao Cui <cuitao@kylinos.cn>
selftests/bpf: add iocost cost model test
This commit adds two BPF struct_ops iocost cost models and selftests that
attach them to a configfs-created null_blk device and verify model
selection, IO pricing, tracepoint firing, and device removal.
> diff --git a/tools/testing/selftests/bpf/prog_tests/iocost_model.c b/tools/testing/selftests/bpf/prog_tests/iocost_model.c
> new file mode 100644
> index 0000000000000..0fbaeb7c5b7ef
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/prog_tests/iocost_model.c
[ ... ]
> +/* open the nullb device node, creating it manually when udev hasn't */
> +static int open_nullb_dev(const char *name, int flags)
> +{
> + char path[32], syspath[64], devtok[32];
> + unsigned int maj, min;
> + dev_t dev;
> + struct stat st;
> + int fd;
> +
> + snprintf(syspath, sizeof(syspath), "/sys/block/%s/dev", name);
> + if (read_file_token(syspath, devtok, sizeof(devtok)) ||
> + sscanf(devtok, "%u:%u", &maj, &min) != 2)
> + return -1;
> + dev = makedev(maj, min);
> +
> + snprintf(path, sizeof(path), "/dev/%s", name);
> + fd = open(path, flags);
> + if (fd >= 0 && fstat(fd, &st) == 0 && S_ISBLK(st.st_mode) &&
> + st.st_rdev == dev)
> + return fd;
> + if (fd >= 0) {
> + /* a stale node from a previous run, drop and recreate */
> + close(fd);
> + fd = -1;
> + }
> + if (unlink(path) && errno != ENOENT)
> + return -1;
> + if (mknod(path, S_IFBLK | 0600, dev))
> + return -1;
> + fd = open(path, flags);
> + if (fd < 0)
> + unlink(path);
> + return fd;
> +}
[Severity: Low]
Can ASSERT_OK(devfd < 0 ? -errno : 0, ...) in serial_test_iocost_model()
and serial_test_iocost_model_streams() pass when open_nullb_dev() fails?
If read_file_token() hits EOF without setting errno, sscanf() fails to
match two tokens, or unlink(path) succeeds and leaves errno at 0 after
the second open() fails, open_nullb_dev() returns -1 with errno == 0.
The callers then pass 0 to ASSERT_OK() and proceed to call
posix_fadvise(), read(), pread(), and close() on fd -1. Using
ASSERT_OK_FD(devfd, ...) in the callers (or returning a negative error
code from open_nullb_dev()) would catch any failure directly.
[ ... ]
> +static bool iocost_test_env(void)
> +{
> + static bool ready;
> + struct statfs sf;
> + int orig_ns = -1, supported;
> + bool ok = false;
> +
> + if (geteuid() != 0) {
> + printf("%s:SKIP:needs root\n", __func__);
> + test__skip();
> + return false;
> + }
> +
> + if (statfs("/sys/fs/cgroup", &sf) ||
> + sf.f_type != CGROUP2_SUPER_MAGIC) {
> + printf("%s:SKIP:needs cgroup v2 at /sys/fs/cgroup\n",
> + __func__);
> + test__skip();
> + return false;
> + }
[ ... ]
> + /*
> + * Isolate the configfs and cgroup mounts from the host. A
> + * failed setup unwinds the namespace so that a later retry does
> + * not stack private namespaces.
> + */
> + orig_ns = open("/proc/self/ns/mnt", O_RDONLY);
> + if (unshare(CLONE_NEWNS) ||
> + mount(NULL, "/", NULL, MS_REC | MS_PRIVATE, NULL)) {
> + printf("%s:SKIP:cannot make mounts private: %s\n",
> + __func__, strerror(errno));
> + test__skip();
> + goto out;
> + }
> + mkdir("/sys/kernel/config", 0755);
> + (void)mount("none", "/sys/kernel/config", "configfs", 0, NULL);
[Severity: Medium]
Will all three iocost tests skip when /sys/fs/cgroup is not already a
cgroup2 mount or after an earlier cgroup v1 test runs in the same
test_progs process?
iocost_test_env(), cgroupfs_has_controller(), readback_model(),
write_cost_model(), and the io.cost.qos writes all access /sys/fs/cgroup
directly without mounting cgroup2 there, while cgroup_setup_and_join() in
tools/testing/selftests/bpf/cgroup_helpers.c mounts cgroup2 at /mnt
(CGROUP_MOUNT_PATH). In addition, setup_classid_environment() in
cgroup_helpers.c mounts tmpfs at /sys/fs/cgroup (CGROUP_MOUNT_DFLT) and
cleanup_classid_environment() leaves that tmpfs mounted:
tools/testing/selftests/bpf/cgroup_helpers.c:setup_classid_environment() {
...
if (mount("tmpfs", CGROUP_MOUNT_DFLT, "tmpfs", 0, NULL) &&
errno != EBUSY) {
log_err("mount cgroup base");
return 1;
}
...
}
When a test using setup_classid_environment() runs before
serial_test_iocost_model*, statfs("/sys/fs/cgroup", &sf) sees TMPFS_MAGIC
instead of CGROUP2_SUPER_MAGIC and skips the tests. Using the cgroup2
mount managed by cgroup_helpers.c (via get_root_cgroup() or
write_cgroup_file("", ...) / read_cgroup_file("", ...)), or mounting
cgroup2 inside the private mount namespace created by iocost_test_env(),
would avoid relying on the host's /sys/fs/cgroup mount state.
--
Sashiko AI review · https://sashiko.dev/#/patchset/20261118131029.797007-1-cui.tao@linux.dev?part=3
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [RFC PATCH v14 2/4] blk-iocost: add iocost_ioc_tick tracepoint for per-period device summary
2026-11-18 13:10 ` [RFC PATCH v14 2/4] blk-iocost: add iocost_ioc_tick tracepoint for per-period device summary Tao Cui
@ 2026-11-18 13:49 ` bot+bpf-ci
0 siblings, 0 replies; 9+ messages in thread
From: bot+bpf-ci @ 2026-11-18 13:49 UTC (permalink / raw)
To: cui.tao, tj, josef, axboe, ast, alexei.starovoitov
Cc: cgroups, linux-block, linux-kernel, bpf, andrii, eddyz87, daniel,
linux-kselftest, cui.tao, cuitao, ameryhung, ast, andrii, daniel,
martin.lau, eddyz87, yonghong.song, mason, ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 1834 bytes --]
> blk-iocost: add iocost_ioc_tick tracepoint for per-period device summary
TL;DR: The new iocost_ioc_tick tracepoint comment describes vrate as relative
to VTIME_PER_SEC, but it is likely vtime per usec (VTIME_PER_USEC is 100%), so
users following it would get values 10^6 too small.
> diff --git a/include/trace/events/iocost.h b/include/trace/events/iocost.h
> --- a/include/trace/events/iocost.h
> +++ b/include/trace/events/iocost.h
[ ... ]
> +/*
> + * Periodic per-device summary, emitted once per period before the vrate
> [ ... ]
> + * vrate is the current vrate in vtime units, where one IO time second
> + * is VTIME_PER_SEC (2^37) vtime and cost.vrate in io.stat reports the
> + * same value as a percentage of that base.
Is this description of vrate accurate? The field is filled from
ioc->vtime_base_rate, which is vtime per wall-clock microsecond:
__entry->vrate = ioc->vtime_base_rate;
and io.stat computes cost.vrate as a percentage of VTIME_PER_USEC, not
VTIME_PER_SEC:
unsigned vp10k = DIV64_U64_ROUND_CLOSEST(
ioc->vtime_base_rate * 10000,
VTIME_PER_USEC);
ioc_refresh_vrate() does the same with
div64_u64(ioc->vtime_base_rate * 100, VTIME_PER_USEC).
Someone following the comment would compute vrate * 100 / 2^37 and get a
value 10^6 too small, since the nominal 100% value is 137438 (2^37 / 10^6).
The commit message points readers to this header for the exact meaning of
the fields. Could the comment say that vrate is in vtime per wall-clock
usec, with VTIME_PER_USEC being 100% as reported by cost.vrate in io.stat?
[ ... ]
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/44312765238
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [RFC PATCH v14 3/4] selftests/bpf: add iocost cost model test
2026-11-18 13:10 ` [RFC PATCH v14 3/4] selftests/bpf: add iocost cost model test Tao Cui
2026-11-18 13:41 ` sashiko-bot
@ 2026-11-18 13:49 ` bot+bpf-ci
1 sibling, 0 replies; 9+ messages in thread
From: bot+bpf-ci @ 2026-11-18 13:49 UTC (permalink / raw)
To: cui.tao, tj, josef, axboe, ast, alexei.starovoitov
Cc: cgroups, linux-block, linux-kernel, bpf, andrii, eddyz87, daniel,
linux-kselftest, cui.tao, cuitao, ameryhung, ast, andrii, daniel,
martin.lau, eddyz87, yonghong.song, mason, ihor.solodrai
[-- Attachment #1: Type: text/plain, Size: 3374 bytes --]
> selftests/bpf: add iocost cost model test
TL;DR: In the iocost selftest, the setns() unwind on the skip path may reset
test_progs' cwd to "/" (and possibly leave a chroot), so later unrelated tests
could fail with -ENOENT when null_blk is unavailable.
> diff --git a/tools/testing/selftests/bpf/prog_tests/iocost_model.c b/tools/testing/selftests/bpf/prog_tests/iocost_model.c
> new file mode 100644
> --- /dev/null
> +++ b/tools/testing/selftests/bpf/prog_tests/iocost_model.c
[ ... ]
> +static bool iocost_test_env(void)
> +{
[ ... ]
> + /*
> + * Isolate the configfs and cgroup mounts from the host. A
> + * failed setup unwinds the namespace so that a later retry does
> + * not stack private namespaces.
> + */
> + orig_ns = open("/proc/self/ns/mnt", O_RDONLY);
> + if (unshare(CLONE_NEWNS) ||
> + mount(NULL, "/", NULL, MS_REC | MS_PRIVATE, NULL)) {
> + printf("%s:SKIP:cannot make mounts private: %s\n",
> + __func__, strerror(errno));
> + test__skip();
> + goto out;
> + }
> + mkdir("/sys/kernel/config", 0755);
> + (void)mount("none", "/sys/kernel/config", "configfs", 0, NULL);
> + SYS_NOFAIL("modprobe null_blk nr_devices=0");
> + if (access("/sys/kernel/config/nullb", W_OK)) {
> + printf("%s:SKIP:needs null_blk with configfs support\n",
> + __func__);
> + test__skip();
> + goto out;
> + }
> +
> + ready = true;
> + ok = true;
> +
> +out:
> + if (orig_ns >= 0) {
> + if (!ok)
> + setns(orig_ns, CLONE_NEWNS);
> + close(orig_ns);
> + }
> + return ok;
> +}
When a prerequisite check fails after unshare(CLONE_NEWNS), for example
when access("/sys/kernel/config/nullb", W_OK) fails because null_blk is
not built, not installed, or modprobe is missing, this goes back with
setns(orig_ns, CLONE_NEWNS).
Can this change the cwd of the test_progs process? mntns_install() in
fs/namespace.c ends with:
/* Update the pwd and root */
set_fs_pwd(fs, &root);
set_fs_root(fs, &root);
unshare() keeps the cwd, so only this skip path moves the process to "/".
The serial tests run in the test_progs main process, so the new cwd
persists for every test that runs afterwards. Many of those open objects
by cwd-relative path, for example serial_test_kfree_skb() loading
"./test_pkt_access.bpf.o", select_reuseport.c opening
"test_select_reuseport_kern.bpf.o", and l4lb_all.c using "test_l4lb.bpf.o".
The flavored runners also lose the chdir() done by cd_flavor_subdir().
So on a kernel with the iocost BPF support but no usable null_blk, would
the iocost tests report SKIP and then unrelated tests fail with -ENOENT?
If test_progs runs inside a chroot, does the root reset also leave the
chroot?
The comment says the unwind exists "so that a later retry does not stack
private namespaces", but serial_test_iocost_model() and
serial_test_iocost_model_streams() call cgroup_setup_and_join(), and
setup_cgroup_environment() calls unshare(CLONE_NEWNS) every time, so
namespaces stack regardless.
Would it work to drop the setns() unwind and cache the failure in a
static, or to save the cwd with open(".", O_RDONLY | O_DIRECTORY) before
unshare() and fchdir() back to it after setns()?
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/44312765238
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2026-11-18 13:49 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2026-11-18 13:10 [RFC PATCH v14 0/4] blk-iocost: BPF struct_ops cost model Tao Cui
2026-11-18 13:10 ` [RFC PATCH v14 1/4] blk-iocost: add BPF struct_ops cost model support Tao Cui
2026-11-18 13:10 ` [RFC PATCH v14 2/4] blk-iocost: add iocost_ioc_tick tracepoint for per-period device summary Tao Cui
2026-11-18 13:49 ` bot+bpf-ci
2026-11-18 13:10 ` [RFC PATCH v14 3/4] selftests/bpf: add iocost cost model test Tao Cui
2026-11-18 13:41 ` sashiko-bot
2026-11-18 13:49 ` bot+bpf-ci
2026-11-18 13:10 ` [RFC PATCH v14 4/4] docs: cgroup-v2: document the iocost BPF cost model attachment Tao Cui
2026-11-18 13:17 ` [RFC PATCH v14 0/4] blk-iocost: BPF struct_ops cost model Tao Cui
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox