[PATCH] blkcg: Optimize exit to user space

[PATCH] blkcg: Optimize exit to user space

[Thomas Gleixner]

blkcg uses TIF_NOTIFY_RESUME to handle throttling on exit to user
space. TIF_NOTIFY_RESUME is a multiplexing TIF bit, which is utilized by
other entities as well.

This results in a unconditional blkcg_maybe_throttle_current() call for
every invocation of resume_user_mode_work(), which is a pointless exercise
as most of the time there is no throttling work to do.

Optimize this by doing a quick check of the throttling condition before
invoking it.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Cc: Tejun Heo <tj@kernel.org>
Cc: Josef Bacik <josef@toxicpanda.com>
Cc: Jens Axboe <axboe@kernel.dk>
---
 block/blk-cgroup.c         |    4 ++--
 include/linux/blk-cgroup.h |   10 +++++++++-
 2 files changed, 11 insertions(+), 3 deletions(-)

--- a/block/blk-cgroup.c
+++ b/block/blk-cgroup.c
@@ -2001,7 +2001,7 @@ static void blkcg_maybe_throttle_blkg(st
 }
 
 /**
- * blkcg_maybe_throttle_current - throttle the current task if it has been marked
+ * __blkcg_maybe_throttle_current - throttle the current task if it has been marked
  *
  * This is only called if we've been marked with set_notify_resume().  Obviously
  * we can be set_notify_resume() for reasons other than blkcg throttling, so we
@@ -2010,7 +2010,7 @@ static void blkcg_maybe_throttle_blkg(st
  * to be called by people willy-nilly as it will actually do the work to
  * throttle the task if it is setup for throttling.
  */
-void blkcg_maybe_throttle_current(void)
+void __blkcg_maybe_throttle_current(void)
 {
 	struct gendisk *disk = current->throttle_disk;
 	struct blkcg *blkcg;
--- a/include/linux/blk-cgroup.h
+++ b/include/linux/blk-cgroup.h
@@ -14,6 +14,7 @@
  * 	              Nauman Rafique <nauman@google.com>
  */
 
+#include <linux/sched.h>
 #include <linux/types.h>
 
 struct bio;
@@ -26,7 +27,14 @@ struct gendisk;
 extern struct cgroup_subsys_state * const blkcg_root_css;
 
 void blkcg_schedule_throttle(struct gendisk *disk, bool use_memdelay);
-void blkcg_maybe_throttle_current(void);
+void __blkcg_maybe_throttle_current(void);
+
+static inline void blkcg_maybe_throttle_current(void)
+{
+	if (unlikely(current->throttle_disk))
+		__blkcg_maybe_throttle_current();
+}
+
 bool blk_cgroup_congested(void);
 void blkcg_pin_online(struct cgroup_subsys_state *blkcg_css);
 void blkcg_unpin_online(struct cgroup_subsys_state *blkcg_css);

Re: [PATCH] blkcg: Optimize exit to user space

[Tejun Heo]

On Wed, Aug 13, 2025 at 04:59:57PM +0200, Thomas Gleixner wrote:
> blkcg uses TIF_NOTIFY_RESUME to handle throttling on exit to user
> space. TIF_NOTIFY_RESUME is a multiplexing TIF bit, which is utilized by
> other entities as well.
> 
> This results in a unconditional blkcg_maybe_throttle_current() call for
> every invocation of resume_user_mode_work(), which is a pointless exercise
> as most of the time there is no throttling work to do.
> 
> Optimize this by doing a quick check of the throttling condition before
> invoking it.
> 
> Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
> Cc: Tejun Heo <tj@kernel.org>
> Cc: Josef Bacik <josef@toxicpanda.com>
> Cc: Jens Axboe <axboe@kernel.dk>
...
> -void blkcg_maybe_throttle_current(void)
> +void __blkcg_maybe_throttle_current(void)
>  {
>  	struct gendisk *disk = current->throttle_disk;
>  	struct blkcg *blkcg;

A nit: __blkcg_maybe_throttle_current() ends up doing another NULL check.
Maybe drop that? Other than that,

Acked-by: Tejun Heo <tj@kernel.org>

Thanks.

-- 
tejun