[PATCH] A tentative patch to bypass REQ_META in blk throttle

[PATCH] A tentative patch to bypass REQ_META in blk throttle

[Benlong Zhang]

One problem with cgwb is how fs should treat metadata bios.
For example in xfs, the log might be partially stuck in one
group, leaving threads in other groups waiting for too long.
Please refer to the linux-xfs discussion:
"[PATCH V2] xfs: implement cgroup writeback support"

One approach is to correctly tag bio->bi_css from within the
filesystems (for xfs log should be blkcg_root), and the other
is to skip them, but relies on REQ_META being set.

It works with xfs on a cgwb porting implementation to 3.10.0.
But really not sure about other filesystems...
---
 block/blk-throttle.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/block/blk-throttle.c b/block/blk-throttle.c
index c5a1316..e2abf6e 100644
--- a/block/blk-throttle.c
+++ b/block/blk-throttle.c
@@ -2159,6 +2159,10 @@ bool blk_throtl_bio(struct request_queue *q, struct blkcg_gq *blkg,
 	if (bio_flagged(bio, BIO_THROTTLED) || !tg->has_rules[rw])
 		goto out;
 
+	/* bypass REQ_META for fs global resource */
+	if (bio->bi_opf & REQ_META)
+		goto out;
+
 	spin_lock_irq(q->queue_lock);
 
 	throtl_update_latency_buckets(td);
-- 
1.8.3.1

Re: [PATCH] A tentative patch to bypass REQ_META in blk throttle

[Tejun Heo]

Hello,

On Fri, Apr 20, 2018 at 06:06:01PM +0800, Benlong Zhang wrote:
> One problem with cgwb is how fs should treat metadata bios.
> For example in xfs, the log might be partially stuck in one
> group, leaving threads in other groups waiting for too long.
> Please refer to the linux-xfs discussion:
> "[PATCH V2] xfs: implement cgroup writeback support"
> 
> One approach is to correctly tag bio->bi_css from within the
> filesystems (for xfs log should be blkcg_root), and the other
> is to skip them, but relies on REQ_META being set.
> 
> It works with xfs on a cgwb porting implementation to 3.10.0.
> But really not sure about other filesystems...

Yeah, Josef (cc'd) is working on a similar approach but in a more
generic way.

Thanks.

-- 
tejun

Re: [PATCH] A tentative patch to bypass REQ_META in blk throttle

[Brian Foster]

On Thu, Apr 26, 2018 at 12:01:18PM -0700, Tejun Heo wrote:
> Hello,
> 
> On Fri, Apr 20, 2018 at 06:06:01PM +0800, Benlong Zhang wrote:
> > One problem with cgwb is how fs should treat metadata bios.
> > For example in xfs, the log might be partially stuck in one
> > group, leaving threads in other groups waiting for too long.
> > Please refer to the linux-xfs discussion:
> > "[PATCH V2] xfs: implement cgroup writeback support"
> > 
> > One approach is to correctly tag bio->bi_css from within the
> > filesystems (for xfs log should be blkcg_root), and the other
> > is to skip them, but relies on REQ_META being set.
> > 
> > It works with xfs on a cgwb porting implementation to 3.10.0.
> > But really not sure about other filesystems...
> 
> Yeah, Josef (cc'd) is working on a similar approach but in a more
> generic way.
> 

I thought Josef's work had more to do with tracking/managing dirty
metadata (i.e., driving writeback/reclaim and whatnot). Perhaps I just
haven't followed it close enough... how does that relate to cgroup bio
ownership?

The issue addressed by this patch (wrt to XFS) is that log I/O that
happens to be submitted via a throttled cgroup ends up blocking
unthrottled tasks because the log is essentially a global/shared
mechanism. We therefore want to somehow exempt metadata I/O from
throttling in that particular case.

Note that I'm not familiar enough with the block layer code to know
whether this patch is the right way to do that, but I think (as noted in
the commit log above) that not doing the default bio association if the
fs hasn't (or associating with some root group) for metadata I/O would
also be sufficient. Hm?

Brian

> Thanks.
> 
> -- 
> tejun

Re: [PATCH] A tentative patch to bypass REQ_META in blk throttle

[Tejun Heo]

Hello, Brian.

On Fri, Apr 27, 2018 at 08:20:06AM -0400, Brian Foster wrote:
> I thought Josef's work had more to do with tracking/managing dirty
> metadata (i.e., driving writeback/reclaim and whatnot). Perhaps I just
> haven't followed it close enough... how does that relate to cgroup bio
> ownership?

Oh, we've been working on it for a while now but haven't posted the
patches yet.  We should be able to post the patches in several weeks.

Thanks.

-- 
tejun

Re: [PATCH] A tentative patch to bypass REQ_META in blk throttle

[Benlong Zhang]

Hi Tejun,

On Thu, Apr 26, 2018 at 12:01:18PM -0700, Tejun Heo wrote:
> Hello,
> 
> On Fri, Apr 20, 2018 at 06:06:01PM +0800, Benlong Zhang wrote:
> > One problem with cgwb is how fs should treat metadata bios.
> > For example in xfs, the log might be partially stuck in one
> > group, leaving threads in other groups waiting for too long.
> > Please refer to the linux-xfs discussion:
> > "[PATCH V2] xfs: implement cgroup writeback support"
> > 
> > One approach is to correctly tag bio->bi_css from within the
> > filesystems (for xfs log should be blkcg_root), and the other
> > is to skip them, but relies on REQ_META being set.
> > 
> > It works with xfs on a cgwb porting implementation to 3.10.0.
> > But really not sure about other filesystems...
> 
> Yeah, Josef (cc'd) is working on a similar approach but in a more
> generic way.

Good to know and looking forward to the code.

Thank you for your reply!

Benlong

> 
> Thanks.
> 
> -- 
> tejun