cgroups.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Tejun Heo <tj-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
To: Jens Axboe <axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
Cc: Paolo Valente
	<paolo.valente-rcYM44yAMweonA0d6jMUrA@public.gmane.org>,
	Li Zefan <lizefan-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>,
	Fabio Checconi
	<fchecconi-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>,
	Arianna Avanzini
	<avanzini.arianna-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>,
	Paolo Valente <posta_paolo-whZMOeQn8C0@public.gmane.org>,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org,
	cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Subject: Re: [PATCH RFC - TAKE TWO - 00/12] New version of the BFQ I/O Scheduler
Date: Sat, 31 May 2014 01:16:35 -0400	[thread overview]
Message-ID: <20140531051635.GA19925@mtj.dyndns.org> (raw)
In-Reply-To: <538926F6.7080409-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>

Hello, Jens.

On Fri, May 30, 2014 at 06:48:54PM -0600, Jens Axboe wrote:
> What I really like about the implementation is, as Tejun highlights, that
> the algorithm is detailed and characterized. Nobody ever wrote any detailed
> documentation on CFQ - I think the closest is a talk I gave at LCA in 2007
> or so. That said, the devil is _always_ in the details when it comes to nice
> algorithms. When theory meets practice, then the little tweaks and tunings
> required to not drop 10% there or 20% here is when it gets ugly. And that's
> where CFQ has the history going for it, at least. Which is another reason

That's the thing I like about the new paper.  It looks like the
original BFQ was the naive ideal implementation but the new paper
basically takes most, if not all, heuristics implemented in cfq,
properly characterizes them and applies the improved versions.  The
end result, AFAICS, really is an evolution of cfq with the core
round-robin + preemption scheduler replaced by something a lot firmer.
It doesn't really lose much of what cfq has accumulated over time.

> for turning patch #2 into a series of changes for CFQ instead. We need to
> end up with something where we can potentially bisect our way down to
> whatever caused any given regression. The worst possible situation is "CFQ
> works fine for this workload, but BFQ does not" or vice versa. Or hangs, or
> whatever it might be.

It's likely that there will be some workloads out there which may be
affected adversely, which is true for any change really but with both
the core scheduling and heuristics properly characterized at least
finding a reasonable trade-off should be much less of a crapshoot and
the expected benefits seem to easily outweigh the risks as long as we
can properly sequence the changes.

Thanks.

-- 
tejun

  parent reply	other threads:[~2014-05-31  5:16 UTC|newest]

Thread overview: 107+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2014-05-27 12:42 [PATCH RFC RESEND 00/14] New version of the BFQ I/O Scheduler paolo
     [not found] ` <1401194558-5283-1-git-send-email-paolo.valente-rcYM44yAMweonA0d6jMUrA@public.gmane.org>
2014-05-27 12:42   ` [PATCH RFC RESEND 01/14] block: kconfig update and build bits for BFQ paolo
     [not found]     ` <1401194558-5283-2-git-send-email-paolo.valente-rcYM44yAMweonA0d6jMUrA@public.gmane.org>
2014-05-28 22:19       ` Tejun Heo
     [not found]         ` <20140528221929.GG1419-Gd/HAXX7CRxy/B6EtB590w@public.gmane.org>
2014-05-29  9:05           ` [PATCH RFC - TAKE TWO - 00/12] New version of the BFQ I/O Scheduler Paolo Valente
2014-05-29  9:05             ` [PATCH RFC - TAKE TWO - 06/12] block, bfq: improve responsiveness Paolo Valente
     [not found]               ` <1401354343-5527-7-git-send-email-paolo.valente-rcYM44yAMweonA0d6jMUrA@public.gmane.org>
2014-05-30 15:41                 ` Tejun Heo
     [not found]             ` <1401354343-5527-1-git-send-email-paolo.valente-rcYM44yAMweonA0d6jMUrA@public.gmane.org>
2014-05-29  9:05               ` [PATCH RFC - TAKE TWO - 01/12] block: introduce the BFQ-v0 I/O scheduler Paolo Valente
     [not found]                 ` <1401354343-5527-2-git-send-email-paolo.valente-rcYM44yAMweonA0d6jMUrA@public.gmane.org>
2014-05-30 15:36                   ` Tejun Heo
2014-05-29  9:05               ` [PATCH RFC - TAKE TWO - 02/12] block, bfq: add full hierarchical scheduling and cgroups support Paolo Valente
     [not found]                 ` <1401354343-5527-3-git-send-email-paolo.valente-rcYM44yAMweonA0d6jMUrA@public.gmane.org>
2014-05-30 15:37                   ` Tejun Heo
     [not found]                     ` <20140530153718.GB24871-Gd/HAXX7CRxy/B6EtB590w@public.gmane.org>
2014-05-30 15:39                       ` Tejun Heo
     [not found]                         ` <20140530153943.GC24871-Gd/HAXX7CRxy/B6EtB590w@public.gmane.org>
2014-05-30 21:49                           ` Paolo Valente
2014-05-30 21:49                       ` Paolo Valente
2014-05-29  9:05               ` [PATCH RFC - TAKE TWO - 03/12] block, bfq: improve throughput boosting Paolo Valente
2014-05-29  9:05               ` [PATCH RFC - TAKE TWO - 04/12] block, bfq: modify the peak-rate estimator Paolo Valente
2014-05-29  9:05               ` [PATCH RFC - TAKE TWO - 05/12] block, bfq: add more fairness to boost throughput and reduce latency Paolo Valente
2014-05-29  9:05               ` [PATCH RFC - TAKE TWO - 07/12] block, bfq: reduce I/O latency for soft real-time applications Paolo Valente
2014-05-29  9:05               ` [PATCH RFC - TAKE TWO - 08/12] block, bfq: preserve a low latency also with NCQ-capable drives Paolo Valente
     [not found]                 ` <1401354343-5527-9-git-send-email-paolo.valente-rcYM44yAMweonA0d6jMUrA@public.gmane.org>
2014-05-31 13:48                   ` Tejun Heo
     [not found]                     ` <20140531134823.GB24557-9pTldWuhBndy/B6EtB590w@public.gmane.org>
2014-06-02  9:58                       ` Paolo Valente
2014-05-29  9:05               ` [PATCH RFC - TAKE TWO - 10/12] block, bfq: add Early Queue Merge (EQM) Paolo Valente
     [not found]                 ` <1401354343-5527-11-git-send-email-paolo.valente-rcYM44yAMweonA0d6jMUrA@public.gmane.org>
2014-06-01  0:03                   ` Tejun Heo
     [not found]                     ` <20140601000331.GA29085-Gd/HAXX7CRxy/B6EtB590w@public.gmane.org>
2014-06-02  9:46                       ` Paolo Valente
     [not found]                         ` <3B7B1A46-46EB-4C52-A52C-4F79C71D14C2-rcYM44yAMweonA0d6jMUrA@public.gmane.org>
2014-06-03 16:28                           ` Tejun Heo
     [not found]                             ` <20140603162844.GD26210-Gd/HAXX7CRxy/B6EtB590w@public.gmane.org>
2014-06-04 11:47                               ` Paolo Valente
2014-06-04 13:04                                 ` Tejun Heo
     [not found]                                   ` <20140604130446.GA5004-Gd/HAXX7CRxy/B6EtB590w@public.gmane.org>
2014-06-16 11:23                                     ` Paolo Valente
2014-05-29  9:05               ` [PATCH RFC - TAKE TWO - 11/12] block, bfq: boost the throughput on NCQ-capable flash-based devices Paolo Valente
     [not found]                 ` <1401354343-5527-12-git-send-email-paolo.valente-rcYM44yAMweonA0d6jMUrA@public.gmane.org>
2014-05-30 15:46                   ` Tejun Heo
     [not found]                     ` <20140530154654.GE24871-Gd/HAXX7CRxy/B6EtB590w@public.gmane.org>
2014-05-30 22:01                       ` Paolo Valente
2014-05-31 11:52                   ` Tejun Heo
     [not found]                     ` <20140531115216.GB5057-Gd/HAXX7CRxy/B6EtB590w@public.gmane.org>
2014-06-02  9:26                       ` Paolo Valente
     [not found]                         ` <36BFDB73-AEC2-4B87-9FD6-205E9431E722-rcYM44yAMweonA0d6jMUrA@public.gmane.org>
2014-06-03 17:11                           ` Tejun Heo
     [not found]                             ` <20140603171124.GE26210-Gd/HAXX7CRxy/B6EtB590w@public.gmane.org>
2014-06-04  7:29                               ` Paolo Valente
     [not found]                                 ` <03CDD106-DB18-4E8F-B3D6-2AAD45782A06-rcYM44yAMweonA0d6jMUrA@public.gmane.org>
2014-06-04 13:56                                   ` Tejun Heo
     [not found]                                     ` <20140604135627.GB5004-Gd/HAXX7CRxy/B6EtB590w@public.gmane.org>
2014-06-16 10:46                                       ` Paolo Valente
     [not found]                                         ` <D163E069-ED77-4BF5-A488-9A90C41C60C1-rcYM44yAMweonA0d6jMUrA@public.gmane.org>
2014-06-19  1:14                                           ` Tejun Heo
2014-05-29  9:05               ` [PATCH RFC - TAKE TWO - 12/12] block, bfq: boost the throughput with random I/O on NCQ-capable HDDs Paolo Valente
     [not found]                 ` <1401354343-5527-13-git-send-email-paolo.valente-rcYM44yAMweonA0d6jMUrA@public.gmane.org>
2014-05-30 15:51                   ` Tejun Heo
2014-05-31 13:34                   ` Tejun Heo
2014-05-30 16:07               ` [PATCH RFC - TAKE TWO - 00/12] New version of the BFQ I/O Scheduler Tejun Heo
     [not found]                 ` <20140530160712.GG24871-Gd/HAXX7CRxy/B6EtB590w@public.gmane.org>
2014-05-30 22:23                   ` Paolo Valente
     [not found]                     ` <464F6CBE-A63E-46EF-A90D-BF8450430444-rcYM44yAMweonA0d6jMUrA@public.gmane.org>
2014-05-30 23:28                       ` Tejun Heo
     [not found]                         ` <20140530232804.GA5057-Gd/HAXX7CRxy/B6EtB590w@public.gmane.org>
2014-05-30 23:54                           ` Paolo Valente
2014-06-02 11:14                           ` Pavel Machek
     [not found]                             ` <20140602111432.GA3737-tWAi6jLit6GreWDznjuHag@public.gmane.org>
2014-06-02 13:02                               ` Pavel Machek
     [not found]                                 ` <20140602130226.GA14654-tWAi6jLit6GreWDznjuHag@public.gmane.org>
2014-06-03 16:54                                   ` Paolo Valente
     [not found]                                     ` <FCFE0106-A4DD-4DEF-AAAE-040F3823A447-rcYM44yAMweonA0d6jMUrA@public.gmane.org>
2014-06-03 20:40                                       ` Pavel Machek
2014-06-04  8:39                                       ` Pavel Machek
2014-06-04  9:08                                       ` Pavel Machek
2014-06-04 10:03                                       ` BFQ speed tests [was Re: [PATCH RFC - TAKE TWO - 00/12] New version of the BFQ I/O Scheduler] Pavel Machek
     [not found]                                         ` <20140604100358.GA4799-tWAi6jLit6GreWDznjuHag@public.gmane.org>
2014-06-04 10:24                                           ` Paolo Valente
     [not found]                                             ` <4888F93F-D58D-48DD-81A6-A6D61C452D92-rcYM44yAMweonA0d6jMUrA@public.gmane.org>
2014-06-04 11:59                                               ` Takashi Iwai
     [not found]                                                 ` <s5hsink3mxk.wl%tiwai-l3A5Bk7waGM@public.gmane.org>
2014-06-04 12:12                                                   ` Paolo Valente
2014-06-11 20:45                                                   ` Paolo Valente
     [not found]                                                     ` <6A4905B2-ACAA-419D-9C83-659BE9A5B20B-rcYM44yAMweonA0d6jMUrA@public.gmane.org>
2014-06-13 16:21                                                       ` Takashi Iwai
2014-06-11 20:39                                           ` Paolo Valente
2014-06-02 17:33                               ` [PATCH RFC - TAKE TWO - 00/12] New version of the BFQ I/O Scheduler Tejun Heo
     [not found]                                 ` <20140602173332.GB8912-Gd/HAXX7CRxy/B6EtB590w@public.gmane.org>
2014-06-03  4:12                                   ` Mike Galbraith
2014-06-04 22:31                                   ` Pavel Machek
     [not found]                                     ` <20140604223152.GA7881-tWAi6jLit6GreWDznjuHag@public.gmane.org>
2014-06-05  2:14                                       ` Jens Axboe
2014-05-31  0:48                   ` Jens Axboe
     [not found]                     ` <538926F6.7080409-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
2014-05-31  5:16                       ` Tejun Heo [this message]
     [not found]                         ` <20140531051635.GA19925-9pTldWuhBndy/B6EtB590w@public.gmane.org>
2014-06-02 14:29                           ` Jens Axboe
     [not found]                             ` <538C8A47.1050502-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
2014-06-02 17:24                               ` Tejun Heo
     [not found]                                 ` <20140602172454.GA8912-Gd/HAXX7CRxy/B6EtB590w@public.gmane.org>
2014-06-02 17:32                                   ` Jens Axboe
     [not found]                                     ` <538CB515.3090700-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
2014-06-02 17:42                                       ` Tejun Heo
     [not found]                                         ` <20140602174250.GC8912-Gd/HAXX7CRxy/B6EtB590w@public.gmane.org>
2014-06-02 17:46                                           ` Jens Axboe
     [not found]                                             ` <538CB87C.7030600-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
2014-06-02 18:51                                               ` Tejun Heo
     [not found]                                                 ` <20140602185138.GD8912-Gd/HAXX7CRxy/B6EtB590w@public.gmane.org>
2014-06-02 20:57                                                   ` Jens Axboe
     [not found]                                                     ` <20140602205713.GB8357-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
2014-06-04 14:31                                                       ` Christoph Hellwig
     [not found]                                                         ` <20140604143136.GA1920-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2014-06-04 14:50                                                           ` Tejun Heo
     [not found]                                                             ` <20140604145053.GE5004-Gd/HAXX7CRxy/B6EtB590w@public.gmane.org>
2014-06-04 14:53                                                               ` Christoph Hellwig
     [not found]                                                                 ` <20140604145330.GA2955-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>
2014-06-04 14:58                                                                   ` Tejun Heo
     [not found]                                                                     ` <20140604145829.GF5004-Gd/HAXX7CRxy/B6EtB590w@public.gmane.org>
2014-06-04 17:51                                                                       ` Christoph Hellwig
2014-06-17 15:55                               ` Paolo Valente
     [not found]                                 ` <0A5218F8-0215-4B4F-959B-EE5AAEFC164A-rcYM44yAMweonA0d6jMUrA@public.gmane.org>
2014-06-19  1:46                                   ` Tejun Heo
     [not found]                                     ` <20140619014600.GA20100-9pTldWuhBndy/B6EtB590w@public.gmane.org>
2014-06-19  1:49                                       ` Tejun Heo
2014-06-19  2:29                                       ` Jens Axboe
     [not found]                                         ` <53A24B1C.1070004-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org>
2014-06-23 13:53                                           ` Paolo Valente
     [not found]                                             ` <8F719638-0CD7-4BD2-8F4F-088913A0EE2D-rcYM44yAMweonA0d6jMUrA@public.gmane.org>
2014-06-23 19:20                                               ` Tejun Heo
     [not found]                                                 ` <20140623192022.GA19660-9pTldWuhBndy/B6EtB590w@public.gmane.org>
2014-07-09 20:54                                                   ` Paolo Valente
2014-05-29  9:05             ` [PATCH RFC - TAKE TWO - 09/12] block, bfq: reduce latency during request-pool saturation Paolo Valente
     [not found]               ` <1401354343-5527-10-git-send-email-paolo.valente-rcYM44yAMweonA0d6jMUrA@public.gmane.org>
2014-05-31 13:54                 ` Tejun Heo
     [not found]                   ` <20140531135402.GC24557-9pTldWuhBndy/B6EtB590w@public.gmane.org>
2014-06-02  9:54                     ` Paolo Valente
2014-05-27 12:42   ` [PATCH RFC RESEND 02/14] block: introduce the BFQ-v0 I/O scheduler paolo
2014-05-27 12:42   ` [PATCH RFC RESEND 03/14] block: add hierarchical-support option to kconfig paolo
2014-05-27 12:42   ` [PATCH RFC RESEND 04/14] block, bfq: add full hierarchical scheduling and cgroups support paolo
2014-05-27 12:42   ` [PATCH RFC RESEND 05/14] block, bfq: improve throughput boosting paolo
2014-05-27 12:42   ` [PATCH RFC RESEND 06/14] block, bfq: modify the peak-rate estimator paolo
2014-05-27 12:42   ` [PATCH RFC RESEND 07/14] block, bfq: add more fairness to boost throughput and reduce latency paolo
2014-05-27 12:42   ` [PATCH RFC RESEND 08/14] block, bfq: improve responsiveness paolo
2014-05-27 12:42   ` [PATCH RFC RESEND 09/14] block, bfq: reduce I/O latency for soft real-time applications paolo
2014-05-27 12:42   ` [PATCH RFC RESEND 10/14] block, bfq: preserve a low latency also with NCQ-capable drives paolo
2014-05-27 12:42   ` [PATCH RFC RESEND 11/14] block, bfq: reduce latency during request-pool saturation paolo
2014-05-27 12:42   ` [PATCH RFC RESEND 13/14] block, bfq: boost the throughput on NCQ-capable flash-based devices paolo
2014-05-27 12:42   ` [PATCH RFC RESEND 14/14] block, bfq: boost the throughput with random I/O on NCQ-capable HDDs paolo
2014-05-30 15:32   ` [PATCH RFC RESEND 00/14] New version of the BFQ I/O Scheduler Vivek Goyal
     [not found]     ` <20140530153228.GE16605-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2014-05-30 16:16       ` Tejun Heo
     [not found]         ` <20140530161650.GH24871-Gd/HAXX7CRxy/B6EtB590w@public.gmane.org>
2014-05-30 17:09           ` Vivek Goyal
     [not found]             ` <20140530170958.GF16605-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2014-05-30 17:26               ` Tejun Heo
     [not found]                 ` <20140530172609.GI24871-Gd/HAXX7CRxy/B6EtB590w@public.gmane.org>
2014-05-30 17:55                   ` Vivek Goyal
     [not found]                     ` <20140530175527.GH16605-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2014-05-30 17:59                       ` Tejun Heo
2014-05-30 23:33               ` Paolo Valente
2014-05-30 17:31   ` Vivek Goyal
     [not found]     ` <20140530173146.GG16605-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
2014-05-30 17:39       ` Tejun Heo
2014-05-27 12:42 ` [PATCH RFC RESEND 12/14] block, bfq: add Early Queue Merge (EQM) paolo

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20140531051635.GA19925@mtj.dyndns.org \
    --to=tj-dgejt+ai2ygdnm+yrofe0a@public.gmane.org \
    --cc=avanzini.arianna-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \
    --cc=axboe-tSWWG44O7X1aa/9Udqfwiw@public.gmane.org \
    --cc=cgroups-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=containers-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org \
    --cc=fchecconi-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org \
    --cc=linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=lizefan-hv44wF8Li93QT0dZR+AlfA@public.gmane.org \
    --cc=paolo.valente-rcYM44yAMweonA0d6jMUrA@public.gmane.org \
    --cc=posta_paolo-whZMOeQn8C0@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).