Re: [PATCH V4 00/15] blk-throttle: add .high limit

[Shaohua Li <shli@fb.com>]

On Mon, Nov 14, 2016 at 05:18:28PM -0800, Bart Van Assche wrote:
> On 11/14/2016 04:49 PM, Shaohua Li wrote:
> > On Mon, Nov 14, 2016 at 04:41:33PM -0800, Bart Van Assche wrote:
> > > Thank you for pointing me to the discussion thread about v3 of this patch
> > > series. Did I see correctly that one of the conclusions was that for users
> > > this mechanism is hard to configure? Are we providing a good service to
> > > Linux users by providing a mechanism that is hard to configure?
> > 
> > Yes, this is a kind of low level knob and is expected to be configured by
> > experienced users. This sucks, but we really don't have good solutions. If
> > anybody has better ideas, I'm happy to try.
> 
> Hello Shaohua,
> 
> An approach I have been considering to analyze further is as follows:
> * For rotational media use an algorithm like BFQ to preserve sequentiality
> of workloads and to guarantee fairness. This means that one application
> submits I/O per time slot.
> * For SSDs, multiplex I/O from multiple applications during a single time
> slot to keep the queue depth high. Throttle I/O if needed to realize
> fairness.
> 
> Implementing this approach requires an approach for estimating I/O cost
> based on the request characteristics (offset and size) and the device type
> (rotational or SSD). This may require measuring the time that was needed to
> process past requests and to use that information in a learning algorithm.
> 
> Unless someone can convince me of the opposite I think that coming up with
> an algorithm for estimating I/O cost is essential to guarantee I/O fairness
> without requesting users to perform complicated parameter configurations.

That's what I tried before:
http://marc.info/?l=linux-kernel&m=145617863208940&w=2

Unfortunately estimating I/O cost and disk capability is very hard if not
impossible. People objected using bandwidth or iops to estimate I/O cost.

Thanks,
Shaohua