[LSF/MM TOPIC] [ATTEND] Throttling I/O

[Suresh Jayaraman <sjayaraman@suse.com>]

Hello,

I'd like to discuss again[1] the problem of throttling buffered writes
and a throttle mechanism that works for all kinds of I/O.

Some background information.

During last year's LSF/MM, Fengguang discussed his proportional I/O
controller patches as part of the writeback session. The limitations
that were seen of his approach were a) non-handling of bursty IO
submission in the flusher thread b) sharing config variables among
different policies c) and that it violates layering and lacking
long-term design. Tejun proposed back-pressure approach to the problem
i.e. apply pressure where the problem is (block layer) and propagate
upwards.

The general opinion at that time was that we needed more
inputs/consensus needed on the natural, flexible, extensible
"interface". The discussion thread that Vivek started[2] to collect the
inputs on "interface", though resulted in good collection of inputs,
not sure whether it represents inputs from all the interested parties.

At Kernel Summit last year, I learned from LWN[3] that the topic was
discussed again. Tejun, apparently proposed a solution that splits up
the global async CFQ queue by cgroup, so that the CFQ scheduler can
easily schedule the per-cgroup sync/async queues according to the
per-cgroup I/O weights. Fengguang proposed a solution by supporting the
per-cgroup buffered write weights in balance_dirty_pages() and running a
user-space daemon that updates the CFQ/BDP weights every second. There
doesn't seem to be consensus towards either of the proposed approaches.

Looking at the possibility of prototyping Tejun's proposed idea lead to
many questions (but my understanding may not be complete here as it is
based only on LWN's mem-cg mini-summit coverage, so please correct me if
wrong).

- Making cfq schedule the per cgroup sync/async queues according to I/O
  weights would mean that we'll need to use per cgroup cfqq's instead
  of per process? What will the impact on sync latencies if for example
  we have many sync only tasks in one cgroup and many async tasks in
  another?  What if BLK_CGROUP is not configured, what would be the
  fallback behavior?

- Suppose if we have 100 cgroups and we are to have one cfqq per
  priority per cgroup, this would mean we'll be requiring 100 x 3 x 8 =
  2400 cfqq's (3 classes and 8 priorities) in the worst case (as
  opposed to current 24 cfqqs)? This may not be as drastic as it sounds
  as we create cfqq's only on demand and we normally won't have tasks
  with every priority and every class?

I'm primarily interested in having the ability to limit/throttle
buffered I/O on a multiuser system where one heavy I/O user shouldn't be
impacting others and everyone should be getting their allocated share. I
understand thought there are different possible use-cases and the agreed
approach should be limiting any potential use-case and hence having a
consensus is quite important. So, I think a discussion on the topic
might help.

I would also be interested in other Network filesystem topics that have
been already proposed including NFS Ganesha, readdirplus syscall etc. I
have been working on Network filesystems for many years now and recently
started looking into block layer side of things too.


[1] http://comments.gmane.org/gmane.linux.kernel.mm/74805 (Last year's
proposal)
[2] http://www.spinics.net/lists/linux-fsdevel/msg53171.html
[3] http://lwn.net/Articles/516540/


Thanks

-- 
Suresh Jayaraman