From mboxrd@z Thu Jan 1 00:00:00 1970 From: Andrea Righi Subject: [RFC PATCH 1/3] fsio-throttle: documentation Date: Fri, 18 Jan 2019 11:31:25 +0100 Message-ID: <20190118103127.325-2-righi.andrea@gmail.com> References: <20190118103127.325-1-righi.andrea@gmail.com> Return-path: DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20161025; h=from:to:cc:subject:date:message-id:in-reply-to:references; bh=ydxFy/2Bm4LOh73jyZznaFZU/ozC3N50GLlK25DN1HY=; b=uZF7UkoWqAOS8InqNGoU+Xy56e0FXChmxbtUDcxGtitnfvoq5J69RWhz0T4GLa1XN7 ic2JWQNYvNwQ7FNdzgTmFlYPuyuMYePIvJOTAkt1rq1qv3npOIlJKnEnuduR5WWdHEMU MlEV6xYWBmZqHHmJfgKOCTyUrPl9QLTy0f69SqrgJg39k0nzLx+lxkK10qryRH6si66h SstsLC8pwuIVUh/PIvbFTJRHxY8UMxObgeOZiAcdSLK0QxKEj/cCnxFDfynRiCt2uUpM bEcteokvSKR89Sej9dLKUFm5hBLlwGls8IDK7sEpaqI46nshsh4x2VRpezr1CsIKYPsC DZKw== In-Reply-To: <20190118103127.325-1-righi.andrea@gmail.com> Sender: linux-kernel-owner@vger.kernel.org List-ID: MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: Tejun Heo , Li Zefan , Johannes Weiner Cc: Jens Axboe , Vivek Goyal , Josef Bacik , Dennis Zhou , cgroups@vger.kernel.org, linux-block@vger.kernel.org, linux-kernel@vger.kernel.org, Andrea Righi Document the filesystem I/O controller: description, usage, design, etc. Signed-off-by: Andrea Righi --- Documentation/cgroup-v1/fsio-throttle.txt | 142 ++++++++++++++++++++++ 1 file changed, 142 insertions(+) create mode 100644 Documentation/cgroup-v1/fsio-throttle.txt diff --git a/Documentation/cgroup-v1/fsio-throttle.txt b/Documentation/cgroup-v1/fsio-throttle.txt new file mode 100644 index 000000000000..4f33cae2adea --- /dev/null +++ b/Documentation/cgroup-v1/fsio-throttle.txt @@ -0,0 +1,142 @@ + + Filesystem I/O throttling controller + +---------------------------------------------------------------------- +1. OVERVIEW + +This controller allows to limit filesystem I/O of mounted devices of specific +process containers (cgroups [1]) enforcing delays to the processes that exceed +the limits defined for their cgroup. + +The goal of the filesystem I/O controller is to improve performance +predictability from applications' point of view and provide performance +isolation of different control groups sharing the same filesystems. + +---------------------------------------------------------------------- +2. DESIGN + +I/O activity generated by READs is evaluated at the block layer, WRITEs are +evaluated when a page changes from clear to dirty (rewriting a page that was +already dirty doesn't generate extra I/O activity). + +Throttling is always performed at the VFS layer. + +This solution has the advantage of always being able to determine the +task/cgroup that originally generated the I/O request and it prevents +filesystem locking contention and potential priority inversion problems +(example: journal I/O being throttled that may slow down the entire system). + +The downside of this solution is that the controller is more fuzzy (compared to +the blkio controller) and it allows I/O bursts that may happen at the I/O +scheduler layer. + +---------------------------------------------------------------------- +2.1. TOKEN BUCKET THROTTLING + + Tokens (I/O rate) - + o + o + o + ....... <--. + \ / | Bucket size (burst limit) + \ooo/ | + --- <--' + |ooo + Incoming --->|---> Conforming + I/O |oo I/O + requests -->|--> requests + | + ---->| + +Token bucket [2] throttling: tokens are added to the bucket +every seconds; the bucket can hold at the most tokens; I/O +requests are accepted if there are available tokens in the bucket; when a +request of N bytes arrives, N tokens are removed from the bucket; if less than +N tokens are available in the bucket, the request is delayed until a sufficient +amount of token is available again in the bucket. + +---------------------------------------------------------------------- +3. USER INTERFACE + +A new I/O limit (in MB/s) can be defined using the file: +- fsio.max_mbs + +The syntax of a throttling policy is the following: + +": " + +Examples: + +- set a maximum I/O rate of 10MB/s on /dev/sda (8:0), bucket size = 10MB: + + # echo "8:0 10 10" > /sys/fs/cgroup/cg1/fsio.max_mbs + +- remove the I/O limit defined for /dev/sda (8:0): + + # echo "8:0 0 0" > /sys/fs/cgroup/cg1/fsio.max_mbs + +---------------------------------------------------------------------- +4. Additional parameters + +---------------------------------------------------------------------- +4.1. Sleep timeslice + +Sleep timeslice is a configurable parameter that allows to decide the minimum +time of sleep to enforce to throttled tasks. Tasks will never be put to sleep +for less than the sleep timeslice. Moreover wakeup timers will be always +aligned to a multiple of the sleep timeslice. + +Increasing the sleep timeslice has the advantage of reducing the overhead of +the controller: with a more coarse-grained control, less timers are created to +wake-up tasks, that means less softirq pressure in the system and less overhead +introduced. However, a bigger sleep timeslice makes the controller more fuzzy +since throttled tasks are going to receive less throttling events with larger +sleeps. + +The parameter can be changed via: +/sys/module/fsio_throttle/parameters/throttle_timeslice_ms + +The default value is 250 ms. + +Example: + - set the sleep timeslice to 1s: + + # echo 1000 > /sys/module/fsio_throttle/parameters/throttle_timeslice_ms + +---------------------------------------------------------------------- +4.2. Sleep timeframe + +This parameter defines maximum time to sleep for a throttled task. + +The parameter can be changed via: +/sys/module/fsio_throttle/parameters/throttle_timeslice_ms + +The default value is 2 sec. + +Example: + - set the sleep timeframe to 5s: + + # echo 5000 > /sys/module/fsio_throttle/parameters/throttle_timeframe_ms + +4.3. Throttle kernel threads + +By default kernel threads are never throttled or accounted for any I/O +activity. It is possible to change this behavior by setting 1 to: + +/sys/module/fsio_throttle/parameters/throttle_kernel_threads + +It is strongly recommended to not change this setting unless you know what you +are doing. + +---------------------------------------------------------------------- +5. TODO + +- Integration with the blkio controller +- Provide distinct read/write limits, as well as MBs vs iops +- Provide additional statistics in cgroupfs + +---------------------------------------------------------------------- +6. REFERENCES + +[1] Documentation/admin-guide/cgroup-v2.rst +[2] http://en.wikipedia.org/wiki/Token_bucket -- 2.17.1