From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1752181AbcELBLp (ORCPT ); Wed, 11 May 2016 21:11:45 -0400 Received: from szxga01-in.huawei.com ([58.251.152.64]:4660 "EHLO szxga01-in.huawei.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751807AbcELBLn (ORCPT ); Wed, 11 May 2016 21:11:43 -0400 Subject: Re: [BUG]Writeback Cgroup/Dirty Throttle: very small buffered write thoughput caused by writeback cgroup and dirty thottle To: Fengguang Wu , Tejun Heo References: <57333E75.3080309@huawei.com> CC: From: Miao Xie Message-ID: <5733D845.2030709@huawei.com> Date: Thu, 12 May 2016 09:11:33 +0800 User-Agent: Mozilla/5.0 (Windows NT 6.1; WOW64; rv:38.0) Gecko/20100101 Thunderbird/38.1.0 MIME-Version: 1.0 In-Reply-To: <57333E75.3080309@huawei.com> Content-Type: text/plain; charset="gbk"; format=flowed Content-Transfer-Encoding: 7bit X-Originating-IP: [10.177.234.67] X-CFilter-Loop: Reflected X-Mirapoint-Virus-RAPID-Raw: score=unknown(0), refid=str=0001.0A020201.5733D84B.0144,ss=1,re=0.000,recu=0.000,reip=0.000,cl=1,cld=1,fgs=0, ip=0.0.0.0, so=2013-06-18 04:22:30, dmn=2013-03-21 17:37:32 X-Mirapoint-Loop-Id: 92955e6da1a5553ed5600e498944c684 Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Cc linux-kernel mail list on 2016/5/11 at 22:15, Miao Xie wrote: > Hi, Tejun and Fengguang > > I found that buffered write thoughput was dropped down by writeback cgroup and dirty thottle on > 4.6-rc7 kernel. If I ran benchmark on the top block cgroup, the thoughput was more than 1500MB/s. > If I ran benchmark on a new block cgroup, the thoughput was down to 4MB/s. > > Steps to reproduce: > # mount -t cgroup2 cgroup > # echo "+io +memory" > /cgroup.subtree_control > # mkdir /aaa > # echo $$ > /aaa/cgroup.procs > # fio test.config > job0: (g=0): rw=write, bs=4K-4K/4K-4K/4K-4K, ioengine=psync, iodepth=1 > fio-2.2.8 > Starting 1 thread > Jobs: 1 (f=1): [W(1)] [3.7% done] [0KB/4000KB/0KB /s] [0/1000/0 iops] [eta 04m:50s] > > Fio configuration is: > [global] > bs=4K > direct=0 > ioengine=psync > iodepth=1 > directory=/mnt/ext4/tstdir0 > time_based > runtime=300 > group_reporting > size=16G > sync=0 > max_latency=120000000 > thread > > [job0] > numjobs=1 > rw=write > > My box has 48 cores and 188GB memory, but I set > vm.dirty_background_bytes = 268435456 > vm.dirty_bytes = 536870912 > > if I set vm.dirty_background_bytes and vm.dirty_bytes to be a large number(vm.dirty_background_bytes = 3GB, > vm.dirty_bytes = 4GB), then fio thoughput would be more than 1500MB/s. and then if I reset them to the original > value(the above ones), the thoughout would be down to 500MB/s. > > And according my debug, I found fio sleeped for 1ms every time we dirty a page(balance dirty pages) when > the thoughput was down to 4MB/s, it might be a bug of dirty throttle when we open write back cgroup, I think. > > Tejun and Fengguang, please let me know what you guys think about this issue, and if you have > any suggestions for possible solutions, Any input is greatly appreciated! > > Thanks > Miao