From mboxrd@z Thu Jan 1 00:00:00 1970 From: Vivek Goyal Subject: Re: IO less throttling and cgroup aware writeback (Was: Re: [Lsf] Preliminary Agenda and Activities for LSF) Date: Thu, 31 Mar 2011 18:25:46 -0400 Message-ID: <20110331222546.GD11139@redhat.com> References: <1301373398.2590.20.camel@mulgrave.site> <20110330041802.GA20849@dastard> <20110330153757.GD1291@redhat.com> <20110330222002.GB20849@dastard> <20110331141637.GA11139@redhat.com> <1301581251-sup-987@think> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: Dave Chinner , Chad Talbott , James Bottomley , lsf , linux-fsdevel To: Chris Mason Return-path: Received: from mx1.redhat.com ([209.132.183.28]:3033 "EHLO mx1.redhat.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1751664Ab1CaW0a (ORCPT ); Thu, 31 Mar 2011 18:26:30 -0400 Content-Disposition: inline In-Reply-To: <1301581251-sup-987@think> Sender: linux-fsdevel-owner@vger.kernel.org List-ID: On Thu, Mar 31, 2011 at 10:34:03AM -0400, Chris Mason wrote: [..] > > > > For me knowing this is also important in another context of absolute IO > > throttling. > > > > - If a fsync is in progress and gets throttled at device, what impact it > > has on other file system operations. What gets serialized behind it. > > It depends. atime updates log inodes and logging needs a transaction > and transactions sometimes need to wait for the last transaction to > finish. So its very possible you'll make anything using the FS appear > to stop. I think I have run into this. I created a cgroup and gave ridiculously low limit of 1bytes/sec and did a fsync. This process blocks. Later I did "ls" in the directory where fsync process is blocked and ls also hangs. Following is backtrace. Looks like atime update led to some kind of blocking in do_get_write_access(). ls D ffffffff8160b060 0 5936 5192 0x00000000 ffff880138729c48 0000000000000086 0000000000000000 0000000100000010 0000000000000000 ffff88013fc40100 ffff88013ac7ac00 000000012e5d70f3 ffff8801353d7af8 ffff880138729fd8 000000000000f558 ffff8801353d7af8 Call Trace: [] do_get_write_access+0x29d/0x500 [jbd2] [] ? wake_bit_function+0x0/0x50 [] jbd2_journal_get_write_access+0x31/0x50 [jbd2] [] __ext4_journal_get_write_access+0x38/0x80 [ext4] [] ext4_reserve_inode_write+0x73/0xa0 [ext4] [] ? call_filldir+0x78/0xe0 [ext4] [] ext4_mark_inode_dirty+0x4c/0x1d0 [ext4] [] ? __do_page_fault+0x1e4/0x480 [] ext4_dirty_inode+0x40/0x60 [ext4] [] __mark_inode_dirty+0x3b/0x160 [] touch_atime+0x12d/0x170 [] ? filldir+0x0/0xe0 [] vfs_readdir+0xd6/0xe0 [] sys_getdents+0x89/0xf0 [] ? page_fault+0x25/0x30 [] system_call_fastpath+0x16/0x1b The vim process doing fsync trace is as follows. This is waiting for some IO to finish which has been throttled at the device. vim D ffffffff8110d1f0 0 5934 4452 0x00000000 ffff880107e2dcc8 0000000000000086 0000000100000000 0000000000000003 ffff8801351f4538 0000000000000000 ffff880107e2dc68 ffffffff810e7da2 ffff8801353d70b8 ffff880107e2dfd8 000000000000f558 ffff8801353d70b8 Call Trace: [] ? ring_buffer_lock_reserve+0xa2/0x160 [] ? ktime_get_ts+0xa9/0xe0 [] ? sync_page+0x0/0x50 [] io_schedule+0x73/0xc0 [] sync_page+0x3d/0x50 [] __wait_on_bit+0x5f/0x90 [] wait_on_page_bit+0x73/0x80 [] ? wake_bit_function+0x0/0x50 [] ? pagevec_lookup_tag+0x25/0x40 [] wait_on_page_writeback_range+0xfb/0x190 [] ? do_writepages+0x21/0x40 [] ? __filemap_fdatawrite_range+0x5b/0x60 [] filemap_write_and_wait_range+0x78/0x90 [] vfs_fsync_range+0x7e/0xe0 [] vfs_fsync+0x1d/0x20 [] do_fsync+0x3e/0x60 [] sys_fsync+0x10/0x20 [] system_call_fastpath+0x16/0x1b Thanks Vivek