From mboxrd@z Thu Jan 1 00:00:00 1970 From: Jens Axboe Subject: Re: [PATCH 0/11] Per-bdi writeback flusher threads v9 Date: Fri, 29 May 2009 19:08:33 +0200 Message-ID: <20090529170833.GJ11363@kernel.dk> References: <1243511204-2328-1-git-send-email-jens.axboe@oracle.com> <4A200846.5050109@gmail.com> Mime-Version: 1.0 Content-Type: text/plain; charset=us-ascii Cc: linux-kernel@vger.kernel.org, linux-fsdevel@vger.kernel.org, tytso@mit.edu, chris.mason@oracle.com, david@fromorbit.com, hch@infradead.org, akpm@linux-foundation.org, jack@suse.cz, yanmin_zhang@linux.intel.com, richard@rsk.demon.co.uk, damien.wyart@free.fr To: Artem Bityutskiy Return-path: Content-Disposition: inline In-Reply-To: <4A200846.5050109@gmail.com> Sender: linux-kernel-owner@vger.kernel.org List-Id: linux-fsdevel.vger.kernel.org On Fri, May 29 2009, Artem Bityutskiy wrote: > Jens Axboe wrote: >> Hi, >> >> Here's the 9th version of the writeback patches. Changes since v8: >> >> - Fix a bdi_work on-stack allocation hang. I hope this fixes Ted's >> issue. >> - Get rid of the explicit wait queues, we can just use wake_up_process() >> since it's just for that one task. >> - Add separate "sync_supers" thread that makes sure that the dirty >> super blocks get written. We cannot safely do this from bdi_forker_task(), >> as that risks deadlocking on ->s_umount. Artem, I implemented this >> by doing the wake ups from a timer so that it would be easier for you >> to just deactivate the timer when there are no super blocks. >> >> For ease of patching, I've put the full diff here: >> >> http://kernel.dk/writeback-v9.patch >> >> and also stored this in a writeback-v9 branch that will not change, >> you can pull that into Linus tree from here: >> >> git://git.kernel.dk/linux-2.6-block.git writeback-v9 > > I'm working with the above branch. Got the following twice. > Not sure what triggers this, probably if I do nothing and > cpufreq starts doing its magic, this is triggered. > > And I'm not sure it has something to do with your changes, > it is just that I saw this only with your tree. Please, > ignore if this is not relevant. OK, doesn't look related, but if it only triggers with the writeback patches, something fishy is going on. I'll check up on it. > > ======================================================= > scaling: [ INFO: possible circular locking dependency detected ] > 2.6.30-rc7-block-2.6 #1 > ------------------------------------------------------- > K99cpuspeed/9923 is trying to acquire lock: > (&(&dbs_info->work)->work){+.+...}, at: [] > __cancel_work_timer+0xd9/0x21d > > but task is already holding lock: > (dbs_mutex){+.+.+.}, at: [] cpufreq_governor_dbs+0x23c/0x2cc [cpufreq_ondemand] > > which lock already depends on the new lock. > > > the existing dependency chain (in reverse order) is: > > -> #2 (dbs_mutex){+.+.+.}: > [] __lock_acquire+0xa63/0xbeb > [] lock_acquire+0xee/0x112 > [] __mutex_lock_common+0x5a/0x419 > [] mutex_lock_nested+0x30/0x35 > [] cpufreq_governor_dbs+0x86/0x2cc [cpufreq_ondemand] > [] __cpufreq_governor+0x84/0xc2 > [] __cpufreq_set_policy+0x195/0x211 > [] store_scaling_governor+0x1e7/0x223 > [] store+0x5f/0x83 > [] sysfs_write_file+0xe4/0x119 > [] vfs_write+0xab/0x105 > [] sys_write+0x47/0x70 > [] > system_call_fastpath+0x16/0x1b > [] 0xffffffffffffffff > > -> #1 (&per_cpu(cpu_policy_rwsem, cpu)){+++++.}: > [] __lock_acquire+0xa63/0xbeb > [] lock_acquire+0xee/0x112 > [] down_write+0x3d/0x49 [] > lock_policy_rwsem_write+0x48/0x78 > [] do_dbs_timer+0x5f/0x27f [cpufreq_ondemand] > [] worker_thread+0x24b/0x367 > [] kthread+0x56/0x83 > [] child_rip+0xa/0x20 > [] 0xffffffffffffffff > > -> #0 (&(&dbs_info->work)->work){+.+...}: > [] __lock_acquire+0x957/0xbeb > [] lock_acquire+0xee/0x112 > [] __cancel_work_timer+0x10d/0x21d > [] cancel_delayed_work_sync+0xd/0xf > [] cpufreq_governor_dbs+0x24f/0x2cc [cpufreq_ondemand] > [] __cpufreq_governor+0x84/0xc2 > [] __cpufreq_set_policy+0x17f/0x211 > [] store_scaling_governor+0x1e7/0x223 > [] store+0x5f/0x83 > [] sysfs_write_file+0xe4/0x119 > [] vfs_write+0xab/0x105 > [] sys_write+0x47/0x70 > [] > system_call_fastpath+0x16/0x1b > [] 0xffffffffffffffff > > other info that might help us debug this: > > 3 locks held by K99cpuspeed/9923: > #0: (&buffer->mutex){+.+.+.}, at: [] sysfs_write_file+0x38/0x119 > #1: (&per_cpu(cpu_policy_rwsem, cpu)){+++++.}, at: [] lock_policy_rwsem_write+0x48/0x78 > #2: (dbs_mutex){+.+.+.}, at: [] cpufreq_governor_dbs+0x23c/0x2cc [cpufreq_ondemand] > > stack backtrace: > Pid: 9923, comm: K99cpuspeed Not tainted 2.6.30-rc7-block-2.6 #1 > Call Trace: > [] print_circular_bug_tail+0x71/0x7c > [] __lock_acquire+0x957/0xbeb > [] lock_acquire+0xee/0x112 > [] ? __cancel_work_timer+0xd9/0x21d > [] __cancel_work_timer+0x10d/0x21d > [] ? __cancel_work_timer+0xd9/0x21d > [] ? __mutex_lock_common+0x3c2/0x419 > [] ? cpufreq_governor_dbs+0x23c/0x2cc [cpufreq_ondemand] > [] ? mark_held_locks+0x4d/0x6b > [] ? cpufreq_governor_dbs+0x23c/0x2cc [cpufreq_ondemand] > [] cancel_delayed_work_sync+0xd/0xf > [] cpufreq_governor_dbs+0x24f/0x2cc [cpufreq_ondemand] > [] ? up_read+0x26/0x2b > [] __cpufreq_governor+0x84/0xc2 > [] __cpufreq_set_policy+0x17f/0x211 > [] store_scaling_governor+0x1e7/0x223 > [] ? handle_update+0x0/0x33 > [] ? down_write+0x45/0x49 > [] store+0x5f/0x83 > [] sysfs_write_file+0xe4/0x119 > [] vfs_write+0xab/0x105 > [] sys_write+0x47/0x70 > [] system_call_fastpath+0x16/0x1b > -- Jens Axboe