From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S1756521AbYEKDs3 (ORCPT ); Sat, 10 May 2008 23:48:29 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S1754382AbYEKDsS (ORCPT ); Sat, 10 May 2008 23:48:18 -0400 Received: from E23SMTP06.au.ibm.com ([202.81.18.175]:56868 "EHLO e23smtp06.au.ibm.com" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1754271AbYEKDsQ (ORCPT ); Sat, 10 May 2008 23:48:16 -0400 Message-ID: <48266C77.3040102@linux.vnet.ibm.com> Date: Sun, 11 May 2008 09:18:07 +0530 From: Kamalesh Babulal User-Agent: Thunderbird 1.5.0.14ubu (X11/20080505) MIME-Version: 1.0 To: pvp-lsts@fs.ru.acad.bg CC: Alexander Beregalov , kernel-testers@vger.kernel.org, kernel list , Ingo Molnar , peterz@infradead.org, xfs@oss.sgi.com, David Chinner Subject: Re: 2.6.26-rc1: possible circular locking dependency with xfs filesystem References: <4825DF71.1030209@linux.vnet.ibm.com> In-Reply-To: <4825DF71.1030209@linux.vnet.ibm.com> Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 7bit Sender: linux-kernel-owner@vger.kernel.org List-ID: X-Mailing-List: linux-kernel@vger.kernel.org Kamalesh Babulal wrote: > Adding the cc to kernel-list, Ingo Molnar and Peter Zijlstra > > Alexander Beregalov wrote: >> [ INFO: possible circular locking dependency detected ] >> 2.6.26-rc1-00279-g28a4acb #13 >> ------------------------------------------------------- >> nfsd/3087 is trying to acquire lock: >> (iprune_mutex){--..}, at: [] shrink_icache_memory+0x38/0x19b >> >> but task is already holding lock: >> (&(&ip->i_iolock)->mr_lock){----}, at: [] xfs_ilock+0xa2/0xd6 >> >> which lock already depends on the new lock. >> >> >> the existing dependency chain (in reverse order) is: >> >> -> #1 (&(&ip->i_iolock)->mr_lock){----}: >> [] __lock_acquire+0xa0c/0xbc6 >> [] lock_acquire+0x6a/0x86 >> [] down_write_nested+0x33/0x6a >> [] xfs_ilock+0x7b/0xd6 >> [] xfs_ireclaim+0x1d/0x59 >> [] xfs_finish_reclaim+0x173/0x195 >> [] xfs_reclaim+0xb3/0x138 >> [] xfs_fs_clear_inode+0x55/0x8e >> [] clear_inode+0x83/0xd2 >> [] dispose_list+0x3c/0xc1 >> [] shrink_icache_memory+0x173/0x19b >> [] shrink_slab+0xda/0x14e >> [] try_to_free_pages+0x1e4/0x2a2 >> [] __alloc_pages_internal+0x23a/0x39d >> [] __alloc_pages+0xa/0xc >> [] __do_page_cache_readahead+0xaa/0x16a >> [] force_page_cache_readahead+0x4a/0x74 >> [] sys_madvise+0x308/0x400 >> [] sysenter_past_esp+0x6a/0xb1 >> [] 0xffffffff >> >> -> #0 (iprune_mutex){--..}: >> [] __lock_acquire+0x929/0xbc6 >> [] lock_acquire+0x6a/0x86 >> [] mutex_lock_nested+0xb4/0x226 >> [] shrink_icache_memory+0x38/0x19b >> [] shrink_slab+0xda/0x14e >> [] try_to_free_pages+0x1e4/0x2a2 >> [] __alloc_pages_internal+0x23a/0x39d >> [] __alloc_pages+0xa/0xc >> [] __do_page_cache_readahead+0xaa/0x16a >> [] ondemand_readahead+0x119/0x127 >> [] page_cache_async_readahead+0x52/0x5d >> [] generic_file_splice_read+0x290/0x4a8 >> [] xfs_splice_read+0x4b/0x78 >> [] xfs_file_splice_read+0x24/0x29 >> [] do_splice_to+0x45/0x63 >> [] splice_direct_to_actor+0xab/0x150 >> [] nfsd_vfs_read+0x1ed/0x2d0 >> [] nfsd_read+0x82/0x99 >> [] nfsd3_proc_read+0xdf/0x12a >> [] nfsd_dispatch+0xcf/0x19e >> [] svc_process+0x3b3/0x68b >> [] nfsd+0x168/0x26b >> [] kernel_thread_helper+0x7/0x10 >> [] 0xffffffff >> >> other info that might help us debug this: >> >> 3 locks held by nfsd/3087: >> #0: (hash_sem){..--}, at: [] exp_readlock+0xd/0xf >> #1: (&(&ip->i_iolock)->mr_lock){----}, at: [] xfs_ilock+0xa2/0xd6 >> #2: (shrinker_rwsem){----}, at: [] shrink_slab+0x24/0x14e >> >> stack backtrace: >> Pid: 3087, comm: nfsd Not tainted 2.6.26-rc1-00279-g28a4acb #13 >> [] print_circular_bug_tail+0x5a/0x65 >> [] ? print_circular_bug_header+0xa8/0xb3 >> [] __lock_acquire+0x929/0xbc6 >> [] ? native_sched_clock+0x8b/0x9f >> [] lock_acquire+0x6a/0x86 >> [] ? shrink_icache_memory+0x38/0x19b >> [] mutex_lock_nested+0xb4/0x226 >> [] ? shrink_icache_memory+0x38/0x19b >> [] ? shrink_icache_memory+0x38/0x19b >> [] shrink_icache_memory+0x38/0x19b >> [] shrink_slab+0xda/0x14e >> [] try_to_free_pages+0x1e4/0x2a2 >> [] ? _spin_unlock_irqrestore+0x36/0x58 >> [] ? isolate_pages_global+0x0/0x3e >> [] __alloc_pages_internal+0x23a/0x39d >> [] __alloc_pages+0xa/0xc >> [] __do_page_cache_readahead+0xaa/0x16a >> [] ondemand_readahead+0x119/0x127 >> [] page_cache_async_readahead+0x52/0x5d >> [] generic_file_splice_read+0x290/0x4a8 >> [] ? _spin_unlock+0x27/0x3c >> [] ? _atomic_dec_and_lock+0x25/0x30 >> [] ? iput+0x24/0x4e >> [] ? __lock_acquire+0xbaa/0xbc6 >> [] ? exportfs_decode_fh+0x9b/0x1a1 >> [] ? spd_release_page+0x0/0xf >> [] xfs_splice_read+0x4b/0x78 >> [] xfs_file_splice_read+0x24/0x29 >> [] do_splice_to+0x45/0x63 >> [] splice_direct_to_actor+0xab/0x150 >> [] ? nfsd_direct_splice_actor+0x0/0xf >> [] nfsd_vfs_read+0x1ed/0x2d0 >> [] nfsd_read+0x82/0x99 >> [] nfsd3_proc_read+0xdf/0x12a >> [] nfsd_dispatch+0xcf/0x19e >> [] svc_process+0x3b3/0x68b >> [] nfsd+0x168/0x26b >> [] ? nfsd+0x0/0x26b >> [] kernel_thread_helper+0x7/0x10 >> -- Adding the trimmed forward message of syslog from Plamen Petrov May 9 02:16:46 nomad64 kernel: [42951853.992912] May 9 02:16:46 nomad64 kernel: [42951853.992913] ======================================================= May 9 02:16:46 nomad64 kernel: [42951853.992920] [ INFO: possible circular locking dependency detected ] May 9 02:16:46 nomad64 kernel: [42951853.992922] 2.6.26-rc1-00243-g46e4965 #1 May 9 02:16:46 nomad64 kernel: [42951853.992924] ------------------------------------------------------- May 9 02:16:46 nomad64 kernel: [42951853.992927] kio_http/3813 is trying to acquire lock: May 9 02:16:46 nomad64 kernel: [42951853.992930] (&mm->mmap_sem){----}, at: [] do_page_fault+0xdd/0x890 May 9 02:16:46 nomad64 kernel: [42951853.992944] May 9 02:16:46 nomad64 kernel: [42951853.992944] but task is already holding lock: May 9 02:16:46 nomad64 kernel: [42951853.992947] (&(&ip->i_iolock)->mr_lock){----}, at: [] xfs_ilock+0x65/0xa0 May 9 02:16:46 nomad64 kernel: [42951853.992960] May 9 02:16:46 nomad64 kernel: [42951853.992960] which lock already depends on the new lock. May 9 02:16:46 nomad64 kernel: [42951853.992961] May 9 02:16:46 nomad64 kernel: [42951853.992964] May 9 02:16:46 nomad64 kernel: [42951853.992965] the existing dependency chain (in reverse order) is: May 9 02:16:46 nomad64 kernel: [42951853.992967] May 9 02:16:46 nomad64 kernel: [42951853.992968] -> #1 (&(&ip->i_iolock)->mr_lock){----}: May 9 02:16:46 nomad64 kernel: [42951853.992974] [] __lock_acquire+0xf92/0x1080 May 9 02:16:46 nomad64 kernel: [42951853.992989] [] lock_acquire+0xa2/0xd0 May 9 02:16:46 nomad64 kernel: [42951853.993002] [] down_write_nested+0x46/0x80 May 9 02:16:46 nomad64 kernel: [42951853.993018] [] xfs_ilock+0x99/0xa0 May 9 02:16:46 nomad64 kernel: [42951853.993034] [] xfs_free_eofblocks+0x1c7/0x250 May 9 02:16:46 nomad64 kernel: [42951853.993049] [] xfs_release+0x186/0x1d0 May 9 02:16:46 nomad64 kernel: [42951853.993062] [] xfs_file_release+0x10/0x20 May 9 02:16:46 nomad64 kernel: [42951853.993076] [] __fput+0xcc/0x1c0 May 9 02:16:46 nomad64 kernel: [42951853.993091] [] fput+0x16/0x20 May 9 02:16:46 nomad64 kernel: [42951853.993105] [] remove_vma+0x4a/0x80 May 9 02:16:46 nomad64 kernel: [42951853.993120] [] do_munmap+0x281/0x2e0 May 9 02:16:46 nomad64 kernel: [42951853.993134] [] sys_munmap+0x4b/0x70 May 9 02:16:46 nomad64 kernel: [42951853.993148] [] system_call_after_swapgs+0x7b/0x80 May 9 02:16:46 nomad64 kernel: [42951853.993161] [] 0xffffffffffffffff May 9 02:16:46 nomad64 kernel: [42951853.993178] May 9 02:16:46 nomad64 kernel: [42951853.993178] -> #0 (&mm->mmap_sem){----}: May 9 02:16:46 nomad64 kernel: [42951853.993185] [] __lock_acquire+0xdb0/0x1080 May 9 02:16:46 nomad64 kernel: [42951853.993197] [] lock_acquire+0xa2/0xd0 May 9 02:16:46 nomad64 kernel: [42951853.993213] [] down_read+0x3b/0x70 May 9 02:16:46 nomad64 kernel: [42951853.993228] [] do_page_fault+0xdd/0x890 May 9 02:16:46 nomad64 kernel: [42951853.993241] [] error_exit+0x0/0xa9 May 9 02:16:46 nomad64 kernel: [42951853.993256] [] 0xffffffffffffffff May 9 02:16:46 nomad64 kernel: [42951853.993269] May 9 02:16:46 nomad64 kernel: [42951853.993270] other info that might help us debug this: May 9 02:16:46 nomad64 kernel: [42951853.993270] May 9 02:16:46 nomad64 kernel: [42951853.993273] 1 lock held by kio_http/3813: May 9 02:16:46 nomad64 kernel: [42951853.993275] #0: (&(&ip->i_iolock)->mr_lock){----}, at: [] xfs_ilock+0x65/0xa0 May 9 02:16:46 nomad64 kernel: [42951853.993286] May 9 02:16:46 nomad64 kernel: [42951853.993287] stack backtrace: May 9 02:16:46 nomad64 kernel: [42951853.993290] Pid: 3813, comm: kio_http Not tainted 2.6.26-rc1-00243-g46e4965 #1 May 9 02:16:46 nomad64 kernel: [42951853.993292] May 9 02:16:46 nomad64 kernel: [42951853.993293] Call Trace: May 9 02:16:46 nomad64 kernel: [42951853.993297] [] print_circular_bug_tail+0x83/0x90 May 9 02:16:46 nomad64 kernel: [42951853.993302] [] __lock_acquire+0xdb0/0x1080 May 9 02:16:46 nomad64 kernel: [42951853.993306] [] ? do_page_fault+0xdd/0x890 May 9 02:16:46 nomad64 kernel: [42951853.993310] [] lock_acquire+0xa2/0xd0 May 9 02:16:46 nomad64 kernel: [42951853.993313] [] ? do_page_fault+0xdd/0x890 May 9 02:16:46 nomad64 kernel: [42951853.993317] [] down_read+0x3b/0x70 May 9 02:16:46 nomad64 kernel: [42951853.993320] [] do_page_fault+0xdd/0x890 May 9 02:16:46 nomad64 kernel: [42951853.993324] [] error_exit+0x0/0xa9 May 9 02:16:46 nomad64 kernel: [42951853.993328] [] ? file_read_actor+0x46/0x1b0 May 9 02:16:46 nomad64 kernel: [42951853.993331] [] ? _read_unlock_irq+0x36/0x60 May 9 02:16:46 nomad64 kernel: [42951853.993335] [] ? generic_file_aio_read+0x2cc/0x5d0 May 9 02:16:46 nomad64 kernel: [42951853.993339] [] ? get_lock_stats+0x19/0x70 May 9 02:16:46 nomad64 kernel: [42951853.993343] [] ? xfs_read+0x139/0x220 May 9 02:16:46 nomad64 kernel: [42951853.993347] [] ? xfs_file_aio_read+0x4d/0x60 May 9 02:16:46 nomad64 kernel: [42951853.993350] [] ? do_sync_read+0xf1/0x130 May 9 02:16:46 nomad64 kernel: [42951853.993354] [] ? autoremove_wake_function+0x0/0x40 May 9 02:16:46 nomad64 kernel: [42951853.993358] [] ? trace_hardirqs_on+0xda/0x170 May 9 02:16:46 nomad64 kernel: [42951853.993361] [] ? __rcu_read_unlock+0xb5/0xc0 May 9 02:16:46 nomad64 kernel: [42951853.993365] [] ? trace_hardirqs_on+0xda/0x170 May 9 02:16:46 nomad64 kernel: [42951853.993369] [] ? security_file_permission+0x11/0x20 May 9 02:16:46 nomad64 kernel: [42951853.993374] [] ? vfs_read+0xc4/0x160 May 9 02:16:46 nomad64 kernel: [42951853.993377] [] ? sys_read+0x50/0x90 May 9 02:16:46 nomad64 kernel: [42951853.993380] [] ? system_call_after_swapgs+0x7b/0x80 -- Thanks & Regards, Kamalesh Babulal, Linux Technology Center, IBM, ISTL.