From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from mail-ph.de-nserver.de ([85.158.179.214]:49213 "EHLO mail-ph.de-nserver.de" rhost-flags-OK-OK-OK-OK) by vger.kernel.org with ESMTP id S1752592AbcCXIKt (ORCPT ); Thu, 24 Mar 2016 04:10:49 -0400 Subject: Re: xfs trace in 4.4.2 / also in 4.3.3 WARNING fs/xfs/xfs_aops.c:1232 xfs_vm_releasepage To: Brian Foster References: <56C81D94.7090603@profihost.ag> <20160220144533.GA36182@bfoster.bfoster> <56D9D834.2000303@profihost.ag> <20160304191329.GC3758@bfoster.bfoster> <56D9E9BE.40101@profihost.ag> <20160304210341.GA8035@bfoster.bfoster> <20160305224845.GR30721@dastard> <56F299E3.4020703@profihost.ag> <20160323140736.GD43073@bfoster.bfoster> Cc: Dave Chinner , linux-fsdevel@vger.kernel.org, "xfs-masters@oss.sgi.com" , "xfs@oss.sgi.com" From: Stefan Priebe - Profihost AG Message-ID: <56F3A101.1020300@profihost.ag> Date: Thu, 24 Mar 2016 09:10:41 +0100 MIME-Version: 1.0 In-Reply-To: <20160323140736.GD43073@bfoster.bfoster> Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 7bit Sender: linux-fsdevel-owner@vger.kernel.org List-ID: Am 23.03.2016 um 15:07 schrieb Brian Foster: > On Wed, Mar 23, 2016 at 02:28:03PM +0100, Stefan Priebe - Profihost AG wrote: >> sorry new one the last one got mangled. Comments inside. >> >> Am 05.03.2016 um 23:48 schrieb Dave Chinner: >>> On Fri, Mar 04, 2016 at 04:03:42PM -0500, Brian Foster wrote: >>>> On Fri, Mar 04, 2016 at 09:02:06PM +0100, Stefan Priebe wrote: >>>>> Am 04.03.2016 um 20:13 schrieb Brian Foster: >>>>>> On Fri, Mar 04, 2016 at 07:47:16PM +0100, Stefan Priebe wrote: >>>>>>> Am 20.02.2016 um 19:02 schrieb Stefan Priebe - Profihost AG: >>>>>>>> >>>>>>>>> Am 20.02.2016 um 15:45 schrieb Brian Foster : >>>>>>>>> >>>>>>>>>> On Sat, Feb 20, 2016 at 09:02:28AM +0100, Stefan Priebe wrote: > ... >> >> This has happened again on 8 different hosts in the last 24 hours >> running 4.4.6. >> >> All of those are KVM / Qemu hosts and are doing NO I/O except the normal >> OS stuff as the VMs have remote storage. So no database, no rsync on >> those hosts - just the OS doing nearly nothing. >> >> All those show: >> [153360.287040] WARNING: CPU: 0 PID: 109 at fs/xfs/xfs_aops.c:1234 >> xfs_vm_releasepage+0xe2/0xf0() >> > > Ok, well at this point the warning isn't telling us anything beyond > you're reproducing the problem. We can't really make progress without > more information. We don't necessarily know what application or > operations caused this by the time it occurs, but perhaps knowing what > file is affected could give us a hint. > > We have the xfs_releasepage tracepoint, but that's unconditional and so > might generate a lot of noise by default. Could you enable the > xfs_releasepage tracepoint and hunt for instances where delalloc != 0? > E.g., we could leave a long running 'trace-cmd record -e > "xfs:xfs_releasepage" ' command on several boxes and wait for the > problem to occur. Alternatively (and maybe easier), run 'trace-cmd start > -e "xfs:xfs_releasepage"' and leave something like 'cat > /sys/kernel/debug/tracing/trace_pipe | grep -v "delalloc 0" > > ~/trace.out' running to capture instances. > > If we can get a tracepoint hit, it will include the inode number and > something like 'find / -inum ' can point us at the file. thanks - need to compile trace-cmd first. Do you know if and how it influences performance? Stefan > > Brian > >> Stefan >> >>> >>> -Dave. >>> >> >> _______________________________________________ >> xfs mailing list >> xfs@oss.sgi.com >> http://oss.sgi.com/mailman/listinfo/xfs