Re: xfs trace in 4.4.2 / also in 4.3.3 WARNING fs/xfs/xfs_aops.c:1232 xfs_vm_releasepage - Stefan Priebe

From: Stefan Priebe - Profihost AG <s.priebe@profihost.ag>
To: Brian Foster <bfoster@redhat.com>
Cc: linux-fsdevel@vger.kernel.org,
	"xfs-masters@oss.sgi.com" <xfs-masters@oss.sgi.com>,
	"xfs@oss.sgi.com" <xfs@oss.sgi.com>
Subject: Re: xfs trace in 4.4.2 / also in 4.3.3 WARNING fs/xfs/xfs_aops.c:1232 xfs_vm_releasepage
Date: Thu, 24 Mar 2016 09:10:41 +0100	[thread overview]
Message-ID: <56F3A101.1020300@profihost.ag> (raw)
In-Reply-To: <20160323140736.GD43073@bfoster.bfoster>

Am 23.03.2016 um 15:07 schrieb Brian Foster:
> On Wed, Mar 23, 2016 at 02:28:03PM +0100, Stefan Priebe - Profihost AG wrote:
>> sorry new one the last one got mangled. Comments inside.
>>
>> Am 05.03.2016 um 23:48 schrieb Dave Chinner:
>>> On Fri, Mar 04, 2016 at 04:03:42PM -0500, Brian Foster wrote:
>>>> On Fri, Mar 04, 2016 at 09:02:06PM +0100, Stefan Priebe wrote:
>>>>> Am 04.03.2016 um 20:13 schrieb Brian Foster:
>>>>>> On Fri, Mar 04, 2016 at 07:47:16PM +0100, Stefan Priebe wrote:
>>>>>>> Am 20.02.2016 um 19:02 schrieb Stefan Priebe - Profihost AG:
>>>>>>>>
>>>>>>>>> Am 20.02.2016 um 15:45 schrieb Brian Foster <bfoster@redhat.com>:
>>>>>>>>>
>>>>>>>>>> On Sat, Feb 20, 2016 at 09:02:28AM +0100, Stefan Priebe wrote:
> ...
>>
>> This has happened again on 8 different hosts in the last 24 hours
>> running 4.4.6.
>>
>> All of those are KVM / Qemu hosts and are doing NO I/O except the normal
>> OS stuff as the VMs have remote storage. So no database, no rsync on
>> those hosts - just the OS doing nearly nothing.
>>
>> All those show:
>> [153360.287040] WARNING: CPU: 0 PID: 109 at fs/xfs/xfs_aops.c:1234
>> xfs_vm_releasepage+0xe2/0xf0()
>>
> 
> Ok, well at this point the warning isn't telling us anything beyond
> you're reproducing the problem. We can't really make progress without
> more information. We don't necessarily know what application or
> operations caused this by the time it occurs, but perhaps knowing what
> file is affected could give us a hint.
> 
> We have the xfs_releasepage tracepoint, but that's unconditional and so
> might generate a lot of noise by default. Could you enable the
> xfs_releasepage tracepoint and hunt for instances where delalloc != 0?
> E.g., we could leave a long running 'trace-cmd record -e
> "xfs:xfs_releasepage" <cmd>' command on several boxes and wait for the
> problem to occur. Alternatively (and maybe easier), run 'trace-cmd start
> -e "xfs:xfs_releasepage"' and leave something like 'cat
> /sys/kernel/debug/tracing/trace_pipe | grep -v "delalloc 0" >
> ~/trace.out' running to capture instances.
> 
> If we can get a tracepoint hit, it will include the inode number and
> something like 'find / -inum <ino>' can point us at the file.

thanks - need to compile trace-cmd first. Do you know if and how it
influences performance?

Stefan

> 
> Brian
> 
>> Stefan
>>
>>>
>>> -Dave.
>>>
>>
>> _______________________________________________
>> xfs mailing list
>> xfs@oss.sgi.com
>> http://oss.sgi.com/mailman/listinfo/xfs

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs