Re: Still seeing hangs in xlog_grant_log_space

From: Dave Chinner <david@fromorbit.com>
To: Chris J Arges <chris.j.arges@canonical.com>
Cc: xfs@oss.sgi.com
Subject: Re: Still seeing hangs in xlog_grant_log_space
Date: Thu, 7 Jun 2012 10:49:18 +1000	[thread overview]
Message-ID: <20120607004918.GO22848@dastard> (raw)
In-Reply-To: <33970511.post@talk.nabble.com>

On Wed, Jun 06, 2012 at 08:00:00AM -0700, Chris J Arges wrote:
> 
> 
> Dave Chinner wrote:
> > 
> > It seems unlikely, but if you turn on kmemleak it might find a
> > memory leak or overwrite that is causing this.
> > 
> 
> Running kmemleak on a 3.0.0 kernel results in the following:
> 
> dmesg:
> [ 3855.751393] XFS (sda2): xlog_verify_grant_tail: space >
> BBTOB(tail_blocks)
> [22987.932317] kmemleak: 1 new suspected memory leaks (see
> /sys/kernel/debug/kmemleak)
> 
> kmemleak:
> unreferenced object 0xffff88015612a340 (size 208):
>  comm "copy-files", pid 1483, jiffies 4310560285 (age 16571.656s)
>  hex dump (first 32 bytes):
>     00 f3 1b 57 01 88 ff ff 00 08 00 58 01 88 ff ff  ...W.......X....
>     a0 a7 32 a0 ff ff ff ff 04 00 00 00 00 00 00 00  ..2.............
>  backtrace:
>     [<ffffffff815c6196>] kmemleak_alloc+0x26/0x50
>     [<ffffffff811542e3>] kmem_cache_alloc+0x123/0x190
>     [<ffffffffa0313967>] kmem_zone_alloc+0x67/0xe0 [xfs]
>     [<ffffffffa03139fd>] kmem_zone_zalloc+0x1d/0x50 [xfs]
>     [<ffffffffa02b033f>] xfs_allocbt_init_cursor+0xdf/0x130 [xfs]
>     [<ffffffffa02adb4c>] xfs_alloc_ag_vextent_near+0x6c/0xd80 [xfs]
>     [<ffffffffa02aea88>] xfs_alloc_ag_vextent+0x228/0x290 [xfs]
>     [<ffffffffa02af7d9>] xfs_alloc_vextent+0x649/0x8c0 [xfs]
>     [<ffffffffa02bcfc6>] xfs_bmap_btalloc+0x286/0x7c0 [xfs]
>     [<ffffffffa02bd521>] xfs_bmap_alloc+0x21/0x40 [xfs]
>     [<ffffffffa02c6ba3>] xfs_bmapi+0xdc3/0x1950 [xfs]
>     [<ffffffffa02f5059>] xfs_iomap_write_allocate+0x179/0x340 [xfs]
>     [<ffffffffa03147d5>] xfs_map_blocks+0x215/0x380 [xfs]
>     [<ffffffffa0315792>] xfs_vm_writepage+0x1b2/0x510 [xfs]
>     [<ffffffff811142e7>] __writepage+0x17/0x40
>     [<ffffffff8111485d>] write_cache_pages+0x20d/0x460
> 
> analysis:
>     I turned on kmemleak, function tracing, and xfs debugging in the build
> in which I ran this. So far I’ve been able to run the copy-files script for
> about 24 hrs  without failure. I’m not sure if this is because all these
> features are turned on and it has slowed something down (so it takes longer
> to reproduce), or if the debugging code is changing the behavior. I’m not
> sure if this backtrace is valid, so I’m attaching an annotated objdump of my
> xfs module.

OK, this is not what we are looking for. Yes, there's a cursor leak
that I can find just by looking at the code, but that's definitely
not related the AIL issue....

I'll send a patch to fix this leak soon.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs