2.6.34-rc3: inode 0x401afe0 background reclaim flush failed with 11

2.6.34-rc3: inode 0x401afe0 background reclaim flush failed with 11

[Christian Kujau]

Hi,

while running some filesystem benchmarks, this happend in my logs when 
bonnie++ was running:

[14610.114155] Filesystem "md0": inode 0x401afe0 background reclaim flush failed with 11
[14610.114171] Filesystem "md0": inode 0x401afe1 background reclaim flush failed with 11
[14610.114183] Filesystem "md0": inode 0x401afe2 background reclaim flush failed with 11
[...]

...and so forth for a couple of inodes.

I can reproduce this pretty reliably with bonnie++ now. This did not 
happen with 2.6.33, but the bonnie++ version has been upgraded too, so I'm 
still not sure if this is a real regression.

I've put a few details on http://nerdbynature.de/bits/2.6.34-rc3/xfs/

Is this something to worry about?

Thanks,
Christian.

PS: Why is the inode shown in hex and not in decimal? Would something 
like this do:

diff --git a/fs/xfs/linux-2.6/xfs_sync.c b/fs/xfs/linux-2.6/xfs_sync.c
index 05cd853..30c7bcb 100644
--- a/fs/xfs/linux-2.6/xfs_sync.c
+++ b/fs/xfs/linux-2.6/xfs_sync.c
@@ -825,7 +825,7 @@ xfs_reclaim_inode(
 	 */
 	if (error && !XFS_FORCED_SHUTDOWN(ip->i_mount)) {
 		xfs_fs_cmn_err(CE_WARN, ip->i_mount,
-			"inode 0x%llx background reclaim flush failed with %d",
+			"inode 0x%llu background reclaim flush failed with %d",
 			(long long)ip->i_ino, error);
 	}
 out:


[0] http://nerdbynature.de/benchmarks/v40z/2010-04-06/
-- 
BOFH excuse #329:

Server depressed, needs Prozac

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: 2.6.34-rc3: inode 0x401afe0 background reclaim flush failed with 11

[Dave Chinner]

On Fri, Apr 09, 2010 at 02:05:14PM -0700, Christian Kujau wrote:
> Hi,
> 
> while running some filesystem benchmarks, this happend in my logs when 
> bonnie++ was running:
> 
> [14610.114155] Filesystem "md0": inode 0x401afe0 background reclaim flush failed with 11
> [14610.114171] Filesystem "md0": inode 0x401afe1 background reclaim flush failed with 11
> [14610.114183] Filesystem "md0": inode 0x401afe2 background reclaim flush failed with 11
> [...]
> 
> ...and so forth for a couple of inodes.
> 
> I can reproduce this pretty reliably with bonnie++ now. This did not 
> happen with 2.6.33, but the bonnie++ version has been upgraded too, so I'm 
> still not sure if this is a real regression.
> 
> I've put a few details on http://nerdbynature.de/bits/2.6.34-rc3/xfs/
> 
> Is this something to worry about?

No.

http://git.kernel.org/?p=linux/kernel/git/dgc/xfs.git;a=commit;h=7bb6049804717d4aa1f43f2abb50691c0df1d9f2

> 
> Thanks,
> Christian.
> 
> PS: Why is the inode shown in hex and not in decimal? Would something 
> like this do:

Because I find that large inode numbers in hex are much easier to
understand than huge decimal numbers. The inode number is a direct
encoding of it's location on disk and these days I can generally
decode them in my head direct from the hex value. IOWs, the first
thing I almost always do when looking at an inode number is convert
it to hex, so I don't see any point in printing them in decimal...

e.g. without knowing the geometry of the filesystem, I'd guess that
inode 0x401afe0 is inode 0x20 (32) of an inode allocation chunk,
it's AG 2, 4, 8 or 16 (depends on the size of the AGs), and the
block offset into the AG is 0xd7e (agbno 3454). From that I know a
lot about the inode - it's the first in an inode cluster buffer and
the other inodes reported are in the same buffer hence it's only one
one busy buffer that caused the warnings, the agbno is small so it's
near the start of the AG so there probably aren't a large number of
inodes in the filesystem, etc.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: 2.6.34-rc3: inode 0x401afe0 background reclaim flush failed with 11

[Christian Kujau]

On Tue, 13 Apr 2010 at 12:49, Dave Chinner wrote:
> No.
> 
> http://git.kernel.org/?p=linux/kernel/git/dgc/xfs.git;a=commit;h=7bb6049804717d4aa1f43f2abb50691c0df1d9f2

Ah, thanks. It's not in -rc4 yet, will it be included in the next release?

> > PS: Why is the inode shown in hex and not in decimal? Would something 
> > like this do:
> 
> Because I find that large inode numbers in hex are much easier to
> understand than huge decimal numbers. The inode number is a direct
> encoding of it's location on disk and these days I can generally
> decode them in my head direct from the hex value.

Hehe, OK. Will learn how to do that too :-)

> IOWs, the first
> thing I almost always do when looking at an inode number is convert
> it to hex, so I don't see any point in printing them in decimal...

I was tempted to "find . -inum" to find out which data might be in trouble 
but then noticed that I had to convert it to decimal first.

> e.g. without knowing the geometry of the filesystem, I'd guess that
> inode 0x401afe0 is inode 0x20 (32) of an inode allocation chunk,
> it's AG 2, 4, 8 or 16 (depends on the size of the AGs), and the
> block offset into the AG is 0xd7e (agbno 3454). From that I know a
> lot about the inode - it's the first in an inode cluster buffer and
> the other inodes reported are in the same buffer hence it's only one
> one busy buffer that caused the warnings, the agbno is small so it's
> near the start of the AG so there probably aren't a large number of
> inodes in the filesystem, etc.

Wow, OK. I respectfully withdraw my proposal then. Thanks for the 
explanation.

Christian.
-- 
BOFH excuse #267:

The UPS is on strike.

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: 2.6.34-rc3: inode 0x401afe0 background reclaim flush failed with 11

[Dave Chinner]

On Mon, Apr 12, 2010 at 07:57:26PM -0700, Christian Kujau wrote:
> On Tue, 13 Apr 2010 at 12:49, Dave Chinner wrote:
> > No.
> > 
> > http://git.kernel.org/?p=linux/kernel/git/dgc/xfs.git;a=commit;h=7bb6049804717d4aa1f43f2abb50691c0df1d9f2
> 
> Ah, thanks. It's not in -rc4 yet, will it be included in the next release?

It's not even in the xfs-dev tree yet. I'm about to gather all the
outstanding reviewed patches into a branch so that they can get
there, but there's a few patches that need to go to linus, too...

> > > PS: Why is the inode shown in hex and not in decimal? Would something 
> > > like this do:
> > 
> > Because I find that large inode numbers in hex are much easier to
> > understand than huge decimal numbers. The inode number is a direct
> > encoding of it's location on disk and these days I can generally
> > decode them in my head direct from the hex value.
> 
> Hehe, OK. Will learn how to do that too :-)
> 
> > IOWs, the first
> > thing I almost always do when looking at an inode number is convert
> > it to hex, so I don't see any point in printing them in decimal...
> 
> I was tempted to "find . -inum" to find out which data might be in trouble 
> but then noticed that I had to convert it to decimal first.

Hmmm - I'd never realised that find doesn't decode hex numbers.
I guess I rarely use find for that purpose. I'll keep that in mind
when adding new output.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: 2.6.34-rc3: inode 0x401afe0 background reclaim flush failed with 11

[Christian Kujau]

For the sake of the archives:

On Tue, 13 Apr 2010 at 14:24, Dave Chinner wrote:
> Hmmm - I'd never realised that find doesn't decode hex numbers.
> I guess I rarely use find for that purpose. I'll keep that in mind
> when adding new output.

No, but standard /bin/sh can do it:

   $ find . -inum $((0xff)) -ls
   255    0 -rw-r--r--   1 root     root      0 Apr 12 21:35 ./0156

I'll have to remember this next time...

Thanks,
Christian.
-- 
BOFH excuse #276:

U.S. Postal Service

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs