Re: xfs_iunlink_remove: xfs_inotobp() returned error 22 -- debugging

[Eric Sandeen <sandeen@sandeen.net>]

On 4/18/13 8:23 AM, 符永涛 wrote:
> Hi Brian and Eric,
> The shutdown is not easy to produce but finally right now 2 of our servers in our test cluster xfs was shutdown.
> 
> the trace output as following
> https://docs.google.com/file/d/0B7n2C4T5tfNCLXRYUWJ0b19JcWc/edit?usp=sharing
> 
> Sorry but the systemtap is interrupt and I didn't noticed that so I didn't get systemtap logs.
> 
> /var/log/message is same as before
> Apr 18 22:43:14 10 kernel: XFS (sdb): 	: xfs_inotobp() returned error 22.
> Apr 18 22:43:14 10 kernel: XFS (sdb): xfs_inactive: xfs_ifree returned error 22
> Apr 18 22:43:14 10 kernel: XFS (sdb): xfs_do_force_shutdown(0x1) called from line 1184 of file fs/xfs/xfs_vnodeops.c.  Return address = 0xffffffffa02d44aa
> Apr 18 22:43:14 10 kernel: XFS (sdb): I/O Error Detected. Shutting down filesystem
> Apr 18 22:43:14 10 kernel: XFS (sdb): Please umount the filesystem and rectify the problem(s)
> Apr 18 22:43:20 10 kernel: XFS (sdb): xfs_log_force: error 5 returned.
> 
> The metadump file is large I'll share it to you soon.
> 

Thanks, we'll take a look.  Just to double check, in the kernel that ran the tracepoints, did you use brian's 2nd version of the patch?  I want to make sure the tracepoints were at the top of the function.

Since you're patching xfs anyway, can you add something like this for next time:

diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c
index 796edce..cad0e8e 100644
--- a/fs/xfs/xfs_inode.c
+++ b/fs/xfs/xfs_inode.c
@@ -1777,8 +1777,9 @@ xfs_iunlink_remove(
 					    &last_ibp, &last_offset, 0);
 			if (error) {
 				xfs_warn(mp,
-					"%s: xfs_inotobp() returned error %d.",
-					__func__, error);
+					"%s: xfs_inotobp() returned error %d "
+					"for inode 0x%llx ag %d agino %x\n",
+					__func__, error, ip->i_ino, agno, agino);
 				return error;
 			}
 			next_agino = be32_to_cpu(last_dip->di_next_unlinked);

so that when we encounter the error we're sure to have the problematic inode number.

Thanks,
-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs