Re: xfs_iunlink_remove: xfs_inotobp() returned error 22 -- debugging

符永涛
2013-04-17T03:15:42Z

[-- Attachment #1.1: Type: text/plain, Size: 6262 bytes --]

Hi Brain,
If it is because NULLAGINO is passed in to xfs_inotobp().
Can I move the following two lines before xfs_inotobp?

For example:

1767 while (next_agino != agino) {
1768 /*
1769 * If the last inode wasn't the one pointing to
1770 * us, then release its buffer since we're not
1771 * going to do anything with it.
1772 */
1773 if (last_ibp != NULL) {
1774 xfs_trans_brelse(tp, last_ibp);
1775 }
1776 next_ino = XFS_AGINO_TO_INO(mp, agno,
next_agino);
+ ASSERT(next_agino != NULLAGINO);
+ ASSERT(next_agino != 0);
1777 error = xfs_inotobp(mp, tp, next_ino,
&last_dip,
1778 &last_ibp, &last_offset,
0);
1779 if (error) {
1780 xfs_warn(mp,
1781 "%s: xfs_inotobp() returned
error %d.",
1782 __func__, error);
1783 return error;
1784 }
1785 next_agino =
be32_to_cpu(last_dip->di_next_unlinked);
- //ASSERT(next_agino != NULLAGINO);
- //ASSERT(next_agino != 0);
1788 }
I don't understand xfs well and correct me if I'm totally wrong.
Thank you very much.

2013/4/17 符永涛 <yongtaofu@gmail.com>

> Hi Brain,
> I want to ask a question, according to the shutdown trace. The ino in xfs_iunlink_remove
> is 0x113, why xfs_imap got ino=0xffffffff ?
>
> --- xfs_imap -- module("xfs").function("xfs_imap@fs/xfs/xfs_ialloc.c:1257").return
> -- return=0x16
> vars: mp=0xffff882017a50800 tp=0xffff881c81797c70 ino=0xffffffff
>
> --- xfs_iunlink_remove -- module("xfs").function("xfs_iunlink_remove@fs
> /xfs/xfs_inode.c:1680").return -- return=0x16
> vars: tp=0xffff881c81797c70 ip=0xffff881003c13c00 next_ino=? mp=? agi=?
> dip=? agibp=0xffff880109b47e20 ibp=? agno=? agino=? next_agino=? last_ibp=?
> last_dip=0xffff882000000000 bucket_index=? offset=?
> last_offset=0xffffffffffff8810 error=? __func__=[...]
> ip: i_ino = 0x113, i_flags = 0x0
>
> Thank you.
>
>
>
> 2013/4/17 符永涛 <yongtaofu@gmail.com>
>
>> Hi Brain,
>> Thank you for your update, and I have applied your last kernel patch.
>> However it is not easy to reproduce especially in out test environment.
>> Till now is not happens again. I'll update the kernel patch now. BTW is
>> there any findings in the logs of previous thread?
>> http://oss.sgi.com/archives/xfs/2013-04/msg00327.html
>> I guess it tend to happen during glusterfs rebalance because glusterfs
>> moves a lot of file from one server to another and then unlink it.
>>
>> Thank you.
>>
>>
>> 2013/4/17 Brian Foster <bfoster@redhat.com>
>>
>>> On 04/16/2013 12:24 PM, Dave Chinner wrote:
>>> > On Mon, Apr 15, 2013 at 07:14:39PM -0400, Brian Foster wrote:
>>> >> Hi,
>>> >>
>>> >> Thanks for the data in the previous thread:
>>> >>
>>> >> http://oss.sgi.com/archives/xfs/2013-04/msg00327.html
>>> >>
>>> ...
>>> >>
>>> >> echo 1 > /sys/kernel/debug/tracing/events/xfs/xfs_iunlink/enable
>>> >> echo 1 >
>>> /sys/kernel/debug/tracing/events/xfs/xfs_iunlink_remove/enable
>>> >> ... reproduce ...
>>> >> cat /sys/kernel/debug/tracing/trace > trace.output
>>> >
>>> > It's better to use trace-cmd for this. it will result in less
>>> > dropped events. i.e.:
>>> >
>>> > $ trace-cmd record -e xfs_iunlink\*
>>> > ... reproduce ...
>>> > ^C
>>> > $ trace-cmd report > trace.output
>>> >
>>> >> --- a/fs/xfs/linux-2.6/xfs_trace.h
>>> >> +++ b/fs/xfs/linux-2.6/xfs_trace.h
>>> >> @@ -581,6 +581,8 @@ DEFINE_INODE_EVENT(xfs_file_fsync);
>>> ...
>>> >
>>> > I would suggest that the the tracing shoul dbe at entry of the
>>> > function, otherwise we won't get a tracepoint for the operation that
>>> > triggers the shutdown. (That's the reason most tracepoints in XFS
>>> > are at function entry...)
>>> >
>>>
>>> Good points, thanks Dave. A v2 that pulls up the tracepoints towards
>>> function entry is appended.
>>>
>>> Brian
>>>
>>> From 280943e78ebe0b97a774cba51e7815c42f044b55 Mon Sep 17 00:00:00 2001
>>> From: Brian Foster <bfoster@redhat.com>
>>> Date: Mon, 15 Apr 2013 18:16:24 -0400
>>> Subject: [PATCH v2] xfs: add tracepoints for xfs_iunlink and
>>> xfs_iunlink_remove
>>>
>>> ---
>>> fs/xfs/linux-2.6/xfs_trace.h | 2 ++
>>> fs/xfs/xfs_inode.c | 4 ++++
>>> 2 files changed, 6 insertions(+), 0 deletions(-)
>>>
>>> diff --git a/fs/xfs/linux-2.6/xfs_trace.h b/fs/xfs/linux-2.6/xfs_trace.h
>>> index adc6ec4..338a0f9 100644
>>> --- a/fs/xfs/linux-2.6/xfs_trace.h
>>> +++ b/fs/xfs/linux-2.6/xfs_trace.h
>>> @@ -583,6 +583,8 @@ DEFINE_INODE_EVENT(xfs_file_fsync);
>>> DEFINE_INODE_EVENT(xfs_destroy_inode);
>>> DEFINE_INODE_EVENT(xfs_dirty_inode);
>>> DEFINE_INODE_EVENT(xfs_clear_inode);
>>> +DEFINE_INODE_EVENT(xfs_iunlink);
>>> +DEFINE_INODE_EVENT(xfs_iunlink_remove);
>>>
>>> DEFINE_INODE_EVENT(xfs_dquot_dqalloc);
>>> DEFINE_INODE_EVENT(xfs_dquot_dqdetach);
>>> diff --git a/fs/xfs/xfs_inode.c b/fs/xfs/xfs_inode.c
>>> index 19900f0..d705c77 100644
>>> --- a/fs/xfs/xfs_inode.c
>>> +++ b/fs/xfs/xfs_inode.c
>>> @@ -1615,6 +1615,8 @@ xfs_iunlink(
>>>
>>> mp = tp->t_mountp;
>>>
>>> + trace_xfs_iunlink(ip);
>>> +
>>> /*
>>> * Get the agi buffer first. It ensures lock ordering
>>> * on the list.
>>> @@ -1694,6 +1696,8 @@ xfs_iunlink_remove(
>>> mp = tp->t_mountp;
>>> agno = XFS_INO_TO_AGNO(mp, ip->i_ino);
>>>
>>> + trace_xfs_iunlink_remove(ip);
>>> +
>>> /*
>>> * Get the agi buffer first. It ensures lock ordering
>>> * on the list.
>>> --
>>> 1.7.7.6
>>>
>>>
>>
>>
>> --
>> 符永涛
>>
>
>
>
> --
> 符永涛
>

--
符永涛

[-- Attachment #1.2: Type: text/html, Size: 13559 bytes --]