linux xfs filesystem corruption

linux xfs filesystem corruption

[Robert]

Dear all,
      I have encoutered linux xfs filesystem corruption, I search this topic 
on the
web, but I can't find any useful information to fix my problem.
      I build a raid5 with 'mdadm' on my own embeded system which is based 
on
arm, and format it with 'mkfs.xfs',  mount it to a directory to be set for 
nfs
share. On the host side, I mount the raid, and start writing data to this 
raid,  for
about 20hours later, I start to delete the data , but the following message 
shows:

XFS internal error XFS_WANT_CORRUPTED_GOTO at line 1610 of file
fs/xfs/xfs_alloc.c, xfs_force_shutdown(md0,0x8) called from line 4073 of 
file fs/xfs/xfs_bmap.c
Filesystem "md0" corruption of in-memory data detected. Shutting down
filesystem.Please umount the filesystem and rectify the problem.

     Although the filesystem crashed, the system did not hang. So I cd to 
this directory,
and ls, and find that all the data seems to be lost,  but 'df' reports that 
the most of
raid has been used, I try to repair it with xfs_repair, and everything is 
OK.
     But now I don't want to fix this problem by xfs_repair, if the problem 
happens again,
I have to reparit it again, it's too trouble, so if there is any way to 
solve this prolbem?
I want to find out what causes this error happend,
   Anybody has experience in this problem?
   The following is my system parameter:
  linux version: 2.6.12.6
  CPU: arm926ej
  memory: 128M
  If you answer this question ,Could you please cc to me? I didn't subscribe 
this mailing list.
 Thanks in advance!

 Robert 

Re: linux xfs filesystem corruption

[Eric Sandeen]

Robert wrote:
> Dear all,
>       I have encoutered linux xfs filesystem corruption, I search this topic 
> on the
> web, but I can't find any useful information to fix my problem.
>       I build a raid5 with 'mdadm' on my own embeded system which is based 
> on
> arm, and format it with 'mkfs.xfs',  mount it to a directory to be set for 
> nfs
> share. On the host side, I mount the raid, and start writing data to this 
> raid,  for
> about 20hours later, I start to delete the data , but the following message 
> shows:
> 
> XFS internal error XFS_WANT_CORRUPTED_GOTO at line 1610 of file
> fs/xfs/xfs_alloc.c, xfs_force_shutdown(md0,0x8) called from line 4073 of 
> file fs/xfs/xfs_bmap.c
> Filesystem "md0" corruption of in-memory data detected. Shutting down
> filesystem.Please umount the filesystem and rectify the problem.

arm has been problematic on several fronts.  The first was with the v2
directory code...

http://oss.sgi.com/archives/xfs/2008-06/msg00064.html

but that doesn't appear to be what you are hitting.  There have been
other problems with cache flushing that I don't remember all the details
of now ...


>      Although the filesystem crashed, the system did not hang. So I cd to 
> this directory,
> and ls, and find that all the data seems to be lost,  but 'df' reports that 
> the most of
> raid has been used, I try to repair it with xfs_repair, and everything is 
> OK.
>      But now I don't want to fix this problem by xfs_repair, if the problem 
> happens again,

and what did xfs_repair find?  Do you have a record of it?

> I have to reparit it again, it's too trouble, so if there is any way to 
> solve this prolbem?
> I want to find out what causes this error happend,
>    Anybody has experience in this problem?
>    The following is my system parameter:
>   linux version: 2.6.12.6

any other patches applied?  Can you try again with a more recent kernel?

-Eric

>   CPU: arm926ej
>   memory: 128M
>   If you answer this question ,Could you please cc to me? I didn't subscribe 
> this mailing list.
>  Thanks in advance!
> 
>  Robert 
> 
> 
> 

Re: linux xfs filesystem corruption

[Robert]

Hi, Eric,
 Thanks for your reply!

>> XFS internal error XFS_WANT_CORRUPTED_GOTO at line 1610 of file
>> fs/xfs/xfs_alloc.c, xfs_force_shutdown(md0,0x8) called from line 4073 of
>> file fs/xfs/xfs_bmap.c
>> Filesystem "md0" corruption of in-memory data detected. Shutting down
>> filesystem.Please umount the filesystem and rectify the problem.
>
> arm has been problematic on several fronts.  The first was with the v2
> directory code...
>
> http://oss.sgi.com/archives/xfs/2008-06/msg00064.html
>
> but that doesn't appear to be what you are hitting.  There have been
> other problems with cache flushing that I don't remember all the details
> of now ...
>

I will try this method to see if there will be effective.

>>      Although the filesystem crashed, the system did not hang. So I cd to
>> this directory,
>> and ls, and find that all the data seems to be lost,  but 'df' reports 
>> that
>> the most of
>> raid has been used, I try to repair it with xfs_repair, and everything is
>> OK.
>>      But now I don't want to fix this problem by xfs_repair, if the 
>> problem
>> happens again,
>
> and what did xfs_repair find?  Do you have a record of it?

I'm sorry that I didn't  capture the outputs, today I try to replay the 
issue again,
and use "xfs_repair" to repair filesytem, but this time it seems that the
filesystem is not repaired successfully, but it still can be used for a 
while.
Here is the output of xfs_repair :

~ # /mnt/xfs_repair -n /dev/md0
        - creating 2 worker thread(s)
Phase 1 - find and verify superblock...
        - reporting progress in intervals of 15 minutes
Killed

>> I have to reparit it again, it's too trouble, so if there is any way to
>> solve this prolbem?
>> I want to find out what causes this error happend,
>>    Anybody has experience in this problem?
>>    The following is my system parameter:
>>   linux version: 2.6.12.6
>
> any other patches applied?  Can you try again with a more recent kernel?
>
 I haven't found any patch of this issue.
 Athough  I find some patches related to this problem from the web,
it has already been applied in my linux kernel.   I have no way to try the 
recent
kernel, because our embeded system is based on Marvell development board,
and if I use the recent kernel,  the Marvell's patch of this kernel is 
required.

Robert

Re: linux xfs filesystem corruption

[Russell Cattelan]

Robert wrote:
> Dear all,
>       I have encoutered linux xfs filesystem corruption, I search this topic 
> on the
> web, but I can't find any useful information to fix my problem.
>       I build a raid5 with 'mdadm' on my own embeded system which is based 
> on
> arm, and format it with 'mkfs.xfs',  mount it to a directory to be set for 
> nfs
> share. On the host side, I mount the raid, and start writing data to this 
> raid,  for
> about 20hours later, I start to delete the data , but the following message 
> shows:
>
>   
Would it be possible to get a xfs meta dump of the corrupted image?

There have been other reports of corrupted fs when running on non x86 
architectures.

It might be helpful to examine the file system to see what has gone wrong.

> XFS internal error XFS_WANT_CORRUPTED_GOTO at line 1610 of file
> fs/xfs/xfs_alloc.c, xfs_force_shutdown(md0,0x8) called from line 4073 of 
> file fs/xfs/xfs_bmap.c
> Filesystem "md0" corruption of in-memory data detected. Shutting down
> filesystem.Please umount the filesystem and rectify the problem.
>
>      Although the filesystem crashed, the system did not hang. So I cd to 
> this directory,
> and ls, and find that all the data seems to be lost,  but 'df' reports that 
> the most of
> raid has been used, I try to repair it with xfs_repair, and everything is 
> OK.
>      But now I don't want to fix this problem by xfs_repair, if the problem 
> happens again,
> I have to reparit it again, it's too trouble, so if there is any way to 
> solve this prolbem?
> I want to find out what causes this error happend,
>    Anybody has experience in this problem?
>    The following is my system parameter:
>   linux version: 2.6.12.6
>   CPU: arm926ej
>   memory: 128M
>   If you answer this question ,Could you please cc to me? I didn't subscribe 
> this mailing list.
>  Thanks in advance!
>
>  Robert 
>
>
>