Corruption of in-memory data

Corruption of in-memory data

[Florian Hines]

Hey everybody,

Over the last few day's I've been getting a rash (8 so far) of disk's
throwing the error "Filesystem "sdxX": Corruption of in-memory data
detected.  Shutting down filesystem: sdxX".  xfs_check never seem's to
find anything and just unmounting and remounting solves the issue at
least for awhile.   Is this usually caused by bad ram ?  It's happened
on 6 systems so far (all using Debian Etch AMD64 with the stock 2.6.18
kernel, each system as a 5 sata drives, not raided).

Can anyone shed some light on what this error actually indicates for me  ?

--Full error from dmesg below--
 Filesystem "sda3": XFS internal error xfs_trans_cancel at line 1138 of
file fs/xfs/xfs_trans.c.  Caller 0xffffffff8818ead5

Call Trace:
 [<ffffffff8818fdd0>] :xfs:xfs_trans_cancel+0x5b/0xfe
 [<ffffffff8818ead5>] :xfs:xfs_rename+0xa13/0xa9a
 [<ffffffff881a0aac>] :xfs:xfs_vn_rename+0x2c/0x6f
 [<ffffffff80220104>] __up_read+0x13/0x8a
 [<ffffffff8817c732>] :xfs:xfs_iunlock+0x57/0x79
 [<ffffffff80220104>] __up_read+0x13/0x8a
 [<ffffffff8817c732>] :xfs:xfs_iunlock+0x57/0x79
 [<ffffffff88195286>] :xfs:xfs_access+0x3d/0x46
 [<ffffffff80228cc4>] vfs_rename+0x2d5/0x426
 [<ffffffff802344fb>] sys_renameat+0x180/0x1f9
 [<ffffffff80221605>] sys_newstat+0x28/0x31
 [<ffffffff80257c16>] system_call+0x7e/0x83

xfs_force_shutdown(sda3,0x8) called from line 1139 of file
fs/xfs/xfs_trans.c.  Return address = 0xffffffff8818fdee
Filesystem "sda3": Corruption of in-memory data detected.  Shutting down
filesystem: sda3
Please umount the filesystem, and rectify the problem(s)


--
flo

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Corruption of in-memory data

[Eric Sandeen]

Florian Hines wrote:
> Hey everybody,
> 
> Over the last few day's I've been getting a rash (8 so far) of disk's
> throwing the error "Filesystem "sdxX": Corruption of in-memory data
> detected.  Shutting down filesystem: sdxX".  xfs_check never seem's to
> find anything and just unmounting and remounting solves the issue at
> least for awhile.   

try xfs_repair, with -n if you want a dry run.

It's canceling a dirty transaction, I'm not sure why.  The message is a
little misleading.  Want to try 2.6.29? :)

-Eric

> Is this usually caused by bad ram ?  It's happened
> on 6 systems so far (all using Debian Etch AMD64 with the stock 2.6.18
> kernel, each system as a 5 sata drives, not raided).
> 
> Can anyone shed some light on what this error actually indicates for me  ?
> 
> --Full error from dmesg below--
>  Filesystem "sda3": XFS internal error xfs_trans_cancel at line 1138 of
> file fs/xfs/xfs_trans.c.  Caller 0xffffffff8818ead5
> 
> Call Trace:
>  [<ffffffff8818fdd0>] :xfs:xfs_trans_cancel+0x5b/0xfe
>  [<ffffffff8818ead5>] :xfs:xfs_rename+0xa13/0xa9a
>  [<ffffffff881a0aac>] :xfs:xfs_vn_rename+0x2c/0x6f
>  [<ffffffff80220104>] __up_read+0x13/0x8a
>  [<ffffffff8817c732>] :xfs:xfs_iunlock+0x57/0x79
>  [<ffffffff80220104>] __up_read+0x13/0x8a
>  [<ffffffff8817c732>] :xfs:xfs_iunlock+0x57/0x79
>  [<ffffffff88195286>] :xfs:xfs_access+0x3d/0x46
>  [<ffffffff80228cc4>] vfs_rename+0x2d5/0x426
>  [<ffffffff802344fb>] sys_renameat+0x180/0x1f9
>  [<ffffffff80221605>] sys_newstat+0x28/0x31
>  [<ffffffff80257c16>] system_call+0x7e/0x83
> 
> xfs_force_shutdown(sda3,0x8) called from line 1139 of file
> fs/xfs/xfs_trans.c.  Return address = 0xffffffff8818fdee
> Filesystem "sda3": Corruption of in-memory data detected.  Shutting down
> filesystem: sda3
> Please umount the filesystem, and rectify the problem(s)
> 
> 
> --
> flo
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Corruption of in-memory data

[Florian Hines]

Eric Sandeen wrote:
> Florian Hines wrote:
>> Hey everybody,
>>
>> Over the last few day's I've been getting a rash (8 so far) of disk's
>> throwing the error "Filesystem "sdxX": Corruption of in-memory data
>> detected.  Shutting down filesystem: sdxX".  xfs_check never seem's to
>> find anything and just unmounting and remounting solves the issue at
>> least for awhile.   
> 
> try xfs_repair, with -n if you want a dry run.
> 
> It's canceling a dirty transaction, I'm not sure why.  The message is a
> little misleading.  Want to try 2.6.29? :)

I'll give xfs_repair a run next time I get one.  Thanks!

Florian

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: Corruption of in-memory data

[Martin Steigerwald]

Am Freitag 27 März 2009 schrieb Florian Hines:
> Hey everybody,

Hi,

> Over the last few day's I've been getting a rash (8 so far) of disk's
> throwing the error "Filesystem "sdxX": Corruption of in-memory data
> detected.  Shutting down filesystem: sdxX".  xfs_check never seem's to
> find anything and just unmounting and remounting solves the issue at
> least for awhile.   Is this usually caused by bad ram ?  It's happened
> on 6 systems so far (all using Debian Etch AMD64 with the stock 2.6.18
> kernel, each system as a 5 sata drives, not raided).
>
> Can anyone shed some light on what this error actually indicates for me
>  ?
>
> --Full error from dmesg below--
>  Filesystem "sda3": XFS internal error xfs_trans_cancel at line 1138 of
> file fs/xfs/xfs_trans.c.  Caller 0xffffffff8818ead5
>
> Call Trace:
>  [<ffffffff8818fdd0>] :xfs:xfs_trans_cancel+0x5b/0xfe
>  [<ffffffff8818ead5>] :xfs:xfs_rename+0xa13/0xa9a
>  [<ffffffff881a0aac>] :xfs:xfs_vn_rename+0x2c/0x6f
>  [<ffffffff80220104>] __up_read+0x13/0x8a
>  [<ffffffff8817c732>] :xfs:xfs_iunlock+0x57/0x79
>  [<ffffffff80220104>] __up_read+0x13/0x8a
>  [<ffffffff8817c732>] :xfs:xfs_iunlock+0x57/0x79
>  [<ffffffff88195286>] :xfs:xfs_access+0x3d/0x46
>  [<ffffffff80228cc4>] vfs_rename+0x2d5/0x426
>  [<ffffffff802344fb>] sys_renameat+0x180/0x1f9
>  [<ffffffff80221605>] sys_newstat+0x28/0x31
>  [<ffffffff80257c16>] system_call+0x7e/0x83
>
> xfs_force_shutdown(sda3,0x8) called from line 1139 of file
> fs/xfs/xfs_trans.c.  Return address = 0xffffffff8818fdee
> Filesystem "sda3": Corruption of in-memory data detected.  Shutting
> down filesystem: sda3

Well we had "Corruption of in-memory data detected" errors with Debian AMD 
64 and the 2.6.22 backports.org kernel. They went away after we upgraded 
to 2.6.26 backports.org kernel. Don't remember the trace tough anymore. I 
posted it on the mailinglist... I think not a long time ago.

Well here is:

Is it possible the check an frozen XFS filesytem to avoid downtime? 
2008-07-14 (was longer ago than I expected ;).

http://oss.sgi.com/archives/xfs/2008-07/msg01475.html

Backtrace look different, but that doesn't have to mean much. 

I would try a newer kernel!

I also recommend having a newer version of xfsprogs at hand in case of 
problems. The one in Etch is completely out-dated. I have made a backport 
back then which is still quite recent:

http://people.teamix.net/~ms/debian/etch-backports/xfsprogs/

Ciao,
-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs