How to recover data from an xfs partition

How to recover data from an xfs partition

[Niccolò Belli]

Hi,
I had a (software) problem with my xen server ended ruining my domU 
partitions. I already recovered all my ext4 partitions using fsck.ext4, 
but I can't recover the only xfs partition I have :(

When I try to mount I get:

root@firewall:~# mount -o loop,offset=$((97280*1024)) -t xfs 
/srv/xen/webserver.img /mnt
mount: Structure needs cleaning



While when trying to fsck I get:

root@firewall:~# xfs_check /dev/loop51
xfs_check: /dev/loop51 is not a valid XFS filesystem (unexpected SB 
magic number 0xeb639010)
xfs_check: size check failed
xfs_check: WARNING - filesystem uses v1 dirs,limited functionality provided.
cache_node_purge: refcount was 1, not zero (node=0x11f0820)
xfs_check: cannot read root inode (22)
cache_node_purge: refcount was 1, not zero (node=0x11f0970)
xfs_check: cannot read realtime bitmap inode (22)
xfs_check: size check failed
xfs_check: WARNING - filesystem uses v1 dirs,limited functionality provided.
bad superblock magic number eb639010, giving up



xfs_check:

root@firewall:~# xfs_check /dev/loop51
xfs_check: /dev/loop51 is not a valid XFS filesystem (unexpected SB 
magic number 0xeb639010)
xfs_check: size check failed
xfs_check: WARNING - filesystem uses v1 dirs,limited functionality provided.
cache_node_purge: refcount was 1, not zero (node=0x22f1820)
xfs_check: cannot read root inode (22)
cache_node_purge: refcount was 1, not zero (node=0x22f1970)
xfs_check: cannot read realtime bitmap inode (22)
xfs_check: size check failed
xfs_check: WARNING - filesystem uses v1 dirs,limited functionality provided.
bad superblock magic number eb639010, giving up




xfs_repair:

root@firewall:~# xfs_repair /dev/loop51
Phase 1 - find and verify superblock...
bad primary superblock - bad magic number !!!

attempting to find secondary superblock...
................................................................................found 
candidate secondary superblock...
unable to verify superblock, continuing...
.............
[...]
....................................Sorry, could not find valid 
secondary superblock
Exiting now.


How can I restore my data? I cannot believe I had no problems restoring 
*15* ext4 partitions while I can't restore the only xfs partition I have :(

Hope you can help,
Niccolò

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: How to recover data from an xfs partition

[Ben Myers]

Niccolo,

On Sat, May 12, 2012 at 07:28:31PM +0200, Niccolò Belli wrote:
> root@firewall:~# xfs_repair /dev/loop51
> Phase 1 - find and verify superblock...
> bad primary superblock - bad magic number !!!
> 
> attempting to find secondary superblock...
> ................................................................................found
> candidate secondary superblock...
> unable to verify superblock, continuing...
> .............
> [...]
> ....................................Sorry, could not find valid
> secondary superblock
> Exiting now.
> 
> 
> How can I restore my data?

I recommend you give it a shot with the latest xfsprogs.  However, it may be
that your filesystem has been corrupted beyond repair.

-Ben

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: How to recover data from an xfs partition

[Niccolò Belli]

Hi,

Il 18/05/2012 16:31, Ben Myers ha scritto:
> I recommend you give it a shot with the latest xfsprogs.  However, it may be
> that your filesystem has been corrupted beyond repair.

I succeeded recovering something with xfsprogs 3.1.8, but no more than 
1GB of data (~5%).

Fortunately I managed to save some precious config files, considering I 
had a 6 months old backup the disaster is a bit less catastrophic.

I had a bad time replacing corrupted data from the other ten ext4 
virtual machines, but they didn't give me too much troubles compared to 
xfs. How ironic that the data loss happened during the installation of 
the backup system...

To be honest I don't know if I will use xfs anymore, unfortunately I 
need per-directory quota and it is the only viable solution.

Niccolò

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: How to recover data from an xfs partition

[Eric Sandeen]

On 5/18/12 10:01 AM, Niccolò Belli wrote:
> Hi,
> 
> Il 18/05/2012 16:31, Ben Myers ha scritto:
>> I recommend you give it a shot with the latest xfsprogs.  However, it may be
>> that your filesystem has been corrupted beyond repair.
> 
> I succeeded recovering something with xfsprogs 3.1.8, but no more than 1GB of data (~5%).
> 
> Fortunately I managed to save some precious config files, considering I had a 6 months old backup the disaster is a bit less catastrophic.
> 
> I had a bad time replacing corrupted data from the other ten ext4 virtual machines, but they didn't give me too much troubles compared to xfs. How ironic that the data loss happened during the installation of the backup system...
> 
> To be honest I don't know if I will use xfs anymore, unfortunately I need per-directory quota and it is the only viable solution.

You're welcome to follow your heart on choice of filesystem of course, but without knowing what happened to your storage, it's a little bit of a stretch to blame xfs.

Just based on:

> xfs_check: /dev/loop51 is not a valid XFS filesystem (unexpected SB magic number 0xeb639010) 

it appears that something scribbled all over your storage (i.e. the file backing it - what was that stored on?)  What it was that did the scribbling, and how much it scribbled is unknown, and it may or may not have been more significant than whatever hit your ext4 partitions.

I do not think you encountered an xfs bug; something else severely damaged your block device.

-Eric


> Niccolò
> 
> _______________________________________________
> xfs mailing list
> xfs@oss.sgi.com
> http://oss.sgi.com/mailman/listinfo/xfs
> 

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: How to recover data from an xfs partition

[Niccolò Belli]

Il 18/05/2012 19:46, Eric Sandeen ha scritto:
> the file backing it - what was that stored on?

It was a raw image created with dd and stored on an SSD (ext4).
Some of the virtual machines were on traditional hard disks instead (ext4).

Only the running virtual machines got damaged, the main filesystems are 
perfect.

> What it was that did the scribbling

Rebooting. A simple, stupid reboot after upgrading Xen. While rebooting 
the system I got some kind of kernel panic and I had to use magic sysrq 
keys (remount read-only, sync, reboot).

Niccolò

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: How to recover data from an xfs partition

[Eric Sandeen]

On 5/18/12 3:57 PM, Niccolò Belli wrote:
> Il 18/05/2012 19:46, Eric Sandeen ha scritto:
>> the file backing it - what was that stored on?
> 
> It was a raw image created with dd and stored on an SSD (ext4). Some
> of the virtual machines were on traditional hard disks instead
> (ext4).
> 
> Only the running virtual machines got damaged, the main filesystems
> are perfect.
> 
>> What it was that did the scribbling
> 
> Rebooting. A simple, stupid reboot after upgrading Xen. While
> rebooting the system I got some kind of kernel panic and I had to use
> magic sysrq keys (remount read-only, sync, reboot).

Well - somehow the first sector of the xfs image got junk written into it...
hard to say how that happened, but the magic number lives in the
first sector of the xfs image hosted on ext4, and it was junk post-reboot.

I have never seen xfs corrupt _itself_ this way.  TBH I'd be as likely to
blame the SSD, as anything.  But you also have to have all of the data
integrity bits set up properly all along the virt stack to survive
a panic or power loss w/o corruption after the fact.

-Eric

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: How to recover data from an xfs partition

[Niccolò Belli]

Il 19/05/2012 00:47, Eric Sandeen ha scritto:
> TBH I'd be as likely to
> blame the SSD, as anything.

I didn't even need to fsck the SSD's partition... Something gone mad in 
the Xen code and it corrupted my data, that was the problem.
I managed to rescue the vast majority of the data in my ext4 partitions, 
but not in the xfs one. Who should I blame for this? xfs? the fate? I 
really don't know it, I'm just reporting what happened compared to 
another fs but I cannot guarantee the bug threated all the partitions 
the same way :)

Niccolò

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: How to recover data from an xfs partition

[Michael Monnerie]

[-- Attachment #1.1: Type: text/plain, Size: 444 bytes --]

Am Freitag, 18. Mai 2012, 22:57:15 schrieb Niccolò Belli:
> I had to use magic sysrq 
> keys (remount read-only, sync, reboot).

Shouldn't that be sync, remount r/o, reboot instead? If you remount r/o, 
the cache in the system wouldn't be written anymore.

-- 
mit freundlichen Grüssen,
Michael Monnerie, Ing. BSc

it-management Internet Services: Protéger
http://proteger.at [gesprochen: Prot-e-schee]
Tel: +43 660 / 415 6531

[-- Attachment #1.2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

[-- Attachment #2: Type: text/plain, Size: 121 bytes --]

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs

Re: How to recover data from an xfs partition

[Niccolò Belli]

Il 21/05/2012 09:44, Michael Monnerie ha scritto:
> Shouldn't that be sync, remount r/o, reboot instead? If you remount r/o,
> the cache in the system wouldn't be written anymore.

Yes, that's what I did (S,U,B).

Niccolò

_______________________________________________
xfs mailing list
xfs@oss.sgi.com
http://oss.sgi.com/mailman/listinfo/xfs