Re: XFS disaster recovery

[Dave Chinner <david@fromorbit.com>]

On Tue, Feb 08, 2022 at 10:46:45AM -0500, Sean Caron wrote:
> Hi Dave,
> 
> I'm sorry for some imprecise language. The array is around 450 TB raw
> and I will refer to it as roughly half a petabyte but factoring out
> RAID parity disks and spare disks it should indeed be around 384 TB
> formatted.

Ah, OK, looks like it was a complete dump, then.

> I found that if I ran the dev tree xfs_repair with the -P option, I
> could get xfs_repair to complete a run. It exits with return code 130
> but the resulting loopback image filesystem is mountable and I see
> around 27 TB in lost+found which would represent around 9% loss in
> terms of what was actually on the filesystem.

I'm sure that if that much ended up in lost+found, xfs_repair also
threw away a whole load of metadata which means data will have been
lost. And with this much metadata corruption occurring, it tends to
imply that there will be widespread data corruption, too.  Hence I
think it's worth pointing out (maybe unnecessarily!) that xfs_repair
doesn't tell you about (or fix) data corruption - it just rebuilds
the metadata back into a consistent state.

> Given where we started I think this is acceptable (more than
> acceptable, IMO, I was getting to the point of expecting to have to
> write off the majority of the filesystem) and it seems like a way
> forward to get the majority of the data off this old filesystem.

Yes, but you are still going to have to verify the data you can
still access is not corrupted - random offsets within files could
now contain garbage regardless of whether the file was moved to
lost+found or not.

> Is there anything further I should check or any caveats that I should
> bear in mind applying this xfs_repair to the real filesystem? Or does
> it seem reasonable to go ahead, repair this and start copying off?

Seems reasonable to repeat the process on the real filesystem, but
given the caveat about data corruption above, I suspect that the
entire dataset on the filesystem might still end up being a complete
write-off.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com