xfs_metadump as a backup tool

xfs_metadump as a backup tool

[Gionatan Danti]

Hi all,
I wish to ask if someone is using xfs_metadump as a "real" backup tool - 
ie: in production.

I was wondering how to lower recovery time in case of filesystem 
corruption, and the idea of using xfs_metadump + xfs_mdrestore struck 
me.

I plan to have a single, relatively big XFS volume with an handfull of 
fully preallocated large files (VM disk images). Basically, I was 
thinking about:
- take a LVM snapshot;
- mount the snap volume as read-only;
- copy the user data (via tar/rsync/whatever);
- additionally, xfs_metadump its metadata content.

If a major filesystem corruption occour, as a first step I should be 
able to simply restore its metadata with xfs_mdrestore and re-gain 
access to files/data blocks. A potential pitfall would be related to 
extents that at the time of xfs_metadump where marked as "unwritten" but 
later were written - restoring the old metadata will immediately turn 
user data into zeroes. It is possible to fully allocate a file *without* 
marking the allocated extents as "unwritten"? Yes, I know this has 
security implications...

Is this a good idea? I am missing something?
Thanks.

-- 
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8

Re: xfs_metadump as a backup tool

[Eric Sandeen]

On 8/21/17 2:59 PM, Gionatan Danti wrote:
> Hi all,
> I wish to ask if someone is using xfs_metadump as a "real" backup tool - ie: in production.

Short answer is "No," because:
xfs_metadump is only a diagnostic tool.  It is not tested or supported for any other use.

If nothing else, on a normal live filesystem your metadata and data is constantly changing; if you replay old metadata over a filesystem, you'll end up pointing to file data blocks that are incorrect.

If you really truly have only statically-allocated files from start to finish, then maybe... 

But more than anything else, the tool was not designed or tested for anything other than developer diagnostics.

For disaster recovery, having periodic metadata snapshots might come in handy from a forensic POV, but I would not rely on this as a primary part of your normal backup/recovery scheme.

> I was wondering how to lower recovery time in case of filesystem corruption, and the idea of using xfs_metadump + xfs_mdrestore struck me.
> 
> I plan to have a single, relatively big XFS volume with an handfull of fully preallocated large files (VM disk images). Basically, I was thinking about:
> - take a LVM snapshot;
> - mount the snap volume as read-only;
> - copy the user data (via tar/rsync/whatever);
> - additionally, xfs_metadump its metadata content.

If you've already done a proper backup wit tar/rsync/(xfsdump?) then you have everything you need to restore the filesystem without resorting to xfs_metadump cleverness, right?

> If a major filesystem corruption occour, as a first step I should be able to simply restore its metadata with xfs_mdrestore and re-gain access to files/data blocks. A potential pitfall would be related to extents that at the time of xfs_metadump where marked as "unwritten" but later were written - restoring the old metadata will immediately turn user data into zeroes. It is possible to fully allocate a file *without* marking the allocated extents as "unwritten"? Yes, I know this has security implications...

No, it's not possible to preallocate space on xfs without marking it as unwritten.

-Eric

> Is this a good idea? I am missing something?
> Thanks.
> 

Re: xfs_metadump as a backup tool

[Gionatan Danti]

Il 21-08-2017 22:11 Eric Sandeen ha scritto:
> Short answer is "No," because:
> xfs_metadump is only a diagnostic tool.  It is not tested or supported
> for any other use.
> 
> If nothing else, on a normal live filesystem your metadata and data is
> constantly changing; if you replay old metadata over a filesystem,
> you'll end up pointing to file data blocks that are incorrect.
> 
> If you really truly have only statically-allocated files from start to
> finish, then maybe...
> 
> But more than anything else, the tool was not designed or tested for
> anything other than developer diagnostics.
> 
> For disaster recovery, having periodic metadata snapshots might come
> in handy from a forensic POV, but I would not rely on this as a
> primary part of your normal backup/recovery scheme.

Hi Eric,
thank for your reply. From xfs_metadump man page is read that the tool 
should be only used as a debugging aid, but hey - maybe someone used it 
in more clever ways ;)

> If you've already done a proper backup wit tar/rsync/(xfsdump?) then
> you have everything you need to restore the filesystem without
> resorting to xfs_metadump cleverness, right?

Sure, but restoring metadata on a corrupted, but healty, volume would be 
much faster than restoring both data and metadata. As a first-stop 
recovery, maybe it was worth a try. But if nobody is doing that, it 
should not be a good idea...

> No, it's not possible to preallocate space on xfs without marking it
> as unwritten.

Understood.
Thanks Eric.

-- 
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8

Re: xfs_metadump as a backup tool

[Gionatan Danti]

Il 21-08-2017 22:11 Eric Sandeen ha scritto:
> Short answer is "No," because:
> xfs_metadump is only a diagnostic tool.  It is not tested or supported
> for any other use.
> 
> If nothing else, on a normal live filesystem your metadata and data is
> constantly changing; if you replay old metadata over a filesystem,
> you'll end up pointing to file data blocks that are incorrect.
> 
> If you really truly have only statically-allocated files from start to
> finish, then maybe...
> 
> But more than anything else, the tool was not designed or tested for
> anything other than developer diagnostics.
> 
> For disaster recovery, having periodic metadata snapshots might come
> in handy from a forensic POV, but I would not rely on this as a
> primary part of your normal backup/recovery scheme.

Hi Eric,
thank for your reply. From xfs_metadump man page is read that the tool 
should be only used as a debugging aid, but hey - maybe someone used it 
in more clever ways ;)

> If you've already done a proper backup wit tar/rsync/(xfsdump?) then
> you have everything you need to restore the filesystem without
> resorting to xfs_metadump cleverness, right?

Sure, but restoring metadata on a corrupted, but healty, volume would be 
much faster than restoring both data and metadata. As a first-stop 
recovery, maybe it was worth a try. But if nobody is doing that, it 
should not be a good idea...

> No, it's not possible to preallocate space on xfs without marking it
> as unwritten.

Understood.
Thanks Eric.

-- 
Danti Gionatan
Supporto Tecnico
Assyoma S.r.l. - www.assyoma.it
email: g.danti@assyoma.it - info@assyoma.it
GPG public key ID: FF5F32A8