XFS and block-level snapshots

XFS and block-level snapshots

[Kamil Kisiel]

Hello,

I had a question about XFS integrity and performing block-level  
snapshots.

We currently have a 2TB (but growing soon..) volume mounted by a Linux  
host with kernel 2.6.23 over iSCSI from our SAN. Our SAN unit has the  
capability to perform block-level snapshots, which is done at regular  
intervals.

I know that it is recommended to perform an xfs_freeze before  
performing a snapshot. However, the control of the snapshots is  
independent from the OS, which currently has no knowledge of their  
occurrence. I'm curious as to the repercussions of this. I understand  
that in all likelyhood, the integrity of files which are currently  
being written will not be preserved. However, even with an xfs_freeze  
this is not guaranteed, as an application may require additional disk  
transactions to maintain the file in a valid state (it is not  
necessarily atomic, depending on the application).

As far as metadata transactions are concerned, the journal should make  
these atomic, so there should not be any problem there?

Basically, I'd like to know what is the worst that could happen, and  
why an xfs_freeze is necessary in this scenario.

____________
Kamil Kisiel
HPC Systems Engineer, Zymeworks Inc.
201-1401 West Broadway,
Vancouver, BC, V6H 1H6, Canada
Tel: (604) 678-1388 ext. 135
Fax: (604) 737-7077
www.zymeworks.com

Re: XFS and block-level snapshots

[Dave Chinner]

On Fri, Jun 06, 2008 at 11:33:17AM -0700, Kamil Kisiel wrote:
> Hello,
>
> I had a question about XFS integrity and performing block-level  
> snapshots.
>
> We currently have a 2TB (but growing soon..) volume mounted by a Linux  
> host with kernel 2.6.23 over iSCSI from our SAN. Our SAN unit has the  
> capability to perform block-level snapshots, which is done at regular  
> intervals.
>
> I know that it is recommended to perform an xfs_freeze before performing 
> a snapshot. However, the control of the snapshots is independent from the 
> OS, which currently has no knowledge of their occurrence. I'm curious as 
> to the repercussions of this. I understand that in all likelyhood, the 
> integrity of files which are currently being written will not be 
> preserved. However, even with an xfs_freeze this is not guaranteed, as an 
> application may require additional disk transactions to maintain the file 
> in a valid state (it is not necessarily atomic, depending on the 
> application).

That's from an application POV, not a filesystem POV. When you
freeze the filesystem all the data and metadata is guaranteed to be
consistent on disk. If your application requires further guarantees
of atomicity, then it needs to call xfs_freeze at a time that the
application can guarantee that it'sstate in the filesystem is
consistent.  i.e. not a filesystem problem.

> As far as metadata transactions are concerned, the journal should
> make  these atomic, so there should not be any problem there?

Sure, asssuming that at the time the snapshot is taken that the sum
of the journal contents, the filesystem metadata on disk and the
data on disk = a consistent filesystem image. Which, of course, will
never happen when you randomly snapshot a busy filesystem as it's a
constantly moving target.

e.g. say that while the log is being snapshotted by the block device
it wraps (i.e. the head moves from the end to the start) and
metadata I/O completes so the tail moves forward. now you have a
snapshot with the old tail in it and you've lost the transactions at
the head of the log. i.e. the journal is no longer consistent with
what is on disk in the snapshot. This can happen for data vs
metadata, metadata vs metadata and metadata vs log. IOWs, if you
don't freeze before you snapshot, your snapshot if full of nasty
little inconsistencies just waiting to trip you over....

> Basically, I'd like to know what is the worst that could happen, and why 
> an xfs_freeze is necessary in this scenario.

Worst case? Silent data corruption in the snapshot. Metadata
corruption in the snapshot leading to filesystem shutdowns and
system panics. Choose your poison - they're all bad.

Cheers,

Dave.
-- 
Dave Chinner
david@fromorbit.com