Data safety horror stories?

Data safety horror stories?

[Felix E. Klee]

I heard that, in case of a power failure, XFS may lose data, even data
that was already existing on the disk. For example, I heard horror
stories of files being overwritten with zeros.

Are those stories true?

If so:

* Do you recommend not using XFS on devices that may frequently fail due
  to power failure?

* Is it possible to find out what files have been damaged? If not,
  will only files be affected that have been changed during the last
  couple of hours?

* Are there options to increase data safety? Should one run a regular
  "sync" in a cron job?

* Is it unsafe to use XFS in a virtual machine which may sometimes be
  terminated without proper shutdown? Suppose the virtual machine is
  running under Windows, and Windows may sometimes be terminated without
  proper shutdown.

I currently am using XFS under Ubuntu 7.10 (Kernel 2.6.22), running in a
virtual machine (VMware) under Windows.  The XFS file system is in a
native partition on a second HDD.

-- 
Felix E. Klee
Jabber/Google Talk: feklee@jabber.org, SIP: 9779619@sipgate.de
ICQ: 158124695, Yahoo!: feklee, AIM: felix.klee@inka.de
Gizmo: felixklee, Skype: felix.klee

Re: Data safety horror stories?

[Iustin Pop]

On Mon, Feb 11, 2008 at 05:46:29PM +0100, Felix E. Klee wrote:
> I heard that, in case of a power failure, XFS may lose data, even data
> that was already existing on the disk. For example, I heard horror
> stories of files being overwritten with zeros.
> 
> Are those stories true?

No, XFS will not lose any data that the application has committed to the
disk.

Improperly written applications and/or improperly configured systems
might have issues with recently written files losing data.

FWIW: I have never lost data with XFS, neither on home computers nor in
SAN environments (in the presence of link/path failure).

Be sure the read the FAQ, especially the section about write cache on
dekstop/consumer HDDs.

Just my opinion as an XFS user, your mileage might vary.

iustin

Re: Data safety horror stories?

[Felix E. Klee]

Hi Justin,

thanks for the info!

On Mon, 11 Feb 2008 18:12:09 +0100, "Iustin Pop" <iusty@k1024.org> said:
> No, XFS will not lose any data that the application has committed to
> the disk.

OK, but just to make sure: The following FAQ entry refers only to
*newly* created files - right?

http://oss.sgi.com/projects/xfs/faq.html#nulls

> Improperly written applications and/or improperly configured systems
> might have issues with recently written files losing data.

Again, just to make sure that I understood you correctly: Could you name
an example?

> Be sure the read the FAQ, especially the section about write cache on
> dekstop/consumer HDDs.

You are probably referring to the following entry.  I now disabled the
write cache of the 2nd HDD in Windows (remember my configuration).  I
wonder though: Wouldn't it be possible to design journaling in a way so
that the write cache does never cause problems? Could someone provide an
example which illustrates the write cache problem in simple terms?

http://oss.sgi.com/projects/xfs/faq.html#wcache

> Just my opinion as an XFS user, your mileage might vary.

Hopefully, it's not about opinions ...

- Felix
-- 
Dipl.-Phys. Felix E. Klee
Naunynstr. 2, 76530 Baden-Baden, Germany
Tel.: +49 7221 396961, Fax: +49 7221 396960, Mobile: +49 174 1386060
http://www.linkedin.com/in/feklee

Re: Data safety horror stories?

[Martin Steigerwald]

[-- Attachment #1: Type: text/plain, Size: 1742 bytes --]

Am Montag 11 Februar 2008 schrieb Felix E. Klee:
> Hi Justin,

Hi Felix,

> > Improperly written applications and/or improperly configured systems
> > might have issues with recently written files losing data.
>
> Again, just to make sure that I understood you correctly: Could you
> name an example?

I recommend to read my article about write barriers and journalling 
filesystems for some basic understanding.
 
http://www.linux-magazin.de/heft_abo/sonderheft/2006/04/beschraenktes_schreiben?category=0

its in german tough. Actually I thought it should have been translated to 
english for Linux Magazine under the title "Imposing Order", but I do not 
find it on the net and it might have not been published.

Note: There is a mistake in it regarding history of write barriers. Write 
barriers are not supported via device mapper. Thats why I do not use LVM 
on my notebook for my productive data.

> > Just my opinion as an XFS user, your mileage might vary.
>
> Hopefully, it's not about opinions ...

I had not had any problems since quite some kernel releases. And there 
were some power losses involved ;). At earlier times Akregator (an KDE 
newsreader) said its opml was broken and a backup was restored after such 
a power loss... but not since quite some kernel releases.

I had one nasty problem with corrupt data in files but that was with a 
kernel that was patched with Con Koliva's patchset and didn't happen in 
mainline.

So since 2.6.17.7 XFS has been pretty stable for me. On a ThinkPad T42, a 
ThinkPad T23 and half a dozen of workstations at work ;-).

Ciao,
-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 189 bytes --]

Re: Data safety horror stories?

[Felix E. Klee]

On Mon, 18 Feb 2008 22:28:45 +0100, "Martin Steigerwald"
<Martin@lichtvoll.de> said:
> I recommend to read my article about write barriers and journalling
> filesystems for some basic understanding.
>
> http://www.linux-magazin.de/heft_abo/sonderheft/2006/04/beschraenktes_schreiben?category=0
>
> its in german tough.

Dankeschön - Deutsch ist doch kein Problem. :-)
-- 
Dipl.-Phys. Felix E. Klee
Naunynstr. 2, 76530 Baden-Baden, Germany
Tel.: +49 7221 396961, Fax: +49 7221 396960, Mobile: +49 174 1386060
http://www.linkedin.com/in/feklee

Re: Data safety horror stories?

[Martin Steigerwald]

Am Montag 18 Februar 2008 schrieb Felix E. Klee:
> On Mon, 18 Feb 2008 22:28:45 +0100, "Martin Steigerwald"
>
> <Martin@lichtvoll.de> said:
> > I recommend to read my article about write barriers and journalling
> > filesystems for some basic understanding.
> >
> > http://www.linux-magazin.de/heft_abo/sonderheft/2006/04/beschraenktes
> >_schreiben?category=0
> >
> > its in german tough.
>
> Dankeschön - Deutsch ist doch kein Problem. :-)

One thing to add: We have XFS also running on a web cluster for a customer 
with softraid 1 over two RAID arrays. We made sure that write caching is 
disabled as those RAID arrays are not equipped with NVRAM (could be added 
tough, but the customer did not yet). Runs marvelously. Its also running 
on some individual servers in that cluster as well.

Well and although the cluster is spread across two locations about 1 
kilometer from each other and it was said that power supply could not 
fail... well it failed once for both data centers and the cluster 
survived that just fine ;-).

BZW I didn't broke that GPG signature. It seems the mailinglist software 
being used for the xfs mailinglist mangles mails.

-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7

Re: Data safety horror stories?

[David Chinner]

On Mon, Feb 11, 2008 at 10:23:09PM +0100, Felix E. Klee wrote:
> Hi Justin,
> 
> thanks for the info!
> 
> On Mon, 11 Feb 2008 18:12:09 +0100, "Iustin Pop" <iusty@k1024.org> said:
> > No, XFS will not lose any data that the application has committed to
> > the disk.
> 
> OK, but just to make sure: The following FAQ entry refers only to
> *newly* created files - right?
> 
> http://oss.sgi.com/projects/xfs/faq.html#nulls

No - it can affect an file where the write is extending the file.
The NULLs problem should be very rare now - we've fixed the common
causes of this problem....

> > Improperly written applications and/or improperly configured systems
> > might have issues with recently written files losing data.
> 
> Again, just to make sure that I understood you correctly: Could you name
> an example?

I suggest watching the video of this talk from the 2007 linux.conf.au:

http://lca2007.linux.org.au/talk/278.html

It's all about how applications do stupid stuff that can lose data.

Cheers,

Dave.
-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group

Re: Data safety horror stories?

[David Chinner]

On Mon, Feb 11, 2008 at 05:46:29PM +0100, Felix E. Klee wrote:
> I heard that, in case of a power failure, XFS may lose data, even data
> that was already existing on the disk. For example, I heard horror
> stories of files being overwritten with zeros.
> 
> Are those stories true?
> 
> If so:
> 
> * Do you recommend not using XFS on devices that may frequently fail due
>   to power failure?

Depends on how much you care about your system and data.

I use XFS on write-cache enabled SATA drives without barriers with
no UPS (yes, it's unsafe!) and I lose power at least once a week.  I
haven't had a data loss or corruption in over two years and tens of
power failures.... 

> * Is it possible to find out what files have been damaged?

Not easily.

> If not,
>   will only files be affected that have been changed during the last
>   couple of hours?

Last few seconds before the power fail, actually.

> * Are there options to increase data safety? Should one run a regular
>   "sync" in a cron job?

If you are truly paranoid - turn off drive caching and mount with
the 'wsync' option.

> * Is it unsafe to use XFS in a virtual machine which may sometimes be
>   terminated without proper shutdown?

I do that all the time, too. Corruption is rare and usually
as a result of some bug in the code I'm testing ;)

> I currently am using XFS under Ubuntu 7.10 (Kernel 2.6.22), running in a
> virtual machine (VMware) under Windows.  The XFS file system is in a
> native partition on a second HDD.

Should be just fine. If you are really concerned - test it.

Cheers,

Dave.
> 
> -- 
> Felix E. Klee
> Jabber/Google Talk: feklee@jabber.org, SIP: 9779619@sipgate.de
> ICQ: 158124695, Yahoo!: feklee, AIM: felix.klee@inka.de
> Gizmo: felixklee, Skype: felix.klee
> 

-- 
Dave Chinner
Principal Engineer
SGI Australian Software Group