Re: Linux 2.6.29

Linus Torvalds
2009-03-27T18:02:21Z

On Fri, 27 Mar 2009, Matthew Garrett wrote:
>
> If every application that does a clobbering rename has to call
> fbarrier() first, then the kernel should just guarantee to do so on the
> application's behalf. ext3, ext4 and btrfs all effectively do this, so
> we should just make it explicit that Linux filesystems are expected to
> behave this way. If people want to make their code Linux specific then
> that's their problem, not the kernel's.

It would probably be good to think about something like this, because
there are currently really two totally different cases of "fsync()" users.

(a) The "critical safety" kind (aka the "traditional" fsync user), where
there is a mail server or similar that will reply "all done" to the
sender, and has to _guarantee_ that the file is on disk in order for
data to simply not be lost.

This is a very different case from most desktop uses, and it's a evry
hard "we have to wait until the thing is physically on disk"
situation. And it's the only case where people really traditionally
used "fsync()".

(b) The non-traditional UNIX usage where people historically didn't use
fsync() for: people editing their config files either
programmatically or by hand.

And this one really doesn't need at all the same kind of hard "wait
for it to hit the disk" semantics. It may well want a much softer
kind of "at least don't delete the old version until the new version
is stable" kind of thing.

And Alan - you can argue that fsync() has been around forever, but you
cannot possibly argue that people have used fsync() for file editing.
That's simply not true. It has happened, but it has been very rare. Yes,
some editors (vi, emacs) do it, but even there it's configurable. And
outside of databases, server apps and big editors, fsync is virtually
unheard of. How many sed-scripts have you seen to edit files? None of them
ever used fsync.

And with the ext3 performance profile for it, it sure is not getting any
more common either. If you have a desktop app that uses fsync(), that
application is DEAD IN THE WATER if people are doing anything else on the
machine. Those multi-second pauses aren't going to make people happy.

So the fact is, "people should always use fsync" simply isn't a realistic
expectation, nor is it historically accurate. Claiming it is is just
obviously bogus. And claiming that people _should_ do it is crazy, since
it performs badly enough to simply not be realistic.

Alternatives should be looked at. For desktop apps, the best alternatives
are likely simply stronger default consistency guarantees. Exactly the
"we don't guarantee that your data hits the disk, but we do guarantee that
if you renamed on top of another file, you'll not have lost _both_
contents".

Linus