Re: Atomic non-durable file write API

[Olaf van der Spek <olafvdspek@gmail.com>]

On Fri, Dec 24, 2010 at 10:51 AM, Ted Ts'o <tytso@mit.edu> wrote:
> On Fri, Dec 24, 2010 at 02:00:13AM +0100, Christian Stroetmann wrote:
>> I really do know what you want to say, despite that this example is
>> based on a bug in another system than the FS. But there will be
>> other examples, for sure.
>
> Sure, but this thread started because someone wanted an "atomic
> non-durable file write API", apparently because it was too slow to use
> fsync().  If people use databases, it's not a problem; databases use
> fsync(), but they use it properly and they provide the proper
> transactional interfaces that people want.
>
> The problem comes when people try to implement their own databases
> using small files for each row and column of the database, or for each
> registry variable.  Then they complain when fsync() is to expensive,
> because they need to use fsync() for every single 3 bytes of data they
> store in their badly implemented database.
>
> The bottom line is that if you want atomic updates of state
> information, you need to use fsync() or fdatasync().  If this is a
> performance bottleneck, then you're doing something wrong.  Maybe you
> shouldn't be writing a third of a megabyte on every URL click, on the
> main GUI thread; maybe the user doesn't need to remember every single
> URL that was visited even if the power suddenly fails (maybe it's
> enough if you write that information to disk every 3-5 minutes, and
> less if you're running on battery).  Or maybe you shouldn't be using
> hundreds of small state files, and screw up the dirty flag handling.
> But regardless, you're doing something wrong/stupid.

Hi Ted,

Thanks for taking the time to answer. The thread was started due to
the dpkg issue.
The questions were:
> What is the recommended way for atomic non-durable (complete) file writes?

It seems you're saying fsync is required, but why can't atomic be
provided without durable? Is it just an API issue?

If rename is recommended, how does one preserve meta-data including file owner?

> I'm also wondering why FSs commit after open/truncate but before
write/close. AFAIK this isn't necessary and thus suboptimal.

Olaf
--
To unsubscribe from this list: send the line "unsubscribe linux-ext4" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html