Re: Atomic non-durable file write API

[Christian Stroetmann <stroetmann@ontolinux.com>]

Am 24.12.2010 12:14, schrieb Olaf van der Spek:
> On Fri, Dec 24, 2010 at 10:51 AM, Ted Ts'o<tytso@mit.edu>  wrote:
>> On Fri, Dec 24, 2010 at 02:00:13AM +0100, Christian Stroetmann wrote:
>>> I really do know what you want to say, despite that this example is
>>> based on a bug in another system than the FS. But there will be
>>> other examples, for sure.
>> Sure, but this thread started because someone wanted an "atomic
>> non-durable file write API", apparently because it was too slow to use
>> fsync().  If people use databases, it's not a problem; databases use
>> fsync(), but they use it properly and they provide the proper
>> transactional interfaces that people want.
>>
>> The problem comes when people try to implement their own databases
>> using small files for each row and column of the database, or for each
>> registry variable.  Then they complain when fsync() is to expensive,
>> because they need to use fsync() for every single 3 bytes of data they
>> store in their badly implemented database.
>>
>> The bottom line is that if you want atomic updates of state
>> information, you need to use fsync() or fdatasync().  If this is a
>> performance bottleneck, then you're doing something wrong.  Maybe you
>> shouldn't be writing a third of a megabyte on every URL click, on the
>> main GUI thread; maybe the user doesn't need to remember every single
>> URL that was visited even if the power suddenly fails (maybe it's
>> enough if you write that information to disk every 3-5 minutes, and
>> less if you're running on battery).  Or maybe you shouldn't be using
>> hundreds of small state files, and screw up the dirty flag handling.
>> But regardless, you're doing something wrong/stupid.

see other e-mail

> Hi Ted,
>
> Thanks for taking the time to answer. The thread was started due to
> the dpkg issue.
> The questions were:
>> What is the recommended way for atomic non-durable (complete) file writes?
> It seems you're saying fsync is required, but why can't atomic be
> provided without durable? Is it just an API issue?
>
> If rename is recommended, how does one preserve meta-data including file owner?
>
>> I'm also wondering why FSs commit after open/truncate but before
> write/close. AFAIK this isn't necessary and thus suboptimal.

these are FS actings which could be optimized, as mentioned in the other 
e-mail

> Olaf
> --

Chrisitan Stroetmann