Re: Atomic non-durable file write API

[Boaz Harrosh <bharrosh@panasas.com>]

On 12/26/2010 06:02 PM, Olaf van der Spek wrote:
> On Sun, Dec 26, 2010 at 4:55 PM, Boaz Harrosh <bharrosh@panasas.com> wrote:
>> What if you use a soft link? wouldn't that solve all of your problems?
>>
>> - do your fsync/fdatasync of choice in a *backend thread* then at the return
>> - point set to the new link, fsync the link it's very small, therefore fast.
>> - Then delete the old source file.
>>
>> You need a simple "name-version" schema and the "name" is kept soft linked.
>> (You might even skip the last step above and implement an undo stack, some
>>  background management caps on history size)
>>
>>>>
>>>>> and this way has other
>>>>> issues, like losing file meta-data.
>>>>
>>
>> With soft links this is persevered?
>>
>> Same system can be used with lots of files. where the final switch is
>> the set of a single soft-link say to a folder of related files.
> 
> Are you proposing to turn every single file into a symlink?

Sure, a symlink and a "versioned" file for every object. Something similar
to the silly rename of nfs.

Even if you have 1000 files that need the same atomicity treatment
that's not that bad. You should be able to devise a namespace policy
that makes all this nit and tidy.

> How would that solve the meta-data issue?
> 

That's what I asked. Do you want to preserve the original's file
metat-data, or the meta-data of the owner of the new content?
In the first case you'll need a metat-data copy like tar is
using.

> Olaf

The point is to fsync/fdatasync on a background thread and continue
from there where the application is free to go on to the next step.
As if you had a notification when the commit was done (in the background).
So you make it an async pipeline model. The version-naming schem is so the
pipeline can get arbitrary big.

Boaz