temporary files on mtd file systems

temporary files on mtd file systems

[Ricard Wanderlof]

I came across a problem the other day, which while maybe not directly 
related to ordinary mtd issues, it's something which perhaps someone else 
has come across and has some form of solution to.

When writing to a file on for instance an ubifs file system, there's 
always the problem that the power may fail at any moment, leaving the file 
in an inconsistent state.

The remedy for this is to first write to a temporary file, and then rename 
the file to the real one. Since the rename is an atomic operation, one 
will either get the original file or the new one.

Traditionally when creating a temporary file, it is recommended to use 
mkstemp(3) and friends, which will guarantee that the temporary file will 
have a unique file name that will not clash with anything else.

However, in the case of a file on a persistent file system, the following 
chain of events could occur:

1. A file is to be updated, say, foo, so a new temporary file (say,
    foo.1234567) is created, using mkstemp.
2. Data is written to the file.
3. Due to timeouts, or an explicit sync call, the file is written out to
    the media, so there is a foo.1234567 on the flash.
4. Power failure.
5. System starts up again, and again, the file is to be updated. Thus, a
    new temporary file is created, this time using a new name, say,
    foo.2345678 .
6. Operation goes as planned, and foo.2345678 is renamed to foo, thus
    removing foo.2345678

In this scenario, the old file foo.1234567 remains on the system and is 
never removed. Eventually a lot of such operations will cause the file 
system to fill up with temporary files.

Am I missing something here, or is there a "good" way to deal with this? 
Of course there are several cleanup scenarios that could be imagened, such 
as cleaning out all 'foo.*' files during boot for instance.

For applications that are not multi threaded, i.e. there may only be one 
process updating foo, one way is to use the same temporary file name each 
time, e.g. foo.temp. But things get awkward when using for instance glib, 
which has a nice g_file_set_contents() call, complete with temporary file 
handling and all, but which uses mkstemp as above to create the temporary 
file, so there is no easy way of telling glib what filename to use, short 
of rewriting a local function which indeed uses a constant temporary file 
name.

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: temporary files on mtd file systems

[Bityutskiy, Artem]

On Tue, 2013-08-20 at 16:51 +0300, Artem Bityutskiy wrote:
> Something like having a special chattr which would indicate that the

Sorry, I meant having a special attribute (extended or standard, but I
guess extended is a better idea).

-- 
Best Regards,
Artem Bityutskiy
---------------------------------------------------------------------
Intel Finland Oy
Registered Address: PL 281, 00181 Helsinki 
Business Identity Code: 0357606 - 4 
Domiciled in Helsinki 

This e-mail and any attachments may contain confidential material for
the sole use of the intended recipient(s). Any review or distribution
by others is strictly prohibited. If you are not the intended
recipient, please contact the sender and delete all copies.

Re: temporary files on mtd file systems

[Artem Bityutskiy]

On Mon, 2013-08-19 at 15:55 +0200, Ricard Wanderlof wrote:
> Am I missing something here, or is there a "good" way to deal with this? 
> Of course there are several cleanup scenarios that could be imagened, such 
> as cleaning out all 'foo.*' files during boot for instance.
> 
> For applications that are not multi threaded, i.e. there may only be one 
> process updating foo, one way is to use the same temporary file name each 
> time, e.g. foo.temp. But things get awkward when using for instance glib, 
> which has a nice g_file_set_contents() call, complete with temporary file 
> handling and all, but which uses mkstemp as above to create the temporary 
> file, so there is no easy way of telling glib what filename to use, short 
> of rewriting a local function which indeed uses a constant temporary file 
> name.

Good question, the only thing which comes to my mind is that one could
try to use extended attributes or 'standard' (chattr(1)) attributes.

Something like having a special chattr which would indicate that the
file is up-to-date. When you need to update the file atomically, you
start with removing the attribute, syncing the file, then updating the
file, then syncing it, then creating the attribute, and probably syncing
too.

-- 
Best Regards,
Artem Bityutskiy