UBIFS recovery truncates file to zero size

UBIFS recovery truncates file to zero size

[Iram Shahzad]

Hi

I am having the following problem with UBIFS (Linux 2.6.25 + ubifs-v2.6.25).
Please could you tell me if this is a known issue.

Problem
    If the power is switched off abruptly while a file is being written,
    in the next boot the contents of the file is lost. That is, the file
    becomes zero sized. The problem is reproducable.

Thanks in advance.
Iram

Re: UBIFS recovery truncates file to zero size

[Brijesh Singh]

Hi,

On Wed, Mar 11, 2009 at 2:46 PM, Iram Shahzad
<iram.shahzad@jp.fujitsu.com> wrote:
> Hi
>
> I am having the following problem with UBIFS (Linux 2.6.25 + ubifs-v2.6.25).
> Please could you tell me if this is a known issue.
>
> Problem
>   If the power is switched off abruptly while a file is being written,
>   in the next boot the contents of the file is lost. That is, the file
>   becomes zero sized. The problem is reproducable.

May be because of write-back support.  Try disabling it.mount with sync option.

For details:
http://www.linux-mtd.infradead.org/doc/ubifs.html#L_writeback

Re: UBIFS recovery truncates file to zero size

[Adrian Hunter]

Brijesh Singh wrote:
> On Wed, Mar 11, 2009 at 2:46 PM, Iram Shahzad
> <iram.shahzad@jp.fujitsu.com> wrote:
>> I am having the following problem with UBIFS (Linux 2.6.25 + ubifs-v2.6.25).
>> Please could you tell me if this is a known issue.
>>
>> Problem
>>   If the power is switched off abruptly while a file is being written,
>>   in the next boot the contents of the file is lost. That is, the file
>>   becomes zero sized. The problem is reproducable.
> 
> May be because of write-back support.  Try disabling it.mount with sync option.

There was also a problem with busybox implementation of vi which truncated
the file to zero length.  See revision 21928 here:

	http://sources.busybox.net/index.py/trunk/busybox/editors/vi.c?view=log&pathrev=22471

RE: UBIFS recovery truncates file to zero size

[Reuben Dowle]

The file is kept in the kernel write-back cache, until it is synced by
the user (eg. A sync() or fsync() call) or the kernel decides to write
it back (seems to be after 30 seconds by default).

You can disable write back cache, but if you make lots of small writes
to the file this will cause more wear on the flash. It is better to call
fsync() when you want the data to be saved.

See http://www.linux-mtd.infradead.org/doc/ubifs.html#L_writeback

Also be carefull to note the comment about making your updates atomic.
Just adding the odd fsync() is not going to truely guarenteed your data
is not going to get fried.

Reuben

-----Original Message-----
From: linux-mtd-bounces@lists.infradead.org
[mailto:linux-mtd-bounces@lists.infradead.org] On Behalf Of Iram Shahzad
Sent: Wednesday, 11 March 2009 10:16 p.m.
To: linux-mtd@lists.infradead.org
Subject: UBIFS recovery truncates file to zero size

Hi

I am having the following problem with UBIFS (Linux 2.6.25 +
ubifs-v2.6.25).
Please could you tell me if this is a known issue.

Problem
    If the power is switched off abruptly while a file is being written,
    in the next boot the contents of the file is lost. That is, the file
    becomes zero sized. The problem is reproducable.

Thanks in advance.
Iram



______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: UBIFS recovery truncates file to zero size

[Iram Shahzad]

Thanks Brijesh, Adrian, Reuben for your replies.

I understood, but please let me confirm further.

For the write back enabled case, I was expecting the
following behaviour:
    power went down before the file is sync-ed, so
    on the next booting the file will be in its old state,
    that is it will contain the contents which it had before
    this write.

But instead of being in the old state, it becomes empty file.
Is this really expected(correct) behaviour?

Thanks
Iram

----- Original Message ----- 
From: "Reuben Dowle" <Reuben.Dowle@navico.com>
To: "Iram Shahzad" <iram.shahzad@jp.fujitsu.com>; 
<linux-mtd@lists.infradead.org>
Sent: Thursday, March 12, 2009 3:13 AM
Subject: RE: UBIFS recovery truncates file to zero size


> The file is kept in the kernel write-back cache, until it is synced by
> the user (eg. A sync() or fsync() call) or the kernel decides to write
> it back (seems to be after 30 seconds by default).
>
> You can disable write back cache, but if you make lots of small writes
> to the file this will cause more wear on the flash. It is better to call
> fsync() when you want the data to be saved.
>
> See http://www.linux-mtd.infradead.org/doc/ubifs.html#L_writeback
>
> Also be carefull to note the comment about making your updates atomic.
> Just adding the odd fsync() is not going to truely guarenteed your data
> is not going to get fried.
>
> Reuben
>
> -----Original Message-----
> From: linux-mtd-bounces@lists.infradead.org
> [mailto:linux-mtd-bounces@lists.infradead.org] On Behalf Of Iram Shahzad
> Sent: Wednesday, 11 March 2009 10:16 p.m.
> To: linux-mtd@lists.infradead.org
> Subject: UBIFS recovery truncates file to zero size
>
> Hi
>
> I am having the following problem with UBIFS (Linux 2.6.25 +
> ubifs-v2.6.25).
> Please could you tell me if this is a known issue.
>
> Problem
>    If the power is switched off abruptly while a file is being written,
>    in the next boot the contents of the file is lost. That is, the file
>    becomes zero sized. The problem is reproducable.
>
> Thanks in advance.
> Iram
>
>
>
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
>
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
> 

Re: UBIFS recovery truncates file to zero size

[Adrian Hunter]

Iram Shahzad wrote:
> Thanks Brijesh, Adrian, Reuben for your replies.
> 
> I understood, but please let me confirm further.
> 
> For the write back enabled case, I was expecting the
> following behaviour:
>     power went down before the file is sync-ed, so
>     on the next booting the file will be in its old state,
>     that is it will contain the contents which it had before
>     this write.
> 
> But instead of being in the old state, it becomes empty file.
> Is this really expected(correct) behaviour?

No.  How were you updating the file?

Re: UBIFS recovery truncates file to zero size

[Iram Shahzad]

>> For the write back enabled case, I was expecting the
>> following behaviour:
>>     power went down before the file is sync-ed, so
>>     on the next booting the file will be in its old state,
>>     that is it will contain the contents which it had before
>>     this write.
>> 
>> But instead of being in the old state, it becomes empty file.
>> Is this really expected(correct) behaviour?
> 
> No.  How were you updating the file?

The file is updated by a Java application in the following way.

-----
FileOutputStream str = new FileOutputStream(aFilename);
// some write here
str.flush();
str.close();
-----

I switch off the power after confirming that the str.close() has been
executed.

P.S:
If I add str.getFD().sync(); after the str.flush(), the file looks good in
the next boot. However my question still remains for the case
when I do not call str.getFD().sync:
"why the file becomes empty rather than being in its old state?".

Thanks
Iram

Re: UBIFS recovery truncates file to zero size

[Adrian Hunter]

Iram Shahzad wrote:
>>> For the write back enabled case, I was expecting the
>>> following behaviour:
>>>     power went down before the file is sync-ed, so
>>>     on the next booting the file will be in its old state,
>>>     that is it will contain the contents which it had before
>>>     this write.
>>>
>>> But instead of being in the old state, it becomes empty file.
>>> Is this really expected(correct) behaviour?
>> No.  How were you updating the file?
> 
> The file is updated by a Java application in the following way.
> 
> -----
> FileOutputStream str = new FileOutputStream(aFilename);
> // some write here
> str.flush();
> str.close();
> -----
> 
> I switch off the power after confirming that the str.close() has been
> executed.
> 
> P.S:
> If I add str.getFD().sync(); after the str.flush(), the file looks good in
> the next boot. However my question still remains for the case
> when I do not call str.getFD().sync:
> "why the file becomes empty rather than being in its old state?".
> 
> Thanks
> Iram
> 
> 
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
> 

What makes you think that:

FileOutputStream str = new FileOutputStream(aFilename);

does not truncate the file to zero length?

Re: UBIFS recovery truncates file to zero size

[Artem Bityutskiy]

On Thu, 2009-03-12 at 17:49 +0900, Iram Shahzad wrote:
> >> For the write back enabled case, I was expecting the
> >> following behaviour:
> >>     power went down before the file is sync-ed, so
> >>     on the next booting the file will be in its old state,
> >>     that is it will contain the contents which it had before
> >>     this write.
> >> 
> >> But instead of being in the old state, it becomes empty file.
> >> Is this really expected(correct) behaviour?
> > 
> > No.  How were you updating the file?
> 
> The file is updated by a Java application in the following way.
> 
> -----
> FileOutputStream str = new FileOutputStream(aFilename);
> // some write here
> str.flush();
> str.close();
> -----

Please, check whether str.flush()/str.close() cause fsync()/sync().

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: UBIFS recovery truncates file to zero size

[Jamie Lokier]

Iram Shahzad wrote:
> >>For the write back enabled case, I was expecting the
> >>following behaviour:
> >>    power went down before the file is sync-ed, so
> >>    on the next booting the file will be in its old state,
> >>    that is it will contain the contents which it had before
> >>    this write.
> >>
> >>But instead of being in the old state, it becomes empty file.
> >>Is this really expected(correct) behaviour?
> >
> >No.  How were you updating the file?
> 
> The file is updated by a Java application in the following way.
> 
> -----
> FileOutputStream str = new FileOutputStream(aFilename);

"new FileOutputStream" truncates the file.

> I switch off the power after confirming that the str.close() has been
> executed.
> 
> P.S:
> If I add str.getFD().sync(); after the str.flush(), the file looks good in
> the next boot. However my question still remains for the case
> when I do not call str.getFD().sync:
> "why the file becomes empty rather than being in its old state?".

str.getFD().sync() won't be reliable if you lose power _between_ "new
FileOutputStream" and sync().  If that happens, the file may still be
truncated.

If you are just appending data, use "new FileOutputStream(aFileName,
true)", write, flush, sync, close.

If you are changing the file contents, instead write to a temporary
file in the same directory, then sync the temporary file (after
flush), then do an atomic rename over the original file, then open the
directory and sync that somehow.

-- Jamie