about ubifs

about ubifs

[tommy]

Dear all,i have found a question about ubifs and sync !

Question is following;

Artem,
What i do is just open an existing file in a UBIFS formated UBI volume
on a Nand Flash,

add a few lines to it ,close the file ,then without unmounting the
filesystem and reboot the filesystem.

When my board reboots and i looked at that file ,it's zero length and
has no contents.I suspect it's because

UBIFS does less frequent syncs .it just lost content of the file because
there's no sync before the reboot!

How can i solve this ?

Re: about ubifs

[Corentin Chary]

On Fri, Jun 5, 2009 at 2:13 PM, tommy<jiujin.hong@mid-fun.com> wrote:
> Dear all,i have found a question about ubifs and sync !
>
> Question is following;
>
> Artem,
> What i do is just open an existing file in a UBIFS formated UBI volume
> on a Nand Flash,
>
> add a few lines to it ,close the file ,then without unmounting the
> filesystem and reboot the filesystem.
>
> When my board reboots and i looked at that file ,it's zero length and
> has no contents.I suspect it's because
>
> UBIFS does less frequent syncs .it just lost content of the file because
> there's no sync before the reboot!
>
> How can i solve this ?
>

Hi,
Do you use fsync() before closing the file ?
You can read http://thunk.org/tytso/blog/2009/03/15/dont-fear-the-fsync/
for more information. It's for ext4, but I think the problem is the
same.



-- 
Corentin Chary
http://xf.iksaif.net - http://uffs.org

Re: about ubifs

[tommy]

no,i didn't fsync when i close file !

did ubifs driver can achieve "fsync" when user modify some file and
close with save ?

if system have no power and shutdown system forcely ,user's file will
become no contents in some case ,how did we protect this ?

thanks!


On 五, 2009-06-05 at 14:42 +0200, Corentin Chary wrote:
> On Fri, Jun 5, 2009 at 2:13 PM, tommy<jiujin.hong@mid-fun.com> wrote:
> > Dear all,i have found a question about ubifs and sync !
> >
> > Question is following;
> >
> > Artem,
> > What i do is just open an existing file in a UBIFS formated UBI volume
> > on a Nand Flash,
> >
> > add a few lines to it ,close the file ,then without unmounting the
> > filesystem and reboot the filesystem.
> >
> > When my board reboots and i looked at that file ,it's zero length and
> > has no contents.I suspect it's because
> >
> > UBIFS does less frequent syncs .it just lost content of the file because
> > there's no sync before the reboot!
> >
> > How can i solve this ?
> >
> 
> Hi,
> Do you use fsync() before closing the file ?
> You can read http://thunk.org/tytso/blog/2009/03/15/dont-fear-the-fsync/
> for more information. It's for ext4, but I think the problem is the
> same.
> 
> 
> 

Re: about ubifs

[Corentin Chary]

On Fri, Jun 5, 2009 at 3:03 PM, tommy<jiujin.hong@mid-fun.com> wrote:
> no,i didn't fsync when i close file !
>
> did ubifs driver can achieve "fsync" when user modify some file and
> close with save ?

For critical data, you need to call fsync to be sure the data is
stored on the flash.
There is also a "sync" option for ubifs, but this will hurt performance a lot.

> if system have no power and shutdown system forcely ,user's file will
> become no contents in some case ,how did we protect this ?

What does the application do ?
Only open - write - close ? Do you also rename files ?
I believe that the content should not disapear when you only do a write.

You can activate ubifs debug to check if there is more informations.

-- 
Corentin Chary
http://xf.iksaif.net - http://uffs.org

Re: about ubifs

[Jamie Lokier]

Corentin Chary wrote:
> On Fri, Jun 5, 2009 at 3:03 PM, tommy<jiujin.hong@mid-fun.com> wrote:
> > no,i didn't fsync when i close file !
> >
> > did ubifs driver can achieve "fsync" when user modify some file and
> > close with save ?
> 
> For critical data, you need to call fsync to be sure the data is
> stored on the flash.
> There is also a "sync" option for ubifs, but this will hurt performance a lot.

That reminds me.

A filesystem-generic "sync-on-close" mount option would be handy for
things like that.

Expecting users to call fsync from a shell script is probably a bit much
in a lot of applications.

-- Jamie

Re: about ubifs

[tommy]

yes,i have do it !

thanks for all !

Tommy
On 一, 2009-06-08 at 02:35 +0100, Jamie Lokier wrote:
> Corentin Chary wrote:
> > On Fri, Jun 5, 2009 at 3:03 PM, tommy<jiujin.hong@mid-fun.com> wrote:
> > > no,i didn't fsync when i close file !
> > >
> > > did ubifs driver can achieve "fsync" when user modify some file and
> > > close with save ?
> > 
> > For critical data, you need to call fsync to be sure the data is
> > stored on the flash.
> > There is also a "sync" option for ubifs, but this will hurt performance a lot.
> 
> That reminds me.
> 
> A filesystem-generic "sync-on-close" mount option would be handy for
> things like that.
> 
> Expecting users to call fsync from a shell script is probably a bit much
> in a lot of applications.
> 
> -- Jamie
> 

Re: about ubifs

[Artem Bityutskiy]

On Mon, 2009-06-08 at 02:35 +0100, Jamie Lokier wrote:
> Corentin Chary wrote:
> > On Fri, Jun 5, 2009 at 3:03 PM, tommy<jiujin.hong@mid-fun.com> wrote:
> > > no,i didn't fsync when i close file !
> > >
> > > did ubifs driver can achieve "fsync" when user modify some file and
> > > close with save ?
> > 
> > For critical data, you need to call fsync to be sure the data is
> > stored on the flash.
> > There is also a "sync" option for ubifs, but this will hurt performance a lot.
> 
> That reminds me.
> 
> A filesystem-generic "sync-on-close" mount option would be handy for
> things like that.
> 
> Expecting users to call fsync from a shell script is probably a bit much
> in a lot of applications.

Not sure about sync-on-every-close, but ex4-ish
sync-on-close-if-there-was-truncate makes sense, IMO, and we even plan
to implement this, we just do not have enough time ATM:
http://www.linux-mtd.infradead.org/doc/ubifs.html#L_sync_exceptions

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: about ubifs

[Jamie Lokier]

Artem Bityutskiy wrote:
> > A filesystem-generic "sync-on-close" mount option would be handy for
> > things like that.
> > 
> > Expecting users to call fsync from a shell script is probably a bit much
> > in a lot of applications.
> 
> Not sure about sync-on-every-close, but ex4-ish
> sync-on-close-if-there-was-truncate makes sense, IMO, and we even plan
> to implement this, we just do not have enough time ATM:
> http://www.linux-mtd.infradead.org/doc/ubifs.html#L_sync_exceptions

sync-on-close-if-there-was-a-truncate isn't useful for this case:

    echo 'Stuff' >> logfile
    reboot

I was thinking both sync-on-close and sync-on-close-after-truncate
would be most useful as _generic_ mount options, in the same way that
O_SYNC has generic filesystem support these days.

-- Jamie

Re: about ubifs

[Artem Bityutskiy]

On Mon, 2009-06-08 at 13:01 +0100, Jamie Lokier wrote:
> I was thinking both sync-on-close and sync-on-close-after-truncate
> would be most useful as _generic_ mount options, in the same way that
> O_SYNC has generic filesystem support these days.

May be. On the other hand this would kill any remote possibility of
having user-space fixed :-)

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: about ubifs

[Jamie Lokier]

Artem Bityutskiy wrote:
> On Mon, 2009-06-08 at 13:01 +0100, Jamie Lokier wrote:
> > I was thinking both sync-on-close and sync-on-close-after-truncate
> > would be most useful as _generic_ mount options, in the same way that
> > O_SYNC has generic filesystem support these days.
> 
> May be. On the other hand this would kill any remote possibility of
> having user-space fixed :-)

My point was that "fixing" user-space by requiring every shell script
to run a special (currently non-existing) fsync program after every
thing it does which modifies a file (virtually every shell command)
which has an ordered relationship with other files is stupid.

It's fine for C programs to call fsync() (e.g. I'm good about doing
that) even though it's sometimes breaks performance, but it's
unhelpful to sprinkle every other line in every shell script with it,
as it would defeat the point of shell scripts which is simplicity.

-- Jamie

Re: about ubifs

[Artem Bityutskiy]

On Mon, 2009-06-08 at 13:29 +0100, Jamie Lokier wrote:
> Artem Bityutskiy wrote:
> > On Mon, 2009-06-08 at 13:01 +0100, Jamie Lokier wrote:
> > > I was thinking both sync-on-close and sync-on-close-after-truncate
> > > would be most useful as _generic_ mount options, in the same way that
> > > O_SYNC has generic filesystem support these days.
> > 
> > May be. On the other hand this would kill any remote possibility of
> > having user-space fixed :-)
> 
> My point was that "fixing" user-space by requiring every shell script
> to run a special (currently non-existing) fsync program after every
> thing it does which modifies a file (virtually every shell command)
> which has an ordered relationship with other files is stupid.

> It's fine for C programs to call fsync() (e.g. I'm good about doing
> that) even though it's sometimes breaks performance, but it's
> unhelpful to sprinkle every other line in every shell script with it,
> as it would defeat the point of shell scripts which is simplicity.

Not necessarily. Shell scripts may always just re-create their
files every time. Or re-create them if they are empty. Or for
large files use /bin/sync. Or even a /bin/fsync program can be
created.

But yes, I see what you mean, of course.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: about ubifs

[Artem Bityutskiy]

Hi,

On Fri, 2009-06-05 at 20:13 +0800, tommy wrote:
> Artem,
> What i do is just open an existing file in a UBIFS formated UBI volume
> on a Nand Flash,
> 
> add a few lines to it ,close the file ,then without unmounting the
> filesystem and reboot the filesystem.

> When my board reboots and i looked at that file ,it's zero length and
> has no contents.I suspect it's because
> 
> UBIFS does less frequent syncs .it just lost content of the file because
> there's no sync before the reboot!
> 
> How can i solve this ?

I've realized that the best way to answer your question is to
write a FAQ entry. There is a short answer for your question -
use fsync(). And there is a long answer, which you will find here:

http://www.linux-mtd.infradead.org/faq/ubifs.html#L_empty_file
and the above link also refers this one:
http://www.linux-mtd.infradead.org/doc/ubifs.html#L_sync_exceptions
which I've also created for you.

Enjoy!

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

About UBIFS

[Qiang Wang]

hello, every body
I am a newbie here.
I want to try UBIFS , and want to study it.

So , here is my first question.
how to download the
git://git.infradead.org/~dedekind/ubi-2.6.git
I try the
git fetch git://git.infradead.org/~dedekind/ubi-2.6.git
but got the error message as below
fatal: Not a git repository
Failed to find a valid git directory.

thank you very much.

best regards
wagnqiang

Re: About UBIFS

[Artem Bityutskiy]

On Tue, 2008-07-08 at 18:08 +0900, Qiang Wang wrote:
> hello, every body
> I am a newbie here.
> I want to try UBIFS , and want to study it.
> 
> So , here is my first question.
> how to download the
> git://git.infradead.org/~dedekind/ubi-2.6.git
> I try the
> git fetch git://git.infradead.org/~dedekind/ubi-2.6.git
> but got the error message as below
> fatal: Not a git repository
> Failed to find a valid git directory.

It is very similar to what is described here:

http://www.linux-mtd.infradead.org/source.html


git-clone --reference linux-2.6
git://git.infradead.org/~dedekind/ubi-2.6.git

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)