JFFS2 determine writing state

JFFS2 determine writing state

[Jeff S]

How would one go about checking if JFFS2 is done with a file operation, or all current pending operations?  Is there something in the proc file system or sys file system that allows you to determine the JFFS2 state or something similar.  I need to be able to notify a user that the flash has been written successfully after open, write, and close operations. Thank you.

- Jeff




      ____________________________________________________________________________________
Never miss a thing.  Make Yahoo your home page. 
http://www.yahoo.com/r/hs

Re: JFFS2 determine writing state

[Josh Boyer]

On Thu, 10 Jan 2008 15:38:50 -0800 (PST)
Jeff S <jsolman33@yahoo.com> wrote:

> How would one go about checking if JFFS2 is done with a file operation, or all current pending operations?  Is there something in the proc file system or sys file system that allows you to determine the JFFS2 state or something similar.  I need to be able to notify a user that the flash has been written successfully after open, write, and close operations. Thank you.

If you are using regular NOR flash and have the write buffer disabled,
anything you write via the write system call will be written to flash
before it returns.  The open call doesn't cause any writes, and close
is supposed to flush all pending writes before it returns.

There is always fsync or sync as well.

josh

Re: JFFS2 determine writing state

[Jamie Lokier]

Josh Boyer wrote:
> The open call doesn't cause any writes, and close
> is supposed to flush all pending writes before it returns.

Oh, that's interesting.  So on JFFS2, fsync() is unnecessary before close()?

(On other filesystems it's necessary, of course).

-- Jamie

Re: JFFS2 determine writing state

[Josh Boyer]

On Sat, 12 Jan 2008 10:03:43 +0000
Jamie Lokier <jamie@shareable.org> wrote:

> Josh Boyer wrote:
> > The open call doesn't cause any writes, and close
> > is supposed to flush all pending writes before it returns.
> 
> Oh, that's interesting.  So on JFFS2, fsync() is unnecessary before close()?
> 
> (On other filesystems it's necessary, of course).

To be honest, it doesn't really matter if it's necessary or not.
Writing an application to do as little as possible based on implicit
knowledge of the underlying filesystem seems like a really bad idea.
Particularly with the behavior of the filesystem can change based on
which config options you have set (writebuffer, etc).

Write you applications to be portable and cautious and you shouldn't
have a problem.

josh

Re: JFFS2 determine writing state

[Jamie Lokier]

Josh Boyer wrote:
> On Sat, 12 Jan 2008 10:03:43 +0000 Jamie Lokier <jamie@shareable.org> wrote:
> 
> > Josh Boyer wrote:
> > > The open call doesn't cause any writes, and close
> > > is supposed to flush all pending writes before it returns.
> > 
> > Oh, that's interesting.  So on JFFS2, fsync() is unnecessary
> > before close()?
> > 
> > (On other filesystems it's necessary, of course).
> 
> To be honest, it doesn't really matter if it's necessary or not.
> Writing an application to do as little as possible based on implicit
> knowledge of the underlying filesystem seems like a really bad idea.
> Particularly with the behavior of the filesystem can change based on
> which config options you have set (writebuffer, etc).
> 
> Write you applications to be portable and cautious and you shouldn't
> have a problem.

That's quite reasonable and of course I do call fsync, both on a file
before closing it, and then on its parent directory after renaming a
replacement file into place (just like usual mail software).

But the particular fs behaviour is relevant the other way around: I
have a program which calls open/write/close with small writes
moderately often (because it calls another program which actually
operates on the file).

If JFFS2 commits pending writes on every close, I should change things
to keep the file open between writes so they are coalesced and faster,
when I don't need the individual writes to be committed separately.
When I do need the data committed I can use fsync of course.

In which case, if a program A opens a file on JFFS2, and forks/execs
program B which writes data, then does an implicit close (when it
exits), while B's writes be committed immediately (which isn't wanted)
even though A still has the descriptor open?

Thanks,
-- Jamie

Re: JFFS2 determine writing state

[Jörn Engel]

On Tue, 15 January 2008 14:08:00 +0000, Jamie Lokier wrote:
> 
> But the particular fs behaviour is relevant the other way around: I
> have a program which calls open/write/close with small writes
> moderately often (because it calls another program which actually
> operates on the file).
> 
> If JFFS2 commits pending writes on every close, I should change things
> to keep the file open between writes so they are coalesced and faster,
> when I don't need the individual writes to be committed separately.

JFFS2 also commits on every write.  So you need userspace caching if you
want to coalesce things.  fwrite might be enough for that...

> When I do need the data committed I can use fsync of course.

...as long as you also fflush (or fclose) before fsync.

Jörn

-- 
Everything should be made as simple as possible, but not simpler.
-- Albert Einstein

Re: JFFS2 determine writing state

[Jamie Lokier]

Jörn Engel wrote:
> On Tue, 15 January 2008 14:08:00 +0000, Jamie Lokier wrote:
> > But the particular fs behaviour is relevant the other way around: I
> > have a program which calls open/write/close with small writes
> > moderately often (because it calls another program which actually
> > operates on the file).
> > 
> > If JFFS2 commits pending writes on every close, I should change things
> > to keep the file open between writes so they are coalesced and faster,
> > when I don't need the individual writes to be committed separately.
> 
> JFFS2 also commits on every write.  So you need userspace caching if you
> want to coalesce things.  fwrite might be enough for that...

Oh, ok, I thought it at least merged things in the page cache like
most other filesystems.  It seems it's more like O_DSYNC - or more
like O_SYNC (commits metadata changed for every write too)?

Is there a particular advantage, in terms of flash overhead,
compression, or mount/gc times, to writing blocks with a particular
size and alignment, such as 4k size and 4k offset-alignment-in-file?

Also, does LogFS (which I want to try later) have similar
characteristics?

> > When I do need the data committed I can use fsync of course.
> 
> ...as long as you also fflush (or fclose) before fsync.

Sure.  Thanks for your response.

-- Jamie

Re: JFFS2 determine writing state

[Jörn Engel]

On Tue, 15 January 2008 17:35:10 +0000, Jamie Lokier wrote:
> 
> Oh, ok, I thought it at least merged things in the page cache like
> most other filesystems.  It seems it's more like O_DSYNC - or more
> like O_SYNC (commits metadata changed for every write too)?

I don't think there is a really good equivalent.  Writes happen
immediatly, but the changes may not become effective immediatly.  In a
way, the worst of both worlds. :)

> Is there a particular advantage, in terms of flash overhead,
> compression, or mount/gc times, to writing blocks with a particular
> size and alignment, such as 4k size and 4k offset-alignment-in-file?

4k size and 4k alignment is best for all the reasons you cited above.

> Also, does LogFS (which I want to try later) have similar
> characteristics?

At the moment, yes.  Most likely I won't get the caching part done
before LCA.

Jörn

-- 
Victory in war is not repetitious.
-- Sun Tzu

Re: JFFS2 determine writing state

[Jamie Lokier]

Jörn Engel wrote:
> On Tue, 15 January 2008 17:35:10 +0000, Jamie Lokier wrote:
> > 
> > Oh, ok, I thought it at least merged things in the page cache like
> > most other filesystems.  It seems it's more like O_DSYNC - or more
> > like O_SYNC (commits metadata changed for every write too)?
> 
> I don't think there is a really good equivalent.  Writes happen
> immediatly, but the changes may not become effective immediatly.  In a
> way, the worst of both worlds. :)

I'm not sure if I understand what you mean by that.  Let me guess:

Do you mean that MTD write operations are started by write(), but
write() returns before the writes are committed reliably?

If so does fsync/fdatasync/sync_page_range ensure that the preceding
writes are committed reliably?  And does that include fsync on a
directory following an atomic rename (for reliable replacement of a
file)?

> > Also, does LogFS (which I want to try later) have similar
> > characteristics?
> 
> At the moment, yes.  Most likely I won't get the caching part done
> before LCA.

I guess write caching is an interesting beast for such a structured
filesystem.  It would seem like a good opportunity to coalesce some
tree node updates, something applications can't do themselves.
Looking forward to it :-)

-- Jamie

Re: JFFS2 determine writing state

[Jörn Engel]

On Tue, 15 January 2008 21:51:44 +0000, Jamie Lokier wrote:
> Jörn Engel wrote:
> > 
> > I don't think there is a really good equivalent.  Writes happen
> > immediatly, but the changes may not become effective immediatly.  In a
> > way, the worst of both worlds. :)
> 
> I'm not sure if I understand what you mean by that.  Let me guess:
> 
> Do you mean that MTD write operations are started by write(), but
> write() returns before the writes are committed reliably?

Yes.  Write() may return before or after reliable commit, depending on
hardware, config options and sheer luck.

> If so does fsync/fdatasync/sync_page_range ensure that the preceding
> writes are committed reliably?

Yes.

> And does that include fsync on a
> directory following an atomic rename (for reliable replacement of a
> file)?

Yes.

> I guess write caching is an interesting beast for such a structured
> filesystem.  It would seem like a good opportunity to coalesce some
> tree node updates, something applications can't do themselves.

That bit is actually done already, modulo some bugs.  Once those bugs
are sorted out, caching actual data is a straight-forward extention.

Jörn

-- 
The wise man seeks everything in himself; the ignorant man tries to get
everything from somebody else.
-- unknown