jffs2: too few erase blocks

jffs2: too few erase blocks

[Duke]

Hi All,
What's the deal with "too few erase blocks" message?
Is one block too few for a jffs2 partition?
My block size if 256K and I created a small jffs2 256K partition for
read/writing.
Is this really too small for jffs2? If so, is there anything I can do for this?

Thanks
D.

Re: jffs2: too few erase blocks

[Josh Boyer]

On Thu, 25 Oct 2007 09:06:29 -0500
Duke <ezbonites@gmail.com> wrote:

> Hi All,
> What's the deal with "too few erase blocks" message?
> Is one block too few for a jffs2 partition?
> My block size if 256K and I created a small jffs2 256K partition for
> read/writing.
> Is this really too small for jffs2? If so, is there anything I can do for this?

Yes, it's too small.  JFFS2 requires a certain percentage of blocks in
the partition to be reserved for it's usage.  At one point, it was 5
eraseblocks.

The only solution that I know of is to grow your partition to cope with
that limitation.

josh

Re: jffs2: too few erase blocks

[Jamie Lokier]

Josh Boyer wrote:
> > Hi All,
> > What's the deal with "too few erase blocks" message?
> > Is one block too few for a jffs2 partition?
> > My block size if 256K and I created a small jffs2 256K partition for
> > read/writing.
> > Is this really too small for jffs2? If so, is there anything I can do for this?
> 
> Yes, it's too small.  JFFS2 requires a certain percentage of blocks in
> the partition to be reserved for it's usage.  At one point, it was 5
> eraseblocks.

Fwiw, 5 blocks worth of free space is no guarantee that it'll be fine.
I have a 1MB JFFS2 partition with 64kB eraseblocks.  That's 320kB in 5
blocks.  But it can return disk full errors when reporting more than
320kB free with 'df' (somewhere between 5 and 6 blocks worth), and it
can also do strange things shortly before it returns disk full errors,
like spending lots of CPU on GC.

-- Jamie

Re: jffs2: too few erase blocks

[Duke]

On 10/25/07, Jamie Lokier <jamie@shareable.org> wrote:
> Josh Boyer wrote:
> > > Hi All,
> > > What's the deal with "too few erase blocks" message?
> > > Is one block too few for a jffs2 partition?
> > > My block size if 256K and I created a small jffs2 256K partition for
> > > read/writing.
> > > Is this really too small for jffs2? If so, is there anything I can do for this?
> >
> > Yes, it's too small.  JFFS2 requires a certain percentage of blocks in
> > the partition to be reserved for it's usage.  At one point, it was 5
> > eraseblocks.
>
> Fwiw, 5 blocks worth of free space is no guarantee that it'll be fine.
> I have a 1MB JFFS2 partition with 64kB eraseblocks.  That's 320kB in 5
> blocks.  But it can return disk full errors when reporting more than
> 320kB free with 'df' (somewhere between 5 and 6 blocks worth), and it
> can also do strange things shortly before it returns disk full errors,
> like spending lots of CPU on GC.

Is there another read/write filesystem I can use in this space?

If I were to create a subdirectory and rw to that dir only? and can I
do this with mtd_debug tool?

Re: jffs2: too few erase blocks

[Duke]

On 10/26/07, Duke <ezbonites@gmail.com> wrote:
> On 10/25/07, Jamie Lokier <jamie@shareable.org> wrote:
> > Josh Boyer wrote:
> > > > Hi All,
> > > > What's the deal with "too few erase blocks" message?
> > > > Is one block too few for a jffs2 partition?
> > > > My block size if 256K and I created a small jffs2 256K partition for
> > > > read/writing.
> > > > Is this really too small for jffs2? If so, is there anything I can do for this?
> > >
> > > Yes, it's too small.  JFFS2 requires a certain percentage of blocks in
> > > the partition to be reserved for it's usage.  At one point, it was 5
> > > eraseblocks.
> >
> > Fwiw, 5 blocks worth of free space is no guarantee that it'll be fine.
> > I have a 1MB JFFS2 partition with 64kB eraseblocks.  That's 320kB in 5
> > blocks.  But it can return disk full errors when reporting more than
> > 320kB free with 'df' (somewhere between 5 and 6 blocks worth), and it
> > can also do strange things shortly before it returns disk full errors,
> > like spending lots of CPU on GC.
>
> Is there another read/write filesystem I can use in this space?
>
> If I were to create a subdirectory and rw to that dir only? and can I
> do this with mtd_debug tool?

Just to clarity myself a bit here. There is nothing stopping me from
using the cp or mv (or any other file modification command for that
matter) to update one directory but I would like to be able to verify
the data is actually written and I thought perhaps mtd_debug or some
other mtd tool might be able to verify this. Maybe it's not maybe it
is, I would like to find out.
Thanks.
-D

Re: jffs2: too few erase blocks

[Jamie Lokier]

Duke wrote:
> Just to clarity myself a bit here. There is nothing stopping me from
> using the cp or mv (or any other file modification command for that
> matter) to update one directory but I would like to be able to verify
> the data is actually written and I thought perhaps mtd_debug or some
> other mtd tool might be able to verify this. Maybe it's not maybe it
> is, I would like to find out.

If you call "fsync()" or "fdatasync()" after writing the data to a
file, that is suppoe to ensure the data is written when the function
returns.

That's supposed to happen, but I don't know if JFFS2 actually does
that.

If you call fsync() on a directory handle, that traditionally ensures
the file names in that directory are written.  So to be sure of
creating a file and writing to it, you would create the file, write
it, fsync() it, then open the directory containing the file and
fsync() the directory.  For example, email delivery programs use that
strategy before reporting to a sender that the email is safely stored.

But again, I don't know if JFFS2 actually does provide those assurances.

-- Jamie

Re: jffs2: too few erase blocks

[David Woodhouse]

On Sun, 2007-10-28 at 18:02 +0000, Jamie Lokier wrote:
> 
> If you call "fsync()" or "fdatasync()" after writing the data to a
> file, that is suppoe to ensure the data is written when the function
> returns.
> 
> That's supposed to happen, but I don't know if JFFS2 actually does
> that.
> 
> If you call fsync() on a directory handle, that traditionally ensures
> the file names in that directory are written.  So to be sure of
> creating a file and writing to it, you would create the file, write
> it, fsync() it, then open the directory containing the file and
> fsync() the directory.  For example, email delivery programs use that
> strategy before reporting to a sender that the email is safely stored.
> 
> But again, I don't know if JFFS2 actually does provide those
> assurances.

Yes, it does. In fact, on NOR flash you get entirely synchronous
behaviour even _without_ explicit syncs -- effectively the same as
MS_SYNC.

-- 
dwmw2

Re: jffs2: too few erase blocks

[Jörn Engel]

On Sun, 28 October 2007 22:59:07 -0400, David Woodhouse wrote:
> 
> Yes, it does. In fact, on NOR flash you get entirely synchronous
> behaviour even _without_ explicit syncs -- effectively the same as
> MS_SYNC.

On older ones.  There appears to be a tendency for newer flashes to
require wbuf and hence sync.

Jörn

-- 
Joern's library part 4:
http://www.paulgraham.com/spam.html

Re: jffs2: too few erase blocks

[Jamie Lokier]

Jörn Engel wrote:
> On Sun, 28 October 2007 22:59:07 -0400, David Woodhouse wrote:
> > 
> > Yes, it does. In fact, on NOR flash you get entirely synchronous
> > behaviour even _without_ explicit syncs -- effectively the same as
> > MS_SYNC.
> 
> On older ones.  There appears to be a tendency for newer flashes to
> require wbuf and hence sync.

When you say "sync", do fdatasync() / fsync() on open files and
directories flush the necessary blocks from wbuf for those
files/directories data and their inode metadata, for JFFS2?

I have been calling sync(), but I wonder if fdatasync/fsync do as they
are supposed to?

(If you know about other fses like LogFS/Yaffs/UBIFS that would be
handy, too).

Probably all are unnecessary in my specific application, as I use a
JFFS2 with cfi_cmdset_0002, which I get the impression doesn't buffer
any writes anyway.  But I like to get the application code right, in
case I change to another device.

-- Jamie

Re: jffs2: too few erase blocks

[David Woodhouse]

On Mon, 2007-10-29 at 20:51 +0000, Jamie Lokier wrote:
> When you say "sync", do fdatasync() / fsync() on open files and
> directories flush the necessary blocks from wbuf for those
> files/directories data and their inode metadata, for JFFS2?
> 
> I have been calling sync(), but I wonder if fdatasync/fsync do as they
> are supposed to?

They certainly should; if you find otherwise, please tell me.

-- 
dwmw2

Re: jffs2: too few erase blocks

[Jamie Lokier]

David Woodhouse wrote:
> On Mon, 2007-10-29 at 20:51 +0000, Jamie Lokier wrote:
> > When you say "sync", do fdatasync() / fsync() on open files and
> > directories flush the necessary blocks from wbuf for those
> > files/directories data and their inode metadata, for JFFS2?
> > 
> > I have been calling sync(), but I wonder if fdatasync/fsync do as they
> > are supposed to?
> 
> They certainly should; if you find otherwise, please tell me.

So far, with JFFS2 on cfi_cmdset_0002 NOR flash, on a Linux 2.4.26-uc0
kernel, I haven't seen any data failed to be committed after sync()
returns.

As JFFS2 is synchronous with this type of flash we shouldn't
be surprised, and that makes fsync/fdatasync nops.

If I ever do use another type of flash with JFFS2 (or any of the other
flash fses) then I'll be sure to report any problems with
fsync/fdatasync not committing data.  See also my earlier message on
what you might add to a test suite for these fses, to check
fsync/fdatasync implementation much better than random application
usage and system crashes can possibly do.

Thanks,
 -- Jamie

Re: jffs2: too few erase blocks

[Jörn Engel]

On Mon, 29 October 2007 20:51:25 +0000, Jamie Lokier wrote:
> 
> When you say "sync", do fdatasync() / fsync() on open files and
> directories flush the necessary blocks from wbuf for those
> files/directories data and their inode metadata, for JFFS2?
> 
> I have been calling sync(), but I wonder if fdatasync/fsync do as they
> are supposed to?
> 
> (If you know about other fses like LogFS/Yaffs/UBIFS that would be
> handy, too).

fsync/fdatasync should work on logfs.  If you know a good way to test
this, I'd add it to my suite of regression tests.  Right now I have to
rely on reading the code.

> Probably all are unnecessary in my specific application, as I use a
> JFFS2 with cfi_cmdset_0002, which I get the impression doesn't buffer
> any writes anyway.  But I like to get the application code right, in
> case I change to another device.

Any filesystem should follow the standards here.  Anything else is a
bug.

Jörn

-- 
Victory in war is not repetitious.
-- Sun Tzu

Re: jffs2: too few erase blocks

[Jamie Lokier]

Jörn Engel wrote:
> fsync/fdatasync should work on logfs.  If you know a good way to test
> this, I'd add it to my suite of regression tests.  Right now I have to
> rely on reading the code.

Here's an idea for testing:

    1. Open file w/O_CREAT|O_EXCL, write some data, then after write()
       returns, immediately force the MTD device read-only somehow (so
       anything not committed cannot be committed after this point).
       Then unmount and mount, and read the file.  Then open the file:
       either it should't exist, or the written data shouldn't be
       present, if buffering is working.

    2. If the file doesn't exist in test 1, call fsync() after opening
       the file and before writing, and also call fsync() on the
       directory containing the created file.  Afterwards, the file
       should exist, but not contain the written data.

    3. Variation: create file, write data, then fsync(), then
       overwrite with new data (not changing the length).  Afterwards,
       the read data should be the first written data, not the second.

    4. Same as tests 1, 2 and 3, but call fsync() after writing.  This
       time, the written data should always be read back.  This tests
       that fsync() makes a difference.

    5. Same as test 3, but call fdatasync() instead of fsync().  The
       second written data should always be read back.

    6. Create the file, set modtime to some fixed time A, call
       fsync(), set modtime to fixed time B, then set MTD read-only,
       unmount, mount.  stat() the file: you might get time A.

    7. As test 6, but call fsync() following the second setting of
       modtime.  Afterwards, stat() should return time B, proving that
       fsync() commits the inode change.

    8. As test 7 but using fdatasync(): if it is really different from
       fsync(), you should get time A the same as test 6.

    9. I'm not sure if fdatasync() is required to update the length of
       a file, when the file is extended by write() or ftruncate().

    10. Variations on the above involving small writes, large writes,
       and scattered writes.

    10. fsync() tests on a directory involve doing changes to
       directories (changing the modtime, and
       creating/linking/unlinking/renaming in them) and checking the
       name change after marking the MTD device read-only and
       unmounting + mount.  Quite similar to the above tests, but
       using directory operations.  I'm not sure if fdatasync() has
       any meaning for directories (indeed I'm not sure if fsync()
       does formally).

> > Probably all are unnecessary in my specific application, as I use a
> > JFFS2 with cfi_cmdset_0002, which I get the impression doesn't buffer
> > any writes anyway.  But I like to get the application code right, in
> > case I change to another device.
> 
> Any filesystem should follow the standards here.  Anything else is a
> bug.

True.  But JFFS2 is a bit buggy from an application point of view (*),
and we care about what an application can actually rely on in
practice, not what the standard says :-)

(*) Hence the subject of this thread, and the uncertainty of the
answer to that question.  Do any of the flash filesystems in
development guarantee a specific amount of the partition can actually
be used for data, and return "disk full" deterministically at that
point, for a given set of file contents?  Does the answer change if
compression is disable?  Do any of them not go suspiciously crazy with
the CPU for a whole minute when it's nearly full, as JFFS2 GC threads
do occasionally?  I see lots of nice things in the white papers about
the new fses, but GC problems, and the amount of space required, seems
curiously glossed over.  Does this mean they're fixed?  I have been
bitten hard by these areas of JFFS2.  It wasn't pretty.

Thanks :-)
-- Jamie

Re: jffs2: too few erase blocks

[Jörn Engel]

On Mon, 29 October 2007 22:46:38 +0000, Jamie Lokier wrote:
> 
> Here's an idea for testing:
> 
> [...]

Thank you.  I won't get hacking on this right away, but it is
appreciated.

> > Any filesystem should follow the standards here.  Anything else is a
> > bug.
> 
> True.  But JFFS2 is a bit buggy from an application point of view (*),
> and we care about what an application can actually rely on in
> practice, not what the standard says :-)

Notice the magical word "here". :)

Jffs2 and logfs do diverge from the standards, or at least from other
filesystems.  I am aware of two differences:
- all mounts implicitly set noatime,
- rewriting data in existing non-sparse files can return -ENOSPC.

The first makes at least some sense and the second cannot be avoided for
compressed files.  Logfs allows files to be non-compressing, though.
Any further differences are likely bugs.

> (*) Hence the subject of this thread, and the uncertainty of the
> answer to that question.  Do any of the flash filesystems in
> development guarantee a specific amount of the partition can actually
> be used for data, and return "disk full" deterministically at that
> point, for a given set of file contents?

I'm not sure if I interpret this correctly.  With logfs, compression is
optional.  Disabling compression will remove one uncertainty.  Also both
data and metadata draw from the same pool.  If you create a million
empty files, the space for file content is reduced.  After those two
uncertainties, logfs is deterministic.  Garbage collection will neither
increase nor reduce the amount of available free space.

> Does the answer change if
> compression is disable?  Do any of them not go suspiciously crazy with
> the CPU for a whole minute when it's nearly full, as JFFS2 GC threads
> do occasionally?

Depending on the write pattern, they all will.  When your filesystem is
99% full and your writes are completely random, you end up garbage
collecting 99% old data for every 1% new data.

Well, maybe they won't go crazy for a whole minute.  That can be
avoided.  But they will become slow as molasses.

Two things can be done about this.  First, logfs will let you specify
some amount of space reserved for GC.  With 5% reserved, the
pathological case is much better than with 1% or 0% reserved.

Secondly, the applications should have some amount of structure in their
writes.  If 70% of the filesystem is never moved, the filesystem can
actually get some usage spread.  Having some segments 100% full and some
100% empty translates to good performance.  Having all equally full, as
happens for completely random writes, will give you molasses.

Jörn

-- 
You ain't got no problem, Jules. I'm on the motherfucker. Go back in
there, chill them niggers out and wait for the Wolf, who should be
coming directly.
-- Marsellus Wallace

Re: jffs2: too few erase blocks

[Duke]

On 10/30/07, Jörn Engel <joern@logfs.org> wrote:
> On Mon, 29 October 2007 22:46:38 +0000, Jamie Lokier wrote:
> >
> > Here's an idea for testing:
> >
> > [...]
>
> Thank you.  I won't get hacking on this right away, but it is
> appreciated.
>
> > > Any filesystem should follow the standards here.  Anything else is a
> > > bug.
> >
> > True.  But JFFS2 is a bit buggy from an application point of view (*),
> > and we care about what an application can actually rely on in
> > practice, not what the standard says :-)
>
> Notice the magical word "here". :)
>
> Jffs2 and logfs do diverge from the standards, or at least from other
> filesystems.  I am aware of two differences:
> - all mounts implicitly set noatime,
> - rewriting data in existing non-sparse files can return -ENOSPC.
>
> The first makes at least some sense and the second cannot be avoided for
> compressed files.  Logfs allows files to be non-compressing, though.
> Any further differences are likely bugs.
>
> > (*) Hence the subject of this thread, and the uncertainty of the
> > answer to that question.  Do any of the flash filesystems in
> > development guarantee a specific amount of the partition can actually
> > be used for data, and return "disk full" deterministically at that
> > point, for a given set of file contents?
>
> I'm not sure if I interpret this correctly.  With logfs, compression is
> optional.  Disabling compression will remove one uncertainty.  Also both
> data and metadata draw from the same pool.  If you create a million
> empty files, the space for file content is reduced.  After those two
> uncertainties, logfs is deterministic.  Garbage collection will neither
> increase nor reduce the amount of available free space.
>
> > Does the answer change if
> > compression is disable?  Do any of them not go suspiciously crazy with
> > the CPU for a whole minute when it's nearly full, as JFFS2 GC threads
> > do occasionally?
>
> Depending on the write pattern, they all will.  When your filesystem is
> 99% full and your writes are completely random, you end up garbage
> collecting 99% old data for every 1% new data.
>
> Well, maybe they won't go crazy for a whole minute.  That can be
> avoided.  But they will become slow as molasses.
>
> Two things can be done about this.  First, logfs will let you specify
> some amount of space reserved for GC.  With 5% reserved, the
> pathological case is much better than with 1% or 0% reserved.
>
> Secondly, the applications should have some amount of structure in their
> writes.  If 70% of the filesystem is never moved, the filesystem can
> actually get some usage spread.  Having some segments 100% full and some
> 100% empty translates to good performance.  Having all equally full, as
> happens for completely random writes, will give you molasses.
>
> Jörn


After all is said and done, can the minimum number of jffs2 blocks be
determined? Or it varies from system to system? Or flash system type
(NOR flash in my case).

--D.

Re: jffs2: too few erase blocks

[Jörn Engel]

On Tue, 30 October 2007 15:08:11 -0500, Duke wrote:
> 
> After all is said and done, can the minimum number of jffs2 blocks be
> determined? Or it varies from system to system? Or flash system type
> (NOR flash in my case).

It can.  Please read jffs2_calc_trigger_levels() in fs/jffs2/build.c.
Maybe those numbers can even be lowered a little.  But you get to do the
math, if that is your wish.

Jörn

-- 
Joern's library part 15:
http://www.knosof.co.uk/cbook/accu06a.pdf

Re: jffs2: too few erase blocks

[Duke]

On 10/30/07, Jörn Engel <joern@logfs.org> wrote:
> On Tue, 30 October 2007 15:08:11 -0500, Duke wrote:
> >
> > After all is said and done, can the minimum number of jffs2 blocks be
> > determined? Or it varies from system to system? Or flash system type
> > (NOR flash in my case).
>
> It can.  Please read jffs2_calc_trigger_levels() in fs/jffs2/build.c.
> Maybe those numbers can even be lowered a little.  But you get to do the
> math, if that is your wish.
>
Ok. From what I've calculated and tested I wouldn't be able to meet
the minimum so I'm trying to use one big jffs2 partition and devide by
dirs.
The problem I'm having with this is not being able to untar files into
a directory essentially overwriting the old files. This seem to be
because untaring takes too much memory. Is there a better way to
replace these directory files (which is about 200 files)?