[UBIFS] Filesystem capacity

[UBIFS] Filesystem capacity

[Adam S. Turowski]

Hi,
Can anyone explain differences in filesystem capacity between jffs2 and 
ubifs?
Kernel 2.6.28
mtd3 29MB nor flash
mtd4 31MB nand flash
File created by dd-ing from /dev/urandom:
jffs2:
nor: 28361 kB
nand: 31200 kB

ubifs:
one volume created on mtd3
UBIFS: mounted UBI device 0, volume 0, name "root"
UBIFS: file system size:   28676736 bytes (28004 KiB, 27 MiB, 219 LEBs)
UBIFS: journal size:       1440384 bytes (1406 KiB, 1 MiB, 11 LEBs)
UBIFS: media format:       4 (latest is 4)
UBIFS: default compressor: LZO
UBIFS: reserved for root:  1417227 bytes (1384 KiB)

one volume created on mtd4
UBIFS: mounted UBI device 1, volume 0, name "data"
UBIFS: file system size:   31870976 bytes (31124 KiB, 30 MiB, 2008 LEBs)
UBIFS: journal size:       1603072 bytes (1565 KiB, 1 MiB, 101 LEBs)
UBIFS: media format:       4 (latest is 4)
UBIFS: default compressor: LZO
UBIFS: reserved for root:  1575089 bytes (1538 KiB)

nor: 26960 kB (I can live with that)
nand: 23552 kB (With that I cannot)

Any suggestions?

--
Regards,
Adam Turowski

Re: [UBIFS] Filesystem capacity

[Adrian Hunter]

Adam S. Turowski wrote:
> Hi,
> Can anyone explain differences in filesystem capacity between jffs2 and
> ubifs?
> Kernel 2.6.28
> mtd3 29MB nor flash
> mtd4 31MB nand flash
> File created by dd-ing from /dev/urandom:
> jffs2:
> nor: 28361 kB
> nand: 31200 kB
> 
> ubifs:
> one volume created on mtd3
> UBIFS: mounted UBI device 0, volume 0, name "root"
> UBIFS: file system size:   28676736 bytes (28004 KiB, 27 MiB, 219 LEBs)
> UBIFS: journal size:       1440384 bytes (1406 KiB, 1 MiB, 11 LEBs)
> UBIFS: media format:       4 (latest is 4)
> UBIFS: default compressor: LZO
> UBIFS: reserved for root:  1417227 bytes (1384 KiB)
> 
> one volume created on mtd4
> UBIFS: mounted UBI device 1, volume 0, name "data"
> UBIFS: file system size:   31870976 bytes (31124 KiB, 30 MiB, 2008 LEBs)
> UBIFS: journal size:       1603072 bytes (1565 KiB, 1 MiB, 101 LEBs)
> UBIFS: media format:       4 (latest is 4)
> UBIFS: default compressor: LZO
> UBIFS: reserved for root:  1575089 bytes (1538 KiB)
> 
> nor: 26960 kB (I can live with that)
> nand: 23552 kB (With that I cannot)
> 
> Any suggestions?

It is because the LEB size is relatively small, and UBIFS does not
fit data into the ends of eraseblocks the way JFFS2 does.  Your options
are:
	1. use JFFS2
	2. amend your NAND driver to pretend that eraseblocks are bigger
than they really are, by treating 2 (or 4 or 8 etc) as one eraseblock
	3. create another MTD driver that sits on top of the NAND driver
and does the same as 2

The disadvantage of 2 or 3 is that it also multiples the effective number
of bad blocks.

Note that the problem is exacerbated by writing un-compressible (e.g. random)
data because you end up fitting only 3 x 4.1K data nodes per 16K eraseblock.

If you are interested in 3, I could help.

Re: [UBIFS] Filesystem capacity

[Jamie Lokier]

Adrian Hunter wrote:
> Adam S. Turowski wrote:
> > jffs2:
> > nor: 28361 kB
> > nand: 31200 kB
> > 
> > [ubifs]:
> > nor: 26960 kB (I can live with that)
> > nand: 23552 kB (With that I cannot)
> 
> It is because the LEB size is relatively small, and UBIFS does not
> fit data into the ends of eraseblocks the way JFFS2 does.  Your options
> are:
> 	1. use JFFS2
> 	2. amend your NAND driver to pretend that eraseblocks are bigger
> than they really are, by treating 2 (or 4 or 8 etc) as one eraseblock
> 	3. create another MTD driver that sits on top of the NAND driver
> and does the same as 2
> 
> The disadvantage of 2 or 3 is that it also multiples the effective number
> of bad blocks.

Is this a major flaw of UBIFS?  I was thinking of using UBIFS for a
project but this makes me wonder if it's a good idea.

What does "UBIFS does not fit data into the ends of eraseblocks the
way JFFS2 does" mean?

Thanks,
-- Jamie

Re: [UBIFS] Filesystem capacity

[Artem Bityutskiy]

On Tue, 2009-02-17 at 00:39 +0000, Jamie Lokier wrote:
> > The disadvantage of 2 or 3 is that it also multiples the effective number
> > of bad blocks.
> 
> Is this a major flaw of UBIFS?  I was thinking of using UBIFS for a
> project but this makes me wonder if it's a good idea.

I do not see any major flaw.

> What does "UBIFS does not fit data into the ends of eraseblocks the
> way JFFS2 does" mean?

JFFS2 can split nodes on 2 parts and put 1 part to the end of one
eraseblock, and the second part to the beginning of another eraseblocks.
UBIFS does not split data blocks. This is the difference.

However, UBIFS tries to write small data nodes to the ends of
eraseblocks to minimize the wastage. And I am not sure JFFS2 is doing
better job, but I did not deliberately compared them.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: [UBIFS] Filesystem capacity

[Artem Bityutskiy]

On Tue, 2009-02-17 at 00:39 +0000, Jamie Lokier wrote:
> Adrian Hunter wrote:
> > Adam S. Turowski wrote:
> > > jffs2:
> > > nor: 28361 kB
> > > nand: 31200 kB
> > > 
> > > [ubifs]:
> > > nor: 26960 kB (I can live with that)
> > > nand: 23552 kB (With that I cannot)
> > 
> > It is because the LEB size is relatively small, and UBIFS does not
> > fit data into the ends of eraseblocks the way JFFS2 does.  Your options
> > are:
> > 	1. use JFFS2
> > 	2. amend your NAND driver to pretend that eraseblocks are bigger
> > than they really are, by treating 2 (or 4 or 8 etc) as one eraseblock
> > 	3. create another MTD driver that sits on top of the NAND driver
> > and does the same as 2
> > 
> > The disadvantage of 2 or 3 is that it also multiples the effective number
> > of bad blocks.
> 
> Is this a major flaw of UBIFS?  I was thinking of using UBIFS for a

I'd call this a drawback. We mostly desinged UBIFS for modern NANDs,
which have larger eraseblocks (128KiB or more). 16KiB eraseblocks are
very small so UBIFS wastes more space than JFFS2. Adrian pointed the
ways how one can make UBIFS waste less for flashes like this. We just
did not do this.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: [UBIFS] Filesystem capacity

[Artem Bityutskiy]

On Mon, 2009-02-16 at 17:54 +0200, Adrian Hunter wrote:
> Adam S. Turowski wrote:
> > Hi,
> > Can anyone explain differences in filesystem capacity between jffs2 and
> > ubifs?
> > Kernel 2.6.28
> > mtd3 29MB nor flash
> > mtd4 31MB nand flash
> > File created by dd-ing from /dev/urandom:
> > jffs2:
> > nor: 28361 kB
> > nand: 31200 kB
> > 
> > ubifs:
> > one volume created on mtd3
> > UBIFS: mounted UBI device 0, volume 0, name "root"
> > UBIFS: file system size:   28676736 bytes (28004 KiB, 27 MiB, 219 LEBs)
> > UBIFS: journal size:       1440384 bytes (1406 KiB, 1 MiB, 11 LEBs)
> > UBIFS: media format:       4 (latest is 4)
> > UBIFS: default compressor: LZO
> > UBIFS: reserved for root:  1417227 bytes (1384 KiB)
> > 
> > one volume created on mtd4
> > UBIFS: mounted UBI device 1, volume 0, name "data"
> > UBIFS: file system size:   31870976 bytes (31124 KiB, 30 MiB, 2008 LEBs)
> > UBIFS: journal size:       1603072 bytes (1565 KiB, 1 MiB, 101 LEBs)
> > UBIFS: media format:       4 (latest is 4)
> > UBIFS: default compressor: LZO
> > UBIFS: reserved for root:  1575089 bytes (1538 KiB)
> > 
> > nor: 26960 kB (I can live with that)
> > nand: 23552 kB (With that I cannot)
> > 
> > Any suggestions?
> 
> It is because the LEB size is relatively small, and UBIFS does not
> fit data into the ends of eraseblocks the way JFFS2 does.  Your options
> are:

Well, UBIFS does try to squeeze small nodes to the ends of eraseblocks.
And I am not convinced it is worse than JFFS2 in this respect, unless
someone shows this with a test. Where did you get those numbers?
Did you try to measure how much you can really fit? Did you read this:
http://www.linux-mtd.infradead.org/faq/ubifs.html#L_df_report
?

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: [UBIFS] Filesystem capacity

[Adam S. Turowski]

Artem Bityutskiy pisze:
> Well, UBIFS does try to squeeze small nodes to the ends of eraseblocks.
> And I am not convinced it is worse than JFFS2 in this respect, unless
> someone shows this with a test. Where did you get those numbers?
> Did you try to measure how much you can really fit? Did you read this:
> http://www.linux-mtd.infradead.org/faq/ubifs.html#L_df_report
> ?
> 

As I wrote in my first post, I used dd from /dev/urandom to create files 
and those were sizes of those files.
df also reports the same:
Filesystem                Size      Used Available Use% Mounted on
ubi0:root                24.9M     24.9M         0 100% /part_root
ubi1:data                22.4M     22.4M         0 100% /part_data

--
Regards,
Adam Turowski

Re: [UBIFS] Filesystem capacity

[Artem Bityutskiy]

On Tue, 2009-02-17 at 08:39 +0100, Adam S. Turowski wrote:
> Artem Bityutskiy pisze:
> > Well, UBIFS does try to squeeze small nodes to the ends of eraseblocks.
> > And I am not convinced it is worse than JFFS2 in this respect, unless
> > someone shows this with a test. Where did you get those numbers?
> > Did you try to measure how much you can really fit? Did you read this:
> > http://www.linux-mtd.infradead.org/faq/ubifs.html#L_df_report
> > ?
> > 
> 
> As I wrote in my first post, I used dd from /dev/urandom to create files 
> and those were sizes of those files.
> df also reports the same:
> Filesystem                Size      Used Available Use% Mounted on
> ubi0:root                24.9M     24.9M         0 100% /part_root
> ubi1:data                22.4M     22.4M         0 100% /part_data

OK, then indeed Adrian wrote exactly the right thing. You have huge
4144-byte uncompressible nodes. You fit 3 of them to each eraseblock,
and you waste 3440 bytes in each eraseblock. JFFS2 would jam a little
more data, because it can split big blocks on parts.

In real life you will likely have compressible data, and many small
files, so you will have small data nodes and many inode nodes, which
are 160 bytes in size, so you will fit more.

However, I just agree with what Adrian said you in his replay.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)

Re: [UBIFS] Filesystem capacity

[Jamie Lokier]

Artem Bityutskiy wrote:
> OK, then indeed Adrian wrote exactly the right thing. You have huge
> 4144-byte uncompressible nodes. You fit 3 of them to each eraseblock,
> and you waste 3440 bytes in each eraseblock. JFFS2 would jam a little
> more data, because it can split big blocks on parts.
> 
> In real life you will likely have compressible data, and many small
> files, so you will have small data nodes and many inode nodes, which
> are 160 bytes in size, so you will fit more.

Also, in real life, JFFS2 will behave badly when the filesystem is
completely full.  Its garbage collector can use 100% CPU for minutes
even on a small NOR flash, if the filesystem fills up too much.  It
happens because it's difficult to reorganise the data when there's no
spare room.

I wonder if the "lost" space in UBIFS is helpful to prevent long
garbage collection cycles?

-- Jamie

Re: [UBIFS] Filesystem capacity

[Artem Bityutskiy]

On Tue, 2009-02-17 at 09:16 +0000, Jamie Lokier wrote:
> Artem Bityutskiy wrote:
> > OK, then indeed Adrian wrote exactly the right thing. You have huge
> > 4144-byte uncompressible nodes. You fit 3 of them to each eraseblock,
> > and you waste 3440 bytes in each eraseblock. JFFS2 would jam a little
> > more data, because it can split big blocks on parts.
> > 
> > In real life you will likely have compressible data, and many small
> > files, so you will have small data nodes and many inode nodes, which
> > are 160 bytes in size, so you will fit more.
> 
> Also, in real life, JFFS2 will behave badly when the filesystem is
> completely full.  Its garbage collector can use 100% CPU for minutes
> even on a small NOR flash, if the filesystem fills up too much.  It
> happens because it's difficult to reorganise the data when there's no
> spare room.
> 
> I wonder if the "lost" space in UBIFS is helpful to prevent long
> garbage collection cycles?

In JFFS2 GC is random, which means it may:
1. Pick an full EB to GC, this is how JFFS2 does wear-levelling
2. Pick a non-optimal EB to GC, which means that instead of picking EB
with largest amount of dirty space, JFFS2 picks an EB with not so much
dirty space

In UBIFS, we always try pick the most optimal LEB, with most of dirty
space. We have hashtables for this, where we keep track of such LEBs.
This makes UBIFS GC work faster.

And yes, in a way the wasted space helps. And in UBIFS the GC will not
even try GC-ing an LEB if the amount of dirty+free space in it is less
than NAND page size.

However, UBIFS may be optimized and one may teach the GC to try putting
small nodes to the wasted areas at the ends of eraseblocks. We just did
not do this. This is one more way to improve UBIFS.

-- 
Best regards,
Artem Bityutskiy (Битюцкий Артём)