Question about mkfs.jffs2 "-s" option

Question about mkfs.jffs2 "-s" option

[Grant Edwards]

[apologies if this isn't the right forum for mkfs.jffs2 questions]

According the the mkfs.jffs2 man page (from mtd-utils 1.4.8):

  -s, --pagesize=SIZE
  
     Use page size SIZE.  The default is 4 KiB.  This size is the
     maximum size of a data node.

I'm using NAND flash with an erase size of 128K.

I've got a directory tree with 68 files. A few are largish (100KB),
but most of them small (a few hundred bytes or less).  They contain a
total of 680K bytes of data and take up 900K of space on an ext2
filesystem.

When I do a "mkfs.jffs2 -e128 -l -n" I get a jffs2 image with a size
of 300K.  That seems reasonable.

If I specify a page size of 4K (which according to the man page is the
default) by doing "mkfs.jffs2 -e128 -s4KiB -n -l", I get a jffs2 image
with a size of 12MB.

According to the man page the default page size is 4K, so specifying a
page size of 4K shouldn't change anything, right?

Why does specifying a page size of 4K make the jffs2 image 40X larger?

The NAND flash I'm using has to be written in 4K blocks -- isn't that
the "page size"?

-- 
Grant Edwards               grant.b.edwards        Yow! Either CONFESS now or
                                  at               we go to "PEOPLE'S COURT"!!
                              gmail.com            

Re: Question about mkfs.jffs2 "-s" option

[Shmulik Ladkani]

Hi Grant,

On Wed, 2 May 2012 16:08:38 +0000 (UTC) Grant Edwards <grant.b.edwards@gmail.com> wrote:
> [apologies if this isn't the right forum for mkfs.jffs2 questions]
> 
> According the the mkfs.jffs2 man page (from mtd-utils 1.4.8):
> 
>   -s, --pagesize=SIZE
>   
>      Use page size SIZE.  The default is 4 KiB.  This size is the
>      maximum size of a data node.
> 
> I'm using NAND flash with an erase size of 128K.
> 
> I've got a directory tree with 68 files. A few are largish (100KB),
> but most of them small (a few hundred bytes or less).  They contain a
> total of 680K bytes of data and take up 900K of space on an ext2
> filesystem.
> 
> When I do a "mkfs.jffs2 -e128 -l -n" I get a jffs2 image with a size
> of 300K.  That seems reasonable.
> 
> If I specify a page size of 4K (which according to the man page is the
> default) by doing "mkfs.jffs2 -e128 -s4KiB -n -l", I get a jffs2 image
> with a size of 12MB.
> 
> According to the man page the default page size is 4K, so specifying a
> page size of 4K shouldn't change anything, right?
> 
> Why does specifying a page size of 4K make the jffs2 image 40X larger?
> 
> The NAND flash I'm using has to be written in 4K blocks -- isn't that
> the "page size"?

Looks like mkfs.jffs2 sets 
	case 's':
		page_size = strtol(optarg, NULL, 0);

and does not further parse KiB/MiB tokens.

So my buest guess, try to specify '-s 4096' instead of '4KiB'.

Regards,
Shmulik

Re: Question about mkfs.jffs2 "-s" option

[Ricard Wanderlof]

On Wed, 2 May 2012, Grant Edwards wrote:

> The NAND flash I'm using has to be written in 4K blocks -- isn't that
> the "page size"?

The smallest writable unit in a NAND flash is called a 'page', but AFAIK 
this is not what the -s option in mkfs.jffs2 is for. Instead, it is 
related for the kernel memory managament page size, which is 4k for most 
architectures. I think the jffs2 driver determines the smallest writable 
unit runtime; it is not a parameter during the file system creation. 
However the erase block size must be specified to mkfs.jffs2 as it affects 
the structure of the file system image.

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: Question about mkfs.jffs2 "-s" option

[Grant Edwards]

On 2012-05-03, Shmulik Ladkani <shmulik.ladkani@gmail.com> wrote:

> Looks like mkfs.jffs2 sets 
> 	case 's':
> 		page_size = strtol(optarg, NULL, 0);
>
> and does not further parse KiB/MiB tokens.
>
> So my buest guess, try to specify '-s 4096' instead of '4KiB'.

I should have tried that.  Using -s4096 indeed produces the same
result as without the -s option.  I read the description of how the
erase block size was parsed, and assumed that applied to the the page
size as well.

-- 
Grant Edwards               grant.b.edwards        Yow! Gee, I feel kind of
                                  at               LIGHT in the head now,
                              gmail.com            knowing I can't make my
                                                   satellite dish PAYMENTS!

Re: Question about mkfs.jffs2 "-s" option

[Grant Edwards]

On 2012-05-03, Ricard Wanderlof <ricard.wanderlof@axis.com> wrote:
>
> On Wed, 2 May 2012, Grant Edwards wrote:
>
>> The NAND flash I'm using has to be written in 4K blocks -- isn't that
>> the "page size"?
>
> The smallest writable unit in a NAND flash is called a 'page', but AFAIK 
> this is not what the -s option in mkfs.jffs2 is for. Instead, it is 
> related for the kernel memory managament page size, which is 4k for most 
> architectures. I think the jffs2 driver determines the smallest writable 
> unit runtime; it is not a parameter during the file system creation. 
> However the erase block size must be specified to mkfs.jffs2 as it affects 
> the structure of the file system image.

So the '-s' option should always be set to the kernel page size for
the target?

-- 
Grant Edwards               grant.b.edwards        Yow! Hello?  Enema Bondage?
                                  at               I'm calling because I want
                              gmail.com            to be happy, I guess ...

Re: Question about mkfs.jffs2 "-s" option

[Shmulik Ladkani]

Hi Ricard,
	
On Thu, 3 May 2012 09:35:14 +0200 (CEST) Ricard Wanderlof <ricard.wanderlof@axis.com> wrote:
> 
> On Wed, 2 May 2012, Grant Edwards wrote:
> 
> > The NAND flash I'm using has to be written in 4K blocks -- isn't that
> > the "page size"?
> 
> The smallest writable unit in a NAND flash is called a 'page', but AFAIK 
> this is not what the -s option in mkfs.jffs2 is for. Instead, it is 
> related for the kernel memory managament page size, which is 4k for most 
> architectures.

According to MKFS.JFFS2(1),

	-s, --pagesize=SIZE
		Use page size SIZE.  The default is 4 KiB.  This size is
		the maximum size of a data node.

And according to http://linux-mtd.infradead.org/~dwmw2/jffs2.pdf

	In order to facilitate rapid decompression of
	data upon readpage() requests, nodes contain
	no more than a single page of data, according to
	the hardware page size on the target platform.

Wow.
I always assumed I had to set it according to NAND's page size, not
according to target system's mm page size.

Can you tell what's the effect of incorrectly setting it to values such
as: 512 or 2048 bytes (instead of 4KiB)?

Well, I guess as data is written during runtime by the jffs2 itself, and
as the GC gets working, new inodes will be correctly created, and their
sizes will be up to system's page size.

However what happens with these small inodes initially created by
mkfs.jffs2? "Only" a performance penalty or something worse?

Thanks,
Shmulik

Re: Question about mkfs.jffs2 "-s" option

[Grant Edwards]

On 2012-05-03, Shmulik Ladkani <shmulik.ladkani@gmail.com> wrote:
> Hi Ricard,
> 	
> On Thu, 3 May 2012 09:35:14 +0200 (CEST) Ricard Wanderlof <ricard.wanderlof@axis.com> wrote:
>> 
>> On Wed, 2 May 2012, Grant Edwards wrote:
>> 
>> > The NAND flash I'm using has to be written in 4K blocks -- isn't that
>> > the "page size"?
>> 
>> The smallest writable unit in a NAND flash is called a 'page', but
>> AFAIK this is not what the -s option in mkfs.jffs2 is for. Instead,
>> it is related for the kernel memory managament page size,

It would be cool if that's what the man page actually said. :)

>> which is 4k
>> for most architectures.
>
> According to MKFS.JFFS2(1),
>
> 	-s, --pagesize=SIZE
> 		Use page size SIZE.  The default is 4 KiB.  This size is
> 		the maximum size of a data node.
>
> And according to http://linux-mtd.infradead.org/~dwmw2/jffs2.pdf
>
> 	In order to facilitate rapid decompression of
> 	data upon readpage() requests, nodes contain
> 	no more than a single page of data, according to
> 	the hardware page size on the target platform.
>
> Wow.
> I always assumed I had to set it according to NAND's page size, not
> according to target system's mm page size.

To me, that's the obvious interpretation of the man page, and I swear
I've read "howto" pages at other sites (probably ARM SoC and
eval-board vendors), that explictly told you to set the -s option to
be the flash device's page size.

> Can you tell what's the effect of incorrectly setting it to values
> such as: 512 or 2048 bytes (instead of 4KiB)?
>
> Well, I guess as data is written during runtime by the jffs2 itself,
> and as the GC gets working, new inodes will be correctly created, and
> their sizes will be up to system's page size.
>
> However what happens with these small inodes initially created by
> mkfs.jffs2? "Only" a performance penalty or something worse?

When I created them with 4 byte nodes, the filesystem seemd to work
fine.  It was just shockingly large. :)

-- 
Grant Edwards               grant.b.edwards        Yow! TAILFINS!! ... click
                                  at               ...
                              gmail.com            

Re: Question about mkfs.jffs2 "-s" option

[Artem Bityutskiy]

[-- Attachment #1: Type: text/plain, Size: 2015 bytes --]

On Thu, 2012-05-03 at 21:13 +0300, Shmulik Ladkani wrote:
> Hi Ricard,
> 	
> On Thu, 3 May 2012 09:35:14 +0200 (CEST) Ricard Wanderlof <ricard.wanderlof@axis.com> wrote:
> > 
> > On Wed, 2 May 2012, Grant Edwards wrote:
> > 
> > > The NAND flash I'm using has to be written in 4K blocks -- isn't that
> > > the "page size"?
> > 
> > The smallest writable unit in a NAND flash is called a 'page', but AFAIK 
> > this is not what the -s option in mkfs.jffs2 is for. Instead, it is 
> > related for the kernel memory managament page size, which is 4k for most 
> > architectures.
> 
> According to MKFS.JFFS2(1),
> 
> 	-s, --pagesize=SIZE
> 		Use page size SIZE.  The default is 4 KiB.  This size is
> 		the maximum size of a data node.
> 
> And according to http://linux-mtd.infradead.org/~dwmw2/jffs2.pdf
> 
> 	In order to facilitate rapid decompression of
> 	data upon readpage() requests, nodes contain
> 	no more than a single page of data, according to
> 	the hardware page size on the target platform.
> 
> Wow.
> I always assumed I had to set it according to NAND's page size, not
> according to target system's mm page size.

Shmulik, I can give some info about this. First of all because of
compression and very different node sizes (UBIFS and JFFS2) and because
JFFS2 was designed for NOR flash originally - NAND page size is not
something which affects the design of file-systems, or their on-flash
layout. We use write-buffers on I/O level to make it appear like the
eraseblock is byte-writable.

The RAM page size comes from the general linux design - the VFS writes
in RAM page size units, and it is much simpler to make max node size =
PAGE_CACHE_SIZE rather than trying to split it when the PAGE_CACHE_SIZE
is something like 16KiB. This is where this -s stuff comes from. I think
it was a mistake to introduce this parameter because it is very unlikely
JFFS2 is/was used on systems with a different PAGE_CACHE_SIZE.

-- 
Best Regards,
Artem Bityutskiy

[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: Question about mkfs.jffs2 "-s" option

[Shmulik Ladkani]

Hi Artem, Ricard,

On Fri, 04 May 2012 10:15:37 +0300 Artem Bityutskiy <dedekind1@gmail.com> wrote:
> On Thu, 2012-05-03 at 21:13 +0300, Shmulik Ladkani wrote:
> > On Thu, 3 May 2012 09:35:14 +0200 (CEST) Ricard Wanderlof <ricard.wanderlof@axis.com> wrote:
> > > 
> > > The smallest writable unit in a NAND flash is called a 'page', but AFAIK 
> > > this is not what the -s option in mkfs.jffs2 is for. Instead, it is 
> > > related for the kernel memory managament page size, which is 4k for most 
> > > architectures.
> > 
> > According to MKFS.JFFS2(1),
> > 
> > 	-s, --pagesize=SIZE
> > 		Use page size SIZE.  The default is 4 KiB.  This size is
> > 		the maximum size of a data node.
> > 
> > And according to http://linux-mtd.infradead.org/~dwmw2/jffs2.pdf
> > 
> > 	In order to facilitate rapid decompression of
> > 	data upon readpage() requests, nodes contain
> > 	no more than a single page of data, according to
> > 	the hardware page size on the target platform.
> > 
> > Wow.
> > I always assumed I had to set it according to NAND's page size, not
> > according to target system's mm page size.
> 
> Shmulik, I can give some info about this. First of all because of
> compression and very different node sizes (UBIFS and JFFS2) and because
> JFFS2 was designed for NOR flash originally - NAND page size is not
> something which affects the design of file-systems, or their on-flash
> layout. We use write-buffers on I/O level to make it appear like the
> eraseblock is byte-writable.
> 
> The RAM page size comes from the general linux design - the VFS writes
> in RAM page size units, and it is much simpler to make max node size =
> PAGE_CACHE_SIZE rather than trying to split it when the PAGE_CACHE_SIZE
> is something like 16KiB. This is where this -s stuff comes from. I think
> it was a mistake to introduce this parameter because it is very unlikely
> JFFS2 is/was used on systems with a different PAGE_CACHE_SIZE.

Thanks for the explanations.
I pretty much realized this after reading [1]...

BTW, googling [mkfs.jffs2 and "-s 0x200"] or [mkfs.jffs2 and "-s 0x800"]
yielded few howtos, faqs, and alike....
Guess I wasn't the only one fooled by the argument's name :-))

[1] http://linux-mtd.infradead.org/~dwmw2/jffs2.pdf

Regards
Shmulik