JFFS2 - speed up while booting

JFFS2 - speed up while booting

[linux newbie]

Hi,

I am using linux version (linux.2.6.31) on my embedded ARM based target.

We are using JFFS2 filesystem to read/write data to NAND flash
(32MBytes) and our root file system resides in it.

It takes long time during kernel booting, if we have root file system
in NAND, but its very fast if we place our file system in SDRAM.

Is there any way to speed up the booting process?

Eagerly waiting for your valuable suggestion.

Thanking You

Re: JFFS2 - speed up while booting

[Ricard Wanderlof]

On Tue, 5 Jan 2010, linux newbie wrote:

> Hi,
>
> I am using linux version (linux.2.6.31) on my embedded ARM based target.
>
> We are using JFFS2 filesystem to read/write data to NAND flash
> (32MBytes) and our root file system resides in it.
>
> It takes long time during kernel booting, if we have root file system
> in NAND, but its very fast if we place our file system in SDRAM.
>
> Is there any way to speed up the booting process?
>
> Eagerly waiting for your valuable suggestion.

Is it slow during every boot or just the first after you've written your 
file system image to the flash?

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: JFFS2 - speed up while booting

[linux newbie]

Hi,

It is slow during every boot. Seems it is scanning for bad blocks (If
I enable the debug message. Usually debug is OFF).

Thanks

On Tue, Jan 5, 2010 at 7:06 PM, Ricard Wanderlof
<ricard.wanderlof@axis.com> wrote:
>
> On Tue, 5 Jan 2010, linux newbie wrote:
>
>> Hi,
>>
>> I am using linux version (linux.2.6.31) on my embedded ARM based target.
>>
>> We are using JFFS2 filesystem to read/write data to NAND flash
>> (32MBytes) and our root file system resides in it.
>>
>> It takes long time during kernel booting, if we have root file system
>> in NAND, but its very fast if we place our file system in SDRAM.
>>
>> Is there any way to speed up the booting process?
>>
>> Eagerly waiting for your valuable suggestion.
>
> Is it slow during every boot or just the first after you've written your
> file system image to the flash?
>
> /Ricard
> --
> Ricard Wolf Wanderlöf                           ricardw(at)axis.com
> Axis Communications AB, Lund, Sweden            www.axis.com
> Phone +46 46 272 2016                           Fax +46 46 13 61 30
>

Re: JFFS2 - speed up while booting

[Ricard Wanderlof]

On Tue, 5 Jan 2010, linux newbie wrote:

> Hi,
>
> It is slow during every boot. Seems it is scanning for bad blocks (If
> I enable the debug message. Usually debug is OFF).

During mount, JFFS2 will scan for cleanmarkers in the OOB area of each 
block, indicating that the block has been erased and can be used for 
storage. If cleanmarkers can not be found in blocks that appear to be 
empty (= all FFh), it puts the blocks on the erase queue for subsequent 
erasure. The erase process takes time and during erasure the flash cannot 
be read from which can account for slow booting. However, this should only 
happen the first time after mounting a newly flashed file system. 
Subsequent mounts should detect the cleanmarkers, and no erase will be 
performed. It can take a few minutes to complete the erase procedure 
though; if you do a 'top' on your system you'll see the pdflush task at 
the top while it is in progress, so if you reboot your system several 
times in succession JFFS2 might not have time to complete the erase of all 
unused blocks.

I don't know if this has been addressed recently, but the odd thing is 
that if you for instance implement separate erase sequence for instance in 
your flash driver, the total erase+boot time is much less than if you let 
JFFS2 handle it via the pdflush task.

Another option is to write cleanmarkers to empty blocks while writing your 
file system image. I don't think nandwrite has an option for that though.

But at any rate, you should only see this slowdown after first mount. 
Subsequent mounts should find the cleanmarkers in place and no erasure 
should be necessary. So I'm not convinced this is actually your problem.

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

Re: JFFS2 - speed up while booting

[Artem Bityutskiy]

On Tue, 2010-01-05 at 10:23 +0100, Ricard Wanderlof wrote:
> On Tue, 5 Jan 2010, linux newbie wrote:
> 
> > Hi,
> >
> > It is slow during every boot. Seems it is scanning for bad blocks (If
> > I enable the debug message. Usually debug is OFF).
> 
> During mount, JFFS2 will scan for cleanmarkers in the OOB area of each 
> block, indicating that the block has been erased and can be used for 
> storage. If cleanmarkers can not be found in blocks that appear to be 
> empty (= all FFh), it puts the blocks on the erase queue for subsequent 
> erasure. The erase process takes time and during erasure the flash cannot 
> be read from which can account for slow booting. However, this should only 
> happen the first time after mounting a newly flashed file system. 
> Subsequent mounts should detect the cleanmarkers, and no erase will be 
> performed. It can take a few minutes to complete the erase procedure 
> though; if you do a 'top' on your system you'll see the pdflush task at 
> the top while it is in progress, so if you reboot your system several 
> times in succession JFFS2 might not have time to complete the erase of all 
> unused blocks.
> 
> I don't know if this has been addressed recently, but the odd thing is 
> that if you for instance implement separate erase sequence for instance in 
> your flash driver, the total erase+boot time is much less than if you let 
> JFFS2 handle it via the pdflush task.
> 
> Another option is to write cleanmarkers to empty blocks while writing your 
> file system image. I don't think nandwrite has an option for that though.
> 
> But at any rate, you should only see this slowdown after first mount. 
> Subsequent mounts should find the cleanmarkers in place and no erasure 
> should be necessary. So I'm not convinced this is actually your problem.

JFFS2 mount process is split on 2 phases:

1. Scanning. Done synchronously. When finished, JFFS2 is accessible in
R/O more. Some writing is often also possible, if there is substantial
amount of free space. However, generally, the FS is R/O after this
phase.

2. Checking. Done asynchronously in context of the background thread.
During this phase JFFS2 read all nodes of all inodes and checks their
CRC. This is very CPU and I/O consuming process. In essence, this is
needed to figure out which nodes are valide/obsolete and
corrupted/correct. Before this process is done, JFFS2 does not really
know how much free space there is.

On NAND, the erasing is ultra-fast, so the first mount needs erasure
factor is not that important there.

The checking process is always done, on every mount, and it is the most
expensive one.

Some more information about JFFS2 scalability weaknesses can be found in
the UBIFS vs JFFS2 table here:

http://www.linux-mtd.infradead.org/doc/ubifs.html#L_scalability

I'm not trying to promote UBIFS (it is that scalable either because of
UBI), it is just that the table contains a good overview.

-- 
Best Regards,
Artem Bityutskiy (Артём Битюцкий)

RE: JFFS2 - speed up while booting

[Darwin Rambo]

 
> It can take a few minutes to complete the erase procedure 
> though; if you do a 'top' on your system you'll see the pdflush task at 
> the top while it is in progress, so if you reboot your system several 
> times in succession JFFS2 might not have time to complete the erase of all 
> unused blocks.

This pdflush activity can take much longer on very large flash parts and be
almost unnoticable on small devices. The reboot command in busybox (and any 
command that tries to flush data to flash) is also stalled until this operation 
is complete. You might think the system is hung at this point but it isn't.

This is the typical erase cycle for one block:

Starting erase of pending block 0x1b200000
Freeing all node refs for eraseblock offset 0x1b200000
jffs2_erase_block(): erase block 0x1b200000 (range 0x1b200000-0x1b280000)
Erase completed successfully at 0x1b200000
Verifying erase at 0x1b200000
Writing erased marker to block at 0x1b200000


The real reason for the long delay is not the erase time, but the time to 
validate that the erase actually worked and that all the data is 0xff. 
If you comment out the jffs2_block_check_erase function below
as shown, then in my case, the 4.5 minute pdflush delay dropped to 2-3 
seconds! I was using a 16Gbit large page flash for this test.

static void jffs2_mark_erased_block(struct jffs2_sb_info *c, struct jffs2_eraseblock *jeb)
{
	size_t retlen;
	int ret;
	uint32_t uninitialized_var(bad_offset);

#if 0
	switch (jffs2_block_check_erase(c, jeb, &bad_offset)) {
	case -EAGAIN:	goto refile;
	case -EIO:	goto filebad;
	}
#endif


Darwin


______________________________________________________
Linux MTD discussion mailing list
http://lists.infradead.org/mailman/listinfo/linux-mtd/

RE: JFFS2 - speed up while booting

[Joakim Tjernlund]

>
>
> > It can take a few minutes to complete the erase procedure
> > though; if you do a 'top' on your system you'll see the pdflush task at
> > the top while it is in progress, so if you reboot your system several
> > times in succession JFFS2 might not have time to complete the erase of all
> > unused blocks.
>
> This pdflush activity can take much longer on very large flash parts and be
> almost unnoticable on small devices. The reboot command in busybox (and any
> command that tries to flush data to flash) is also stalled until this operation
> is complete. You might think the system is hung at this point but it isn't.

I posted a patch for that quite some while ago. It didn't
get into the repo because it was racy w.r.t module unload. I am still
using it so I guess it applies to the current tree.

>
> This is the typical erase cycle for one block:
>
> Starting erase of pending block 0x1b200000
> Freeing all node refs for eraseblock offset 0x1b200000
> jffs2_erase_block(): erase block 0x1b200000 (range 0x1b200000-0x1b280000)
> Erase completed successfully at 0x1b200000
> Verifying erase at 0x1b200000
> Writing erased marker to block at 0x1b200000
>
>
> The real reason for the long delay is not the erase time, but the time to
> validate that the erase actually worked and that all the data is 0xff.
> If you comment out the jffs2_block_check_erase function below
> as shown, then in my case, the 4.5 minute pdflush delay dropped to 2-3
> seconds! I was using a 16Gbit large page flash for this test.

Ah, that make sense. Some new NOR parts form Numonyx has a new CFI command,
Blank Check, that indicates if it is all ones or not. That could be handy
to use in this case. You could even use that instead of erasing empty
block because they miss a clean marker.

>
> static void jffs2_mark_erased_block(struct jffs2_sb_info *c, struct
> jffs2_eraseblock *jeb)
> {
>    size_t retlen;
>    int ret;
>    uint32_t uninitialized_var(bad_offset);
>
> #if 0
>    switch (jffs2_block_check_erase(c, jeb, &bad_offset)) {
>    case -EAGAIN:   goto refile;
>    case -EIO:   goto filebad;
>    }
> #endif
>
>
> Darwin
>
>
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
>
>
>
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
>
>

RE: JFFS2 - speed up while booting

[Ricard Wanderlof]

On Tue, 5 Jan 2010, Joakim Tjernlund wrote:

>> The real reason for the long delay is not the erase time, but the time to
>> validate that the erase actually worked and that all the data is 0xff.
>> If you comment out the jffs2_block_check_erase function below
>> as shown, then in my case, the 4.5 minute pdflush delay dropped to 2-3
>> seconds! I was using a 16Gbit large page flash for this test.
>
> Ah, that make sense. Some new NOR parts form Numonyx has a new CFI command,
> Blank Check, that indicates if it is all ones or not. That could be handy
> to use in this case. You could even use that instead of erasing empty
> block because they miss a clean marker.

It is my understanding that the reason the blocks are erased when no 
cleanmarker is found, is that a potentially uncompleted previous erase may 
have left the block in an appearantly erased state, making future 
programming unreliable unless the block is erased properly. So the blocks 
would still have to be erased, but of course the checking function (which 
seems to be the main reason for the delay) could be considerably 
optimized.

/Ricard
-- 
Ricard Wolf Wanderlöf                           ricardw(at)axis.com
Axis Communications AB, Lund, Sweden            www.axis.com
Phone +46 46 272 2016                           Fax +46 46 13 61 30

RE: JFFS2 - speed up while booting

[Joakim Tjernlund]

Ricard Wanderlof <ricard.wanderlof@axis.com> wrote on 06/01/2010 20:54:58:
>
>
> On Tue, 5 Jan 2010, Joakim Tjernlund wrote:
>
> >> The real reason for the long delay is not the erase time, but the time to
> >> validate that the erase actually worked and that all the data is 0xff.
> >> If you comment out the jffs2_block_check_erase function below
> >> as shown, then in my case, the 4.5 minute pdflush delay dropped to 2-3
> >> seconds! I was using a 16Gbit large page flash for this test.
> >
> > Ah, that make sense. Some new NOR parts form Numonyx has a new CFI command,
> > Blank Check, that indicates if it is all ones or not. That could be handy
> > to use in this case. You could even use that instead of erasing empty
> > block because they miss a clean marker.
>
> It is my understanding that the reason the blocks are erased when no
> cleanmarker is found, is that a potentially uncompleted previous erase may
> have left the block in an appearantly erased state, making future
> programming unreliable unless the block is erased properly. So the blocks
> would still have to be erased, but of course the checking function (which
> seems to be the main reason for the delay) could be considerably
> optimized.

I know but from a quick reading of the blank check feature, you don't have to.
The blank check is reliable w.r.t power loss and such. Seems like it
was designed for this purpose.

 Jocke

Re: JFFS2 - speed up while booting

[Joakim Tjernlund]

hedwin <hedwin.koning@gmail.com> wrote on 06/01/2010 21:10:31:

> From: hedwin <hedwin.koning@gmail.com>
> To: Ricard Wanderlof <ricard.wanderlof@axis.com>
> Cc: Joakim Tjernlund <joakim.tjernlund@transmode.se>, linux newbie
> <linux.newbie79@gmail.com>, Darwin Rambo <drambo@broadcom.com>, "linux-
> mtd@lists.infradead.org" <linux-mtd@lists.infradead.org>
> Date: 06/01/2010 21:10
> Subject: Re: JFFS2 - speed up while booting
>
> One thing we did to speed up JFFS2 booting is delaying the garbage collection
> process from starting.
>
> Hedwin

Yes, maybe GC should be delayed a minute. It is not really needed initially.

 Jocke

Re: JFFS2 - speed up while booting

[Joakim Tjernlund]

hedwin <hedwin.koning@gmail.com> wrote on 06/01/2010 22:16:27:
>
> Interesting, when was this added to JFFS2? Need to catch up a bit I guess.
>
Please don't top post.

This isn't in JFFS2(yes). Basically the empty check need to be move to the chip driver.
Then the driver knows if the chip has a blank check command and use that or fall
back to old method.

Alternatively one could export a Blank Check command from the driver and
test for it in JFFS2.

> On Wed, Jan 6, 2010 at 9:59 PM, Joakim Tjernlund <joakim.tjernlund@transmode.se> wrote:
> Ricard Wanderlof <ricard.wanderlof@axis.com> wrote on 06/01/2010 20:54:58:
> >
> >
> > On Tue, 5 Jan 2010, Joakim Tjernlund wrote:
> >
> > >> The real reason for the long delay is not the erase time, but the time to
> > >> validate that the erase actually worked and that all the data is 0xff.
> > >> If you comment out the jffs2_block_check_erase function below
> > >> as shown, then in my case, the 4.5 minute pdflush delay dropped to 2-3
> > >> seconds! I was using a 16Gbit large page flash for this test.
> > >
> > > Ah, that make sense. Some new NOR parts form Numonyx has a new CFI command,
> > > Blank Check, that indicates if it is all ones or not. That could be handy
> > > to use in this case. You could even use that instead of erasing empty
> > > block because they miss a clean marker.
> >
> > It is my understanding that the reason the blocks are erased when no
> > cleanmarker is found, is that a potentially uncompleted previous erase may
> > have left the block in an appearantly erased state, making future
> > programming unreliable unless the block is erased properly. So the blocks
> > would still have to be erased, but of course the checking function (which
> > seems to be the main reason for the delay) could be considerably
> > optimized.

> I know but from a quick reading of the blank check feature, you don't have to.
> The blank check is reliable w.r.t power loss and such. Seems like it
> was designed for this purpose.
>
>  Jocke
>
>
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/

Re: JFFS2 - speed up while booting

[Jamie Lokier]

Joakim Tjernlund wrote:
> hedwin <hedwin.koning@gmail.com> wrote on 06/01/2010 21:10:31:
> 
> > From: hedwin <hedwin.koning@gmail.com>
> > To: Ricard Wanderlof <ricard.wanderlof@axis.com>
> > Cc: Joakim Tjernlund <joakim.tjernlund@transmode.se>, linux newbie
> > <linux.newbie79@gmail.com>, Darwin Rambo <drambo@broadcom.com>, "linux-
> > mtd@lists.infradead.org" <linux-mtd@lists.infradead.org>
> > Date: 06/01/2010 21:10
> > Subject: Re: JFFS2 - speed up while booting
> >
> > One thing we did to speed up JFFS2 booting is delaying the garbage collection
> > process from starting.
> >
> > Hedwin
> 
> Yes, maybe GC should be delayed a minute. It is not really needed initially.

Conceivably, the blank check could be delayed for each block
individually until just before using the block too.

-- Jamie