JFFS2: mount time, SUMMARY and gc.c ?

JFFS2: mount time, SUMMARY and gc.c ?

[Steven Scholz]

Hi there,

I am using linux-2.6.10 with recent MTD CVS and the SUMMARY patch on an embedded 
ARM9 system (150MHz core, 48MHz SDRAM) with 8MB NOR flash as root fs.

The SUMMARY patch actually reduced the mount time - the time between

  NET: Registered protocol family 1
  NET: Registered protocol family 17

and

  VFS: Mounted root (jffs2 filesystem).
  Freeing init memory: 72K

- from 6 to less then 1 second (710ms)! :-)

But then before

  init started:  BusyBox v1.00 (2005.01.04-08:42+0000) multi-call binary

appears still "nothing" happens for around 8 seconds.

I noticed that "jffs2_gcd_mtd0" is heavily running after boot consuming much time.

gc.c:jffs2_garbage_collect_pass() gets called very often - 596 times!

Why is that garbage collection running everytime I boot my target!?!?
Is there anything I can do about it?

Thanks a million!

-- 
Steven Scholz

Re: JFFS2: mount time, SUMMARY and gc.c ?

[Steven Scholz]

Hi there,

> I noticed that "jffs2_gcd_mtd0" is heavily running after boot consuming 
> much time.
> 
> gc.c:jffs2_garbage_collect_pass() gets called very often - 596 times!

Well, actually it is only 293 times.
(My mtd0 has 110 x 64KiB sectors = 7040KiB)

But still why is it done everytime I boot my target?

Thanks.

-- 
Steven Scholz

Re: JFFS2: mount time, SUMMARY and gc.c ?

[Artem B. Bityuckiy]

On Fri, 7 Jan 2005, Steven Scholz wrote:

> Hi there,
> 
> > I noticed that "jffs2_gcd_mtd0" is heavily running after boot consuming 
> > much time.
> > 
> > gc.c:jffs2_garbage_collect_pass() gets called very often - 596 times!
> 
> Well, actually it is only 293 times.
> (My mtd0 has 110 x 64KiB sectors = 7040KiB)
> 
> But still why is it done everytime I boot my target?
This is how JFFS2 implemented. It skips nodes CRC checking during mount 
and does this lated in background. Actually, you can't write anything to 
your flash untill the GC thread (jffs2_gcd_mtd0) have not completed 
checking. But you may read from JFFS2 during that checking process.

> 
> Thanks.
> 
> -- 
> Steven Scholz
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
> 

--
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Re: JFFS2: mount time, SUMMARY and gc.c ?

[Steven Scholz]

Artem B. Bityuckiy wrote:

>>But still why is it done everytime I boot my target?
> 
> This is how JFFS2 implemented. It skips nodes CRC checking during mount 
> and does this lated in background. Actually, you can't write anything to 
> your flash untill the GC thread (jffs2_gcd_mtd0) have not completed 
> checking. But you may read from JFFS2 during that checking process.

Well but "in background" means 99.9% of cpu time!
And this slows down the start up of important user applications.

Would it help to enable CONFIG_PREEMPT?

--
Steven

Re: JFFS2: mount time, SUMMARY and gc.c ?

[Artem B. Bityuckiy]

On Fri, 7 Jan 2005, Steven Scholz wrote:

> Artem B. Bityuckiy wrote:
> 
> >>But still why is it done everytime I boot my target?
> > 
> > This is how JFFS2 implemented. It skips nodes CRC checking during mount 
> > and does this lated in background. Actually, you can't write anything to 
> > your flash untill the GC thread (jffs2_gcd_mtd0) have not completed 
> > checking. But you may read from JFFS2 during that checking process.
> 
> Well but "in background" means 99.9% of cpu time!
At first, if *anybody* tries to write to JFFS2 before it has done 
checking, 
it will be blocked. So if boot processes try to write...

But in your case the GC thread eats CPU...

> And this slows down the start up of important user applications.
You may try to increase the nice value... It is currently hardcoded to 
10 in background.c (search for set_user_nice)...

> 
> Would it help to enable CONFIG_PREEMPT
Don't know, try it. Hope JFFS2 works well with it. I didn't try. I tried 
only SMP and JFFS2 has some locking bugs. MAy be UP+preempt will work 
fine - some locking bugs were recently fixed. 

> 
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
> 

--
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Re: JFFS2: mount time, SUMMARY and gc.c ?

[Artem B. Bityuckiy]

On Fri, 7 Jan 2005, Steven Scholz wrote:

> Artem B. Bityuckiy wrote:
> 
> >>But still why is it done everytime I boot my target?
> > 
> > This is how JFFS2 implemented. It skips nodes CRC checking during mount 
> > and does this lated in background. Actually, you can't write anything to 
> > your flash untill the GC thread (jffs2_gcd_mtd0) have not completed 
> > checking. But you may read from JFFS2 during that checking process.
> 
> Well but "in background" means 99.9% of cpu time!
And  the job the thread performs seems to be CPU-boud (it checks CRCs). 
Thus, isn't it normal to eat a lot of CPU cycles? 

> And this slows down the start up of important user applications.
> 
> Would it help to enable CONFIG_PREEMPT?
> 
> --
> Steven
> 
> 
> 
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
> 

--
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Re: JFFS2: mount time, SUMMARY and gc.c ?

[Todd Poynor]

Steven Scholz wrote:

>>And this slows down the start up of important user applications.
>>
>>Would it help to enable CONFIG_PREEMPT?

It sounds like your apps need to write to an fs on flash and are blocked 
on completion of the initial jffs2 gc pass, not that the cpu hogging is 
blocking your apps.  If so, would suggest a separate partition with a 
read-only fs (CramFS, etc.) for read-only data/executables (if not 
already), maybe put /tmp and /var/tmp (or whatever dirs the apps are 
writing to) in a ram fs such as tmpfs, and later copy 
important/persistent data to jffs2 after everything's started up...


-- 
Todd

Re: JFFS2: mount time, SUMMARY and gc.c ?

[Artem B. Bityuckiy]

On Fri, 7 Jan 2005, Todd Poynor wrote:

> Steven Scholz wrote:
> 
> >>And this slows down the start up of important user applications.
> >>
> >>Would it help to enable CONFIG_PREEMPT?
> 
> It sounds like your apps need to write to an fs on flash and are blocked 
> on completion of the initial jffs2 gc pass, not that the cpu hogging is 
> blocking your apps.  If so, would suggest a separate partition with a 
> read-only fs (CramFS, etc.) for read-only data/executables (if not 
> already), maybe put /tmp and /var/tmp (or whatever dirs the apps are 
> writing to) in a ram fs such as tmpfs, and later copy 
> important/persistent data to jffs2 after everything's started up...
> 
If he have written something to fs and because of this have been blocked, 
CPU would have been consumed by the task which tried to write, not by the 
background task.But in his case the GC thread consumes CPU. So I don't 
think the problem is the early writing.

> -- 
> Todd
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
> 

--
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.

Re: JFFS2: mount time, SUMMARY and gc.c ?

[Josh Boyer]

On Fri, 2005-01-07 at 13:11, Steven Scholz wrote:
> Artem B. Bityuckiy wrote:
> 
> >>But still why is it done everytime I boot my target?
> > 
> > This is how JFFS2 implemented. It skips nodes CRC checking during mount 
> > and does this lated in background. Actually, you can't write anything to 
> > your flash untill the GC thread (jffs2_gcd_mtd0) have not completed 
> > checking. But you may read from JFFS2 during that checking process.
> 
> Well but "in background" means 99.9% of cpu time!
> And this slows down the start up of important user applications.

That's odd...  IIRC the thread schedules itself after every call to
jffs2_garbage_collect_pass.  By how much are the startup of these apps
delayed?  If it's using 99.9% of the cpu, maybe there is nothing else
that your system can do at that particular point?

> 
> Would it help to enable CONFIG_PREEMPT?

Maybe...  but since the thread tries to schedule itself already, I'm not
sure by how much.

josh

Re: JFFS2: mount time, SUMMARY and gc.c ?

[Steven Scholz]

Josh Boyer wrote:

>>Well but "in background" means 99.9% of cpu time!
>>And this slows down the start up of important user applications.
> 
> That's odd...  IIRC the thread schedules itself after every call to
> jffs2_garbage_collect_pass.  By how much are the startup of these apps
> delayed?  If it's using 99.9% of the cpu, maybe there is nothing else
> that your system can do at that particular point?

Ok. Maybe I exaggerated a bit. It consumes much time. But it seems to share it 
with my application. Shortly after boot I see something like 70-80% for gc and 
20% for my app. But this changes after a few seconds.

VFS: Mounted root (jffs2 filesystem).
Freeing init memory: 76K

<6 seconds pause>

init started:  BusyBox v1.00 (2005.01.04-08:42+0000) multi-call binary
<1 seconds pause>
Please press Enter to activate this console.

After this gc is still running.

-- 
Steven Scholz

Re: JFFS2: mount time, SUMMARY and gc.c ?

[Artem B. Bityuckiy]

On Fri, 7 Jan 2005, Steven Scholz wrote:

> Hi there,
> 
> I am using linux-2.6.10 with recent MTD CVS and the SUMMARY patch on an embedded 
> ARM9 system (150MHz core, 48MHz SDRAM) with 8MB NOR flash as root fs.
> 
> The SUMMARY patch actually reduced the mount time - the time between
> 
>   NET: Registered protocol family 1
>   NET: Registered protocol family 17
> 
> and
> 
>   VFS: Mounted root (jffs2 filesystem).
>   Freeing init memory: 72K
> 
> - from 6 to less then 1 second (710ms)! :-)
Really? Seems I was wrong. But anyway, there should not be so big 
difference... I may try to explain it how jffs2_scan_medium() implemented. 
It reads PAGE_SIZE bytes to the buffer (jffs3_fill_scan_buf()) and the 
scan this buffer for nodes. Thus, it may end up reading the whole block.
In case of summaries, we only read summary node.

Another difference - without summary we check CRCs of node common headers 
and diretries fully. But with summary we do not do this - only check 
summary CRC once.

So, I wonder if we have fixed the above two issues would the summary data 
needed at all? Would we have better mount speed?

> 
> But then before
> 
>   init started:  BusyBox v1.00 (2005.01.04-08:42+0000) multi-call binary
> 
> appears still "nothing" happens for around 8 seconds.
> 
> I noticed that "jffs2_gcd_mtd0" is heavily running after boot consuming much time.
> 
> gc.c:jffs2_garbage_collect_pass() gets called very often - 596 times!
> 
> Why is that garbage collection running everytime I boot my target!?!?
> Is there anything I can do about it?

It checks all file on your flash. This is normal. I believe when Ferenc 
implement the feature I suggested, the checking time will be significantly reduced.

> 
> Thanks a million!
> 
> -- 
> Steven Scholz
> 
> ______________________________________________________
> Linux MTD discussion mailing list
> http://lists.infradead.org/mailman/listinfo/linux-mtd/
> 

--
Best Regards,
Artem B. Bityuckiy,
St.-Petersburg, Russia.