Update on my problem

Update on my problem

[Chris Fowler]

Many of you guys lately has been halping overcome some issues I've been having during an initrd execution.  I appreciate the efforts.  I'm eamiling you to inform you what I did to fix it in case you ever run into the same problem. 

I have to compile my linuxrc.c program statically.  lib-c is just too big to place on a initrd image.  To keep the size down from 400k I used a program called diet.  It bacsically executes gcc and use some of its own libraries.  My loader 
went from 400k to 22k using it.  This allowed me to stuff more program on the initrd image.  I ran into some major problems using zlib and this program so I wnet back to a normal static binary.  My loader is now 400k but seems to
be working great.  I can download software packages form the network and use those as my roofs just as easy as the package on flash.  I can also restore a backup package that the user has the option to create at flash time.
It appears to be working great.  

Thanks for all your help.
Chris

jffs2_scan_make_ino_cache() in scan.c very slow

[Joakim Tjernlund]

Hi 

I have made some crude measures in jffs2_scan_medium() and friends to
see where most of the time is spent. It turns out that
jffs2_scan_make_ino_cache() is takes about 50 % of the total mount time!

What can be done to decrease this time?

 Jocke

PS.
    I sent a patch (Subject:  cfi_cmdset0001.c: bug fixes and new features) a week ago
    and I haven't got any comments so far.

Re: jffs2_scan_make_ino_cache() in scan.c very slow

[David Woodhouse]

joakim.tjernlund@lumentis.se said:
>  I have made some crude measures in jffs2_scan_medium() and friends to
> see where most of the time is spent. It turns out that
> jffs2_scan_make_ino_cache() is takes about 50 % of the total mount
> time!

How did you do your profiling? Are you sure it's not spending the time in 
jffs2_get_ino_cache() and jffs2_add_ino_cache()? I suspect it is.

> What can be done to decrease this time? 

Try increasing INOCACHE_HASHSIZE - that'll make the hash table more 
efficient - well, it'll make the hash table into a hash table instead of a 
linked list :)

You'll probably find that it's looking up the same inode over and over 
again too - in which case caching the last used inode in 
jffs2_scan_make_ino_cache() would probably be useful.



joakim.tjernlund@lumentis.se said:
>  PS.
>     I sent a patch (Subject:  cfi_cmdset0001.c: bug fixes and new
> features) a week ago
>     and I haven't got any comments so far.


Sorry. Can you make sure it still applies to the current CVS tree and 
resend it?

--
dwmw2

RE: jffs2_scan_make_ino_cache() in scan.c very slow

[Joakim Tjernlund]

> 
> 
> 
> joakim.tjernlund@lumentis.se said:
> >  I have made some crude measures in jffs2_scan_medium() and friends to
> > see where most of the time is spent. It turns out that
> > jffs2_scan_make_ino_cache() is takes about 50 % of the total mount
> > time!
> 
> How did you do your profiling?
I accumulated the difference of jiffies from the entry of jffs2_scan_make_ino_cache()
to it returned.

 Are you sure it's not spending the time in 
> jffs2_get_ino_cache() and jffs2_add_ino_cache()? I suspect it is.

That is so, allmost all time is in jffs2_get_ino_cache().

> 
> > What can be done to decrease this time? 
> 
> Try increasing INOCACHE_HASHSIZE - that'll make the hash table more 
> efficient - well, it'll make the hash table into a hash table instead of a 
> linked list :)
> 
> You'll probably find that it's looking up the same inode over and over 
> again too - in which case caching the last used inode in 
> jffs2_scan_make_ino_cache() would probably be useful.

OK, will try an see what happens.
> 
> 
> 
> joakim.tjernlund@lumentis.se said:
> >  PS.
> >     I sent a patch (Subject:  cfi_cmdset0001.c: bug fixes and new
> > features) a week ago
> >     and I haven't got any comments so far.
> 
> 
> Sorry. Can you make sure it still applies to the current CVS tree and 
> resend it?

Well, I not using the current CVS at the moment. It will take some time
until I upgrade again.

 Jocke

> 
> --
> dwmw2
> 
> 
> 

RE: jffs2_scan_make_ino_cache() in scan.c very slow

[Joakim Tjernlund]

> > 
> > 
> > 
> > joakim.tjernlund@lumentis.se said:
> > >  I have made some crude measures in jffs2_scan_medium() and friends to
> > > see where most of the time is spent. It turns out that
> > > jffs2_scan_make_ino_cache() is takes about 50 % of the total mount
> > > time!
> > 
> > How did you do your profiling?
> I accumulated the difference of jiffies from the entry of jffs2_scan_make_ino_cache()
> to it returned.
> 
>  Are you sure it's not spending the time in 
> > jffs2_get_ino_cache() and jffs2_add_ino_cache()? I suspect it is.
> 
> That is so, allmost all time is in jffs2_get_ino_cache().
> 
> > 
> > > What can be done to decrease this time? 
> > 
> > Try increasing INOCACHE_HASHSIZE - that'll make the hash table more 
> > efficient - well, it'll make the hash table into a hash table instead of a 
> > linked list :)
> > 
> > You'll probably find that it's looking up the same inode over and over 
> > again too - in which case caching the last used inode in 
> > jffs2_scan_make_ino_cache() would probably be useful.
> 
> OK, will try an see what happens.

Setting INOCACHE_HASHSIZE to 14(higher values makes it PANIC at mount, don't know
why) and changed the beginning of jffs2_scan_make_ino_cache() to
{
   	static struct jffs2_inode_cache *ic = NULL;

	if(ic && ic->ino == ino)
		return ic;
...

is it OK to make *ic an static variable?

Now my mount time is just half of what it used to be!!(well almost :-)

Re: jffs2_scan_make_ino_cache() in scan.c very slow

[David Woodhouse]

joakim.tjernlund@lumentis.se said:
>  Setting INOCACHE_HASHSIZE to 14(higher values makes it PANIC at
> mount, don't know why) 

Possibly because the superblock union gets too big. What does it say when 
it panics?

> and changed the beginning of jffs2_scan_make_ino_cache() to {
> is it OK to make *ic an static variable?

No - consider 'mount /opt & mount /usr' - it has to be per-filesystem. If
it's this which really makes a difference, then we can add the pointer to
the jffs2_sb_info. Can you make it keep a count of how many hits and misses
there are, and print the counts when it's finished mounting?

> Now my mount time is just half of what it used to be!!(well almost :-)

Cool - can you try each of these changes on its own and measure the benefit
of each? 

--
dwmw2

RE: jffs2_scan_make_ino_cache() in scan.c very slow

[Joakim Tjernlund]

> 
> joakim.tjernlund@lumentis.se said:
> >  Setting INOCACHE_HASHSIZE to 14(higher values makes it PANIC at
> > mount, don't know why) 
> 
> Possibly because the superblock union gets too big. What does it say when 
> it panics?
Kernel panic: VFS: Unable to mount root fs on 1f:04
> 
> > and changed the beginning of jffs2_scan_make_ino_cache() to {
> > is it OK to make *ic an static variable?
> 
> No - consider 'mount /opt & mount /usr' - it has to be per-filesystem. If
> it's this which really makes a difference, then we can add the pointer to
> the jffs2_sb_info. Can you make it keep a count of how many hits and misses
> there are, and print the counts when it's finished mounting?

Well, I already have counters to measure elapsed jiffies between entry and exit
of jffs2_scan_medium()

Here are the results:
Both optimizations: 1665 jiffies
extra cache and INOCACHE_HASHSIZE = 1: 2040 jiffies
no extra cache and INOCACHE_HASHSIZE = 14: 1813 jiffies
no optimization: ~3000 jiffies 

I think both are needed.

HZ is 200, this is on a mppc860, 80Mhz

 Jocke

> 
> > Now my mount time is just half of what it used to be!!(well almost :-)
> 
> Cool - can you try each of these changes on its own and measure the benefit
> of each? 
> 
> --
> dwmw2
> 
> 
> 

Re: jffs2_scan_make_ino_cache() in scan.c very slow

[Thomas Gleixner]

On Thursday, 21. February 2002 15:34, Joakim Tjernlund wrote:
>
> > Sorry. Can you make sure it still applies to the current CVS tree and
> > resend it?
> Well, I not using the current CVS at the moment. It will take some time
> until I upgrade again.

I tried it with current CVS and there is no difference without or with 
optimizing. Mount time for a 8MB NAND partition in all variants ~11 sec. on a 
74MHz ARM7.
-- 
Thomas
__________________________________________________
Thomas Gleixner, autronix automation GmbH
auf dem berg 3, d-88690 uhldingen-muehlhofen
fon: +49 7556 919891 , fax: +49 7556 919886
mail: gleixner@autronix.de, http://www.autronix.de  

Re: jffs2_scan_make_ino_cache() in scan.c very slow

[Joakim Tjernlund]

> On Thursday, 21. February 2002 15:34, Joakim Tjernlund wrote:
> >
> > > Sorry. Can you make sure it still applies to the current CVS tree and
> > > resend it?
> > Well, I not using the current CVS at the moment. It will take some time
> > until I upgrade again.
>
> I tried it with current CVS and there is no difference without or with
> optimizing. Mount time for a 8MB NAND partition in all variants ~11 sec.
on a
> 74MHz ARM7.

What did you try? My cfi_cmd0001.c changes(I guess not, since you are
on NAND) or the inode cache thing?

Has someone else tried? results?

    Jocke

Re: jffs2_scan_make_ino_cache() in scan.c very slow

[Thomas Gleixner]

On Thursday, 21. February 2002 23:05, Joakim Tjernlund wrote:
> What did you try? My cfi_cmd0001.c changes(I guess not, since you are
> on NAND) or the inode cache thing?
the inode cache thing
-- 
Thomas
__________________________________________________
Thomas Gleixner, autronix automation GmbH
auf dem berg 3, d-88690 uhldingen-muehlhofen
fon: +49 7556 919891 , fax: +49 7556 919886
mail: gleixner@autronix.de, http://www.autronix.de  

Re: jffs2_scan_make_ino_cache() in scan.c very slow

[Joakim Tjernlund]

> On Thursday, 21. February 2002 23:05, Joakim Tjernlund wrote:
> > What did you try? My cfi_cmd0001.c changes(I guess not, since you are
> > on NAND) or the inode cache thing?
> the inode cache thing

hmm, how big is your FS? Maybe, if you have a small FS the inode 
cache thing gets lost in "noise" from the NAND driver?

My FS is 63 MB and df reports 31% used.

      Jocke

Re: jffs2_scan_make_ino_cache() in scan.c very slow

[Thomas Gleixner]

On Thursday, 21. February 2002 23:27, Joakim Tjernlund wrote:
> > On Thursday, 21. February 2002 23:05, Joakim Tjernlund wrote:
> > the inode cache thing
I tried with 8 and 16MB both ~60% used
-- 
Thomas
__________________________________________________
Thomas Gleixner, autronix automation GmbH
auf dem berg 3, d-88690 uhldingen-muehlhofen
fon: +49 7556 919891 , fax: +49 7556 919886
mail: gleixner@autronix.de, http://www.autronix.de  

Re: jffs2_scan_make_ino_cache() in scan.c very slow

[Joakim Tjernlund]

> On Thursday, 21. February 2002 23:27, Joakim Tjernlund wrote:
> > > On Thursday, 21. February 2002 23:05, Joakim Tjernlund wrote:
> > > the inode cache thing
> I tried with 8 and 16MB both ~60% used

Very slow mounts has happened to me in the past. Never found the cause.
Finally
solved it by starting with a fresh copy linux and than add JFFS2&MTD to
it again.

I have seen 1 or 2 reports on this list about very slow mounts, maybe you
should do the same?

           Jocke

Re: jffs2_scan_make_ino_cache() in scan.c very slow, new improvment

[Joakim Tjernlund]

Is the CRC check needed for the data part(if (ri.csize ....) in
jffs2_scan_inode_node ()
in ? Removing this check halves my already improved mount time :-)

The CRC check of ri.node_crc and rd.node_crc can be removed since it has
already
been checked in jffs2_scan_eraseblock(). OK ?

         Jocke

RE: jffs2_scan_make_ino_cache() in scan.c very slow, new improvment

[Joakim Tjernlund]

Hi again

I done some more digging and as far as I can tell it's not possible to omit the
CRC check. Questions:
Is it possible to change the writeout of inode's in such away that the
CRC check can be omitted?
Idea: write the whole node, but replace the data_crc with 0xffffffff, then
write the real data_crc. At scan time you only need to check if the data_crc is 0xfffffff
or not. This wont work if 0xffffffff is a legal CRC value, does someone know?
One could also use the flags field in same way?

Comments? 

Adler32:
I ran a quick test where I swapped crc32 to adler32 in jffs2_scan_inode_node () and
that lowered my mount time with 21 % (from 848 jiffies to 666 jiffies). How 
about swapping crc32 to adler32? This could be a compile time option or stored somewhere
in the FS.

Comments?

 Jocke

> -----Original Message-----
> From: linux-mtd-admin@lists.infradead.org
> [mailto:linux-mtd-admin@lists.infradead.org]On Behalf Of Joakim
> Tjernlund
> Sent: Thursday, February 21, 2002 23:44
> To: David Woodhouse
> Cc: linux-mtd@lists.infradead.org
> Subject: Re: jffs2_scan_make_ino_cache() in scan.c very slow, new
> improvment
> 
> 
> Is the CRC check needed for the data part(if (ri.csize ....) in
> jffs2_scan_inode_node ()
> in ? Removing this check halves my already improved mount time :-)

Re: Update on my problem

[Russ Dill]

On Thu, 2002-02-21 at 06:03, Chris Fowler wrote:
> Many of you guys lately has been halping overcome some issues I've been having during an initrd execution.  I appreciate the efforts.  I'm eamiling you to inform you what I did to fix it in case you ever run into the same problem. 
> 
> I have to compile my linuxrc.c program statically.  lib-c is just too big to place on a initrd image.  To keep the size down from 400k I used a program called diet.  It bacsically executes gcc and use some of its own libraries.  My loader 
> went from 400k to 22k using it.  This allowed me to stuff more program on the initrd image.  I ran into some major problems using zlib and this program so I wnet back to a normal static binary.  My loader is now 400k but seems to
> be working great.  I can download software packages form the network and use those as my roofs just as easy as the package on flash.  I can also restore a backup package that the user has the option to create at flash time.
> It appears to be working great.  

you probably want to check out uClibc at http://uclibc.org