[U-Boot] boot-up time optimization. Where to start?

[U-Boot] boot-up time optimization. Where to start?

[Simon Schwarz]

Hi there,

I just started to work on my bachelor thesis. It is about "Linux boot-up
time optimization". The past days I spend analyzing what consumes the most
time in the boot process.

 I found that u-boot takes pretty much as long as the whole Linux kernel
(the one we are using).

I started digging into the source and I think I have a big picture of what
is going on. I already learned from the mailing list that it is a good idea
to start a discussion early if you plan to change something and want it
upstream. At this point of my thesis I'am free to choose where I start -
only string attached is that if it is platform specific it has to be TI
OMAP3.

So here is my question: Where do you see the most potential to optimize
u-boot?

I already have two bullets on my list (just some ideas - maybe totally
unrealistic *g*):
- Use Hardware specific copy commands
- build the checksum while moving the kernel to RAM

Thanks!
Simon

[U-Boot] boot-up time optimization. Where to start?

[Tabi Timur-B04825]

On Wed, Apr 27, 2011 at 8:39 AM, Simon Schwarz
<simonschwarzcor@googlemail.com> wrote:

> So here is my question: Where do you see the most potential to optimize
> u-boot?

I'm sure many of the timeouts could be optimized.

-- 
Timur Tabi
Linux kernel developer at Freescale

[U-Boot] boot-up time optimization. Where to start?

[Eric Cooper]

On Wed, Apr 27, 2011 at 03:39:18PM +0200, Simon Schwarz wrote:
> So here is my question: Where do you see the most potential to optimize
> u-boot?
> 
> I already have two bullets on my list (just some ideas - maybe totally
> unrealistic *g*):
> - Use Hardware specific copy commands
> - build the checksum while moving the kernel to RAM

This shouldn't be a matter of opinion/guess work.  You should
instrument U-Boot first and find out where the time is really being
spent.  Otherwise you run the classic risk of optimizing something
that isn't really a problem.

Good luck, I look forward to seeing the results of your analysis.

-- 
Eric Cooper             e c c @ c m u . e d u

[U-Boot] boot-up time optimization. Where to start?

[Mike Frysinger]

On Wed, Apr 27, 2011 at 09:39, Simon Schwarz wrote:
> ?I found that u-boot takes pretty much as long as the whole Linux kernel
> (the one we are using).
>
> I started digging into the source and I think I have a big picture of what
> is going on. I already learned from the mailing list that it is a good idea
> to start a discussion early if you plan to change something and want it
> upstream. At this point of my thesis I'am free to choose where I start -
> only string attached is that if it is platform specific it has to be TI
> OMAP3.

make sure caches are enabled.  then do as Eric suggested.
-mike

[U-Boot] boot-up time optimization. Where to start?

[Alexander Stein]

Hi,

Am Mittwoch, 27. April 2011, 15:39:18 schrieb Simon Schwarz:
> I just started to work on my bachelor thesis. It is about "Linux boot-up
> time optimization". The past days I spend analyzing what consumes the most
> time in the boot process.
> 
>  I found that u-boot takes pretty much as long as the whole Linux kernel
> (the one we are using).
> 
> I started digging into the source and I think I have a big picture of what
> is going on. I already learned from the mailing list that it is a good idea
> to start a discussion early if you plan to change something and want it
> upstream. At this point of my thesis I'am free to choose where I start -
> only string attached is that if it is platform specific it has to be TI
> OMAP3.
> 
> So here is my question: Where do you see the most potential to optimize
> u-boot?

Setting stdin, stdout and stderr takes a lot of time (IIRC ~500ms). Which IMO 
is useless on a bootloader without LCD support.

Regards,
Alexander

[U-Boot] boot-up time optimization. Where to start?

[Mike Frysinger]

On Wed, Apr 27, 2011 at 11:59, Alexander Stein wrote:
> Am Mittwoch, 27. April 2011, 15:39:18 schrieb Simon Schwarz:
>> I just started to work on my bachelor thesis. It is about "Linux boot-up
>> time optimization". The past days I spend analyzing what consumes the most
>> time in the boot process.
>>
>> ?I found that u-boot takes pretty much as long as the whole Linux kernel
>> (the one we are using).
>>
>> I started digging into the source and I think I have a big picture of what
>> is going on. I already learned from the mailing list that it is a good idea
>> to start a discussion early if you plan to change something and want it
>> upstream. At this point of my thesis I'am free to choose where I start -
>> only string attached is that if it is platform specific it has to be TI
>> OMAP3.
>>
>> So here is my question: Where do you see the most potential to optimize
>> u-boot?
>
> Setting stdin, stdout and stderr takes a lot of time (IIRC ~500ms). Which IMO
> is useless on a bootloader without LCD support.

eh ?  those are still used with serial consoles.  unless you're
talking about some driver that is specific to the OMAP3 and/or a
board.
-mike

[U-Boot] boot-up time optimization. Where to start?

[Wolfgang Denk]

Dear Alexander Stein,

In message <201104271759.11818.alexander.stein@systec-electronic.com> you wrote:
> 
> Setting stdin, stdout and stderr takes a lot of time (IIRC ~500ms). Which IMO 
> is useless on a bootloader without LCD support.

Statements like this are completely worhtless if you don;t tell
exactly on which architecture and board, and with which exact version
of U-Boot such numbers have been measured.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
Of all the things I've lost, I miss my mind the most.

[U-Boot] boot-up time optimization. Where to start?

[Wolfgang Denk]

Dear Charles Krinke,

please keep the mailing list on Cc:

In message <BANLkTimQzo7mMMb0DjqX0c78pcNHnDXPLg@mail.gmail.com> you wrote:
>
> Also, please help me understand if this is not true, but the notion of
> stdin, stdout, stderr predates lcd displays back to when all communication
> with a computer was via a teletype on a serial port.

I'm not sure what exactly you mean here.

Stdin, stdout, and stderr are first of all terms that relate to the
standard C library, more precisely to it's implementation of standard
I/O streams.  This has obviously little to do with U-Boot itself.

In U-Boot's terminology we use these names as references to I/O
channels which can be assigned to different devices. Such devices may
be serial or network ports (for input and output), keyboards (input
only), LCD screens or other video adapters (output only).

> So, I think that stdin, stdout and stderr are still very much appropriate in
> a non-lcd environment.

"lcd environment" or "non-lcd environment" is not really related at
all to these terms.

> On Apr 27, 2011 12:09 PM, "Wolfgang Denk" <wd@denx.de> wrote:
> > Dear Alexander Stein,
...

Please don't top-post / full quote. You may want to read
http://www.netmeister.org/news/learn2quote.html

> Content-Type: text/html; charset=UTF-8
> Content-Transfer-Encoding: quoted-printable

And never, never ever send HTML. Always send plain text only.

Thanks.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
The Gates in my computer are AND, OR and NOT; they are not Bill.

[U-Boot] boot-up time optimization. Where to start?

[Simon Schwarz]

Thank you all! That was more than I expected!

2011/4/27 Wolfgang Denk <wd@denx.de>

> Dear Charles Krinke,
>
> please keep the mailing list on Cc:
>
> In message <BANLkTimQzo7mMMb0DjqX0c78pcNHnDXPLg@mail.gmail.com> you wrote:
> >
> > Also, please help me understand if this is not true, but the notion of
> > stdin, stdout, stderr predates lcd displays back to when all
> communication
> > with a computer was via a teletype on a serial port.
>
> I'm not sure what exactly you mean here.
>
> Stdin, stdout, and stderr are first of all terms that relate to the
> standard C library, more precisely to it's implementation of standard
> I/O streams.  This has obviously little to do with U-Boot itself.
>
> In U-Boot's terminology we use these names as references to I/O
> channels which can be assigned to different devices. Such devices may
> be serial or network ports (for input and output), keyboards (input
> only), LCD screens or other video adapters (output only).
>
> > So, I think that stdin, stdout and stderr are still very much appropriate
> in
> > a non-lcd environment.
>
> "lcd environment" or "non-lcd environment" is not really related at
> all to these terms.
>
> > On Apr 27, 2011 12:09 PM, "Wolfgang Denk" <wd@denx.de> wrote:
> > > Dear Alexander Stein,
> ...
>
> Please don't top-post / full quote. You may want to read
> http://www.netmeister.org/news/learn2quote.html
>
> > Content-Type: text/html; charset=UTF-8
> > Content-Transfer-Encoding: quoted-printable
>
> And never, never ever send HTML. Always send plain text only.
>
> Thanks.
>
> Best regards,
>
> Wolfgang Denk
>
> --
> DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
> HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
> The Gates in my computer are AND, OR and NOT; they are not Bill.
> _______________________________________________
> U-Boot mailing list
> U-Boot at lists.denx.de
> http://lists.denx.de/mailman/listinfo/u-boot
>

[U-Boot] boot-up time optimization. Where to start?

[Alexander Stein]

Dear Wolfgang,

Am Mittwoch, 27. April 2011, 21:08:50 schrieb Wolfgang Denk:
> In message <201104271759.11818.alexander.stein@systec-electronic.com> you 
wrote:
> > Setting stdin, stdout and stderr takes a lot of time (IIRC ~500ms). Which
> > IMO is useless on a bootloader without LCD support.
> 
> Statements like this are completely worhtless if you don;t tell
> exactly on which architecture and board, and with which exact version
> of U-Boot such numbers have been measured.

Ok, let me be more precise on this.
We used U-Boot v2010.09 on a custom board running on an I.MX35 (ARM1136).
We noticed the following code snippet took relatively long.
From common/console.c in console_init_r(void):

> /* Setting environment variables */
> for (i = 0; i < 3; i++) {
> 	setenv(stdio_names[i], stdio_devices[i]->name);
> }

We added PIN toggling around this part of code and measured something >100ms. 
A collegue said it was ~100ms, I remembered ~500ms. Dunno who is right.

Regards,
Alexander

[U-Boot] boot-up time optimization. Where to start?

[Detlev Zundel]

Hi Alexander,

> Dear Wolfgang,
>
> Am Mittwoch, 27. April 2011, 21:08:50 schrieb Wolfgang Denk:
>> In message <201104271759.11818.alexander.stein@systec-electronic.com> you 
> wrote:
>> > Setting stdin, stdout and stderr takes a lot of time (IIRC ~500ms). Which
>> > IMO is useless on a bootloader without LCD support.
>> 
>> Statements like this are completely worhtless if you don;t tell
>> exactly on which architecture and board, and with which exact version
>> of U-Boot such numbers have been measured.
>
> Ok, let me be more precise on this.
> We used U-Boot v2010.09 on a custom board running on an I.MX35 (ARM1136).
> We noticed the following code snippet took relatively long.
>>From common/console.c in console_init_r(void):
>
>> /* Setting environment variables */
>> for (i = 0; i < 3; i++) {
>> 	setenv(stdio_names[i], stdio_devices[i]->name);
>> }
>
> We added PIN toggling around this part of code and measured something >100ms. 
> A collegue said it was ~100ms, I remembered ~500ms. Dunno who is right.

It doesn't really matter who is right - 100ms is way off for setting
these variables.  Looking into common/cmd_nvedit.c, these variables have
a special handling and there are ifdef's involved, so its not
straightforward to read.  You should really find out, where in there the
time is spent for your board and fix the problem ;)

Cheers
  Detlev

-- 
I can understand that things like user-level libraries have to take crazy people
into account, but the kernel internal libraries definitely do not.
                                     -- Linus Torvalds
--
DENX Software Engineering GmbH,      MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich,  Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: dzu at denx.de

[U-Boot] boot-up time optimization. Where to start?

[Alexander Stein]

Hi Detlev,

Am Montag, 2. Mai 2011, 17:31:15 schrieb Detlev Zundel:
> Hi Alexander,
> 
> > Dear Wolfgang,
> > 
> > Am Mittwoch, 27. April 2011, 21:08:50 schrieb Wolfgang Denk:
> >> In message <201104271759.11818.alexander.stein@systec-electronic.com>
> >> you
> > 
> > wrote:
> >> > Setting stdin, stdout and stderr takes a lot of time (IIRC ~500ms).
> >> > Which IMO is useless on a bootloader without LCD support.
> >> 
> >> Statements like this are completely worhtless if you don;t tell
> >> exactly on which architecture and board, and with which exact version
> >> of U-Boot such numbers have been measured.
> > 
> > Ok, let me be more precise on this.
> > We used U-Boot v2010.09 on a custom board running on an I.MX35 (ARM1136).
> > We noticed the following code snippet took relatively long.
> > 
> >>From common/console.c in console_init_r(void):
> >> /* Setting environment variables */
> >> for (i = 0; i < 3; i++) {
> >> 
> >> 	setenv(stdio_names[i], stdio_devices[i]->name);
> >> 
> >> }
> > 
> > We added PIN toggling around this part of code and measured something
> > >100ms. A collegue said it was ~100ms, I remembered ~500ms. Dunno who is
> > right.
> 
> It doesn't really matter who is right - 100ms is way off for setting
> these variables.  Looking into common/cmd_nvedit.c, these variables have
> a special handling and there are ifdef's involved, so its not
> straightforward to read.  You should really find out, where in there the
> time is spent for your board and fix the problem ;)

Our 'fix' was removing the stated lines at all.

Regards,
Alexander

[U-Boot] boot-up time optimization. Where to start?

[Detlev Zundel]

Hi Alexander,

[...]

>> > Ok, let me be more precise on this.
>> > We used U-Boot v2010.09 on a custom board running on an I.MX35 (ARM1136).
>> > We noticed the following code snippet took relatively long.
>> > 
>> >>From common/console.c in console_init_r(void):
>> >> /* Setting environment variables */
>> >> for (i = 0; i < 3; i++) {
>> >> 
>> >> 	setenv(stdio_names[i], stdio_devices[i]->name);
>> >> 
>> >> }
>> > 
>> > We added PIN toggling around this part of code and measured something
>> > >100ms. A collegue said it was ~100ms, I remembered ~500ms. Dunno who is
>> > right.
>> 
>> It doesn't really matter who is right - 100ms is way off for setting
>> these variables.  Looking into common/cmd_nvedit.c, these variables have
>> a special handling and there are ifdef's involved, so its not
>> straightforward to read.  You should really find out, where in there the
>> time is spent for your board and fix the problem ;)
>
> Our 'fix' was removing the stated lines at all.

That is not a fix but simply ignoring a problem.  Maybe if you find out
why these lines have such an unexpected run-time, you will solve more
problems also.

We have a mantra here on the mailing list, so let me introduce you to
it:

Solve the problems one by one in the order that you encounter them.
Every ignored problem will come back later and catch you.  Really, it
will.

Now repeat after me ;)

Cheers
  Detlev

-- 
There is no need to be  rigid in carrying out policies about what changes
to install.  To do a good job of maintaining Emacs doesn't require acting
like government bureaucrats.
                -- Richard Stallman <E1MIx3Y-0005iZ-OH@fencepost.gnu.org>
--
DENX Software Engineering GmbH,      MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich,  Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: dzu at denx.de

[U-Boot] boot-up time optimization. Where to start?

[Wolfgang Denk]

Dear Alexander Stein,

In message <201105021807.52142.alexander.stein@systec-electronic.com> you wrote:
> 
> Our 'fix' was removing the stated lines at all.

Did you understand what you were doing?

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
"You shouldn't make my toaster angry." - Household security explained
in "Johnny Quest"

[U-Boot] boot-up time optimization. Where to start?

[Wolfgang Denk]

Dear Alexander Stein,

In message <201105021640.27241.alexander.stein@systec-electronic.com> you wrote:
> 
> Ok, let me be more precise on this.
> We used U-Boot v2010.09 on a custom board running on an I.MX35 (ARM1136).

Let's summarize known facts:

1. You are talking about an out-of-tree port, i. e. code which is
   completely unknown here, so it is basicly impossible to comment.
   We can barely speculate.

2. This is an ARM board.

3. This is an old version before cache support for ARM was added.

> We noticed the following code snippet took relatively long.
> From common/console.c in console_init_r(void):
> 
> > /* Setting environment variables */
> > for (i = 0; i < 3; i++) {
> > 	setenv(stdio_names[i], stdio_devices[i]->name);
> > }
> 
> We added PIN toggling around this part of code and measured something >100ms. 
> A collegue said it was ~100ms, I remembered ~500ms. Dunno who is right.

Both numbers are way off.

Let me speculate: (I) you have a _huge_ environment allocated for your
board, probably > 100 KiB or more; (II) you are loading it from a slow
storage device, probably NAND flash; (III) you are running on a narrow
system bus (16 bit) with non-optimal RAM timings; (IV) you do all this
with caches turned off; (V) you measure some numbers but you don;t
understand what they mean.

I bet some beer that at least 3 of these speculations hit the point.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
I don't want to be young again, I just don't want to get any older.

[U-Boot] boot-up time optimization. Where to start?

[Alexander Stein]

Dear Wolfgang,

Am Montag, 2. Mai 2011, 19:00:47 schrieb Wolfgang Denk:
> In message <201105021640.27241.alexander.stein@systec-electronic.com> you 
wrote:
> > Ok, let me be more precise on this.
> > We used U-Boot v2010.09 on a custom board running on an I.MX35 (ARM1136).
> 
> Let's summarize known facts:
> 
> 1. You are talking about an out-of-tree port, i. e. code which is
>    completely unknown here, so it is basicly impossible to comment.
>    We can barely speculate.

I'm aware of that. But on the other hand, cmd_nvedit.c and console.c are both 
generic parts.
As Detlev have written, this is not the cause of the problem. Let's see if I 
find time again for this.

> 2. This is an ARM board.
> 
> 3. This is an old version before cache support for ARM was added.

This specific version was selected due to relocation problems on ARM. But I 
expect the dcache doesn't have that big influence on the named code part as 
the environment is already in RAM.

> > We noticed the following code snippet took relatively long.
> > 
> > From common/console.c in console_init_r(void):
> > > /* Setting environment variables */
> > > for (i = 0; i < 3; i++) {
> > > 
> > > 	setenv(stdio_names[i], stdio_devices[i]->name);
> > > 
> > > }
> > 
> > We added PIN toggling around this part of code and measured something
> > >100ms. A collegue said it was ~100ms, I remembered ~500ms. Dunno who is
> > right.
> 
> Both numbers are way off.
> 
> Let me speculate: (I) you have a _huge_ environment allocated for your
> board, probably > 100 KiB or more;

> Environment size: 2098/131067 bytes

So, no.

> (II) you are loading it from a slow
> storage device, probably NAND flash;

The environment is stored in NOR-Flash. So, no.

> (III) you are running on a narrow
> system bus (16 bit) with non-optimal RAM timings;

It is using a 32-Bit RAM-Bus. So, no.

> (IV) you do all this
> with caches turned off;

dcaches should be off, while icaches are on. So yes and no.

> (V) you measure some numbers but you don;t
> understand what they mean.

These numbers show me that this part of code increases the start time of a 
considerable amount.
The workaround resulted in a faster startup without notable side effect.
I'm aware this is not the fix of the problem.
So yes and no.

> I bet some beer that at least 3 of these speculations hit the point.

You better not want to bet here :-)

Regards,
Alexander

[U-Boot] boot-up time optimization. Where to start?

[Stefano Babic]

On 05/03/2011 08:48 AM, Alexander Stein wrote:

Hi Alexander,

> Am Montag, 2. Mai 2011, 19:00:47 schrieb Wolfgang Denk:
>> In message <201105021640.27241.alexander.stein@systec-electronic.com> you 
> wrote:
>>> Ok, let me be more precise on this.
>>> We used U-Boot v2010.09 on a custom board running on an I.MX35 (ARM1136).

The i.MX35 is supported in u-boot mainline. As I can suppose, you start
from the mx35pdk to port your code, probably you have good chances to
use the last u-boot code. And then to send patches for your board,
too... ;-)

> 
>> 2. This is an ARM board.
>>
>> 3. This is an old version before cache support for ARM was added.
> 
> This specific version was selected due to relocation problems on ARM.

The mx35pdk board is supported in u-boot mainline using relocation.

> expect the dcache doesn't have that big influence on the named code part as 
> the environment is already in RAM.

Check in the mailing list, the numbers reported by people who measured
the influence of cache say there is a big difference, specially when big
chunks of code is copied, that is copying the kernel from storage before
booting.

Best regards,
Stefano Babic

-- 
=====================================================================
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-0 Fax: +49-8142-66989-80  Email: office at denx.de
=====================================================================

[U-Boot] boot-up time optimization. Where to start?

[Wolfgang Denk]

Dear Alexander Stein,

In message <201105030848.17576.alexander.stein@systec-electronic.com> you wrote:
> 
> This specific version was selected due to relocation problems on ARM. But I 
> expect the dcache doesn't have that big influence on the named code part as 
> the environment is already in RAM.

Your expectation is most likely completely wrong.  Reading from /
writing to uncached RAM is painfully slow compared to a system with
caches turned on.  And if you - as I speculate - need to checksum a
huge amount of data, this will delay things without need.


Are you also still using the old environment code in your port, or is
the new, hash table based one?  When using the old code, there are
additional penalties for using a needlessly big environment as each
call to setenv() will recalculate the checksum.


> > Let me speculate: (I) you have a _huge_ environment allocated for your
> > board, probably > 100 KiB or more;
> 
> > Environment size: 2098/131067 bytes
> 
> So, no.

So, yes!  You cannot even read your own numbers correctly.

131067 = 128 KiB which _is_ > 100 KiB.

> > (II) you are loading it from a slow
> > storage device, probably NAND flash;
> 
> The environment is stored in NOR-Flash. So, no.

Especially on NOR flash there is no reason to use an environment size
of 128 KiB when you only use 2 KiB of it.

> > (III) you are running on a narrow
> > system bus (16 bit) with non-optimal RAM timings;
> 
> It is using a 32-Bit RAM-Bus. So, no.

And your NOR flash?

And your memory timings?

> > (IV) you do all this
> > with caches turned off;
> 
> dcaches should be off, while icaches are on. So yes and no.

DC of makes things awfully slow.  See comments of commits c3330e9,
95c6f6d and 7e4a9e6 - for plain RAM bound operations like
copying/uncompressing an image from RAM to RAM switchign on the DC can
accelerate the system by a factor of up to >15.

> > (V) you measure some numbers but you don;t
> > understand what they mean.
> 
> These numbers show me that this part of code increases the start time of a 
> considerable amount.

You don;t even understand that you have > 100 KiB of environment size
which gets checksummed without need.

> The workaround resulted in a faster startup without notable side effect.
> I'm aware this is not the fix of the problem.
> So yes and no.
> 
> > I bet some beer that at least 3 of these speculations hit the point.
> 
> You better not want to bet here :-)

It appears I was right on accounts (I), (IV) and (V), it seems ;-)
And (III) is not clear yet.

Fact is, the code that you claim takes 100 (or 500) ms to run has no
potential for such a long run time unless your system is seriously
misconfigured.  I guess it runs at least 100 times faster on all
systems I have access to.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
"I dislike companies that have a we-are-the-high-priests-of-hardware-
so-you'll-like-what-we-give-you attitude. I like commodity markets in
which iron-and-silicon hawkers know that they exist to  provide  fast
toys for software types like me to play with..."    - Eric S. Raymond

[U-Boot] boot-up time optimization. Where to start?

[Alexander Stein]

Dear Wolfgang,

Am Donnerstag, 5. Mai 2011, 07:32:20 schrieb Wolfgang Denk:
> In message <201105030848.17576.alexander.stein@systec-electronic.com> you 
wrote:
> > This specific version was selected due to relocation problems on ARM. But
> > I expect the dcache doesn't have that big influence on the named code
> > part as the environment is already in RAM.
> 
> Your expectation is most likely completely wrong.  Reading from /
> writing to uncached RAM is painfully slow compared to a system with
> caches turned on.  And if you - as I speculate - need to checksum a
> huge amount of data, this will delay things without need.
> 
> 
> Are you also still using the old environment code in your port, or is
> the new, hash table based one?  When using the old code, there are
> additional penalties for using a needlessly big environment as each
> call to setenv() will recalculate the checksum.

I was digging into this problem for a short time. And yes, the CRC 
checksumcalculation takes about 25ms each run. So setenv is called for each 
stdin,stdout and stderr. which sums up to ~75ms.
So you're right this is the old environment code. Here a dcache will speed up 
the execution of course.
But our standard startup just stars U-Boot and copies the Linux kernel into 
RAM and starts it. There is not much use of dcache during copy here.

> > > (III) you are running on a narrow
> > > system bus (16 bit) with non-optimal RAM timings;
> > 
> > It is using a 32-Bit RAM-Bus. So, no.
> 
> And your NOR flash?

It is connected 16-bit like most devices only support, but it is setup to use 
page read mode.

> And your memory timings?

Should be pretty good.

> > > (IV) you do all this
> > > with caches turned off;
> > 
> > dcaches should be off, while icaches are on. So yes and no.
> 
> DC of makes things awfully slow.  See comments of commits c3330e9,
> 95c6f6d and 7e4a9e6 - for plain RAM bound operations like
> copying/uncompressing an image from RAM to RAM switchign on the DC can
> accelerate the system by a factor of up to >15.

Yes, from RAM to RAM, dcache will help a lot. But we neither copy from RAM to 
RAM nor do we uncompressing.

> > > (V) you measure some numbers but you don;t
> > > understand what they mean.
> > 
> > These numbers show me that this part of code increases the start time of
> > a considerable amount.
> 
> You don;t even understand that you have > 100 KiB of environment size
> which gets checksummed without need.

Mh, this might be an option for further ports.

> Fact is, the code that you claim takes 100 (or 500) ms to run has no
> potential for such a long run time unless your system is seriously
> misconfigured.  I guess it runs at least 100 times faster on all
> systems I have access to.

Well, as already said this is related to CRC calculation of environment. I did 
a fast port to v2011.03 and the setenv is a lot faster, which is due the new 
env code base.
But I also noticed the time until kernel_entry is called is about 30ms later 
after reset than on the old code base. But I didn't investigate any time 
further to see what caused this. But AFAICS also the new U-Boot code doesn't 
enable dcache on ARM1136 either.

Regards,
Alexander

[U-Boot] boot-up time optimization. Where to start?

[Wolfgang Denk]

Dear Alexander Stein,

In message <201105050906.35834.alexander.stein@systec-electronic.com> you wrote:
> 
> > Are you also still using the old environment code in your port, or is
> > the new, hash table based one?  When using the old code, there are
> > additional penalties for using a needlessly big environment as each
> > call to setenv() will recalculate the checksum.
> 
> I was digging into this problem for a short time. And yes, the CRC 
> checksumcalculation takes about 25ms each run. So setenv is called for each 
> stdin,stdout and stderr. which sums up to ~75ms.
> So you're right this is the old environment code. Here a dcache will speed up 
> the execution of course.

Even more so would reducing the environment size to some reasonable
value. Currently you are using some 2 KiB, so say you set the
environment size to 8 KiB. This would be 1/16 of your current size,
which means the ~75ms would shrink to less than 5 ms.  You are wasting
70 ms (only here - there are other places which will add to this
figure) just because this inappropriate configuration.

> But our standard startup just stars U-Boot and copies the Linux kernel into 
> RAM and starts it. There is not much use of dcache during copy here.

You are wrong. There is a huge difference between perrforming a copy
operation in single write cycles to uncached RAM versus writing to a
cached area where the cache flushes willoperate in burst mode. Also,
the U-Boot code will run faster, too, so copying and decompression is
much faster.


You repeat the same mistake again: you make assumptions about  what
may or may not be  fast or slow on your system without actually
measuring it.  Donald Knuth is right again: "Early optimization is
the root of much evil."


> > > It is using a 32-Bit RAM-Bus. So, no.
> > 
> > And your NOR flash?
> 
> It is connected 16-bit like most devices only support, but it is setup to use 
> page read mode.

Well, many systems use two 16 bit chips in parallel to give a 32 bit
bus.

> > DC of makes things awfully slow.  See comments of commits c3330e9,
> > 95c6f6d and 7e4a9e6 - for plain RAM bound operations like
> > copying/uncompressing an image from RAM to RAM switchign on the DC can
> > accelerate the system by a factor of up to >15.
> 
> Yes, from RAM to RAM, dcache will help a lot. But we neither copy from RAM to 
> RAM nor do we uncompressing.

There is still a huge diference in memory bandwith between using plain
single write cycles versus burst mode accesses.

Don't speculate.  Measure yourself!


Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
"Send lawyers, guns and money..."  - Lyrics from a Warren Zevon song

[U-Boot] boot-up time optimization. Where to start?

[Stefano Babic]

On 05/05/2011 09:06 AM, Alexander Stein wrote:

> Well, as already said this is related to CRC calculation of environment. I did 
> a fast port to v2011.03 and the setenv is a lot faster, which is due the new 
> env code base.
> But I also noticed the time until kernel_entry is called is about 30ms later 
> after reset than on the old code base. But I didn't investigate any time 
> further to see what caused this. But AFAICS also the new U-Boot code doesn't 
> enable dcache on ARM1136 either.

This is wrong. The dcache is enabled on several MX31 boards. On the
mx35pdk, it is not enabled.

Add CONFIG_CMD_CACHE to your configuration file, and you will get the
dcache <on|off> command.

Best regards,
Stefano Babic

-- 
=====================================================================
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-0 Fax: +49-8142-66989-80  Email: office at denx.de
=====================================================================

[U-Boot] boot-up time optimization. Where to start?

[Charles Manning]

On Thursday 05 May 2011 17:32:20 Wolfgang Denk wrote:
> Dear Alexander Stein,
>
> In message <201105030848.17576.alexander.stein@systec-electronic.com> you 
wrote:
> > This specific version was selected due to relocation problems on ARM. But
> > I expect the dcache doesn't have that big influence on the named code
> > part as the environment is already in RAM.
>
> Your expectation is most likely completely wrong.  Reading from /
> writing to uncached RAM is painfully slow compared to a system with
> caches turned on.  And if you - as I speculate - need to checksum a
> huge amount of data, this will delay things without need.

Caching has a huge effect on **all** code and is the first thing I'd play with 
in trying to speed things up.

I have been doing some stuff to speed omap3 booting. It was taking approx 4 
seconds from power up until the kernel started spewing boot messages. That is 
now down to less than 2 secs (including the funky omap3 romboot time, loading 
uboot from NAND and then loading the kernel from NAND). Only difference was 
turning on caching in uboot using the caching commands.

-- Charles

[U-Boot] boot-up time optimization. Where to start?

[Simon Glass]

On Thu, May 5, 2011 at 11:10 AM, Charles Manning <manningc2@actrix.gen.nz>wrote:

> On Thursday 05 May 2011 17:32:20 Wolfgang Denk wrote:
> > Dear Alexander Stein,
> >
> > In message <201105030848.17576.alexander.stein@systec-electronic.com>
> you
> wrote:
> > > This specific version was selected due to relocation problems on ARM.
> But
> > > I expect the dcache doesn't have that big influence on the named code
> > > part as the environment is already in RAM.
> >
> > Your expectation is most likely completely wrong.  Reading from /
> > writing to uncached RAM is painfully slow compared to a system with
> > caches turned on.  And if you - as I speculate - need to checksum a
> > huge amount of data, this will delay things without need.
>
> Caching has a huge effect on **all** code and is the first thing I'd play
> with
> in trying to speed things up.
>

Yes agreed. Running U-Boot without caching is a great way to slow boot time.
In fact we should turn on the L2 cache if there is one.

>
> I have been doing some stuff to speed omap3 booting. It was taking approx 4
> seconds from power up until the kernel started spewing boot messages. That
> is
> now down to less than 2 secs (including the funky omap3 romboot time,
> loading
> uboot from NAND and then loading the kernel from NAND). Only difference was
> turning on caching in uboot using the caching commands.
>

I have a Seaboard (Tegra2) networking booting using a USB dongle in about 5s
with console output, about 3s of which is USB and PHY delay. Turning off
caching adds several seconds mainly because the tftp is so much slower. The
same would apply for any boot medium. I have a patch which displays boot
time in microseconds if it is of interest to anyone:

Timer summary in microseconds:
       Mark    Elapsed  Stage
          0          0  awake
    193,298    193,298  usb_start
  1,342,411  1,149,113  eth_start
  3,767,039  2,424,628  bootp_start
  3,790,121     23,082  bootp_stop
  3,790,293        172  tftp start
  4,761,459    971,166  tftp done
  4,761,489         30  bootm_start
  4,892,145    130,656  start_kernel

Regards,
Simon


> -- Charles
>
> _______________________________________________
> U-Boot mailing list
> U-Boot at lists.denx.de
> http://lists.denx.de/mailman/listinfo/u-boot
>

[U-Boot] boot-up time optimization. Where to start?

[Detlev Zundel]

Hi Simon,

[...]

> I have a patch which displays boot time in microseconds if it is of
> interest to anyone:
>
> Timer summary in microseconds:
>        Mark    Elapsed  Stage
>           0          0  awake
>     193,298    193,298  usb_start
>   1,342,411  1,149,113  eth_start
>   3,767,039  2,424,628  bootp_start
>   3,790,121     23,082  bootp_stop
>   3,790,293        172  tftp start
>   4,761,459    971,166  tftp done
>   4,761,489         30  bootm_start
>   4,892,145    130,656  start_kernel

Of course we are interested, please show us the code.  Such code, even
if it does not get into mainline can be a good starter for other people
benchmarking their port.

Thanks
  Detlev

-- 
"One of my most productive days was throwing away 1000 lines of code."
  - Ken Thompson.
--
DENX Software Engineering GmbH,      MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich,  Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: dzu at denx.de

[U-Boot] boot-up time optimization. Where to start?

[Simon Schwarz]

How could I miss this. Of course I am interested!

Thanky you
Simon

2011/5/9 Detlev Zundel <dzu@denx.de>

> Hi Simon,
>
> [...]
>
> > I have a patch which displays boot time in microseconds if it is of
> > interest to anyone:
> >
> > Timer summary in microseconds:
> >        Mark    Elapsed  Stage
> >           0          0  awake
> >     193,298    193,298  usb_start
> >   1,342,411  1,149,113  eth_start
> >   3,767,039  2,424,628  bootp_start
> >   3,790,121     23,082  bootp_stop
> >   3,790,293        172  tftp start
> >   4,761,459    971,166  tftp done
> >   4,761,489         30  bootm_start
> >   4,892,145    130,656  start_kernel
>
> Of course we are interested, please show us the code.  Such code, even
> if it does not get into mainline can be a good starter for other people
> benchmarking their port.
>
> Thanks
>  Detlev
>
> --
> "One of my most productive days was throwing away 1000 lines of code."
>  - Ken Thompson.
> --
> DENX Software Engineering GmbH,      MD: Wolfgang Denk & Detlev Zundel
> HRB 165235 Munich,  Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: dzu at denx.de
>

[U-Boot] boot-up time optimization. Where to start?

[Simon Glass]

Hi,

I will post a patch. Also we have a 'time' command we could upstream.

Regards,
Simon

On Mon, May 9, 2011 at 3:49 AM, Detlev Zundel <dzu@denx.de> wrote:

> Hi Simon,
>
> [...]
>
> > I have a patch which displays boot time in microseconds if it is of
> > interest to anyone:
> >
> > Timer summary in microseconds:
> >        Mark    Elapsed  Stage
> >           0          0  awake
> >     193,298    193,298  usb_start
> >   1,342,411  1,149,113  eth_start
> >   3,767,039  2,424,628  bootp_start
> >   3,790,121     23,082  bootp_stop
> >   3,790,293        172  tftp start
> >   4,761,459    971,166  tftp done
> >   4,761,489         30  bootm_start
> >   4,892,145    130,656  start_kernel
>
> Of course we are interested, please show us the code.  Such code, even
> if it does not get into mainline can be a good starter for other people
> benchmarking their port.
>
> Thanks
>  Detlev
>
> --
> "One of my most productive days was throwing away 1000 lines of code."
>  - Ken Thompson.
> --
> DENX Software Engineering GmbH,      MD: Wolfgang Denk & Detlev Zundel
> HRB 165235 Munich,  Office: Kirchenstr.5, D-82194 Groebenzell, Germany
> Phone: (+49)-8142-66989-40 Fax: (+49)-8142-66989-80 Email: dzu at denx.de
>

[U-Boot] boot-up time optimization. Where to start?

[Wolfgang Denk]

Dear Simon Schwarz,

In message <BANLkTi=HfOsg73_GCC=4aVoi_XUNbcf9nQ@mail.gmail.com> you wrote:
>
> I just started to work on my bachelor thesis. It is about "Linux boot-up
> time optimization". The past days I spend analyzing what consumes the most
> time in the boot process.

A lot of efforts have alreay been spent in this area.  A ton of
specialized solutions exist, some of them even documetned.  I highly
recommend to study the available information first.  A good starting
point might be this page: http://www.elinux.org/Boot_Time

>  I found that u-boot takes pretty much as long as the whole Linux kernel
> (the one we are using).

This depends on a lot of things: architecture, boot device, memory
architecture, etc. etc.

> I started digging into the source and I think I have a big picture of what
> is going on. I already learned from the mailing list that it is a good idea
> to start a discussion early if you plan to change something and want it
> upstream. At this point of my thesis I'am free to choose where I start -
> only string attached is that if it is platform specific it has to be TI
> OMAP3.
> 
> So here is my question: Where do you see the most potential to optimize
> u-boot?
> 
> I already have two bullets on my list (just some ideas - maybe totally
> unrealistic *g*):
> - Use Hardware specific copy commands
> - build the checksum while moving the kernel to RAM

Eventually you will find that these are not _that_ important.

A few thoughts, mostly unsorted:

- Booting fast means running fast, so make sure your CPU can run as
  fast as possible: chose an optimized clock configuration; chose
  optimized memory timings; make sure all caches are enabled.

- Embedded processors usually have tiny caches, so performance is
  often limited by the memory bandwidth: a device with a 32 bit wide
  memory bus will be faster than one with a 16 bit bus.

- Memory technology is imprtant: DDR3 is much faster than plain old
  SDRAM.

- Chose the storage device wisely: using NOR flash as boot device
  allows direct execution of the code from the NOR (often using a 32
  bit wide bus); if you use NAND flash as storage you will have a
  relatively slow download and a way more complicated protocol for
  booting.

- Benchmark your system before you make assumptions.  For example, it
  is not possible to decide in general if a compressed or an
  uncompressed Linux kernel image is a better choice - if you have a
  fast bus and a slow CPU, then the CPU performance for uncompressing
  the code may be the bottleneck, and an uncompressed kernel may be
  way faster; if you have a fast CPU and a relatively slow storage
  device it may be much more efficient to load the smaller, compressed
  kernel image, and the CPU will still be mostly idle uncompressing
  it.

- Keep in mind that you are always trading efforts and costs, boot
  time and reliability/security against each other. You always get a
  maximum of two items, never all three (good, cheap, fast). If you
  optimize for speed, you may have to accept not only higher
  development efforts, but for example also higher production coses
  (like when needing bigger boot devices for uncompressed images or
  file systems), but also have to give up security/reliabilty by for
  example switching off the checksum protection of the images.

- Optimize your configuration: U-Boot in it's general configuration is
  more of a bring-up and debug tool, combined with a powerful machine
  for implementing fancy features like software updates and such, but
  not optimized at all for fast booting.

  Removing unused code (both in U-Boot and Linux) makes the memory
  footprint smaller and thus loading faster.

- Apply the old Antoine de Saint-Exupery aphorism: "Perfection is
  reached, not when there is no longer anything to add, but when
  there is no longer anything to take away."  For example when booting
  form NAND flash, we have a number of steps: some ROM code loads a
  small block of code from NAND into some (usually limited) memory;
  this code (in U-Boot terms the nand_spl code) will load the rest of
  U-Boot and start it; U-Boot will then initialize more of the system
  and relocate itself to the upper end o the RAM; then U-Boot will
  load the Linux kernel and start it.

  This allows for a powerful and very flexible system, but how much
  of this power and flexibility do you really need on a system where
  your primary goal is to boot fast, i. e. to make U-Boot goes out of
  the way as fast as possible?

  If you want to boot fast, why do you go through all these steps,
  then?  Why does not your nand_spl code load the Linux kernel
  directly and start it, instead of wasting time with all these other
  steps and phases?  [Note that this is the approach taken by all
  these systems that report sub-second boot times.]

- Don't stop the optimization with the loading of the Linux kernel.
  The choice of the root file system type, the strategy for driver
  initialization (immediate with statically linked drivers versus lazy
  with dynamically loaded modues) and for starting software services
  is at least as well as important.

  Keep in mind that on most systems it's trivial (with standard
  U-Boot) to have Linux starting the first user space code within 5 or
  6 seconds after port-on.  But on many systems the appliucation
  startup will take much longer than that.

- And on each step of you path benchmark again, and focus on the hot
  spots.  It makes little sense to spend a week of effort on reducing
  the execustion time of function foo() to 5% of it's original value,
  when this cunction contributes only 1% to the total boot time.

Hope this helps.

Best regards,

Wolfgang Denk

-- 
DENX Software Engineering GmbH,     MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
But the only way of discovering the limits  of  the  possible  is  to
venture a little way past them into the impossible.
                         - _Profiles of the Future_ (1962; rev. 1973)
                  ``Hazards of Prophecy: The Failure of Imagination''