[U-Boot] boot-up time optimization. Where to start?

Wolfgang Denk
2011-04-27T19:07:26Z

Dear Simon Schwarz,

In message <BANLkTi=HfOsg73_GCC=4aVoi_XUNbcf9nQ@mail.gmail.com> you wrote:
>
> I just started to work on my bachelor thesis. It is about "Linux boot-up
> time optimization". The past days I spend analyzing what consumes the most
> time in the boot process.

A lot of efforts have alreay been spent in this area. A ton of
specialized solutions exist, some of them even documetned. I highly
recommend to study the available information first. A good starting
point might be this page: http://www.elinux.org/Boot_Time

> I found that u-boot takes pretty much as long as the whole Linux kernel
> (the one we are using).

This depends on a lot of things: architecture, boot device, memory
architecture, etc. etc.

> I started digging into the source and I think I have a big picture of what
> is going on. I already learned from the mailing list that it is a good idea
> to start a discussion early if you plan to change something and want it
> upstream. At this point of my thesis I'am free to choose where I start -
> only string attached is that if it is platform specific it has to be TI
> OMAP3.
>
> So here is my question: Where do you see the most potential to optimize
> u-boot?
>
> I already have two bullets on my list (just some ideas - maybe totally
> unrealistic *g*):
> - Use Hardware specific copy commands
> - build the checksum while moving the kernel to RAM

Eventually you will find that these are not _that_ important.

A few thoughts, mostly unsorted:

- Booting fast means running fast, so make sure your CPU can run as
fast as possible: chose an optimized clock configuration; chose
optimized memory timings; make sure all caches are enabled.

- Embedded processors usually have tiny caches, so performance is
often limited by the memory bandwidth: a device with a 32 bit wide
memory bus will be faster than one with a 16 bit bus.

- Memory technology is imprtant: DDR3 is much faster than plain old
SDRAM.

- Chose the storage device wisely: using NOR flash as boot device
allows direct execution of the code from the NOR (often using a 32
bit wide bus); if you use NAND flash as storage you will have a
relatively slow download and a way more complicated protocol for
booting.

- Benchmark your system before you make assumptions. For example, it
is not possible to decide in general if a compressed or an
uncompressed Linux kernel image is a better choice - if you have a
fast bus and a slow CPU, then the CPU performance for uncompressing
the code may be the bottleneck, and an uncompressed kernel may be
way faster; if you have a fast CPU and a relatively slow storage
device it may be much more efficient to load the smaller, compressed
kernel image, and the CPU will still be mostly idle uncompressing
it.

- Keep in mind that you are always trading efforts and costs, boot
time and reliability/security against each other. You always get a
maximum of two items, never all three (good, cheap, fast). If you
optimize for speed, you may have to accept not only higher
development efforts, but for example also higher production coses
(like when needing bigger boot devices for uncompressed images or
file systems), but also have to give up security/reliabilty by for
example switching off the checksum protection of the images.

- Optimize your configuration: U-Boot in it's general configuration is
more of a bring-up and debug tool, combined with a powerful machine
for implementing fancy features like software updates and such, but
not optimized at all for fast booting.

Removing unused code (both in U-Boot and Linux) makes the memory
footprint smaller and thus loading faster.

- Apply the old Antoine de Saint-Exupery aphorism: "Perfection is
reached, not when there is no longer anything to add, but when
there is no longer anything to take away." For example when booting
form NAND flash, we have a number of steps: some ROM code loads a
small block of code from NAND into some (usually limited) memory;
this code (in U-Boot terms the nand_spl code) will load the rest of
U-Boot and start it; U-Boot will then initialize more of the system
and relocate itself to the upper end o the RAM; then U-Boot will
load the Linux kernel and start it.

This allows for a powerful and very flexible system, but how much
of this power and flexibility do you really need on a system where
your primary goal is to boot fast, i. e. to make U-Boot goes out of
the way as fast as possible?

If you want to boot fast, why do you go through all these steps,
then? Why does not your nand_spl code load the Linux kernel
directly and start it, instead of wasting time with all these other
steps and phases? [Note that this is the approach taken by all
these systems that report sub-second boot times.]

- Don't stop the optimization with the loading of the Linux kernel.
The choice of the root file system type, the strategy for driver
initialization (immediate with statically linked drivers versus lazy
with dynamically loaded modues) and for starting software services
is at least as well as important.

Keep in mind that on most systems it's trivial (with standard
U-Boot) to have Linux starting the first user space code within 5 or
6 seconds after port-on. But on many systems the appliucation
startup will take much longer than that.

- And on each step of you path benchmark again, and focus on the hot
spots. It makes little sense to spend a week of effort on reducing
the execustion time of function foo() to 5% of it's original value,
when this cunction contributes only 1% to the total boot time.

Hope this helps.

Best regards,

Wolfgang Denk

--
DENX Software Engineering GmbH, MD: Wolfgang Denk & Detlev Zundel
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd at denx.de
But the only way of discovering the limits of the possible is to
venture a little way past them into the impossible.
- _Profiles of the Future_ (1962; rev. 1973)
``Hazards of Prophecy: The Failure of Imagination''