Running from ROM

Running from ROM

[Kent Borg]

I am doing some early investigation of the suitability of embedded
Linux.  I am wondering about program loading.

Traditionally, embedded systems run their programming directly out of
ROM.  Linux doesn't seem to have any concept of such a thing.  But if
I have a large embedded application that needs every bit of its RAM
and ROM, I would like to find a way to get Linux to run directly out
of ROM with no moving parts (no disks).  Anyone doing that?


Thanks,

-kb, the Kent who has done some web searching and is close to
concluding no one is doing this, but he hopes he is wrong.

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Running from ROM

[Mark Hatle]

We (MontaVista Software) are working on that.  I'm not sure schedules or
status of the project.

The idea is to have the ROM at 0x000000000 and run the kernel out of
ROM, and then have the writable structures of the kernel is some RAM
space.

My understanding is that this is for a PPC 8xx application.

--Mark Hatle
MontaVista Software, Inc.

Kent Borg wrote:
>
> I am doing some early investigation of the suitability of embedded
> Linux.  I am wondering about program loading.
>
> Traditionally, embedded systems run their programming directly out of
> ROM.  Linux doesn't seem to have any concept of such a thing.  But if
> I have a large embedded application that needs every bit of its RAM
> and ROM, I would like to find a way to get Linux to run directly out
> of ROM with no moving parts (no disks).  Anyone doing that?
>
> Thanks,
>
> -kb, the Kent who has done some web searching and is close to
> concluding no one is doing this, but he hopes he is wrong.
>

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Running from ROM

[Joe Green]

On Mon, 05 Jun 2000, Mark Hatle wrote:
> We (MontaVista Software) are working on that.  I'm not sure schedules or
> status of the project.
>
> The idea is to have the ROM at 0x000000000 and run the kernel out of
> ROM, and then have the writable structures of the kernel is some RAM
> space.

We already have this working for a customer, though the RAM is actually
still mapped at 0, with the ROM at another address.

> Kent Borg wrote:
> > I am doing some early investigation of the suitability of embedded
> > Linux.  I am wondering about program loading.
> >
> > Traditionally, embedded systems run their programming directly out of
> > ROM.  Linux doesn't seem to have any concept of such a thing.  But if
> > I have a large embedded application that needs every bit of its RAM
> > and ROM, I would like to find a way to get Linux to run directly out
> > of ROM with no moving parts (no disks).  Anyone doing that?

The project Mark mentioned is specifically for running the kernel from
ROM.  Are you also asking about running application code from ROM?  We
have customers now who keep compressed file systems in ROM, where pages
are copied into RAM to be executed.  We're also looking into direct
execution of application code from ROM, but keep in mind that this
would mean the code could not be compressed.  If significant amounts of
your code are executed infrequently, a compressed file system with
pages brought in as needed may produce the best overall footprint.

--
Joe Green <jgreen@mvista.com>
MontaVista Software, Inc.

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Running from ROM

[Kent Borg]

Joe Green <jgreen@mvista.com> wrote:
>The project Mark mentioned is specifically for running the kernel
>from ROM.

That doesn't sound horribly complicated.  Groady details to be sure,
but ~conceptually~ easy.

>Are you also asking about running application code from ROM?

Yes.  I realize that it can frequently make sense to compress and load
into RAM, but there is a significant chance that running directly from
ROM could be sensible for us.  Interestingly, part of the reason might
be a "real space" approach, that is, wanting to be deterministic about
whether a given operation can successfully run.  In a system where
sometimes every byte of RAM is needed, I can imagine circumstances
where, depending on the order of operations, some RAM-hungry
operations could sometimes fail and sometimes succeed.  Getting one
more "it's virtual, it does the right thing"-aspect out of the system
could help.  Or maybe not.

So back to the question, it appears there are not scads of people
already doing this.  Suggestions on approaches?  As I let my
imagination run only ugly solution fragments come out on this.


Thanks,

-kb, the Kent who needs to keep reading Rubini's book to learn more
about how the normal case works.

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Running from ROM

[Tom Montgomery]

Our situation is very similar to Kent's.

We are actively evaluating embedded linux for our PPC-850-based product
which currently runs under a commercial RTOS. What I have gathered from
lurking here is that embedded linux operates like PC linux, and that is
not the model we are looking for. Running from ROM makes sense to us,
too.

I think we are at the same stage of development as Kent, and would be
most interested in learning what work is being done in this area, and,
conversely, what amount of work we will be undertaking ourselves...

Tom Montgomery

-tm, the Tom who always enjoyed kb's postings on OpenDTV until he
unsubscribed from that list to save his sanity and lower his blood
pressure!

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Running from ROM

[Dan Malek]

Tom Montgomery wrote:


> .... What I have gathered from
> lurking here is that embedded linux operates like PC linux, and that is
> not the model we are looking for.

Hmmm.  It is quite different from a workstation version from my
perspective.  On the other hand, the workstation version of Linux
contains many features we can (and want to) utilize.

If you want to make Linux look like a primitive RTOS, you can hard
code a bunch of variables and write some simple kernel threads and
be done.  Most people are tired of primitive RTOS environments and
want something with the features and configurability of a workstation
in a smaller package.  That's probably why embedded Linux appears to
look like a workstation to you.  Why not utilize those features when
it is beneficial?

> .... Running from ROM makes sense to us,
> too.

OK, explain this to me.  It doesn't make any sense to me for the
following reasons:

	Flash ROM is more expensive than equivalent sized DRAM
	Flash ROM is more difficult to acquire
	I can compress the kernel and file system into a single
		8-bit device, saving substantial board space.
	Flash ROM is typically slower access than DRAM
	Small DRAM parts are getting harder to find and more
		expensive, so you may as well use more DRAM
	Running the kernel from flash may save 600K or so of RAM
		and require an additional 500K of Flash ROM (because it
		isn't compressed any more)

I have seen some very versatile and cost effective systems that use
a tiny Flash rom and Compact Flash for storage.  Above some size (like
16 Mbytes or so), Compact Flash is more economical than soldered on
board flash.  It is also much easier to manage and adds product
flexibility.

Execution from Flash ROM is harder to debug and configure, increasing
development time.

I like learning new things.....explain why running from Flash is better
for your system.


	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Running from ROM

[Kent Borg]

Dan Malek <dan@netx4.com> responded to a post by Tom Montgomery
<tmontgom@miranda.com> with various reasons for why one should not try
to make Linux look like a primative RTOS, ending with:
>I like learning new things.....explain why running from Flash is
>better for your system.

Well, I am not Tom and I don't know much about his system, but I did
open this up by asking the question, so let me give this a shot.

As a preface, let me say that I don't know that it *is* better for us
to run from ROM, only that I suspect it *might* be.  See if the
following makes any sense.

I think the reasons start out at about two.

First is the particulars of how big ROM requirements are vs. RAM.
Flash and RAM come in specific sizes, they are not available one byte
at a time.  If a system has, say, 8 MB of each, and if much of the ROM
is not very compressable data, compressing the ROM contents maybe
doesn't save any parts, yet it might cost in marginal RAM usage,
possibly requiring 16 MB of RAM instead of 8 MB.

The particulars here will change as sizes change and where they fall
on available parts' boundaries change, but it can matter.  In our case
we might have single digit megabytes of code, single digit megabytes
of data, and a need for mallocable data space that starts in high
single digit megabytes and stretches to triple digits (depending upon
how much work the user throws at it).  I am not talking about merely
running the kernel out of ROM (though that seems the easy part), I am
talking about accessing megabytes of code and data without wanting to
spend the time or space necessary in decompresion and copying to RAM.
In some configurations our products might be populated with lots of
RAM, allowing nastier data to be thrown at them before they fail, but
the low-end still needs to function with little RAM, so
architecturally the design needs to carefully conserve RAM.

Decompressing and copying into RAM will not eliminate the need for
flash, merely reduce the need.  The cost savings in flash parts only
happens if you can drop at least one whole part or move to
significantly lessor parts, yet the "cost" in RAM kicks in as that RAM
can no longer be used for other things, and in our case where every
marginal byte of RAM is potentially useful, the costs are immediate.
Unless a significant substitution in parts happens, the fractional
cost advantage of RAM is never realized.

One way to sooner realize the price advantage of RAM would be to only
page needed code into RAM.  But our RAM needs get nastier as lots of
data gets thrown at us, plus, exactly how a user exercises our
features can be unpredictable.  This frightens me as I fear the use of
an unfortuitous combination of features unpredictably pushing us into
thrashing bits through the decompression and copying routines.

Which slides me into my second reason, which is less precise, but
possibly more interesting.

In realtime systems it is the programmers who are expected to
carefully manage time, and to do so they need simple deterministic
control over time.  In our case, we are trying to squeeze a lot out of
a little RAM, you might say we are doing "realspace" programming as
well as "realtime" programming, and to do that we need clear
deterministic control over much of our memory use.  Not relying on a
paging system to make room in which to run our executable code seems
to me part of carefully managing space.  In fact, we go to significant
effort to predict whether, for a given input, we will meet time
deadlines.  Part of that calculation is assuming we can trade time and
space.  The idea of having an unpredictable amount of space available
makes that calculation really scary if not simply impossible.

A couple other notes.

>Flash ROM is typically slower access than DRAM

Not always, a recent project on which I worked had faster access to
flash than it did to RAM.

>Execution from Flash ROM is harder to debug and configure, increasing
>development time.

Who says that development has to happen in flash ROM?  Certainly the
mechanism for executing directly out of ROM needs to be debugged, and
certainly final code needs to be extensively tested in ROM, but why
not do developmentment executing directly from RAM?  In our current
non-Linux environment we we do both RAM and ROM builds, depending upon
what we are doing.  And, yes, I have spent a lot of time learning
exactly what our gdb set up will and will not let me do with a flash
build.

>Why not utilize those [workstation] features when it is beneficial?

Indeed, why not?  I look forward to dropping in extra RAM, mounting a
network disk, possibly dropping in a local disk, and having rather
nice facilities available when developing--but not in the shipping
product.  That means the extra RAM is gone (or available for the user
to use), the disk is gone (or available for the user to use), and my
other Linux box isn't being mounted.

Some products ship in volumes where a few dollars difference in parts
makes a big difference, and even a few cents is noticed.  We are in
that world.  I would very much like to have workstation while
developing, but we can't afford to ship lots of parts dedicated to
being a workstation.

>I can compress the kernel and file system into a single 8-bit device,
>saving substantial board space.

But how much application code can you fit in that 8-bit device?  I am
looking at running megabytes of applications and megabytes data
directly from ROM.

Did I miss anything?


Finally, I repeat that I am not sure it makes sense try to run code
and data directly from ROM, but neither is it obviously a mistake to
want to.  I am trying to understand all of this.

So I am still wondering how we might do this as I worry about possibly
having to muck in the deep innards of MMU manipulating code.


-kb, the Kent who reminds those who think direct code and data access
in Linux is an obvious oxymoron that the term "embedded Unix" would
rather recently have been considered completely laughable.

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/

Re: Running from ROM

[Dan Malek]

Kent Borg wrote:

> As a preface, let me say that I don't know that it *is* better for us
> to run from ROM, only that I suspect it *might* be.  See if the
> following makes any sense.

Yes, it does.  Count bytes and make trade offs.  What "fudge factor"
do you use :-).

> .....  I am not talking about merely
> running the kernel out of ROM (though that seems the easy part),

I know a few people that would disagree with this statement, having
actually done the work.  There is this minor problem of branch targets
not able to cover 32 bits of address, and it is all uphill from there
when you are trying to span a 4 GByte space in code/data (plus fit
some user applications in the middle).


> Who says that development has to happen in flash ROM?

I didn't mean to imply all development happens there.  Execute from
ROM (in my experience) was an additional step for testing that
didn't always work the first time, and debugging in this environment
is more challenging.  For 8xx development, I typically use NFS root
disk filesystems, then just make these into a compressed ram disk.
In this case, you are done because the final product is still running
from RAM just like the original development.  Further, if you design
the product to use some (carefully) removeable media like compact flash,
your initial guess at the amount of flash rom doesn't have to be
accurate.  You may wish it was less, but it won't stop the shipment of
products and you can squeeze it or update the features over time.
It really sucks when you have soldered down the maximum amount and you
can't make the software fit....


> But how much application code can you fit in that 8-bit device?

Lots, until you fill it up :-).  Seriously, in products I have seen,
you can achieve at least 50% reduction in Flash requirement by
compressing the kernel and file system.  As you said, some things
compress better than others.

> So I am still wondering how we might do this as I worry about possibly
> having to muck in the deep innards of MMU manipulating code.

It's not only the MMU.  When you use the standard Linux file system
features like RAM disks, ROM file systems, and program loaders, your
software just works and your product is out the door.  People also forget
that the kernel is a small part in the product development.  Many of
these "features" affect the libraries, which are substantially more
code and more complex (to someone like me that doesn't really understand
dynamic loading).  It also affects the tools used to develop the
software. You have to also consider if saving that dollar or two of
flash or RAM is worth additional time of not shipping products.

OK, back to real work now......I must write some code.


	-- Dan

** Sent via the linuxppc-embedded mail list. See http://lists.linuxppc.org/