looking for a model for building CRAMFS(?)-based system

looking for a model for building CRAMFS(?)-based system

[Robert P. J. Day]

  i'm hoping someone has an example of the following that they're
willing to share.

  currently, the system i've built for our 850 board incorporates

	- boot loader (sadly, not u-boot, but i'm working on it)
	- standard zImage.initrd.bin kernel+initrd image
	... mountable JFFS2 filesystem with persistent stuff ...

  obviously, with this layout, it's kind of a nuisance to update
anything individually in the initrd portion of the system, so i'd like
to at least experiment with a layout that has separate

	- boot loader (ideally, u-boot)
	- kernel image
	- updateable (normally mounted read-only?) root filesystem
	... rest of stuff the same ...

  i'm going to start over at DENX with their docs since that seems to
be the canonical place to get the scoop on this but, in the meantime,
if anyone has built something like this and is willing to share, say,
their makefile so i can see how it goes together, i'd be thrilled.

  if your example happens to *require* u-boot, well, perhaps so much
the better since that will give me the incentive to switch. :-)

  thanks.

rday

Re: looking for a model for building CRAMFS(?)-based system

[Pantelis Antoniou]

Robert P. J. Day wrote:
>   i'm hoping someone has an example of the following that they're
> willing to share.
> 
>   currently, the system i've built for our 850 board incorporates
> 
> 	- boot loader (sadly, not u-boot, but i'm working on it)
> 	- standard zImage.initrd.bin kernel+initrd image
> 	... mountable JFFS2 filesystem with persistent stuff ...
> 
>   obviously, with this layout, it's kind of a nuisance to update
> anything individually in the initrd portion of the system, so i'd like
> to at least experiment with a layout that has separate
> 
> 	- boot loader (ideally, u-boot)
> 	- kernel image
> 	- updateable (normally mounted read-only?) root filesystem
> 	... rest of stuff the same ...
> 
>   i'm going to start over at DENX with their docs since that seems to
> be the canonical place to get the scoop on this but, in the meantime,
> if anyone has built something like this and is willing to share, say,
> their makefile so i can see how it goes together, i'd be thrilled.
> 
>   if your example happens to *require* u-boot, well, perhaps so much
> the better since that will give me the incentive to switch. :-)

We use a scheme on NAND with redundant cramfs rootfs partitions
that also contain the proper kernel image(s).

It is u-boot specific since it depends on u-boot loading the
cramfs kernel image.

For persistent storage we use a small JFFS2 partition.

> 
>   thanks.
> 
> rday

Regards

Pantelis

Re: looking for a model for building CRAMFS(?)-based system

[Patrick Huesmann]

Hi,

On Wed, 16 Mar 2005 11:33:39 -0500 (EST), Robert P. J. Day
<rpjday@mindspring.com> wrote:
>   currently, the system i've built for our 850 board incorporates
> 
>         - boot loader (sadly, not u-boot, but i'm working on it)
>         - standard zImage.initrd.bin kernel+initrd image
>         ... mountable JFFS2 filesystem with persistent stuff ...
> 
>   obviously, with this layout, it's kind of a nuisance to update
> anything individually in the initrd portion of the system, so i'd like
> to at least experiment with a layout that has separate
> 
>         - boot loader (ideally, u-boot)
>         - kernel image
>         - updateable (normally mounted read-only?) root filesystem
>         ... rest of stuff the same ...

I'm using separate MTD partitions for bootloader, device parameters,
zImage, initrd and persistent storage.

When updating the initrd, the only thing necessary is a
eraseall /dev/mtdX
cat initrd.gz > /dev/mtdX

When booting the kernel, you need something like this in the commandline:
root=/dev/mtdblockX prompt_ramdisk=0 load_ramdisk=1 ramdisk_size=8192

This is bootloader independent, of course.

HTH,
Patrick

Re: looking for a model for building CRAMFS(?)-based system

[Robert P. J. Day]

On Thu, 17 Mar 2005, Patrick Huesmann wrote:

> I'm using separate MTD partitions for bootloader, device parameters,
> zImage, initrd and persistent storage.
>
> When updating the initrd, the only thing necessary is a
> eraseall /dev/mtdX
> cat initrd.gz > /dev/mtdX

now this is the sticky part.  imagine this system out in the field,
where you need to make an update to something in the initrd in the
root filesystem.

one technique would be to, of couse, download an entirely new
initrd.gz and reflash (hoping no one pulls the plug as you're doing
it), as you describe above.

what about sacrificing space and having an uncompressed root
filesystem in flash?  and, if that's possible, would one mount it
read-only from flash (speed?), or just copy it into RAM and mount that
every time?

if all this is covered somewhere, a URL would be fine, thanks.

rday

Re: looking for a model for building CRAMFS(?)-based system

[Patrick Huesmann]

Hi,

On Thu, 17 Mar 2005 10:14:41 -0500 (EST), Robert P. J. Day
<rpjday@mindspring.com> wrote:
> On Thu, 17 Mar 2005, Patrick Huesmann wrote:
> > When updating the initrd, the only thing necessary is a
> > eraseall /dev/mtdX
> > cat initrd.gz > /dev/mtdX
> 
> now this is the sticky part.  imagine this system out in the field,
> where you need to make an update to something in the initrd in the
> root filesystem.

If you know in advance what small parts you will want to change
(perhaps your main application, /var/www or whatever), you can use an
additional small partition, that holds only these files, mount it
somewhere in your rootfs, and you won't have to upload the whole
initrd when updating.

> one technique would be to, of couse, download an entirely new
> initrd.gz and reflash (hoping no one pulls the plug as you're doing
> it), as you describe above.

Power failure is not the only scenario when field updates go wrong.
We once had a stressed-out customer who uploaded the "readme" file to
the zImage partition. (Of course, you can prevent that with sanity
checks),
When something like this happens, it's always good to have a fallback
option, like a small bootloader that is NEVER updated, and can still
be used to flash the rest of the system over slow RS232 or whatever.

Regards,
Patrick

RE: looking for a model for building CRAMFS(?)-based system

[Fillod Stephane]

Robert P. J. Day wrote:
> now this is the sticky part.  imagine this system out in the field,
> where you need to make an update to something in the initrd in the
> root filesystem.

You could get away with mirrored partitions, having version N and N-1
in flash, so a rollback is possible. But static partitions suck,
they are either too big (space wasted), either too small (!!).

Here's an idea I'd enjoy having feedback from: why not dedicating
the whole flash to a jffs2 space, allowing a kind of "dynamic"=20
partitioning with the composing filesystems in image files accessed
trough
loop devices? For example, the base system (libc, busybox, startup=20
scripts, ..) would be packed as a cramfs/squashfs file, put in the jffs2
storage. This image file would be mounted as root fs in 2 steps through=20
a loop device, thanks to a small patch against linux/init/do_mounts.c.=20
I've named this idea "Initial loopback (initlo) support", patch's
available.
Choosing the rootfs image is done through kernel command line.=20
For example, let's say /dev/mtdblock0 is the jffs2 storage, and the real
rootfs is a cramfs image file "rev1/rootfs.img" (from base of
mtdblock0).
The command line would be:
		root=3D/dev/mtdblock0 lofile=3Drev1/rootfs.img
After bootup, the rootfs.img is the rootfs /, and the mtdblock0 space
is pivoted to /initlo, hence still accessible.
It's easy to have whatever number of rootfs versions, switching from=20
one another, and updating the kernel command line only when install
succeeded (=3Dno power fail,..).
During normal operation, the jffs2 storage is kept read-only for safety.

Only during install it is remounted read-write.
Applications can be packaged the same way (image file). jffs2 will be
happy
to store also configuration files/data and at least 2 kernel versions
(for=20
safe update). Great monitors like U-Boot are able to load the kernel
image
(and other firmwares) from jffs2. The jffs2 loader is portable.

Note: these filesystems in a filesystem look inefficient (read slow
startup),
but who cares, you're already relying on compression. Please note that
once
loaded in pagecache, the system runs at full speed. cramfs/squashfs are
at
advantage here over initrd, because only needed files make it into the
pagecache.

Comments welcome

--=20
Stephane

Re: looking for a model for building CRAMFS(?)-based system

[Wolfgang Denk]

Dear Robert,

in message <Pine.LNX.4.61.0503171011220.20335@localhost.localdomain> you wrote:
> 
> now this is the sticky part.  imagine this system out in the field,
> where you need to make an update to something in the initrd in the
> root filesystem.

This is the szenario where an overly filesystem enters the stage.

See http://www.denx.de/e/news.php#MINI_FO

Note that the overlay filesystem was developed specifically for  this
purpose;  it  has been in production use in some telecom products for
over a year now.

Best regards,

Wolfgang Denk

-- 
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de
Only in our dreams we are free.  The rest of the time we need  wages.
                                    - Terry Pratchett, _Wyrd Sisters_

Re: looking for a model for building CRAMFS(?)-based system

[Robert P. J. Day]

On Thu, 17 Mar 2005, Wolfgang Denk wrote:

> Dear Robert,
>
> in message <Pine.LNX.4.61.0503171011220.20335@localhost.localdomain> you wrote:
> >
> > now this is the sticky part.  imagine this system out in the field,
> > where you need to make an update to something in the initrd in the
> > root filesystem.
>
> This is the szenario where an overly filesystem enters the stage.
>
> See http://www.denx.de/e/news.php#MINI_FO

i'd already read that paper a few months back, but wasn't sure if this
was actually being used, or was just a concept idea.  if it's actually
in practise, i'll take another look at it.

rday

Re: looking for a model for building CRAMFS(?)-based system

[Wolfgang Denk]

In message <Pine.LNX.4.61.0503171134230.22767@localhost.localdomain> you wrote:
> 
> > See http://www.denx.de/e/news.php#MINI_FO
> 
> i'd already read that paper a few months back, but wasn't sure if this
> was actually being used, or was just a concept idea.  if it's actually
> in practise, i'll take another look at it.

As mentioned before, it's in use in commercial products. See also
http://www.denx.de/twiki/bin/view/Know/MiniFOHome

Best regards,

Wolfgang Denk

-- 
Software Engineering:  Embedded and Realtime Systems,  Embedded Linux
Phone: (+49)-8142-66989-10 Fax: (+49)-8142-66989-80 Email: wd@denx.de
"Wagner's music is better than it sounds."               - Mark Twain