Generating slightly different images for different IMAGE_FSTYPES

Generating slightly different images for different IMAGE_FSTYPES

[Martin Jansa]

Hi,

on om-gta01 we had problem that even our minimal image
shr-lite-image is too big for its 64MB NAND chip

We're slowly stripping packages not really needed, but it would be 
nice to remove whole /boot with kernel-image.

Problem is that when we're booting on gta01, than /boot/uImage is used so 
it should be installed in .tar.gz image. But when booting from NAND, then
kernel is usually in different NAND partition and /boot on rootfs partition
is useless.

In patch is what I used now to generate images like this, but it's not optimal at all
1) It copies whole rootfs 3 times if you have all jffs2/ubi/ubifs in FSTYPES, 
   instead just once for -no-boot version (it's possible ie if we expect certain order 
   of making FSTYPES or prepare -no-boot always with normal rootfs and don't remove it
2) task-base still depends on kernel-image and I don't see how to easily remove that 
   dependency, without defining different image recipe and building it separately. 
   So with next kernel upgrade user will get new kernel-image which will probably fail
   to install because of lack of space.

Is there better way to solve it or should we try harder to save space somewhere else?

[PATCH] om-gta01: remove /boot directory from jffs2/ubi/ubifs images

[Martin Jansa]

Signed-off-by: Martin Jansa <Martin.Jansa@gmail.com>
---
 conf/machine/om-gta01.conf |    5 +++++
 1 files changed, 5 insertions(+), 0 deletions(-)

diff --git a/conf/machine/om-gta01.conf b/conf/machine/om-gta01.conf
index ccd2b7a..578e90c 100644
--- a/conf/machine/om-gta01.conf
+++ b/conf/machine/om-gta01.conf
@@ -57,6 +57,11 @@ ROOT_FLASH_SIZE = "60"
 
 IMAGE_FSTYPES += "tar.gz jffs2 ubi"
 
+# We don't want /boot directory in jffs2/ubi/ubifs images
+IMAGE_CMD_jffs2 = "cp -ra ${IMAGE_ROOTFS} ${IMAGE_ROOTFS}-no-boot && rm -rf ${IMAGE_ROOTFS}-no-boot/boot && mkfs.jffs2 -x lzo --root=${IMAGE_ROOTFS}-no-boot --faketime --output=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.jffs2 ${EXTRA_IMAGECMD} && rm -rf ${IMAGE_ROOTFS}-no-boot"
+IMAGE_CMD_ubi = "cp -ra ${IMAGE_ROOTFS} ${IMAGE_ROOTFS}-no-boot && rm -rf ${IMAGE_ROOTFS}-no-boot/boot && echo \[ubifs\] > ubinize.cfg ; echo mode=ubi >> ubinize.cfg ; echo image=${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ubifs >> ubinize.cfg ; echo vol_id=0 >> ubinize.cfg ; echo vol_type=dynamic >> ubinize.cfg ; echo vol_name=${UBI_VOLNAME} >> ubinize.cfg ; echo vol_flags=autoresize >> ubinize.cfg;mkfs.ubifs -r ${IMAGE_ROOTFS} -o ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ubifs ${MKUBIFS_ARGS} && ubinize -o ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ubi ${UBINIZE_ARGS} ubinize.cfg && rm -rf ${IMAGE_ROOTFS}-no-boot"
+IMAGE_CMD_ubifs = "cp -ra ${IMAGE_ROOTFS} ${IMAGE_ROOTFS}-no-boot && rm -rf ${IMAGE_ROOTFS}-no-boot/boot && mkfs.ubifs -r ${IMAGE_ROOTFS} -o ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.ubifs.img ${MKUBIFS_ARGS} && rm -rf ${IMAGE_ROOTFS}-no-boot"
+
 # extra jffs2 tweaks
 EXTRA_IMAGECMD_jffs2 = "--little-endian --eraseblock=0x20000 --pagesize=0x800 \
   --no-cleanmarkers --pad -n ; sumtool --eraseblock=0x20000 --no-cleanmarkers \
-- 
1.7.0

Re: Generating slightly different images for different IMAGE_FSTYPES

[Marcin Juszkiewicz]

Dnia piątek, 26 lutego 2010 o 17:39:16 Martin Jansa napisał(a):

> We're slowly stripping packages not really needed, but it would be
> nice to remove whole /boot with kernel-image.

> Is there better way to solve it or should we try harder to save space
>  somewhere else?

Learn about ROOTFS_POSTPROCESS_COMMAND variable present in 
classes/rootfs_*.bbclass files. You can use it to launch your function after 
rootfs is populated.

So for example your shr-lite-image has drop_kernel_from_boot function in which 
you check "MACHINE==om-gta01" and if it is then "rm -rf /boot/uImage". After 
your function all normal ext2/ubifs/jffs2/tarball images will be created 
without kernel in them.

Regards, 
-- 
JID:      hrw@jabber.org
Website:  http://marcin.juszkiewicz.com.pl/
LinkedIn: http://www.linkedin.com/in/marcinjuszkiewicz

Re: Generating slightly different images for different IMAGE_FSTYPES

[Tom Rini]

On Fri, 2010-02-26 at 17:54 +0100, Marcin Juszkiewicz wrote:
> Dnia piątek, 26 lutego 2010 o 17:39:16 Martin Jansa napisał(a):
> 
> > We're slowly stripping packages not really needed, but it would be
> > nice to remove whole /boot with kernel-image.
> 
> > Is there better way to solve it or should we try harder to save space
> >  somewhere else?
> 
> Learn about ROOTFS_POSTPROCESS_COMMAND variable present in 
> classes/rootfs_*.bbclass files. You can use it to launch your function after 
> rootfs is populated.
> 
> So for example your shr-lite-image has drop_kernel_from_boot function in which 
> you check "MACHINE==om-gta01" and if it is then "rm -rf /boot/uImage". After 
> your function all normal ext2/ubifs/jffs2/tarball images will be created 
> without kernel in them.

Or (as there's a number of ways to do this):
PACKAGE_REMOVE = "kernel-image-*"
ROOTFS_POSTPROCESS_COMMAND_om-gta01 += "opkg-cl ${IPKG_ARGS}
-force-depends \
                                remove ${PACKAGE_REMOVE};"

-- 
Tom Rini <tom_rini@mentor.com>
Mentor Graphics Corporation

Re: Generating slightly different images for different IMAGE_FSTYPES

[Martin Jansa]

On Fri, Feb 26, 2010 at 10:07:41AM -0700, Tom Rini wrote:
> On Fri, 2010-02-26 at 17:54 +0100, Marcin Juszkiewicz wrote:
> > Dnia piątek, 26 lutego 2010 o 17:39:16 Martin Jansa napisał(a):
> > 
> > > We're slowly stripping packages not really needed, but it would be
> > > nice to remove whole /boot with kernel-image.
> > 
> > > Is there better way to solve it or should we try harder to save space
> > >  somewhere else?
> > 
> > Learn about ROOTFS_POSTPROCESS_COMMAND variable present in 
> > classes/rootfs_*.bbclass files. You can use it to launch your function after 
> > rootfs is populated.
> > 
> > So for example your shr-lite-image has drop_kernel_from_boot function in which 
> > you check "MACHINE==om-gta01" and if it is then "rm -rf /boot/uImage". After 
> > your function all normal ext2/ubifs/jffs2/tarball images will be created 
> > without kernel in them.
> 
> Or (as there's a number of ways to do this):
> PACKAGE_REMOVE = "kernel-image-*"
> ROOTFS_POSTPROCESS_COMMAND_om-gta01 += "opkg-cl ${IPKG_ARGS}
> -force-depends \
>                                 remove ${PACKAGE_REMOVE};"

This removes it from ALL FSTYPEs... not just jffs2/ubi/ubifs

Regards,

-- 
uin:136542059                jid:Martin.Jansa@gmail.com
Jansa Martin                 sip:jamasip@voip.wengo.fr 
JaMa                         

Re: Generating slightly different images for different IMAGE_FSTYPES

[Tom Rini]

On Fri, 2010-02-26 at 18:14 +0100, Martin Jansa wrote:
> On Fri, Feb 26, 2010 at 10:07:41AM -0700, Tom Rini wrote:
> > On Fri, 2010-02-26 at 17:54 +0100, Marcin Juszkiewicz wrote:
> > > Dnia piątek, 26 lutego 2010 o 17:39:16 Martin Jansa napisał(a):
> > > 
> > > > We're slowly stripping packages not really needed, but it would be
> > > > nice to remove whole /boot with kernel-image.
> > > 
> > > > Is there better way to solve it or should we try harder to save space
> > > >  somewhere else?
> > > 
> > > Learn about ROOTFS_POSTPROCESS_COMMAND variable present in 
> > > classes/rootfs_*.bbclass files. You can use it to launch your function after 
> > > rootfs is populated.
> > > 
> > > So for example your shr-lite-image has drop_kernel_from_boot function in which 
> > > you check "MACHINE==om-gta01" and if it is then "rm -rf /boot/uImage". After 
> > > your function all normal ext2/ubifs/jffs2/tarball images will be created 
> > > without kernel in them.
> > 
> > Or (as there's a number of ways to do this):
> > PACKAGE_REMOVE = "kernel-image-*"
> > ROOTFS_POSTPROCESS_COMMAND_om-gta01 += "opkg-cl ${IPKG_ARGS}
> > -force-depends \
> >                                 remove ${PACKAGE_REMOVE};"
> 
> This removes it from ALL FSTYPEs... not just jffs2/ubi/ubifs

I would strongly argue (especially after my last email) that you either
always want a kernel in your rootfs, because it's actually booted from,
or you never do and it's just wasting space (and just not critically so
on other devices).

-- 
Tom Rini <tom_rini@mentor.com>
Mentor Graphics Corporation

Re: Generating slightly different images for different IMAGE_FSTYPES

[Martin Jansa]

On Fri, Feb 26, 2010 at 10:20:08AM -0700, Tom Rini wrote:
> On Fri, 2010-02-26 at 18:14 +0100, Martin Jansa wrote:
> > On Fri, Feb 26, 2010 at 10:07:41AM -0700, Tom Rini wrote:
> > > On Fri, 2010-02-26 at 17:54 +0100, Marcin Juszkiewicz wrote:
> > > > Dnia piątek, 26 lutego 2010 o 17:39:16 Martin Jansa napisał(a):
> > > > 
> > > > > We're slowly stripping packages not really needed, but it would be
> > > > > nice to remove whole /boot with kernel-image.
> > > > 
> > > > > Is there better way to solve it or should we try harder to save space
> > > > >  somewhere else?
> > > > 
> > > > Learn about ROOTFS_POSTPROCESS_COMMAND variable present in 
> > > > classes/rootfs_*.bbclass files. You can use it to launch your function after 
> > > > rootfs is populated.
> > > > 
> > > > So for example your shr-lite-image has drop_kernel_from_boot function in which 
> > > > you check "MACHINE==om-gta01" and if it is then "rm -rf /boot/uImage". After 
> > > > your function all normal ext2/ubifs/jffs2/tarball images will be created 
> > > > without kernel in them.
> > > 
> > > Or (as there's a number of ways to do this):
> > > PACKAGE_REMOVE = "kernel-image-*"
> > > ROOTFS_POSTPROCESS_COMMAND_om-gta01 += "opkg-cl ${IPKG_ARGS}
> > > -force-depends \
> > >                                 remove ${PACKAGE_REMOVE};"
> > 
> > This removes it from ALL FSTYPEs... not just jffs2/ubi/ubifs
> 
> I would strongly argue (especially after my last email) that you either
> always want a kernel in your rootfs, because it's actually booted from,
> or you never do and it's just wasting space (and just not critically so
> on other devices).

OK so more info:

Qi (alternative bootloader) usually used on om-gta02 (because it safer
to reflash it there with u-boot still as fail-safe in NOR)
looks for kernel on first partitions on uSD in /boot, then it loads
kernel from specified (different than rootfs) partition in nand and pass
right rootfs params. Qi doesn't read jffs2 (because nobody really needed
it so much to implement it).

u-boot env shipped with device by default looks for kernel in nand and
in menu is option to load kernel from first small fat32 partition on uSD
and rootfs in second partition.

I know that I can change flash layout/u-boot env (and I would do it on
my om-gta01 if I had one :)), but I'm trying to make smaller images
without forcing gta users to mess with their bootloader/flash layout.

As I don't even have gta01 (gta02 has bigger flash), than I can say that 
I'll wait for someone else to do it, or someone can sort out which icons 
are not realy needed in image and remove them instead of unused kernel
(which is as useless as not used icons but harder to remove).

Kind regards,

I really appreciate your comments/advise (if it looks different).

-- 
uin:136542059                jid:Martin.Jansa@gmail.com
Jansa Martin                 sip:jamasip@voip.wengo.fr 
JaMa                         

Re: Generating slightly different images for different IMAGE_FSTYPES

[Denys Dmytriyenko]

On Fri, Feb 26, 2010 at 10:07:41AM -0700, Tom Rini wrote:
> On Fri, 2010-02-26 at 17:54 +0100, Marcin Juszkiewicz wrote:
> > Dnia pi??tek, 26 lutego 2010 o 17:39:16 Martin Jansa napisa??(a):
> > 
> > > We're slowly stripping packages not really needed, but it would be
> > > nice to remove whole /boot with kernel-image.
> > 
> > > Is there better way to solve it or should we try harder to save space
> > >  somewhere else?
> > 
> > Learn about ROOTFS_POSTPROCESS_COMMAND variable present in 
> > classes/rootfs_*.bbclass files. You can use it to launch your function after 
> > rootfs is populated.
> > 
> > So for example your shr-lite-image has drop_kernel_from_boot function in which 
> > you check "MACHINE==om-gta01" and if it is then "rm -rf /boot/uImage". After 
> > your function all normal ext2/ubifs/jffs2/tarball images will be created 
> > without kernel in them.
> 
> Or (as there's a number of ways to do this):
> PACKAGE_REMOVE = "kernel-image-*"
> ROOTFS_POSTPROCESS_COMMAND_om-gta01 += "opkg-cl ${IPKG_ARGS}
> -force-depends \
>                                 remove ${PACKAGE_REMOVE};"

This worked for me:

RDEPENDS_kernel-base = ""


As kernel.bbclass suggests the following:

# Allow machines to override this dependency if kernel image files are
# not wanted in images as standard
RDEPENDS_kernel-base ?= "kernel-image"

-- 
Denys

Re: Generating slightly different images for different IMAGE_FSTYPES

[Martin Jansa]

On Fri, Feb 26, 2010 at 01:48:54PM -0500, Denys Dmytriyenko wrote:
> On Fri, Feb 26, 2010 at 10:07:41AM -0700, Tom Rini wrote:
> > On Fri, 2010-02-26 at 17:54 +0100, Marcin Juszkiewicz wrote:
> > > Dnia pi??tek, 26 lutego 2010 o 17:39:16 Martin Jansa napisa??(a):
> > > 
> > > > We're slowly stripping packages not really needed, but it would be
> > > > nice to remove whole /boot with kernel-image.
> > > 
> > > > Is there better way to solve it or should we try harder to save space
> > > >  somewhere else?
> > > 
> > > Learn about ROOTFS_POSTPROCESS_COMMAND variable present in 
> > > classes/rootfs_*.bbclass files. You can use it to launch your function after 
> > > rootfs is populated.
> > > 
> > > So for example your shr-lite-image has drop_kernel_from_boot function in which 
> > > you check "MACHINE==om-gta01" and if it is then "rm -rf /boot/uImage". After 
> > > your function all normal ext2/ubifs/jffs2/tarball images will be created 
> > > without kernel in them.
> > 
> > Or (as there's a number of ways to do this):
> > PACKAGE_REMOVE = "kernel-image-*"
> > ROOTFS_POSTPROCESS_COMMAND_om-gta01 += "opkg-cl ${IPKG_ARGS}
> > -force-depends \
> >                                 remove ${PACKAGE_REMOVE};"
> 
> This worked for me:
> 
> RDEPENDS_kernel-base = ""
> 
> 
> As kernel.bbclass suggests the following:
> 
> # Allow machines to override this dependency if kernel image files are
> # not wanted in images as standard
> RDEPENDS_kernel-base ?= "kernel-image"

That's also for ALL fstypes of that image.

> > > > Problem is that when we're booting on gta01, than /boot/uImage is used
> > > > so it should be installed in .tar.gz image.

ahh I'm missing "booting from uSD on gta01".

So once more and hopefully better:

so with rootfs in NAND, kernel from different NAND partition is used
(and is ignored in /boot on its jffs2/ubi/ubifs rootfs).

with rootfs on some uSD partition, then kernel is used from /boot/uImage
on the same partition

User can choose which rootfs is booting with every reboot and I don't
want to force every user to modify/reflash bootloader.

Regards,

-- 
uin:136542059                jid:Martin.Jansa@gmail.com
Jansa Martin                 sip:jamasip@voip.wengo.fr 
JaMa                         

Re: Generating slightly different images for different IMAGE_FSTYPES

[Martin Jansa]

On Fri, Feb 26, 2010 at 05:54:25PM +0100, Marcin Juszkiewicz wrote:
> Dnia piątek, 26 lutego 2010 o 17:39:16 Martin Jansa napisał(a):
> 
> > We're slowly stripping packages not really needed, but it would be
> > nice to remove whole /boot with kernel-image.
> 
> > Is there better way to solve it or should we try harder to save space
> >  somewhere else?
> 
> Learn about ROOTFS_POSTPROCESS_COMMAND variable present in 
> classes/rootfs_*.bbclass files. You can use it to launch your function after 
> rootfs is populated.
> 
> So for example your shr-lite-image has drop_kernel_from_boot function in which 
> you check "MACHINE==om-gta01" and if it is then "rm -rf /boot/uImage". After 
> your function all normal ext2/ubifs/jffs2/tarball images will be created 
> without kernel in them.
> 
> Regards, 

with additional IRC chat about hrw's umbaumba-wonders :P moved to
recipes/images/shr-image.inc in faster version (which moves only /boot)

# We don't want /boot directory in jffs2/ubi/ubifs images
IMAGE_CMD_jffs2 = "
mv ${IMAGE_ROOTFS}/boot ${IMAGE_ROOTFS}-boot 
&& mkfs.jffs2 ....
&& mv ${IMAGE_ROOTFS}-boot ${IMAGE_ROOTFS}/boot"

Any better idea?

-- 
uin:136542059                jid:Martin.Jansa@gmail.com
Jansa Martin                 sip:jamasip@voip.wengo.fr 
JaMa                         

Re: Generating slightly different images for different IMAGE_FSTYPES

[Tom Rini]

On Fri, 2010-02-26 at 17:39 +0100, Martin Jansa wrote:
> Hi,
> 
> on om-gta01 we had problem that even our minimal image
> shr-lite-image is too big for its 64MB NAND chip
> 
> We're slowly stripping packages not really needed, but it would be 
> nice to remove whole /boot with kernel-image.
> 
> Problem is that when we're booting on gta01, than /boot/uImage is used so 
> it should be installed in .tar.gz image. But when booting from NAND, then
> kernel is usually in different NAND partition and /boot on rootfs partition
> is useless.
> 
> In patch is what I used now to generate images like this, but it's not optimal at all
> 1) It copies whole rootfs 3 times if you have all jffs2/ubi/ubifs in FSTYPES, 
>    instead just once for -no-boot version (it's possible ie if we expect certain order 
>    of making FSTYPES or prepare -no-boot always with normal rootfs and don't remove it
> 2) task-base still depends on kernel-image and I don't see how to easily remove that 
>    dependency, without defining different image recipe and building it separately. 
>    So with next kernel upgrade user will get new kernel-image which will probably fail
>    to install because of lack of space.

Here's one more thing you could try.  Assuming U-Boot, apparently you
can have it load a kernel from jffs2 or ubifs, so rework flash layout a
little bit and load /boot/uImage.

-- 
Tom Rini <tom_rini@mentor.com>
Mentor Graphics Corporation