initramfs generation with OE

initramfs generation with OE

[Richard Purdie]

Hi,

So if recursive bitbake isn't an option (its not), how do we generate an
initramfs with OE? I've had a think about this and I have an idea. Its
totally untested but its also neat and simple so here goes.

We don't actually care if we generate a normal kernel as a side effect
and Pauls code already does this. Its even really close to whats needed,
it just misses some task dependency magic. My untested solution would
be:

"""

INITRAMFS_SYMLINK_NAME ?= "initramfs-${MACHINE}"
INITRAMFS_IMAGE_TARGET ?= "initramfs-image"

do_builtin_initramfs() {
       unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
       cp "${DEPLOY_DIR_IMAME}/${INITRAMFS_SYMLINK_NAME}" usr/initramfs_data.cpio.gz
       oe_runmake ${KERNEL_IMAGETYPE} CC="${KERNEL_CC}" LD="${KERNEL_LD}"
       install -d ${DEPLOY_DIR_IMAGE}
       install -m 0644 ${KERNEL_OUTPUT} ${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGE_BASE_NAME}-initramfs.bin
       # Make sure to kill injected initramfs, in case someone will do "-c compile -f"
       rm usr/initramfs_data.cpio.gz
}
addtask builtin_initramfs after do_deploy before do_build
do_builtin_initramfs[nostamp] = "1"
do_builtin_initramfs[depends] = "${INITRAMFS_IMAGE_TARGET}:do_rootfs"

"""

of which the last line is the magic key. From that, bitbake should be
able to handle the rest. In addition to this you need to add something
to initramfs-image that:

* symlinks from ${DEPLOY_DIR_IMAME}/${INITRAMFS_SYMLINK_NAME} to the initramfs image
* makes sure IMAGE_FSTYPES has cpio.gz in it

Also note that task should move to a separate .bbclass since it is now
added before do_build and not everything will want to build this.
Alternatively, we can drop the "before do_build" bit and mean users
manually run:

bitbake kernel -c builtin_initramfs

Anyhow, this is something we can discuss and find a way forward with if
the basic idea works.

So to summarise what should happen, the kernel will compile and build as
normal. It should then trigger the do_rootfs task in
INITRAMFS_IMAGE_TARGET to run. Once that has run, the builtin_initramfs
task should run.

Bitbake should be clever enough to work out that the package tasks need
to run before the image is generated and all the dependencies should
fall into place. One potential problem is if anything in the image
generation has a recursive dependency on do_build in which case it will
break. I don't see any reason an image should have that dependency (meta
tasks and toolchains do) but I haven't tested anything.

If this doesn't work for any reason I will do what I can to fix it. Note
that I'm not around tomorrow and whilst I may find time on Sunday, I
can't guarantee it so it may be Monday before I can look at any
problems. Please bear with me on that.

Cheers,

Richard

Re: initramfs generation with OE

[Rolf Leggewie]

Richard Purdie wrote:
> how do we generate an initramfs with OE?

Sorry to jump in with another question that does not relate to solving
the problem but me understanding the scope and importance of it.

Would the initramfs thing allow for example NFS boot over wifi?

Re: initramfs generation with OE

[Paul Sokolovsky]

Hello,

On Sat, 16 Feb 2008 08:09:41 +0100
Rolf Leggewie <no2spam@nospam.arcornews.de> wrote:

> Richard Purdie wrote:
> > how do we generate an initramfs with OE?
> 
> Sorry to jump in with another question that does not relate to solving
> the problem but me understanding the scope and importance of it.
> 
> Would the initramfs thing allow for example NFS boot over wifi?

Initramfs as technology allows that and anything else, and
initramfs-uniboot (packages/initrdscripts/) is intended to make that all
practically and easily achievable for realistic usecases. It is built
in modular fashion, allowing developer to select needed functionality
from available choices, and add needed functionality easily (again, at
least for realistic usecases).

For example, there's already module for NFS boot (it has implicit
support for USB net so far). So, to add NFS-over-WiFi functionality,
just a module which will up (load modules, configure ip) WiFi i/f needs
to be added.

[]

-- 
Best regards,
 Paul                          mailto:pmiscml@gmail.com

Re: initramfs generation with OE

[Paul Sokolovsky]

Hello,

On Fri, 15 Feb 2008 23:40:16 +0000
Richard Purdie <rpurdie@rpsys.net> wrote:

> Hi,
> 
> So if recursive bitbake isn't an option (its not), how do we generate
> an initramfs with OE? I've had a think about this and I have an idea.
> Its totally untested but its also neat and simple so here goes.
> 
> We don't actually care if we generate a normal kernel as a side effect
> and Pauls code already does this. Its even really close to whats
> needed, it just misses some task dependency magic. My untested
> solution would be:
> 
> """
> 
> INITRAMFS_SYMLINK_NAME ?= "initramfs-${MACHINE}"
> INITRAMFS_IMAGE_TARGET ?= "initramfs-image"
> 
> do_builtin_initramfs() {
>        unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
>        cp "${DEPLOY_DIR_IMAME}/${INITRAMFS_SYMLINK_NAME}"
> usr/initramfs_data.cpio.gz oe_runmake ${KERNEL_IMAGETYPE}
> CC="${KERNEL_CC}" LD="${KERNEL_LD}" install -d ${DEPLOY_DIR_IMAGE}
>        install -m 0644 ${KERNEL_OUTPUT}
> ${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGE_BASE_NAME}-initramfs.bin # Make
> sure to kill injected initramfs, in case someone will do "-c compile
> -f" rm usr/initramfs_data.cpio.gz }
> addtask builtin_initramfs after do_deploy before do_build
> do_builtin_initramfs[nostamp] = "1"
> do_builtin_initramfs[depends] = "${INITRAMFS_IMAGE_TARGET}:do_rootfs"
> 
> """
> 
> of which the last line is the magic key. From that, bitbake should be
> able to handle the rest. 

What would be really needed is:

do_builtin_initramfs[depends] = "${INITRAMFS_IMAGE_TARGET}:do_rootfs
(MACHINE=${INITRAMFS_MACHINE} SYSTYPE=${INITRAMFS_SYSTYPE} ...)"

Where stuff in braces after dependent task specification means: run
the task specified and all its subtasks with environment modified with
those settings (environment is restored after the task finishes).
Without that, it doesn't solve directly my usecase (need to build uclibc
initramfs for glibc-built kernel). My recursive bitbake invocation is
exactly attempt to emulate that at the OS level (and thanks to Unix
nature, where process recursion is first-class phenomena, the attempt
is very successful I would add, as it even works, even though bitbake
was not designed for such use ;-) ).

Well, back to pragmatics, there's of course no difference with
glibc-built kernel and uclibc-built kernel (and its kind of
inefficiency the same kernel is built twice). So, supposedly that can
be worked around with building just uclibc-kernel and then shuffling
deploy dirs.

> In addition to this you need to add something
> to initramfs-image that:
> 
> * symlinks from ${DEPLOY_DIR_IMAME}/${INITRAMFS_SYMLINK_NAME} to the
> initramfs image
> * makes sure IMAGE_FSTYPES has cpio.gz in it
> 
> Also note that task should move to a separate .bbclass since it is now
> added before do_build and not everything will want to build this.
> Alternatively, we can drop the "before do_build" bit and mean users
> manually run:
> 
> bitbake kernel -c builtin_initramfs
> 
> Anyhow, this is something we can discuss and find a way forward with
> if the basic idea works.

The most interesting thing with all this is package writing. That's
what took me lot of time to make it work, and still only "somehow".

[]

> 
> Cheers,
> 
> Richard
> 
> 



-- 
Best regards,
 Paul                          mailto:pmiscml@gmail.com