Re: [PATCH] kernel.bbclass, image.bbclass: Implement kernel INITRAMFS dependency and bundling

[Bruce Ashfield <bruce.ashfield@windriver.com>]

On 13-08-20 11:03 AM, Andrea Adami wrote:
> On Tue, Aug 20, 2013 at 3:51 PM, Bruce Ashfield
> <bruce.ashfield@windriver.com> wrote:
>> On 13-08-19 03:11 PM, Jason Wessel wrote:
>>>
>>> This patch aims to fix the following two cases for the INITRAMFS
>>> generation.
>>>     1) Allow an image recipe to specify a paired INITRAMFS recipe such
>>>        as core-image-minimal-initramfs.  This allows building a base
>>>        image which always generates the needed initramfs image in one step
>>>     2) Allow building a single binary which contains a kernel and
>>>        the initramfs.
>>>
>>> A key requirement of the initramfs is to be able to add kernel
>>> modules.  The current implementation of the INITRAMFS_IMAGE variable
>>> has a circular dependency when using kernel modules in the initramfs
>>> image.bb file that is caused by kernel.bbclass trying to build the
>>> initramfs before the kernel's do_install rule.
>>>
>>> The solution for this problem is to have the kernel's
>>> do_bundle_initramfs_image task depend on the do_rootfs from the
>>> INITRAMFS_IMAGE and not some intermediate point.  The image.bbclass
>>> will also sets up dependencies to make the initramfs creation task run
>>> last.
>>>
>>> The code to bundle the kernel and initramfs together has been added.
>>> At a high level, all it is doing is invoking a second compilation of
>>> the kernel but changing the value of CONFIG_INITRAMFS_SOURCE to point
>>> to the generated initramfs from the image recipe.
>>
>>
>> In case it wasn't obvious with Jason cc'ing me, I've seen the patch
>> before, and I'm good with the change.
>>
>> So definitely: Acked-by: Bruce Ashfield <bruce.ashfield@windriver.com>
>>
>> I'm adding Andrea to the cc list as well, since we've been talking
>> about initramfs creation for years now, and I'd want his opinion on
>> whether this looks good, and would work for meta-handheld.
>>
>> Cheers,
>>
>> Bruce
>>
>>
>>>
>>> Signed-off-by: Jason Wessel <jason.wessel@windriver.com>
>>> ---
>>>    meta/classes/image.bbclass  |   12 ++++++
>>>    meta/classes/kernel.bbclass |   96
>>> +++++++++++++++++++++++++++++++++++++------
>>>    meta/conf/local.conf.sample |   20 +++++++++
>>>    3 files changed, 116 insertions(+), 12 deletions(-)
>>>
>>> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
>>> index 116bd22..78c25db 100644
>>> --- a/meta/classes/image.bbclass
>>> +++ b/meta/classes/image.bbclass
>>> @@ -140,6 +140,10 @@ python () {
>>>        d.setVar('MULTILIB_VENDORS', ml_vendor_list)
>>>
>>>        check_image_features(d)
>>> +    initramfs_image = d.getVar('INITRAMFS_IMAGE', True) or ""
>>> +    if initramfs_image != "":
>>> +        d.appendVarFlag('do_build', 'depends', " %s:do_bundle_initramfs"
>>> %  d.getVar('PN', True))
>>> +        d.appendVarFlag('do_bundle_initramfs', 'depends', " %s:do_rootfs"
>>> % initramfs_image)
>>>    }
>>>
>>>    #
>>> @@ -614,3 +618,11 @@ do_package_write_deb[noexec] = "1"
>>>    do_package_write_rpm[noexec] = "1"
>>>
>>>    addtask rootfs before do_build
>>> +# Allow the kernel to be repacked with the initramfs and boot image file
>>> as a single file
>>> +do_bundle_initramfs[depends] += "virtual/kernel:do_bundle_initramfs"
>>> +do_bundle_initramfs[nostamp] = "1"
>>> +do_bundle_initramfs[noexec] = "1"
>>> +do_bundle_initramfs () {
>>> +       :
>>> +}
>>> +addtask bundle_initramfs after do_rootfs
>>> diff --git a/meta/classes/kernel.bbclass b/meta/classes/kernel.bbclass
>>> index e039dfc..8cf66ce 100644
>>> --- a/meta/classes/kernel.bbclass
>>> +++ b/meta/classes/kernel.bbclass
>>> @@ -9,6 +9,7 @@ INHIBIT_DEFAULT_DEPS = "1"
>>>    KERNEL_IMAGETYPE ?= "zImage"
>>>    INITRAMFS_IMAGE ?= ""
>>>    INITRAMFS_TASK ?= ""
>>> +INITRAMFS_IMAGE_BUNDLE ?= ""
>>>
>>>    python __anonymous () {
>>>        kerneltype = d.getVar('KERNEL_IMAGETYPE', True) or ''
>>> @@ -19,7 +20,15 @@ python __anonymous () {
>>>
>>>        image = d.getVar('INITRAMFS_IMAGE', True)
>>>        if image:
>>> -        d.setVar('INITRAMFS_TASK', '${INITRAMFS_IMAGE}:do_rootfs')
>>> +        d.appendVarFlag('do_bundle_initramfs', 'depends', '
>>> ${INITRAMFS_IMAGE}:do_rootfs')
>>> +
>>> +    # NOTE: setting INITRAMFS_TASK is for backward compatibility
>>> +    #       The preferred method is to set INITRAMFS_IMAGE, because
>>> +    #       this INITRAMFS_TASK has circular dependency problems
>>> +    #       if the initramfs requires kernel modules
>>> +    image_task = d.getVar('INITRAMFS_TASK', True)
>>> +    if image_task:
>>> +        d.appendVarFlag('do_configure', 'depends', ' ${INITRAMFS_TASK}')
>>>    }
>>>
>>>    inherit kernel-arch deploy
>>> @@ -72,9 +81,82 @@ KERNEL_SRC_PATH = "/usr/src/kernel"
>>>
>>>    KERNEL_IMAGETYPE_FOR_MAKE = "${@(lambda s: s[:-3] if s[-3:] == ".gz"
>>> else s)(d.getVar('KERNEL_IMAGETYPE', True))}"
>>>
>>> +copy_initramfs() {
>>> +       echo "Copying initramfs into ./usr ..."
>>> +       # Find and use the first initramfs image archive type we find
>>> +       rm -f ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.cpio
>>> +       for img in cpio.gz cpio.lzo cpio.lzma cpio.xz; do
>>> +               if [ -e
>>> "${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img" ]; then
>>> +                       cp
>>> ${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img ${B}/usr/.
>>> +                       case $img in
>>> +                       *gz)
>>> +                               echo "gzip decompressing image"
>>> +                               gunzip -f
>>> ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
>>> +                               break
>>> +                               ;;
>>> +                       *lzo)
>>> +                               echo "lzo decompressing image"
>>> +                               lzop -df
>>> ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
>>> +                               break
>>> +                               ;;
>>> +                       *lzma)
>>> +                               echo "lzma decompressing image"
>>> +                               lzmash -df
>>> ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
>>> +                               break
>>> +                               ;;
>>> +                       *xz)
>>> +                               echo "xz decompressing image"
>>> +                               xz -df
>>> ${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.$img
>>> +                               break
>>> +                               ;;
>>> +                       esac
>>> +               fi
>>> +       done
>>> +       echo "Finished copy of initramfs into ./usr"
>>> +}
>>> +
>>> +INITRAMFS_BASE_NAME =
>>> "${KERNEL_IMAGETYPE}-initramfs-${PV}-${PR}-${MACHINE}-${DATETIME}"
>>> +INITRAMFS_BASE_NAME[vardepsexclude] = "DATETIME"
>>> +do_bundle_initramfs () {
>>> +       if [ ! -z "${INITRAMFS_IMAGE}" -a x"${INITRAMFS_IMAGE_BUNDLE}" =
>>> x1 ]; then
>>> +               echo "Creating a kernel image with a bundled initramfs..."
>>> +               copy_initramfs
>>> +               if [ -e ${KERNEL_OUTPUT} ] ; then
>>> +                       mv -f ${KERNEL_OUTPUT} ${KERNEL_OUTPUT}.bak
>>> +               fi
>>> +
>>> use_alternate_initrd=CONFIG_INITRAMFS_SOURCE=${B}/usr/${INITRAMFS_IMAGE}-${MACHINE}.cpio
>>> +               kernel_do_compile
>>> +               mv -f ${KERNEL_OUTPUT} ${KERNEL_OUTPUT}.initramfs
>>> +               mv -f ${KERNEL_OUTPUT}.bak ${KERNEL_OUTPUT}
>>> +               # Update install area
>>> +               echo "There is kernel image bundled with initramfs:
>>> ${B}/${KERNEL_OUTPUT}.initramfs"
>>> +               install -m 0644 ${B}/${KERNEL_OUTPUT}.initramfs
>>> ${D}/boot/${KERNEL_IMAGETYPE}-initramfs-${MACHINE}.bin
>>> +               echo "${B}/${KERNEL_OUTPUT}.initramfs"
>>> +               cd ${B}
>>> +               # Update deploy directory
>>> +               if [ -e "${KERNEL_OUTPUT}.initramfs" ]; then
>>> +                       echo "Copying deploy kernel-initramfs image and
>>> setting up links..."
>>> +                       initramfs_base_name=${INITRAMFS_BASE_NAME}
>>> +
>>> initramfs_symlink_name=${KERNEL_IMAGETYPE}-initramfs-${MACHINE}
>>> +                       install -m 0644 ${KERNEL_OUTPUT}.initramfs
>>> ${DEPLOY_DIR_IMAGE}/${initramfs_base_name}.bin
>>> +                       cd ${DEPLOY_DIR_IMAGE}
>>> +                       ln -sf ${initramfs_base_name}.bin
>>> ${initramfs_symlink_name}.bin
>>> +               fi
>>> +       fi
>>> +}
>>> +do_bundle_initramfs[nostamp] = "1"
>>> +addtask bundle_initramfs after do_compile
>>> +
>>>    kernel_do_compile() {
>>>          unset CFLAGS CPPFLAGS CXXFLAGS LDFLAGS MACHINE
>>> -       oe_runmake ${KERNEL_IMAGETYPE_FOR_MAKE} ${KERNEL_ALT_IMAGETYPE}
>>> CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
>>> +       # The $use_alternate_initrd is only set from
>>> +       # do_bundle_initramfs() This variable is specifically for the
>>> +       # case where we are making a second pass at the kernel
>>> +       # compilation and we want to force the kernel build to use a
>>> +       # different initramfs image.  The way to do that in the kernel
>>> +       # is to specify:
>>> +       # make ...args...
>>> CONFIG_INITRAMFS_SOURCE=some_other_initramfs.cpio
>>> +       oe_runmake ${KERNEL_IMAGETYPE_FOR_MAKE} ${KERNEL_ALT_IMAGETYPE}
>>> CC="${KERNEL_CC}" LD="${KERNEL_LD}" ${KERNEL_EXTRA_ARGS}
>>> $use_alternate_initrd
>>>          if test "${KERNEL_IMAGETYPE_FOR_MAKE}.gz" = "${KERNEL_IMAGETYPE}";
>>> then
>>>                  gzip -9c < "${KERNEL_IMAGETYPE_FOR_MAKE}" >
>>> "${KERNEL_OUTPUT}"
>>>          fi
>>> @@ -219,18 +301,8 @@ kernel_do_configure() {
>>>                  cp "${WORKDIR}/defconfig" "${B}/.config"
>>>          fi
>>>          yes '' | oe_runmake oldconfig
>>> -
>>> -       if [ ! -z "${INITRAMFS_IMAGE}" ]; then
>>> -               for img in cpio.gz cpio.lzo cpio.lzma cpio.xz; do
>>> -               if [ -e
>>> "${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img" ]; then
>>> -                       cp
>>> "${DEPLOY_DIR_IMAGE}/${INITRAMFS_IMAGE}-${MACHINE}.$img" initramfs.$img
>>> -               fi
>>> -               done
>>> -       fi
>>>    }
>>>
>>> -do_configure[depends] += "${INITRAMFS_TASK}"
>>> -
>>>    do_savedefconfig() {
>>>          oe_runmake savedefconfig
>>>    }
>>> diff --git a/meta/conf/local.conf.sample b/meta/conf/local.conf.sample
>>> index 2b078d0..17733ab 100644
>>> --- a/meta/conf/local.conf.sample
>>> +++ b/meta/conf/local.conf.sample
>>> @@ -146,6 +146,26 @@ EXTRA_IMAGE_FEATURES = "debug-tweaks"
>>>    USER_CLASSES ?= "buildstats image-mklibs image-prelink"
>>>
>>>    #
>>> +# Kernel image features
>>> +#
>>> +# The INITRAMFS_IMAGE image variable will cause an additional recipe to
>>> +# be built as a dependency to the what ever rootfs recipe you might be
>>> +# using such as core-image-sato.  The initramfs might be needed for
>>> +# the initial boot of of the target system such as to load kernel
>>> +# modules prior to mounting the root file system.
>>> +#
>>> +# INITRAMFS_IMAGE_BUNDLE variable controls if the image recipe
>>> +# specified by the INITRAMFS_IMAGE will be run through an extra pass
>>> +# through the kernel compilation in order to build a single binary
>>> +# which contains both the kernel image and the initramfs.  The
>>> +# combined binary will be deposited into the tmp/deploy directory.
>>> +# NOTE: You can set INITRAMFS_IMAGE in an image recipe, but
>>> +#       INITRAMFS_IMAGE_BUNDLE can only be set in a conf file.
>>> +#
>>> +#INITRAMFS_IMAGE = "core-image-minimal-initramfs"
>>> +#INITRAMFS_IMAGE_BUNDLE = "1"
>>> +
>>> +#
>>>    # Runtime testing of images
>>>    #
>>>    # The build system can test booting virtual machine images under qemu
>>> (an emulator)
>>>
>>
>
> I plan to test this as soon as possible.
>
> Though, it seems to me a bit too special-cased: you only change
> CONFIG_INITRAMFS_SOURCE=some_other_initramfs.cpio.
> I mean, our necessity is to have a *different* second kernel embedding
> the initramfs and in fact we have linux-yocto as standard kernel and a
> linux-yocto-tiny derived special-purpose kernel+cpio.

This is exactly what I was hoping to hear as feedback, and yes,
this solution does stop short of the kexec solution or a two
kernel variant. It does allow you to have a kernel module for a
driver in your initramfs, and a unique filesystem for that
initramfs.

Your document/steps are one of the cases that I had in mind, it
would be nice to make it simpler as well .. if possible.

>
> (in case you missed it:
> http://kexecboot.org/documentation/crosscompiling/oe-yocto )
>
> About building all in one-pass, well, we were there and we did step back.
> It was preferred to build only the 'production' kernel when building
> images and to build separately the special kernel+cpio.
> The logic was to create a dummy recipe and let the option to build it
> automatically i.e. adding it as # EXTRA_IMAGEDEPENDS += "dummy" in
> your machine.conf or another conf file

Remind me on this part. When you assemble and include that second
kernel into the production kernel's initramfs, how are the build
artifacts shared ? i.e. it's just a second build with a shared
deploy directory ? (your my-installer.bb recipe) ?

Cheers,

Bruce

>
> I'll test soon anyway, thx.
>
> Cheers
>
> Andrea
>