Re: [PATCH 1/2] V2 Incremental image generation(rpm based rootfs)

[Richard Purdie <richard.purdie@linuxfoundation.org>]

On Sat, 2011-12-31 at 16:10 +0800, Robert Yang wrote:
> Incremental image generation, the rootfs would be totally removed and
> re-created in the second generation by default, but with INC_IMAGE_GEN =
> "1", the rootfs would be kept, and will do update(remove/add some pkgs)
> on it.
> NOTE: This is not suggested when you want to create a productive rootfs
> 
> For example:
>   1) Add the follow config option to a conf file:
>      INC_IMAGE_GEN = "1"
> 
>   2) bitbake core-image-sato
>      modify a package
>      bitbake core-image-sato
> 
> The rootfs would not be totally removed and re-created in the second
> generation, it would be simply updated based on the "package".
> 
> Implatation:
> 1) Figure out the pkg which need to be removed or re-installed, then use
> 'rpm -e to remove the old one. Use the rpm's BUILDTIME to determine
> which pkg has been rebuilt.
> 
> 2) Figure out the pkg which is newly added, and use 'rpm -U' to install
> it.
> 
> This only for the rpm based rootfs, I don't know whether we also need
> this for ipk or deb  based rootfs.

As Koen points out, this should be called INC_RPM_IMAGE_GEN since it
only works for rpm at present. It would be good to make this work on the
other package backends. If we keep the generic variable name, the other
package backends could error out saying they don't support it.

> [YOCTO #1651]
> 
> Signed-off-by: Robert Yang <liezhi.yang@windriver.com>
> ---
>  meta/classes/image.bbclass       |   11 +++++-
>  meta/classes/package_rpm.bbclass |   72 ++++++++++++++++++++++++++++++++-----
>  2 files changed, 71 insertions(+), 12 deletions(-)
> 
> diff --git a/meta/classes/image.bbclass b/meta/classes/image.bbclass
> index 865d430..71b60f3 100644
> --- a/meta/classes/image.bbclass
> +++ b/meta/classes/image.bbclass
> @@ -134,15 +134,22 @@ do_rootfs[umask] = 022
>  
>  fakeroot do_rootfs () {
>  	#set -x
> -	rm -rf ${IMAGE_ROOTFS}
> +    # When use the incremental image generation, don't remove the rootfs
> +    if [ "${INC_IMAGE_GEN}" != "1" ]; then
> +        rm -rf ${IMAGE_ROOTFS}
> +    fi
>  	rm -rf ${MULTILIB_TEMP_ROOTFS}
>  	mkdir -p ${IMAGE_ROOTFS}
>  	mkdir -p ${DEPLOY_DIR_IMAGE}
>  
>  	cp ${COREBASE}/meta/files/deploydir_readme.txt ${DEPLOY_DIR_IMAGE}/README_-_DO_NOT_DELETE_FILES_IN_THIS_DIRECTORY.txt
>  
> -	if [ "${USE_DEVFS}" != "1" ]; then
> +    # If "${IMAGE_ROOTFS}/dev" exists, then the device had been made by
> +    # the previous build
> +	if [ "${USE_DEVFS}" != "1" -a ! -r "${IMAGE_ROOTFS}/dev" ]; then
>  		for devtable in ${@get_devtable_list(d)}; do
> +            # Always return ture since there maybe already one when use the
> +            # incremental image generation
>  			makedevs -r ${IMAGE_ROOTFS} -D $devtable
>  		done
>  	fi
> diff --git a/meta/classes/package_rpm.bbclass b/meta/classes/package_rpm.bbclass
> index d03dc3f..541d351 100644
> --- a/meta/classes/package_rpm.bbclass
> +++ b/meta/classes/package_rpm.bbclass
> @@ -147,6 +147,66 @@ resolve_package_rpm () {
>  	echo $pkg_name
>  }
>  
> +# rpm common command and options
> +rpm_common_comand () {
> +
> +    local target_rootfs="${INSTALL_ROOTFS_RPM}"
> +    local extra_args="$@"
> +
> +    ${RPM} --root ${target_rootfs} \
> +        --predefine "_rpmds_sysinfo_path ${target_rootfs}/etc/rpm/sysinfo" \
> +        --predefine "_rpmrc_platform_path ${target_rootfs}/etc/rpm/platform" \
> +        -D "_var ${localstatedir}" \
> +        -D "_dbpath ${rpmlibdir}" \
> +        --noparentdirs --nolinktos \
> +        -D "__dbi_txn create nofsync private" \
> +        -D "_cross_scriptlet_wrapper ${WORKDIR}/scriptlet_wrapper" $extra_args
> +}
> +
> +# install or remove the pkg
> +rpm_update_pkg () {
> +
> +    local target_rootfs="${INSTALL_ROOTFS_RPM}"
> +
> +    # Save the rpm's build time for incremental image generation, and the file
> +    # would be moved to ${T}
> +    rm -f ${target_rootfs}/install/total_solution_bt.manifest
> +    for i in `cat ${target_rootfs}/install/total_solution.manifest`; do
> +        # Use "rpm" rather than "${RPM}" here, since we don't need the
> +        # '--dbpath' option
> +        echo "$i `rpm -qp --qf '%{BUILDTIME}\n' $i`" >> \
> +            ${target_rootfs}/install/total_solution_bt.manifest
> +    done
> +
> +    # Only install the different pkgs if incremental image generation is set
> +    if [ "${INC_IMAGE_GEN}" = "1" -a -f ${T}/total_solution_bt.manifest ]; then
> +        cur_list="${target_rootfs}/install/total_solution_bt.manifest"
> +        pre_list="${T}/total_solution_bt.manifest"
> +        sort -u $cur_list -o $cur_list
> +        sort -u $pre_list -o $pre_list
> +        comm -1 -3 $cur_list $pre_list | sed 's#.*/\(.*\)\.rpm .*#\1#' > \
> +            ${target_rootfs}/install/remove.manifest
> +        comm -2 -3 $cur_list $pre_list | awk '{print $1}' > \
> +            ${target_rootfs}/install/incremental.manifest
> +
> +        # Attempt to remove unwanted pkgs, the scripts(pre, post, etc.) has not
> +        # been run by now, so don't have to run them(preun, postun, etc.) when
> +        # erase the pkg

Really? I'd have thought it possible the scripts have run during the
previous do_rootfs?

Otherwise the patch looks reasonable to me.

Cheers,

Richard