Re: [yocto-patches] [meta-rockchip][PATCH v7] enable stored U-Boot environment

[Trevor Woerner <twoerner@gmail.com>]

On Thu 2024-05-23 @ 03:03:53 PM, Quentin Schulz via lists.yoctoproject.org wrote:
> Hi Trevor,
> 
> On 5/23/24 12:53 AM, Trevor Woerner via lists.yoctoproject.org wrote:
> > U-Boot has the ability to store its environment variables to a permanent
> > storage device. Whether or not it does so for any one specific device
> > depends on whatever settings are enabled in that specific device's
> > defconfig. In order to definitively configure U-Boot to be able to store
> > its environment into the device from which it boots, for any device
> > supported in this BSP, simply add the following to MACHINE_FEATURES:
> > 
> > 	rk-u-boot-env
> > 
> > If enabled, there is now a second choice to make: should the build also
> > include the U-Boot environment in the image or not? The default environment,
> > as generated by U-Boot, can be included in the generated wic image. If it
> > is included, then flashing the image will also flash the default U-Boot
> > environment variables and settings, wiping out anything that might have been
> > there already. If it is not included then your device will either continue
> > using whatever environment happens to be there (if valid), or will not use any
> > stored environment if the stored environment has not been set or is invalid.
> > The variable which governs this behaviour is:
> > 
> > 	RK_IMAGE_INCLUDES_UBOOT_ENV
> > 
> > By default this is set to "0", meaning that by default the image does not
> > contain the U-Boot environment. To enable this behaviour, enable this
> > variable. This variable only takes effect if rk-u-boot-env is listed in
> > MACHINE_FEATURES, and has no effect otherwise.
> > 
> > The script:
> > 
> > 	scripts/dump-uboot-env-from-yocto-image.sh
> > 
> > can be used on a rockchip wic image to see the contents of the U-Boot
> > environment partition at build time.
> > 
> > Tested by booting the same image on both eMMC and SDcard with the following
> > devices, verifying the ability to read and write the U-Boot environment in
> > both U-Boot and Linux user-space, and that changes made in one are seen in the
> > other:
> > 	rock-3a
> > 	rock-5a
> > 	rock-5b
> > 	rock-pi-4b
> > 	rock-pi-e
> > 	rock64
> > 
> > Signed-off-by: Trevor Woerner <twoerner@gmail.com>
> > ---
> > v7 changes:
> > - move u-boot's extra image install to machine/include (it wasn't having
> >    any effect)
> 
> Couldn't find this in the diff between v6 and v7, are you sure you sent the
> proper version or is this outdated? You did add the inc file to the
> rock2-square though, is that what you meant?

Oops, sorry. I have about 4 different versions of v7 and I probably got
confused between some of them. In any case the extra packages are listed in
the feature's inc file and it works there.

> > - only call u-boot's rk_generate_env() postfunc if the feature is in
> >    MACHINEOVERRIDES (in which it is only *conditionally* found, unlike its
> >    presence in MACHINE_FEATURES)
> > - add a warning if the user is trying to build an image whereby they've
> >    enabled the rk-u-boot-env mechanism but it isn't supported in a given
> >    specific build
> > 
> > v6 changes:
> > - separate out the MACHINEOVERRIDES handling into its own include so
> >    that it can be included more flexibly into different scenarios; i.e.
> >    rock2-square build was failing because it is not wic-based
> > 
> > v5 changes:
> > - fix a bug in v4 to handle RK_IMAGE_INCLUDES_UBOOT_ENV correctly in the
> >    bash test in postfuncs of u-boot bbappend
> > - add extra check in do_deploy() of u-boot bbappend to only copy the
> >    environment if RK_IMAGE_INCLUDES_UBOOT_ENV is true
> > 
> > v4 changes:
> > - sort WICVARS alphabetically
> > - tweak dump-uboot-env-from-yocto-image.sh script to not just check for
> >    the existence of the user-supplied parameter, but that it is a file
> >    specifically
> > - u-boot bbappend:
> >    - use a do_compile[postfuncs] for generating the fw_env.config instead of
> >      a do_compile:append
> >    - move the conversion of the U-Boot text-based environment to the
> >      required binary representation from a do_deploy:append to the newly-created
> >      do_compile[postfuncs] created above, this will allow the user to inject
> >      any extra U-Boot environment variables they might wish as part of a
> >      do_compile:append and have that occur before the binary is generated
> >      so that those additional user-supplied values are included
> >    - add a lot of extra checking (instead of assuming various resources exist)
> >      before making use of said resources
> > 
> > v3 changes:
> > - switch from using a bbclass so the `rk-u-boot-env` override can be used
> >    directly
> > - add SPDX header to script
> > - drop `inherit deploy` in U-Boot bbappend
> > 
> > v2 changes:
> > - re-word the commit message and README for clarity
> > - use bash's built-in math handling instead of depending on `bc`
> > - in anticipation of the upcoming feature in U-Boot whereby rockchip
> >    devices can automatically select the environment storage device to be
> >    the same as the boot device, use a more recent SRCREV for builds that do
> >    environment handling, instead of hardcoding the environment storage device
> >    by SoC family
> > - handle RK_IMAGE_INCLUDES_UBOOT_ENV as a boolean
> > ---
> >   README                                        | 20 +++++++++
> >   .../include/rockchip-rk-u-boot-env.inc        |  4 ++
> >   conf/machine/include/rockchip-wic.inc         |  7 +++
> >   conf/machine/rock2-square.conf                |  1 -
> >   .../rockchip-enable-environment-mmc.cfg       |  6 +++
> >   recipes-bsp/u-boot/u-boot_%.bbappend          | 45 +++++++++++++++++++
> >   scripts/dump-uboot-env-from-yocto-image.sh    | 29 ++++++++++++
> >   wic/rockchip.wks                              |  2 +-
> >   8 files changed, 112 insertions(+), 2 deletions(-)
> >   create mode 100644 conf/machine/include/rockchip-rk-u-boot-env.inc
> >   create mode 100644 recipes-bsp/u-boot/files/rk-u-boot-env/rockchip-enable-environment-mmc.cfg
> >   create mode 100755 scripts/dump-uboot-env-from-yocto-image.sh
> > 
> > diff --git a/README b/README
> > index c76fc9131276..605773d4ecd3 100644
> > --- a/README
> > +++ b/README
> > @@ -67,6 +67,26 @@ Notes:
> >   	
> >   	in the configuration (e.g. conf/local.conf).
> > +U-Boot Environment:
> > +------------------
> > +	In order to configure U-Boot to be able to store its environment into the
> > +	device from which it was booted, for any device supported in this BSP,
> > +	simply add the following to MACHINE_FEATURES:
> > +
> > +		rk-u-boot-env
> > +
> > +	If enabled, to additionally have the U-Boot environment generated and
> > +	stored in the image, also enable the following variable (default: off):
> > +
> > +		RK_IMAGE_INCLUDES_UBOOT_ENV
> > +
> > +	The script:
> > +
> > +		scripts/dump-uboot-env-from-yocto-image.sh
> > +
> > +	can be used on a rockchip wic image to see the contents of the U-Boot
> > +	environment partition at build time.
> > +
> >   Maintenance:
> >   -----------
> >   	Please send pull requests, patches, comments, or questions to the
> > diff --git a/conf/machine/include/rockchip-rk-u-boot-env.inc b/conf/machine/include/rockchip-rk-u-boot-env.inc
> > new file mode 100644
> > index 000000000000..4eb877f77c48
> > --- /dev/null
> > +++ b/conf/machine/include/rockchip-rk-u-boot-env.inc
> > @@ -0,0 +1,4 @@
> > +# 'rk-u-boot-env' indicates the user wants to be able to save their U-Boot
> > +# environment back to the drive from which the device was booted
> > +MACHINEOVERRIDES .= "${@bb.utils.contains('MACHINE_FEATURES', 'rk-u-boot-env', ':rk-u-boot-env', '', d)}"
> > +IMAGE_INSTALL:append:rk-u-boot-env = " u-boot-fw-utils u-boot-env"
> > diff --git a/conf/machine/include/rockchip-wic.inc b/conf/machine/include/rockchip-wic.inc
> > index 147a36685d7d..b5ee6e0c2724 100644
> > --- a/conf/machine/include/rockchip-wic.inc
> > +++ b/conf/machine/include/rockchip-wic.inc
> > @@ -1,6 +1,7 @@
> >   # common meta-rockchip wic/wks items
> >   require conf/machine/include/rockchip-extlinux.inc
> > +require conf/machine/include/rockchip-rk-u-boot-env.inc
> >   SPL_BINARY ?= "idbloader.img"
> > @@ -11,7 +12,13 @@ WKS_FILE_DEPENDS ?= " \
> >   	virtual/bootloader \
> >   	"
> > +RK_IMAGE_INCLUDES_UBOOT_ENV ?= "no"
> > +RK_UBOOT_ENV = " "
> > +RK_UBOOT_ENV:rk-u-boot-env = "${@ '--source rawcopy --sourceparams=file=u-boot.env' \
> > +	if bb.utils.to_boolean(d.getVar('RK_IMAGE_INCLUDES_UBOOT_ENV'), False) else ' '}"
> > +
> >   WICVARS:append = " \
> > +	RK_UBOOT_ENV \
> >   	SPL_BINARY \
> >   	UBOOT_SUFFIX \
> >   	"
> > diff --git a/conf/machine/rock2-square.conf b/conf/machine/rock2-square.conf
> > index 9468b9a6b559..b31eb574015f 100644
> > --- a/conf/machine/rock2-square.conf
> > +++ b/conf/machine/rock2-square.conf
> > @@ -16,4 +16,3 @@ UBOOT_MACHINE = "rock2_defconfig"
> >   # image class
> >   IMAGE_FSTYPES += "rockchip-gpt-img"
> >   IMAGE_CLASSES += "rockchip-gpt-img"
> > -
> > diff --git a/recipes-bsp/u-boot/files/rk-u-boot-env/rockchip-enable-environment-mmc.cfg b/recipes-bsp/u-boot/files/rk-u-boot-env/rockchip-enable-environment-mmc.cfg
> > new file mode 100644
> > index 000000000000..778772d27767
> > --- /dev/null
> > +++ b/recipes-bsp/u-boot/files/rk-u-boot-env/rockchip-enable-environment-mmc.cfg
> > @@ -0,0 +1,6 @@
> > +CONFIG_ENV_SIZE=0x8000
> > +CONFIG_ENV_OFFSET=0x3f8000
> > +# CONFIG_ENV_IS_NOWHERE is not set
> > +CONFIG_ENV_IS_IN_MMC=y
> > +CONFIG_DM_SEQ_ALIAS=y
> > +CONFIG_SPL_DM_SEQ_ALIAS=y
> > diff --git a/recipes-bsp/u-boot/u-boot_%.bbappend b/recipes-bsp/u-boot/u-boot_%.bbappend
> > index a83179a9f007..33b521021a1c 100644
> > --- a/recipes-bsp/u-boot/u-boot_%.bbappend
> > +++ b/recipes-bsp/u-boot/u-boot_%.bbappend
> > @@ -1,7 +1,13 @@
> > +FILESEXTRAPATHS:prepend := "${THISDIR}/files:"
> > +
> > +SRC_URI:append:rk-u-boot-env = " file://rockchip-enable-environment-mmc.cfg"
> > +SRCREV:rk-u-boot-env = "cdfcc37428e06f4730ab9a17cc084eeb7676ea1a"
> > +
> >   # various machines require the pyelftools library for parsing dtb files
> >   DEPENDS:append = " python3-pyelftools-native"
> >   DEPENDS:append:rk3308 = " u-boot-tools-native"
> >   DEPENDS:append:rock-pi-4 = " gnutls-native"
> > +DEPENDS:append:rk-u-boot-env = " u-boot-mkenvimage-native"
> >   EXTRA_OEMAKE:append:px30 = " BL31=${DEPLOY_DIR_IMAGE}/bl31-px30.elf"
> >   EXTRA_OEMAKE:append:rk3308 = " \
> > @@ -34,3 +40,42 @@ do_compile:append:rock2-square () {
> >   		cp ${B}/spl/${SPL_BINARY} ${B}
> >   	fi
> >   }
> > +
> > +python rk_no_env() {
> > +    if bb.utils.contains('MACHINE_FEATURES', 'rk-u-boot-env', True, False, d):
> > +        bb.warn("the rk-u-boot-env MACHINE_FEATURE is not supported for this build")
> > +}
> > +
> > +rk_generate_env() {
> > +	if [ ! -f "${B}/.config" ]; then
> > +		echo "U-Boot .config not found, can't determine environment size"
> > +		return 1
> > +	fi
> > +	cat ${B}/.config | grep "^CONFIG_ENV_SIZE=" > /dev/null
> > +	if [ $? -ne 0 ]; then
> > +		echo "can not find CONFIG_ENV_SIZE value in U-Boot .config"
> > +		return 1
> > +	fi
> > +
> > +	UBOOT_ENV_SIZE="$(cat ${B}/.config | grep "^CONFIG_ENV_SIZE=" | cut -d'=' -f2)"
> > +
> > +	# linux user-space U-Boot env config file
> > +	echo "/dev/disk/by-partlabel/uboot_env 0x0000 ${UBOOT_ENV_SIZE}" > ${WORKDIR}/fw_env.config
> > +
> > +	# convert text-based environment to binary suitable for image
> > +	if [ "${@bb.utils.to_boolean(d.getVar('RK_IMAGE_INCLUDES_UBOOT_ENV'), False)}" = "True" ]; then
> > +		if [ ! -f ${B}/u-boot-initial-env ]; then
> > +			echo "initial, text-formatted U-Boot environment file \"${B}/u-boot-initial-env\" not found"
> > +			return 1
> > +		fi
> > +		mkenvimage -s ${UBOOT_ENV_SIZE} ${B}/u-boot-initial-env -o ${WORKDIR}/u-boot.env
> > +	fi
> > +}
> > +do_compile[postfuncs] += "${@'rk_generate_env' if 'rk-u-boot-env' in d.getVar('MACHINEOVERRIDES').split(':') else 'rk_no_env'}"
> > +
> 
> Could be simplified with:
> 
> """
> rk_generate_env() {
>     if bb.utils.contains('MACHINE_FEATURES', 'rk-u-boot-env', True, False,
> d):
>         bb.warn("the rk-u-boot-env MACHINE_FEATURE is not supported for this
> build")
> }
> 
> rk_generate_env:rk-u-boot-env() {
> [content of rk_generate_env in v7]
> }
> 
> do_compile[postfuncs] += "rk_generate_env;"
> """

"...eye of the beholder..." I guess. You've suggested this before as a
simplification, but I guess I've never quite gotten accustomed to bitbake's
override mechanism. To this day I still find it odd... and unique.

What I've proposed is a simple: "IF <string> IN <object> THEN <generate_env>
ELSE <no_env>", which is mostly plain English.

With overrides you're saying "always call this function... oh
wait! unless overrides are in place, then call this one or this other one...
wait! are overrides in place? what are the overrides?..."

In any case what you've proposed doesn't work. The non-override
rk_generate_env() function would need a "python" prefix and try as I might, it
seems you can't have a set of override functions with one being python and the
other one not. Without the "python" bitbake chokes if the non-override one is
called. With the "python" bitbake will still choke if the overridden version
is called:

Try making the change you suggest, both with and without the "python"
qualifier for the non-overridden version, then try building with both
MACHINE=rock2-square and MACHINE=rock-5a; one or the other won't work with or
without the "python".

In any case I like my version :-)
I'm going to apply it. If you can change my mind we can always take a later
patch.

Thanks for the review and the discussion!!