[PATCH v2] u-boot: Generate default U-Boot environment images

[PATCH v2] u-boot: Generate default U-Boot environment images

[Lukasz Majewski]

This change provides tasks to generate default U-Boot environment images
from built U-Boot (via. get_default_envs.sh script).

Those images then can be used to generate wic images (with e.g. eMMC layout).
With such approach the end user doesn't see the "CRC environment" error
after the first boot.

Moreover, those are built per MACHINE (as u-boot itself is) so then could
be used in SWUpdate scenarios with single tar'ed archive with multiple
MACHINE specific *.swu images.

The UBOOT_ENVS_SIZE must be defined in machine specific conf file as well
as it is also possible to adjust (optionally) the *_ENVS_* variables.

Test:
Newest master-next for poky repo - SHA1: eb5b0a0b5e53a6e55a09e66489d3f24d0c6232ee
MACHINE = "beaglebone-yocto" in local.conf
bitbake virtual/bootloader


As a result following links are available in deploy directory:
u-boot-env.img{_r}.

Signed-off-by: Lukasz Majewski <lukma@denx.de>

---
Changes for v2:

- Move the content of deploy_default_envs task to do_deploy
- Move the content of gen_default_envs task to do_compile
- Create the u-boot-env.img{r} conditionally, only when UBOOT_ENVS_SIZE is
  defined
- Change the commit title to reflect changes done for v2
---
 meta/recipes-bsp/u-boot/u-boot.inc | 34 ++++++++++++++++++++++++++++++++++
 1 file changed, 34 insertions(+)

diff --git a/meta/recipes-bsp/u-boot/u-boot.inc b/meta/recipes-bsp/u-boot/u-boot.inc
index 9a754fd09b..7ede2da0eb 100644
--- a/meta/recipes-bsp/u-boot/u-boot.inc
+++ b/meta/recipes-bsp/u-boot/u-boot.inc
@@ -68,6 +68,13 @@ UBOOT_EXTLINUX_INSTALL_DIR ?= "/boot/extlinux"
 UBOOT_EXTLINUX_CONF_NAME ?= "extlinux.conf"
 UBOOT_EXTLINUX_SYMLINK ?= "${UBOOT_EXTLINUX_CONF_NAME}-${MACHINE}-${PR}"
 
+# U-Boot variables necessary for extracting default envs from build U-Boot
+DEFAULT_UBOOT_ENVS_FILE ?= "u-boot-env"
+DEFAULT_ENVS ?= "${DEFAULT_UBOOT_ENVS_FILE}.txt"
+UBOOT_ENVS_DEFAULT ?= "${DEFAULT_UBOOT_ENVS_FILE}-${MACHINE}-${PV}-${PR}.img"
+# The UBOOT_ENVS_SIZE needs to defined in the <machine>.conf file
+# UBOOT_ENVS_SIZE = "65536"
+
 # returns all the elements from the src uri that are .cfg files
 def find_cfgs(d):
     sources=src_patches(d, True)
@@ -132,6 +139,19 @@ do_compile () {
         oe_runmake -C ${S} O=${B} ${UBOOT_MAKE_TARGET}
     fi
 
+    # Generate default environment
+    if [ -n "${UBOOT_ENVS_SIZE}" ]
+    then
+        ${B}/source/scripts/get_default_envs.sh ${B} > ${B}/${DEFAULT_ENVS}
+
+        # Generate env image
+        ${B}/tools/mkenvimage -s ${UBOOT_ENVS_SIZE} -o ${B}/${UBOOT_ENVS_DEFAULT} ${B}/${DEFAULT_ENVS}
+
+        # Generate redundant env image
+        ${B}/tools/mkenvimage -r -s ${UBOOT_ENVS_SIZE} -o ${B}/${UBOOT_ENVS_DEFAULT}_r ${B}/${DEFAULT_ENVS}
+
+        rm ${B}/${DEFAULT_ENVS}
+    fi
 }
 
 do_install () {
@@ -328,6 +348,20 @@ do_deploy () {
         ln -sf ${UBOOT_EXTLINUX_SYMLINK} ${DEPLOYDIR}/${UBOOT_EXTLINUX_CONF_NAME}-${MACHINE}
         ln -sf ${UBOOT_EXTLINUX_SYMLINK} ${DEPLOYDIR}/${UBOOT_EXTLINUX_CONF_NAME}
     fi
+
+    if [ -n "${UBOOT_ENVS_SIZE}" ]
+    then
+        install -d ${DEPLOYDIR}
+        install ${B}/${UBOOT_ENVS_DEFAULT} ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}
+        install ${B}/${UBOOT_ENVS_DEFAULT}_r ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}_r
+
+        cd ${DEPLOYDIR}
+        ln -sf ${UBOOT_ENVS_DEFAULT} ${DEFAULT_UBOOT_ENVS_FILE}.img
+        ln -sf ${UBOOT_ENVS_DEFAULT}_r ${DEFAULT_UBOOT_ENVS_FILE}.img_r
+
+        rm ${B}/${UBOOT_ENVS_DEFAULT}
+        rm ${B}/${UBOOT_ENVS_DEFAULT}_r
+    fi
 }
 
 addtask deploy before do_build after do_compile
-- 
2.11.0

Re: [PATCH v2] u-boot: Generate default U-Boot environment images

[Stefano Babic]

Hi Lukasz,

On 26/07/19 09:40, Lukasz Majewski wrote:
> This change provides tasks to generate default U-Boot environment images
> from built U-Boot (via. get_default_envs.sh script).
> 
> Those images then can be used to generate wic images (with e.g. eMMC layout).
> With such approach the end user doesn't see the "CRC environment" error
> after the first boot.
> 
> Moreover, those are built per MACHINE (as u-boot itself is) so then could
> be used in SWUpdate scenarios with single tar'ed archive with multiple
> MACHINE specific *.swu images.

By working together with SWUpdate, I am just wondering why this is
necessary. I guess that "default" U-Boot environment images means
"initial" environment. The issue I think you see is if there is no
environment in the storage and boards boot with the linked-in
environment. SWUpdate cannot find such as an environment.

For that, there is a target in u-boot: make u-boot-initial-env. This
generates an ASCII file that can be used by SWUpdate if no environment
is stored and U-Boot starts with the linked-in environemnt. It is enough
to put this file into rootfs and to enable SWUpdate (and libubootenv) to
use it. Create specific image to be stored into a .wic seems
overkilling, and it does not help if for some reason the board comes up
without environment in the flash.

Best regards,
Stefano Babic

> 
> The UBOOT_ENVS_SIZE must be defined in machine specific conf file as well
> as it is also possible to adjust (optionally) the *_ENVS_* variables.
> 
> Test:
> Newest master-next for poky repo - SHA1: eb5b0a0b5e53a6e55a09e66489d3f24d0c6232ee
> MACHINE = "beaglebone-yocto" in local.conf
> bitbake virtual/bootloader
> 
> 
> As a result following links are available in deploy directory:
> u-boot-env.img{_r}.
> 
> Signed-off-by: Lukasz Majewski <lukma@denx.de>
> 
> ---
> Changes for v2:
> 
> - Move the content of deploy_default_envs task to do_deploy
> - Move the content of gen_default_envs task to do_compile
> - Create the u-boot-env.img{r} conditionally, only when UBOOT_ENVS_SIZE is
>   defined
> - Change the commit title to reflect changes done for v2
> ---
>  meta/recipes-bsp/u-boot/u-boot.inc | 34 ++++++++++++++++++++++++++++++++++
>  1 file changed, 34 insertions(+)
> 
> diff --git a/meta/recipes-bsp/u-boot/u-boot.inc b/meta/recipes-bsp/u-boot/u-boot.inc
> index 9a754fd09b..7ede2da0eb 100644
> --- a/meta/recipes-bsp/u-boot/u-boot.inc
> +++ b/meta/recipes-bsp/u-boot/u-boot.inc
> @@ -68,6 +68,13 @@ UBOOT_EXTLINUX_INSTALL_DIR ?= "/boot/extlinux"
>  UBOOT_EXTLINUX_CONF_NAME ?= "extlinux.conf"
>  UBOOT_EXTLINUX_SYMLINK ?= "${UBOOT_EXTLINUX_CONF_NAME}-${MACHINE}-${PR}"
>  
> +# U-Boot variables necessary for extracting default envs from build U-Boot
> +DEFAULT_UBOOT_ENVS_FILE ?= "u-boot-env"
> +DEFAULT_ENVS ?= "${DEFAULT_UBOOT_ENVS_FILE}.txt"
> +UBOOT_ENVS_DEFAULT ?= "${DEFAULT_UBOOT_ENVS_FILE}-${MACHINE}-${PV}-${PR}.img"
> +# The UBOOT_ENVS_SIZE needs to defined in the <machine>.conf file
> +# UBOOT_ENVS_SIZE = "65536"
> +
>  # returns all the elements from the src uri that are .cfg files
>  def find_cfgs(d):
>      sources=src_patches(d, True)
> @@ -132,6 +139,19 @@ do_compile () {
>          oe_runmake -C ${S} O=${B} ${UBOOT_MAKE_TARGET}
>      fi
>  
> +    # Generate default environment
> +    if [ -n "${UBOOT_ENVS_SIZE}" ]
> +    then
> +        ${B}/source/scripts/get_default_envs.sh ${B} > ${B}/${DEFAULT_ENVS}
> +
> +        # Generate env image
> +        ${B}/tools/mkenvimage -s ${UBOOT_ENVS_SIZE} -o ${B}/${UBOOT_ENVS_DEFAULT} ${B}/${DEFAULT_ENVS}
> +
> +        # Generate redundant env image
> +        ${B}/tools/mkenvimage -r -s ${UBOOT_ENVS_SIZE} -o ${B}/${UBOOT_ENVS_DEFAULT}_r ${B}/${DEFAULT_ENVS}
> +
> +        rm ${B}/${DEFAULT_ENVS}
> +    fi
>  }
>  
>  do_install () {
> @@ -328,6 +348,20 @@ do_deploy () {
>          ln -sf ${UBOOT_EXTLINUX_SYMLINK} ${DEPLOYDIR}/${UBOOT_EXTLINUX_CONF_NAME}-${MACHINE}
>          ln -sf ${UBOOT_EXTLINUX_SYMLINK} ${DEPLOYDIR}/${UBOOT_EXTLINUX_CONF_NAME}
>      fi
> +
> +    if [ -n "${UBOOT_ENVS_SIZE}" ]
> +    then
> +        install -d ${DEPLOYDIR}
> +        install ${B}/${UBOOT_ENVS_DEFAULT} ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}
> +        install ${B}/${UBOOT_ENVS_DEFAULT}_r ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}_r
> +
> +        cd ${DEPLOYDIR}
> +        ln -sf ${UBOOT_ENVS_DEFAULT} ${DEFAULT_UBOOT_ENVS_FILE}.img
> +        ln -sf ${UBOOT_ENVS_DEFAULT}_r ${DEFAULT_UBOOT_ENVS_FILE}.img_r
> +
> +        rm ${B}/${UBOOT_ENVS_DEFAULT}
> +        rm ${B}/${UBOOT_ENVS_DEFAULT}_r
> +    fi
>  }
>  
>  addtask deploy before do_build after do_compile
> 

-- 
=====================================================================
DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: +49-8142-66989-53 Fax: +49-8142-66989-80 Email: sbabic@denx.de
=====================================================================

Re: [PATCH v2] u-boot: Generate default U-Boot environment images

[Lukasz Majewski]

[-- Attachment #1: Type: text/plain, Size: 6850 bytes --]

Hi Stefano,

> Hi Lukasz,
> 
> On 26/07/19 09:40, Lukasz Majewski wrote:
> > This change provides tasks to generate default U-Boot environment
> > images from built U-Boot (via. get_default_envs.sh script).
> > 
> > Those images then can be used to generate wic images (with e.g.
> > eMMC layout). With such approach the end user doesn't see the "CRC
> > environment" error after the first boot.
> > 
> > Moreover, those are built per MACHINE (as u-boot itself is) so then
> > could be used in SWUpdate scenarios with single tar'ed archive with
> > multiple MACHINE specific *.swu images.  
> 
> By working together with SWUpdate, I am just wondering why this is
> necessary. I guess that "default" U-Boot environment images means
> "initial" environment. 

The main use case is with eMMC/SPI-NOR/NAND memory images produced with
*.wic files for different machines.

In this case - the whole layout of eMMC or SPI-NOR is flashed. With the
default images generated with this u-boot recipe modification it is
possible to create this layout with properly set environment images.

As a result from the first run of the flashed devices there is no "CRC
error" and default (in U-boot binary) envs are not used.

> The issue I think you see is if there is no
> environment in the storage and boards boot with the linked-in
> environment.

On my use case eMMC layouts for two MACHINEs are bundled together and
then updated by SWUpdate depending on the MACHINE.

> SWUpdate cannot find such as an environment.
> 
> For that, there is a target in u-boot: make u-boot-initial-env.

Yes, I've noticed that it was added recently to U-Boot.

> This
> generates an ASCII file that can be used by SWUpdate if no environment
> is stored and U-Boot starts with the linked-in environemnt. 
> It is
> enough to put this file into rootfs and to enable SWUpdate (and
> libubootenv) to use it. 

And if we do have this ASCII file, shall the mkimage with some SWUpdate
script be run to generate default envs, which then would be stored to
eMMC/SPI-NOR to avoid after reset "CRC env read error" ?

> Create specific image to be stored into
> a .wic seems overkilling,

The advantages for such approach:

- I do have a single, per MACHINE *.wic file which I can write either
  via u-boot HUSH script or with SWUpdate and after reset I do have
  access to right envs (no adjustments needed).

- This is helpful for factory flashing.

- The correct *.wic image is created solely with OE/Yocto - no extra
  hacks needed.


> and it does not help if for some reason the
> board comes up without environment in the flash.

As I update the whole layout (including GPT/MBR, boot, rootfs, custom,
envs, etc) - all elements are updated.

(For incremental updates I can provide *.swu file with only fitImage or
rootfs).


In short:

What I need ? 

I do need env images generated (with mkimage) during u-boot build, so I
can then use then during *.wic per machine image creation.

> 
> Best regards,
> Stefano Babic
> 
> > 
> > The UBOOT_ENVS_SIZE must be defined in machine specific conf file
> > as well as it is also possible to adjust (optionally) the *_ENVS_*
> > variables.
> > 
> > Test:
> > Newest master-next for poky repo - SHA1:
> > eb5b0a0b5e53a6e55a09e66489d3f24d0c6232ee MACHINE =
> > "beaglebone-yocto" in local.conf bitbake virtual/bootloader
> > 
> > 
> > As a result following links are available in deploy directory:
> > u-boot-env.img{_r}.
> > 
> > Signed-off-by: Lukasz Majewski <lukma@denx.de>
> > 
> > ---
> > Changes for v2:
> > 
> > - Move the content of deploy_default_envs task to do_deploy
> > - Move the content of gen_default_envs task to do_compile
> > - Create the u-boot-env.img{r} conditionally, only when
> > UBOOT_ENVS_SIZE is defined
> > - Change the commit title to reflect changes done for v2
> > ---
> >  meta/recipes-bsp/u-boot/u-boot.inc | 34
> > ++++++++++++++++++++++++++++++++++ 1 file changed, 34 insertions(+)
> > 
> > diff --git a/meta/recipes-bsp/u-boot/u-boot.inc
> > b/meta/recipes-bsp/u-boot/u-boot.inc index 9a754fd09b..7ede2da0eb
> > 100644 --- a/meta/recipes-bsp/u-boot/u-boot.inc
> > +++ b/meta/recipes-bsp/u-boot/u-boot.inc
> > @@ -68,6 +68,13 @@ UBOOT_EXTLINUX_INSTALL_DIR ?= "/boot/extlinux"
> >  UBOOT_EXTLINUX_CONF_NAME ?= "extlinux.conf"
> >  UBOOT_EXTLINUX_SYMLINK ?=
> > "${UBOOT_EXTLINUX_CONF_NAME}-${MACHINE}-${PR}" 
> > +# U-Boot variables necessary for extracting default envs from
> > build U-Boot +DEFAULT_UBOOT_ENVS_FILE ?= "u-boot-env"
> > +DEFAULT_ENVS ?= "${DEFAULT_UBOOT_ENVS_FILE}.txt"
> > +UBOOT_ENVS_DEFAULT ?=
> > "${DEFAULT_UBOOT_ENVS_FILE}-${MACHINE}-${PV}-${PR}.img" +# The
> > UBOOT_ENVS_SIZE needs to defined in the <machine>.conf file +#
> > UBOOT_ENVS_SIZE = "65536" +
> >  # returns all the elements from the src uri that are .cfg files
> >  def find_cfgs(d):
> >      sources=src_patches(d, True)
> > @@ -132,6 +139,19 @@ do_compile () {
> >          oe_runmake -C ${S} O=${B} ${UBOOT_MAKE_TARGET}
> >      fi
> >  
> > +    # Generate default environment
> > +    if [ -n "${UBOOT_ENVS_SIZE}" ]
> > +    then
> > +        ${B}/source/scripts/get_default_envs.sh ${B} >
> > ${B}/${DEFAULT_ENVS} +
> > +        # Generate env image
> > +        ${B}/tools/mkenvimage -s ${UBOOT_ENVS_SIZE} -o
> > ${B}/${UBOOT_ENVS_DEFAULT} ${B}/${DEFAULT_ENVS} +
> > +        # Generate redundant env image
> > +        ${B}/tools/mkenvimage -r -s ${UBOOT_ENVS_SIZE} -o
> > ${B}/${UBOOT_ENVS_DEFAULT}_r ${B}/${DEFAULT_ENVS} +
> > +        rm ${B}/${DEFAULT_ENVS}
> > +    fi
> >  }
> >  
> >  do_install () {
> > @@ -328,6 +348,20 @@ do_deploy () {
> >          ln -sf ${UBOOT_EXTLINUX_SYMLINK}
> > ${DEPLOYDIR}/${UBOOT_EXTLINUX_CONF_NAME}-${MACHINE} ln -sf
> > ${UBOOT_EXTLINUX_SYMLINK} ${DEPLOYDIR}/${UBOOT_EXTLINUX_CONF_NAME}
> > fi +
> > +    if [ -n "${UBOOT_ENVS_SIZE}" ]
> > +    then
> > +        install -d ${DEPLOYDIR}
> > +        install ${B}/${UBOOT_ENVS_DEFAULT}
> > ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}
> > +        install ${B}/${UBOOT_ENVS_DEFAULT}_r
> > ${DEPLOYDIR}/${UBOOT_ENVS_DEFAULT}_r +
> > +        cd ${DEPLOYDIR}
> > +        ln -sf ${UBOOT_ENVS_DEFAULT} ${DEFAULT_UBOOT_ENVS_FILE}.img
> > +        ln -sf ${UBOOT_ENVS_DEFAULT}_r
> > ${DEFAULT_UBOOT_ENVS_FILE}.img_r +
> > +        rm ${B}/${UBOOT_ENVS_DEFAULT}
> > +        rm ${B}/${UBOOT_ENVS_DEFAULT}_r
> > +    fi
> >  }
> >  
> >  addtask deploy before do_build after do_compile
> >   
> 




Best regards,

Lukasz Majewski

--

DENX Software Engineering GmbH,      Managing Director: Wolfgang Denk
HRB 165235 Munich, Office: Kirchenstr.5, D-82194 Groebenzell, Germany
Phone: (+49)-8142-66989-59 Fax: (+49)-8142-66989-80 Email: lukma@denx.de

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 499 bytes --]