Re: [PATCH] doc: Add documentation about devicetree usage

[Sean Anderson <seanga2@gmail.com>]

On 8/27/21 11:23 PM, Simon Glass wrote:
> At present some of the ideas and techniques behind devicetree in U-Boot
> are assumed, implied or unsaid. Add some documentation to cover how
> devicetree is build, how it can be modified and the rules about using
> the various CONFIG_OF_... options.
> 
> Signed-off-by: Simon Glass <sjg@chromium.org>
> ---
> 
>   doc/develop/index.rst              |   1 +
>   doc/develop/package/devicetree.rst | 315 +++++++++++++++++++++++++++++
>   doc/develop/package/index.rst      |   1 +
>   3 files changed, 317 insertions(+)
>   create mode 100644 doc/develop/package/devicetree.rst
> 
> diff --git a/doc/develop/index.rst b/doc/develop/index.rst
> index 83c929babda..d5ad8f9fe53 100644
> --- a/doc/develop/index.rst
> +++ b/doc/develop/index.rst
> @@ -36,6 +36,7 @@ Packaging
>      :maxdepth: 1
>   
>      package/index
> +   package/devicetree
>   
>   Testing
>   -------
> diff --git a/doc/develop/package/devicetree.rst b/doc/develop/package/devicetree.rst
> new file mode 100644
> index 00000000000..fccbb182f3e
> --- /dev/null
> +++ b/doc/develop/package/devicetree.rst
> @@ -0,0 +1,315 @@
> +.. SPDX-License-Identifier: GPL-2.0+
> +
> +Updating the devicetree
> +=======================
> +
> +U-Boot uses devicetree for runtime configuration and storing required blobs or
> +any other information it needs to operate. It is possible to update the
> +devicetree separately from actually building U-Boot. This provides a good degree
> +of control and flexibility for firmware that uses U-Boot in conjunction with
> +other project.

nit: projects

> +
> +There are many reasons why it is useful to modify the devicetree after building
> +it:
> +
> +- Configuration can be changed, e.g. which UART to use
> +- A serial number can be added
> +- Public keys can be added to allow image verification
> +- Console output can be changed (e.g. to select serial or vidconsole)
> +
> +This section describes how to work with devicetree to accomplish your goals.
> +
> +See also :doc:`../devicetree/control` for a basic summary of the available
> +features.
> +
> +
> +Devicetree source
> +-----------------
> +
> +Every board in U-Boot must include a devicetree sufficient to build and boot
> +that board on suitable hardware (or emulation). This is specified using the
> +`CONFIG DEFAULT_DEVICE_TREE` option.
> +
> +
> +Current situation (August 2021)
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +As an aside, at present U-Boot allows `CONFIG_DEFAULT_DEVICE_TREE` to be empty,
> +e.g. if `CONFIG_OF_BOARD` or `CONFIG_OF_PRIOR_STAGE` are used. This has
> +unfortunately created an enormous amount of confusion and some wasted effort.
> +This was not intended and this bug will be fixed soon. Specifically:
> +
> +- `CONFIG_OF_BOARD` was added in rpi_patch_ for Raspberry Pi, which does have
> +  an in-tree devicetree, but this feature has since been used for boards that
> +  don't
> +- `CONFIG_OF_PRIOR_STAGE` was added in bcm_patch_ as part of a larger Broadcom
> +  change with a tag indicating it only affected one board, so the change in
> +  behaviour was not noticed at the time. It has since been used by RISC-V qemu
> +  boards.
> +
> +Once this bug is fixed, CONFIG_OF_BOARD and CONFIG_OF_PRIOR_STAGE will override
> +(at runtime) the devicetree suppled with U-Boot, but will otherwise use
> +CONFIG_OF_SEPARATE for the in-tree build. So these two will become options,
> +moving out of the 'choice' in `dts/Kconfig`
> +
> +Offending boards are:
> +
> +- bcm7260
> +- bcm7445
> +- qemu_arm64
> +- qemu_arm
> +- qemu-ppce500
> +- qemu-riscv32
> +- qemu-riscv32_smode
> +- qemu-riscv64
> +- qemu-riscv64_smode
> +
> +All of these need to have a devicetree added in-tree. This is targeted to be
> +fixed in the 2022.01 release.
> +
> +
> +Building the devicetree
> +-----------------------
> +
> +U-Boot automatically builds the devicetree for a board, from the
> +`arch/<arch>/dts` directory. The Makefile in those directories has rules for
> +building devicetree files. It is preferable to avoid target-specific rules in
> +those files: i.e. all boards for a particular SoC should be built at once,
> +where practical. Apart from simplifying the Makefile, this helps to efficiently
> +(and immediately) ensure that changes in one board's DT do not break others that
> +are related. Building devicetrees is fast, so performance is seldom a concern
> +here.
> +
> +
> +Overriding the default devicetree
> +---------------------------------
> +
> +When building U-Boot, the `DEVICE_TREE` environment variable allows the
> +default devicetree file to be overridden at build time. This can be useful if
> +modifications have to be made to the in-tree devicetree file, for the benefit
> +of a downstream build system. Note that the in-tree devicetree must be
> +sufficient to build and boot, so this is not a way to bypass that requirement.
> +
> +
> +Modifying the devicetree after building
> +---------------------------------------
> +
> +While it is generally painful and hacky to modify the code or rodata of a
> +program after it is built, in many cases it is useul to do so, e.g. to add

nit: useful

> +configuration information like serial numbers, enabling/disabling features, etc.
> +
> +Devicetree provides a very nice solution to these problems since it is
> +structured data and it is relatively easy to change it, even in binary form
> +(see fdtput).
> +
> +U-Boot takes care that the devicetree is easily accessible after the build
> +process. In fact it is placed in a separate file called `u-boot.dtb`. If the
> +build system wants to modify or replace that file, it can do so. Then all that
> +is needed is to run `binman update` to update the file inside the image. If
> +binman is not used, then `u-boot-nodtb.bin` and the new `u-boot.dtb` can simply
> +be concatenated to achieve the desired result. U-Boot happily copes with the
> +devicetree growing or shrinking.
> +
> +The `u-boot.bin` image contains both pieces. While it is possible to locate the
> +devicetree within the image using the signature at the start of the file, this
> +is a bit messy.
> +
> +This is why `CONFIG_OF_SEPARATE` should always be used when building U-Boot.
> +The `CONFIG_OF_EMBED` option embeds the devicetree somewhere in the U-Boot ELF
> +image as rodata, meaning that it is hard to find it and it cannot increase in
> +size.
> +
> +When modifying the devicetree, the different cases to consider are as follows:
> +
> +- CONFIG_OF_SEPARATE
> +    This is easy, described above. Just change, replace or rebuild the
> +    devicetree so it suits your needs, then rerun binman or redo the `cat`
> +    operation to join `u-boot-nodtb.bin` and the new `u-boot.dtb`
> +
> +- CONFIG_OF_EMBD
> +    This is tricky, since the devicetree cannot easily be located. If the EFL
> +    file is available, then the _dtb_dt_begin and __dtb_dt_end symbols can be
> +    examined to find it. While it is possible to contract the file, it is not
> +    possible to expand the file since that would involve re-linking
> +
> +- CONFIG_OF_PRIOR_STAGE
> +    In this case the devicetree must be modified in the project which provides
> +    it, as described below
> +
> +- CONFIG_OF_BOARD
> +    This is a board-specific situation, so needs to be considered on a
> +    case-by-case base. The devicetree must be modified so that the correct
> +    one is provided to U-Boot. How this is done depends entirely on the
> +    implementation of this option for the board. It might require injecting the
> +    changes into a different project somehow using tooling available there, or
> +    it might involve merging an overlay file at runtime to obtain the desired
> +    result.
> +
> +
> +Devicetree in another project
> +-----------------------------
> +
> +In some cases U-Boot receive its devicetree at runtime from a program that calls
> +it. For example ARM's Trusted Firmware A (`TF-A`_) may have a devicetree that it
> +passes to U-Boot. This overrides any devicetree build by U-Boot. When packaging
> +the firmware, the U-Boot devicetree may in fact be left out if it can be
> +guaranteed that it will receive one from another project.
> +
> +In this case, the devicetree in the other project must track U-Boot's use of
> +device tree. It must provide a way to add configuration and other information to
> +the devicetree for use by U-Boot, such as the /config node. Note that the
> +U-Boot in-tree devicetree must be sufficient to build and boot, so this is not a
> +way to bypass that requirement.
> +
> +If binman is used, the in-tree U-Boot devicetree must contain the binman
> +definition so that a valid image can be build.
> +
> +If verified boot is used, the project must provide a way to inject a public key,
> +certificate or other material into the U-Boot devicetree so that it is available
> +to U-Boot at runtime. See `Signing with U-Boot devicetree`_. This may be
> +through tooling in the project itself or by making use of U-Boot's tooling.
> +
> +
> +Devicetree generated on-the-fly in another project
> +--------------------------------------------------
> +
> +In some rare cases, another project may wish to create a devicetree for U-Boot
> +entirely on-the-fly, then pass it to U-Boot at runtime. The only known example
> +of this at the time of writing (2021) is qemu, for ARM (`QEMU ARM`_) and
> +RISC-V (`QEMU RISC-V`_).
> +
> +In this case, the devicetree in the other project must track U-Boot's use of
> +device tree, so that it remains compatible. If a particular version of the
> +project is needed for a particular version of U-Boot, that must be documented
> +in both projects.
> +
> +Further, it must provide a way to add configuration and other information to
> +the devicetree for use by U-Boot, such as the `/config` node. Note that the
> +U-Boot in-tree devicetree must be sufficient to build and boot, so this is not a
> +way to bypass that requirement.
> +
> +More specifically, tooling or command-line arguments must provide a way to
> +add a `/config` node or items within that node, so that U-Boot can receive a
> +suitable configuration. These options can then be used as part of the build
> +process, which puts the firmware image together. For binman, a way must be
> +provided to add the binman definition into the devicetree in the same way.

Can you elaborate on the purpose here? What sort of things go into
/config which can't be set at compile-time?

> +
> +One way to do this is to allow a .dtsi file to be merged in with the generated
> +devicetree.
> +
> +
> +Passing the devicetree through to Linux
> +---------------------------------------
> +
> +Ideally U-Boot and Linux use the same devicetree source, even though it is
> +hosted in separate projects. U-Boot adds some extra pieces, such as the
> +`config/` node and tags like `u-boot,dm-spl`. Linux adds some extra pieces, such
> +as `linux,default-trigger` and `linux,code`. This should not interfere with
> +each other.
> +
> +In principle it is possible for U-Boot's control devicetree to be passed to
> +Linux. This is, after all, one of the goals of devicetree and the original
> +Open Firmware project, to have the firmware provide the hardware description to
> +the Operating System.
> +
> +For boards where this approach is used, care must be taken. U-Boot typically
> +needs to 'fix up' the devicetree before passing it to Linux, e.g. to add
> +information about the memory map, about which serial console is used, provide
> +the root hash for dm-verify or select whether the console should be silenced for

nit: dm-verity

Is the property for this documented anywhere? A quick grep doesn't find
it in Linux.

> +a faster boot.
> +
> +Fix-ups involve modifying the devicetree. If the control devicetree is used,
> +that means the control devicetree could be modified, while U-Boot is using it.
> +Removing a device and reinserting it can cause problems if the devicetree offset
> +has changed, for example, since the device will be unable to locates its
> +devicetree properties at the expected devicetree offset, which is a fixed
> +integer.
> +
> +To deal with this, it is recommended to employ one or more of the following
> +approaches:
> +
> +- Make a copy of the devicetree and 'fix up' the copy, leaving the control
> +  devicetree alone
> +- Enable `CONFIG_OF_LIVE` so that U-Boot makes its own copy of the devicetree
> +  during relocation; fixups then happen on the original flat tree
> +- Ensure that fix-ups happen after all loading has happened and U-Boot has
> +  completed image verification
> +
> +In practice,the last point is typically observed, since boot_prep_linux() is
> +called just before jumping to Linux, long after signature verification, for
> +example. But it is important to make sure that this line is not blurred,
> +particularly if untrusted user data in involved.

nit: is

> +
> +
> +Devicetree use cases that must be supported
> +-------------------------------------------
> +
> +Regardless of how the devicetree is provided to U-Boot at runtime, various
> +U-Boot features must be fully supported. This section describes some of these
> +features and the implications for other projects.
> +
> +If U-Boot uses its own in-tree devicetree these features are supported
> +automatically.
> +
> +
> +Signing with U-Boot devicetree
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +U-Boot supports signing a payload so that it can be verified to have been
> +created by a party owning a private key. This is called verified boot in U-Boot
> +(see doc/uImage.FIT/verified-boot.txt).
> +
> +Typically this works by creating a FIT and then running the `mkimage` tool to
> +add signatures for particular images. As part of this process, `mkimage` writes
> +a public key to the U-Boot devicetree, although this can be done separately.

It would be nice to just allow adding a public key to a device tree
without signing anything. ATM I have a "dummy" devicetree that I sign
just to add the public key.

> +As with all configuration information, if another project is providing the
> +devicetree to U-Boot, it must provide a way to add this public key into the
> +devicetree it passes to U-Boot. This could be via a tooling option, making use
> +of `mkimage`, or alowing a .dtsi file to be merged in with what is generated in
> +the other project.
> +
> +
> +Providing the binman image definition
> +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> +
> +In complex systems U-Boot must locate and make use of other firmware components,
> +such as images for the user interface, files containing peripheral firmware,
> +multiple copies of U-Boot for use with A/B boot, etc. U-Boot uses
> +:doc:`Binman <binman>` as a standard way of putting an image together.
> +
> +Typically this works by running binman with the devicetree as an input, to
> +create the file image. Binman then outputs an updated devicetree which is
> +packed in the firmware image, so U-Boot can access the binman definition and
> +locate all the components.
> +
> +As with all configuration information, if another project is providing the
> +devicetree to U-Boot, it must provide a way to add this binman definition into
> +the devicetree it passes to U-Boot. This could be via a tooling option, making
> +use of `binman`, or alowing a .dtsi file to be merged in with what is generated

nit: allowing

--Sean

> +in the other project.
> +
> +
> +Protecting the devicetree
> +-------------------------
> +
> +U-Boot relies heavily on devicetree for correct operation. A corrupt or invalid
> +device can cause U-Boot to fail to start, behave incorrectly, crash (e.g. if
> +`CONFIG_OF_LIBFDT_ASSUME_MASK` is adjusted, or fail to boot an Operating System.
> +Within U-Boot, the devicetree is as important as any other part of the source
> +code. At ruuntime, the devicetree can be considered to be structured rodata.
> +
> +With secure systems, care must be taken that the devicetree is valid:
> +
> +- If the code / rodata has a hash or signature, the devicetree should also, if
> +  they are packaged separately.
> +- If the code / rodata is write-protected when running, the devicetree should be
> +  also. Note that U-Boot relocates its code and devicetree, so this is not as
> +  simple as it sounds. U-Boot must write-protect these items after relocating.
> +
> +
> +.. _rpi_patch: https://patchwork.ozlabs.org/project/uboot/patch/20170402082520.32546-1-deymo@google.com/
> +.. _bcm_patch: https://patchwork.ozlabs.org/project/uboot/patch/16fc0901f4521d3c399eac950c52a634b2f9473b.1528485916.git.fitzsim@fitzsim.org/
> +.. _`TF-A`: https://www.trustedfirmware.org/projects/tf-a
> +.. _`QEMU ARM`: https://github.com/qemu/qemu/blob/master/hw/arm/virt.c
> +.. _`QEMU RISC-V`: https://github.com/qemu/qemu/blob/master/hw/riscv/virt.c
> diff --git a/doc/develop/package/index.rst b/doc/develop/package/index.rst
> index 9374be2e62c..188c376950e 100644
> --- a/doc/develop/package/index.rst
> +++ b/doc/develop/package/index.rst
> @@ -17,3 +17,4 @@ SPI flash.
>      :maxdepth: 2
>   
>      binman
> +   devicetree
>