Re: [PATCH v9 0/7] env: Allow environment in text files

[Tom Rini <trini@konsulko.com>]

[-- Attachment #1: Type: text/plain, Size: 7698 bytes --]

On Tue, Oct 19, 2021 at 04:44:15PM -0600, Simon Glass wrote:

> One barrier to completing the 7-year-long Kconfig migration is that
> the default environment is implemented using ad-hoc CONFIG options.
> At present U-Boot environment variables, and thus scripts, are defined
> by CONFIG_EXTRA_ENV_SETTINGS.
> 
> It is not really feasible to move the environment to Kconfig as it is
> hundreds of lines of text in some cases.
> 
> Even considering the current situation, it is painful to add large
> amounts of text to the config-header file and dealing with quoting and
> newlines is harder than it should be. It would be better if we could just
> type the script into a text file and have it included by U-Boot.
> 
> This is already supported by the CONFIG_USE_DEFAULT_ENV_FILE feature. but
> that does not support use of CONFIG options or comments, so is best suited
> for use by other build systems wanting to define the U-Boot environment.
> 
> Add a feature that brings in a .env file associated with the board
> config, if present. To use it, create a file board/<vendor>/<board>.env or
> use CONFIG_ENV_SOURCE_FILE to set a filename.
> 
> The environment variables should be of the form "var=value". Values can
> extend to multiple lines. This series converts the existing environment
> documentation to rST and updates it to explain how to use this.
> 
> Note: this series was originally sent eight years ago:
> 
> https://patchwork.ozlabs.org/project/uboot/patch/1382763695-2849-4-git-send-email-sjg@chromium.org/
> 
> It has been updated to work with Kconfig, etc. Some review comments in
> that patch were infeasible so I have not addressed them. I would like
> this series to be considered independently, on its merits.
> 
> Rather than deal with the complexity of rewriting the distro-boot
> script, this is disabled for sandbox. The forthcoming bootmethod approach
> should provide the same functionality without needing the complex
> scripting in the environment.
> 
> Migration needs more thought, although it can be done later. It may be
> possible to do migrate automatically, using buildman to extract the
> built-in environmnent from the ELF file.
> 
> This would produce a pretty ugly conversion though, since it would drop
> all the intermediate variables used to create the environment.
> 
> Better would be to parse the config.h file, figure out the components of
> CONFIG_EXTRA_ENV_SETTINGS then output these as separate pieces in the
> file. It is not clear how easy that would be, nor whether the result would
> be very pretty. Also the __stringify() macro needs to be handled somehow.
> 
> This series is available at u-boot-dm/env-working
> 
> Comments welcome.
> 
> Changes in v9:
> - Drop mention of other strange characters
> - Clarify that the + restriction is on the variable name not its value
> - Add some tests for the script
> - Deal with leading tabs
> - Squash indentation down to one space
> - Convert newlines within strings to spaces, which seems more consistent
> - Handle appending an empty string to an empty var
> - Fix blank line between tags
> - Fix typo in commit message
> - More bikeshedding on env_get_autostart()
> - Fix '<vendor><board>' in cover letter
> - Use env_get_yesno() in env_get_autostart() and update docs
> 
> Changes in v8:
> - Update commit message to avoid mentioning the 'env' subdirectory
> - Update commit message to mention the + restriction, etc.
> - Overwrite the env file each time, to avoid incremental-build problems
> - Fix ambiguity about what is ignored
> - Go into more detail about the change of behaviour with autostart
> 
> Changes in v7:
> - Use 'env' basename instead of 'environment' for intermediate output files
> - Show a message indicating the source text file being used
> - Give an error if CONFIG_EXTRA_ENV_SETTINGS is also defined
> - Use CONFIG_ENV_SOURCE_FILE instead of rules to specify the text-file name
> - Make board.env the default name if CONFIG_ENV_SOURCE_FILE is empty
> - Rewrite the documentation
> - Drop the use of common.env
> - Update awk script to output the whole CONFIG string, or just a comment
> - Add new patch to explain the relationship with DEFAULT_ENV_FILE
> - A few more tweaks
> - Update the cover letter
> 
> Changes in v6:
> - Move all updates to a separate patch
> - Combine the two env2string.awk patches into one
> - Move all updates to a separate patch
> - More updates and improvements
> - Add new patch to tidy up use of autostart env var
> 
> Changes in v5:
> - Minor updates as suggested by Wolfgang
> - Explain how to include the common.env file
> - Explain why variables starting with _ , and / are not supported
> - Expand the definition of how to declare an environment variable
> - Explain what happens to empty variables
> - Update maintainer
> - Move use of += to this patch
> - Explain that environment variables may not end in +
> - Minor updates as suggested by Wolfgang
> 
> Changes in v4:
> - Add new patch to move environment documentation to rST
> - Move this from being part of configuring U-Boot to part of building it
> - Don't put the environment in autoconf.mk as it is not needed
> - Add documentation in rST format instead of README
> - Drop mention of import/export
> - Update awk script to ignore blank lines, as generated by clang
> - Add documentation in rST format instead of README
> - Add new patch to move environment documentation to rST
> 
> Changes in v3:
> - Adjust Makefile to generate the .inc and .h files in separate fules
> - Add more detail in the README about the format of .env files
> - Improve the comment about " in the awk script
> - Correctly terminate environment files with \n
> - Define __UBOOT_CONFIG__ when collecting environment files
> - Add new patch to use a text-based environment for sandbox
> 
> Changes in v2:
> - Move .env file from include/configs to board/
> - Use awk script to process environment since it is much easier on the brain
> - Add information and updated example script to README
> - Add dependency rule so that the environment is rebuilt when it changes
> - Add separate patch to enable C preprocessor for environment files
> - Enable var+=value form to simplify composing variables in multiple steps
> 
> Simon Glass (7):
>   sandbox: Drop distro_boot
>   doc: Move environment documentation to rST
>   env: Allow U-Boot scripts to be placed in a .env file
>   sandbox: Use a text-based environment
>   doc: Mention CONFIG_DEFAULT_ENV_FILE
>   doc: Improve environment documentation
>   bootm: Tidy up use of autostart env var
> 
>  MAINTAINERS               |   7 +
>  Makefile                  |  66 ++++-
>  README                    | 328 -------------------------
>  board/sandbox/sandbox.env |  25 ++
>  cmd/bootm.c               |   4 +-
>  cmd/elf.c                 |   3 +-
>  common/bootm_os.c         |   5 +-
>  config.mk                 |   2 +
>  doc/usage/environment.rst | 490 ++++++++++++++++++++++++++++++++++++++
>  doc/usage/index.rst       |   1 +
>  env/Kconfig               |  18 ++
>  env/common.c              |   5 +
>  env/embedded.c            |   1 +
>  include/configs/sandbox.h |  40 ----
>  include/env.h             |   7 +
>  include/env_default.h     |  11 +
>  scripts/env2string.awk    |  71 ++++++
>  test/py/tests/test_env.py |  93 ++++++++
>  18 files changed, 799 insertions(+), 378 deletions(-)
>  create mode 100644 board/sandbox/sandbox.env
>  create mode 100644 doc/usage/environment.rst
>  create mode 100644 scripts/env2string.awk

Cc'ing the other lists...

-- 
Tom

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 659 bytes --]