Re: [Buildroot] [PATCH] package/micropython: add support for manifest.py in the configuration

["Yann E. MORIN" <yann.morin.1998@free.fr>]

Abilio, All,

On 2024-02-03 22:26 -0800, Abilio Marques spake thusly:
> Micropython can embed packages an modules as frozen bytecode. What code
> gets built this way can be defined by means of a "manifest.py" file.
> 
> This commit exposes the variable FROZEN_MANIFEST to Buildroot users
> through a new variable called BR2_PACKAGE_MICROPYTHON_MANIFEST.
> 
> Please check Micropython's documentation for more information:
> https://docs.micropython.org/en/latest/reference/manifest.html

Minor nit: this blurb should have been in the help text of the new
option, and I could have done that when applying, but I have a much
more involved question, see below.

> Signed-off-by: Abilio Marques <abiliojr@gmail.com>
> ---
>  package/micropython/Config.in      | 8 ++++++++
>  package/micropython/micropython.mk | 5 +++++
>  2 files changed, 13 insertions(+)
> 
> diff --git a/package/micropython/Config.in b/package/micropython/Config.in
> index 26a00baab0..52717d815a 100644
> --- a/package/micropython/Config.in
> +++ b/package/micropython/Config.in
> @@ -17,6 +17,14 @@ config BR2_PACKAGE_MICROPYTHON_LIB
>  	help
>  	  Core Python libraries ported to MicroPython.
>  
> +config BR2_PACKAGE_MICROPYTHON_MANIFEST
> +	string "Path to a manifest.py file"
> +	help
> +	  MicroPython allows Python code to be “frozen” as bytecode
> +	  into its binary, as an alternative to loading code from
> +	  the filesystem. See MicroPython's documentation for more
> +	  information.
> +
>  endif # BR2_PACKAGE_MICROPYTHON
>  
>  comment "micropython needs a toolchain w/ threads, dynamic library"
> diff --git a/package/micropython/micropython.mk b/package/micropython/micropython.mk
> index 125a0edcfb..5a2c136547 100644
> --- a/package/micropython/micropython.mk
> +++ b/package/micropython/micropython.mk
> @@ -43,6 +43,11 @@ else
>  MICROPYTHON_MAKE_OPTS += MICROPY_PY_FFI=0
>  endif
>  
> +ifneq ($(BR2_PACKAGE_MICROPYTHON_MANIFEST),"")
> +MICROPYTHON_MAKE_OPTS += \
> +	FROZEN_MANIFEST=$(BR2_PACKAGE_MICROPYTHON_MANIFEST)
> +endif

So, as I understand this, micropython will grab the files (from modules,
files, etc...) listed from the manifest and bundle them in the
micropython executable. It seems the paths in the manifest can be either
absolute, or relative; in the latter case, it not documented what they
would be relative to; additionally there are placeholders (see below)
that can be used but are absolute paths.

So this has a few implications:

  - the files must be available before micropython is built, so some
    dependency order is needed to ensure that: if files are provided in
    a package, then we need a dependency and it is going to be a little
    bit difficult to do and we need a way to address that;

  - the paths can't be hard-coded to absolute paths in the manifest file
    itself, because we don't know beforehand where the buildroot build
    directory will be, so we need a way to shoehorn $(TOPDIR) or some
    such variable in the manifest;

  - absolute paths must be fixed to accommodate the per-package option,
    because those paths change for eaach package.

From the micropython doc, that you pointed above, there are a few
placeholders that will be replaced in the manifest, but I don't think
we can reuse those:

    Any paths used in manifest files can include the following variables.
    These all resolve to absolute paths.
      * $(MPY_DIR) – path to the micropython repo.
      * $(MPY_LIB_DIR) – path to the micropython-lib submodule. Prefer to use require().
      * $(PORT_DIR) – path to the current port (e.g. ports/stm32)
      * $(BOARD_DIR) – path to the current board (e.g.  ports/stm32/boards/PYBV11)

So we need a bit more information how this feature is to be used, and an
example (or a few) would probably help a lot in this respect. We also
have a runtime test for micropython, in:
    support/testing/tests/package/test_micropython.py

It would be nice to extend that runtime test as well.

Regards,
Yann E. MORIN.

>  define MICROPYTHON_BUILD_CMDS
>  	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D)/mpy-cross
>  	$(TARGET_MAKE_ENV) $(MAKE) -C $(@D)/ports/unix \
> -- 
> 2.43.0
> 
> _______________________________________________
> buildroot mailing list
> buildroot@buildroot.org
> https://lists.buildroot.org/mailman/listinfo/buildroot

-- 
.-----------------.--------------------.------------------.--------------------.
|  Yann E. MORIN  | Real-Time Embedded | /"\ ASCII RIBBON | Erics' conspiracy: |
| +33 662 376 056 | Software  Designer | \ / CAMPAIGN     |  ___               |
| +33 561 099 427 `------------.-------:  X  AGAINST      |  \e/  There is no  |
| http://ymorin.is-a-geek.org/ | _/*\_ | / \ HTML MAIL    |   v   conspiracy.  |
'------------------------------^-------^------------------^--------------------'
_______________________________________________
buildroot mailing list
buildroot@buildroot.org
https://lists.buildroot.org/mailman/listinfo/buildroot