Re: [docs] [PATCH 1/2] image-bootfiles.bbclass: new class, copy boot files to /boot on rfs

Re: [docs] [PATCH 1/2] image-bootfiles.bbclass: new class, copy boot files to /boot on rfs

[Alexander Kanavin]

[-- Attachment #1: Type: text/plain, Size: 3776 bytes --]

Wrong mailing list. Also why not install these from regular packages?

Alex

On Thu 23. May 2024 at 9.33, Marcus Folkesson via lists.yoctoproject.org
<marcus.folkesson=gmail.com@lists.yoctoproject.org> wrote:

> image-bootfiles class copy files listed IMAGE_BOOT_FILES
> to the /boot directory of the root filesystem.
>
> This is useful when there is no explicit boot partition but all boot
> files should instead reside inside the root filesystem.
>
> Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>
> ---
>  meta/classes/image-bootfiles.bbclass | 69 ++++++++++++++++++++++++++++
>  1 file changed, 69 insertions(+)
>  create mode 100644 meta/classes/image-bootfiles.bbclass
>
> diff --git a/meta/classes/image-bootfiles.bbclass
> b/meta/classes/image-bootfiles.bbclass
> new file mode 100644
> index 0000000000..850e14e4bb
> --- /dev/null
> +++ b/meta/classes/image-bootfiles.bbclass
> @@ -0,0 +1,69 @@
> +#
> +# Writes IMAGE_BOOT_FILES to the /boot directory
> +#
> +# Copyright (C) 2024 Marcus Folkesson
> +# Author: Marcus Folkesson <marcus.folkesson@gmail.com>
> +#
> +# Licensed under the MIT license, see COPYING.MIT for details
> +# Inspired of bootimg-partition.py
> +#
> +# Usage: add INHERIT += "image-bootfiles" to your conf file
> +#
> +
> +def bootfiles_populate(d):
> +    import re
> +    from glob import glob
> +    import shutil
> +
> +    boot_files = d.getVar("IMAGE_BOOT_FILES")
> +    deploy_image_dir = d.getVar("DEPLOY_DIR_IMAGE")
> +    boot_dir = d.getVar("IMAGE_ROOTFS") + "/boot"
> +    install_files = []
> +
> +    if boot_files is None:
> +        return
> +
> +    # list of tuples (src_name, dst_name)
> +    deploy_files = []
> +    for src_entry in re.findall(r'[\w;\-\./\*]+', boot_files):
> +        if ';' in src_entry:
> +            dst_entry = tuple(src_entry.split(';'))
> +            if not dst_entry[0] or not dst_entry[1]:
> +                raise bb.parse.SkipRecipe('Malformed boot file entry: %s'
> % src_entry)
> +        else:
> +            dst_entry = (src_entry, src_entry)
> +
> +        deploy_files.append(dst_entry)
> +
> +    for deploy_entry in deploy_files:
> +        src, dst = deploy_entry
> +        if '*' in src:
> +            # by default install files under their basename
> +            entry_name_fn = os.path.basename
> +            if dst != src:
> +                # unless a target name was given, then treat name
> +                # as a directory and append a basename
> +                entry_name_fn = lambda name: \
> +                                os.path.join(dst,
> +                                             os.path.basename(name))
> +
> +
> +                srcs = glob(os.path.join(deploy_image_dir, src))
> +                for entry in srcs:
> +                    src = os.path.relpath(entry, deploy_mage_dir)
> +                    entry_dst_name = entry_name_fn(entry)
> +                    install_files.append((src, entry_dst_name))
> +            else:
> +                install_files.append((src, dst))
> +
> +    for entry in install_files:
> +        src, dst = entry
> +        image_src = os.path.join(deploy_image_dir, src)
> +        image_dst = os.path.join(boot_dir, dst)
> +        shutil.copyfile(image_src, image_dst)
> +
> +python bootfiles () {
> +   bootfiles_populate(d),
> +}
> +
> +IMAGE_PREPROCESS_COMMAND += "bootfiles;"
> --
> 2.44.0
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#5227):
> https://lists.yoctoproject.org/g/docs/message/5227
> Mute This Topic: https://lists.yoctoproject.org/mt/106258620/1686489
> Group Owner: docs+owner@lists.yoctoproject.org
> Unsubscribe: https://lists.yoctoproject.org/g/docs/unsub [
> alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>

[-- Attachment #2: Type: text/html, Size: 5404 bytes --]

[PATCH 1/2] image-bootfiles.bbclass: new class, copy boot files to /boot on rfs

[Marcus Folkesson]

image-bootfiles class copy files listed IMAGE_BOOT_FILES
to the /boot directory of the root filesystem.

This is useful when there is no explicit boot partition but all boot
files should instead reside inside the root filesystem.

Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>
---
 meta/classes/image-bootfiles.bbclass | 69 ++++++++++++++++++++++++++++
 1 file changed, 69 insertions(+)
 create mode 100644 meta/classes/image-bootfiles.bbclass

diff --git a/meta/classes/image-bootfiles.bbclass b/meta/classes/image-bootfiles.bbclass
new file mode 100644
index 0000000000..850e14e4bb
--- /dev/null
+++ b/meta/classes/image-bootfiles.bbclass
@@ -0,0 +1,69 @@
+#
+# Writes IMAGE_BOOT_FILES to the /boot directory
+#
+# Copyright (C) 2024 Marcus Folkesson
+# Author: Marcus Folkesson <marcus.folkesson@gmail.com>
+#
+# Licensed under the MIT license, see COPYING.MIT for details
+# Inspired of bootimg-partition.py
+#
+# Usage: add INHERIT += "image-bootfiles" to your conf file
+#
+
+def bootfiles_populate(d):
+    import re
+    from glob import glob
+    import shutil
+
+    boot_files = d.getVar("IMAGE_BOOT_FILES")
+    deploy_image_dir = d.getVar("DEPLOY_DIR_IMAGE")
+    boot_dir = d.getVar("IMAGE_ROOTFS") + "/boot"
+    install_files = []
+
+    if boot_files is None:
+        return
+
+    # list of tuples (src_name, dst_name)
+    deploy_files = []
+    for src_entry in re.findall(r'[\w;\-\./\*]+', boot_files):
+        if ';' in src_entry:
+            dst_entry = tuple(src_entry.split(';'))
+            if not dst_entry[0] or not dst_entry[1]:
+                raise bb.parse.SkipRecipe('Malformed boot file entry: %s' % src_entry)
+        else:
+            dst_entry = (src_entry, src_entry)
+
+        deploy_files.append(dst_entry)
+
+    for deploy_entry in deploy_files:
+        src, dst = deploy_entry
+        if '*' in src:
+            # by default install files under their basename
+            entry_name_fn = os.path.basename
+            if dst != src:
+                # unless a target name was given, then treat name
+                # as a directory and append a basename
+                entry_name_fn = lambda name: \
+                                os.path.join(dst,
+                                             os.path.basename(name))
+
+
+                srcs = glob(os.path.join(deploy_image_dir, src))
+                for entry in srcs:
+                    src = os.path.relpath(entry, deploy_mage_dir)
+                    entry_dst_name = entry_name_fn(entry)
+                    install_files.append((src, entry_dst_name))
+            else:
+                install_files.append((src, dst))
+
+    for entry in install_files:
+        src, dst = entry
+        image_src = os.path.join(deploy_image_dir, src)
+        image_dst = os.path.join(boot_dir, dst)
+        shutil.copyfile(image_src, image_dst)
+
+python bootfiles () {
+   bootfiles_populate(d),
+}
+
+IMAGE_PREPROCESS_COMMAND += "bootfiles;"
-- 
2.44.0

[PATCH 2/2] ref-manual: classes: add new image-bootfiles class

[Marcus Folkesson]

Describe the newly introduced image-bootfiles class.

Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>
---
 documentation/ref-manual/classes.rst | 14 ++++++++++++++
 1 file changed, 14 insertions(+)

diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst
index 9520d0bf7c..53b3697bee 100644
--- a/documentation/ref-manual/classes.rst
+++ b/documentation/ref-manual/classes.rst
@@ -1169,6 +1169,20 @@ Yocto Project Overview and Concepts Manual.
 
 .. _ref-classes-image-buildinfo:
 
+``image-bootfiles``
+===================
+
+The :ref:`ref-classes-image-buildinfo` class copies over files listed
+in :ref:`IMAGE_BOOT_FILES` to the /boot directory of the root filesystem.
+
+This can be useful if no separate boot partition is used but all boot files
+should be included into the rootfs image.
+
+:ref:`IMAGE_BOOT_FILES` is the same space-separated list of files used
+by the ``bootimg-partition`` source plugin to populate the boot partition.
+
+.. _ref-classes-image_types:
+
 ``image-buildinfo``
 ===================
 
-- 
2.44.0

Re: [docs] [PATCH 1/2] image-bootfiles.bbclass: new class, copy boot files to /boot on rfs

[Marcus Folkesson]

Hi Alex,

On Thu, May 23, 2024 at 09:36:30AM +0200, Alexander Kanavin wrote:
> Wrong mailing list. Also why not install these from regular packages?

Yes it is for the first patch.
The patch series touches both docs and openembedded-core.
I had not subscribed to docs so the patches were rejected, therefor I
resend it to docs only.

I think there is a win in using the same method as bootimg-partition
does.
Then it is the simple task to switch from a separate boot partition to
embed the boot files into the target root filesystem.

> 
> Alex

Best regards,
Marcus Folkesson

Re: [docs] [PATCH 1/2] image-bootfiles.bbclass: new class, copy boot files to /boot on rfs

[Alexander Kanavin]

On Thu, 23 May 2024 at 09:48, Marcus Folkesson
<marcus.folkesson@gmail.com> wrote:
> I think there is a win in using the same method as bootimg-partition
> does.
> Then it is the simple task to switch from a separate boot partition to
> embed the boot files into the target root filesystem.

I don't know. It goes against the established practice for forming the
rootfs. /boot folder is not special.

Alex

Re: [docs] [PATCH 2/2] ref-manual: classes: add new image-bootfiles class

[Michael Opdenacker]

Hi Marcus

Thanks for the documentation patch!

On 5/23/24 at 13:09, Marcus Folkesson wrote:
> Describe the newly introduced image-bootfiles class.
>
> Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>
> ---
>   documentation/ref-manual/classes.rst | 14 ++++++++++++++
>   1 file changed, 14 insertions(+)
>
> diff --git a/documentation/ref-manual/classes.rst b/documentation/ref-manual/classes.rst
> index 9520d0bf7c..53b3697bee 100644
> --- a/documentation/ref-manual/classes.rst
> +++ b/documentation/ref-manual/classes.rst
> @@ -1169,6 +1169,20 @@ Yocto Project Overview and Concepts Manual.
>   
>   .. _ref-classes-image-buildinfo:
>   
> +``image-bootfiles``
> +===================
> +
> +The :ref:`ref-classes-image-buildinfo` class copies over files listed
> +in :ref:`IMAGE_BOOT_FILES` to the /boot directory of the root filesystem.
> +
> +This can be useful if no separate boot partition is used but all boot files
> +should be included into the rootfs image.
> +
> +:ref:`IMAGE_BOOT_FILES` is the same space-separated list of files used
> +by the ``bootimg-partition`` source plugin to populate the boot partition.
> +
> +.. _ref-classes-image_types:
> +
>   ``image-buildinfo``
>   ===================

There are issues though...

cd documentation
make html

reading sources... [100%] releases
/home/mike/work/git/git.yoctoproject.org/yocto-docs/documentation/ref-manual/classes.rst:1206: 
WARNING: Duplicate explicit target name: "ref-classes-image_types".
looking for now-outdated files... none found
pickling environment... done
checking consistency... done
preparing documents... done
/home/mike/work/git/git.yoctoproject.org/yocto-docs/documentation/ref-manual/classes.rst:1175: 
WARNING: undefined label: 'image_boot_files'
/home/mike/work/git/git.yoctoproject.org/yocto-docs/documentation/ref-manual/classes.rst:1181: 
WARNING: undefined label: 'image_boot_files'
/home/mike/work/git/git.yoctoproject.org/yocto-docs/documentation/ref-manual/classes.rst:1211: 
WARNING: undefined label: 'ref-classes-image_types'
/home/mike/work/git/git.yoctoproject.org/yocto-docs/documentation/ref-manual/classes.rst:1217: 
WARNING: undefined label: 'ref-classes-image_types'
/home/mike/work/git/git.yoctoproject.org/yocto-docs/documentation/ref-manual/classes.rst:1234: 
WARNING: undefined label: 'ref-classes-image_types'

You should actually add the new class documentation before " .. 
_ref-classes-image-buildinfo:" and have your own ".. 
_ref-classes-image-bootfiles" definition.

Thanks in advance. You may wait for your class to be merged before 
submitting a V2 of the documentation (though early documentation is 
always useful!), as I won't be able to merge the documentation changes 
before this happens.
Thanks again,
Michael.

-- 
Michael Opdenacker, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com