[PATCH v4 0/2] image-bootfiles: new class

[PATCH v4 0/2] image-bootfiles: new class

[Marcus Folkesson]

The image-bootfiles class is used to put all files listed in
IMAGE_BOOT_FILES into the root filesystem.

IMAGE_BOOT_FILES is used by the bootimg-partition wic plugin to put the
files into a boot partition. 
Be able to list files as "boot files" in e.g. your .conf or image files
instead of install those in every recipe is a good thing.

It is not always desired to have a separate boot partition for boot
files. Sometimes it could be good to have them as a part of the root
filesystem.

For example, if a double copy strategy is used for update the system,
then you probably want to update both the boot files and root filesystem
at the same time as there may be dependencies.

v2:
    - Removed the documentation from the patch series (will be submitted later)
    - Break out the parts in bootimg-partition that is used by
      image-bootfiles to a common library
    - Make the destination directory in root filesystem configurable
v3:
    - See changelog in patches

v4:
    - See changelog in patches

Marcus Folkesson (2):
  bootimg-partition: break out code to a common library.
  image-bootfiles.bbclass: new class, copy boot files to root filesystem

 meta/classes/image-bootfiles.bbclass          | 38 +++++++++++++
 meta/lib/oe/bootfiles.py                      | 57 +++++++++++++++++++
 .../wic/plugins/source/bootimg-partition.py   | 39 +------------
 3 files changed, 97 insertions(+), 37 deletions(-)
 create mode 100644 meta/classes/image-bootfiles.bbclass
 create mode 100644 meta/lib/oe/bootfiles.py

-- 
2.45.1

[PATCH v4 1/2] bootimg-partition: break out code to a common library.

[Marcus Folkesson]

Break out the code that parse IMAGE_BOOT_FILES to a common library.

Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>
Reviewed-by: Quentin Schulz <quentin.schulz@cherry.de>
---
 meta/lib/oe/bootfiles.py                      | 57 +++++++++++++++++++
 .../wic/plugins/source/bootimg-partition.py   | 39 +------------
 2 files changed, 59 insertions(+), 37 deletions(-)
 create mode 100644 meta/lib/oe/bootfiles.py

diff --git a/meta/lib/oe/bootfiles.py b/meta/lib/oe/bootfiles.py
new file mode 100644
index 0000000000..155fe742db
--- /dev/null
+++ b/meta/lib/oe/bootfiles.py
@@ -0,0 +1,57 @@
+#
+# SPDX-License-Identifier: MIT
+#
+# Copyright (C) 2024 Marcus Folkesson
+# Author: Marcus Folkesson <marcus.folkesson@gmail.com>
+#
+# Utility functions handling boot files
+#
+# Look into deploy_dir and search for boot_files.
+# Returns a list of tuples with (original filepath relative to
+# deploy_dir, desired filepath renaming)
+#
+# Heavily inspired of bootimg-partition.py
+#
+def get_boot_files(deploy_dir, boot_files):
+    import re
+    import os
+    from glob import glob
+
+    if boot_files is None:
+        return None
+
+    # list of tuples (src_name, dst_name)
+    deploy_files = []
+    for src_entry in re.findall(r'[\w;\-\./\*]+', boot_files):
+        if ';' in src_entry:
+            dst_entry = tuple(src_entry.split(';'))
+            if not dst_entry[0] or not dst_entry[1]:
+                raise ValueError('Malformed boot file entry: %s' % src_entry)
+        else:
+            dst_entry = (src_entry, src_entry)
+
+        deploy_files.append(dst_entry)
+
+    install_files = []
+    for deploy_entry in deploy_files:
+        src, dst = deploy_entry
+        if '*' in src:
+            # by default install files under their basename
+            entry_name_fn = os.path.basename
+            if dst != src:
+                # unless a target name was given, then treat name
+                # as a directory and append a basename
+                entry_name_fn = lambda name: \
+                                os.path.join(dst,
+                                             os.path.basename(name))
+
+            srcs = glob(os.path.join(deploy_dir, src))
+
+            for entry in srcs:
+                src = os.path.relpath(entry, deploy_dir)
+                entry_dst_name = entry_name_fn(entry)
+                install_files.append((src, entry_dst_name))
+        else:
+            install_files.append((src, dst))
+
+    return install_files
diff --git a/scripts/lib/wic/plugins/source/bootimg-partition.py b/scripts/lib/wic/plugins/source/bootimg-partition.py
index 1071d1af3f..589853a439 100644
--- a/scripts/lib/wic/plugins/source/bootimg-partition.py
+++ b/scripts/lib/wic/plugins/source/bootimg-partition.py
@@ -16,7 +16,7 @@ import logging
 import os
 import re
 
-from glob import glob
+from oe.bootfiles import get_boot_files
 
 from wic import WicError
 from wic.engine import get_custom_config
@@ -66,42 +66,7 @@ class BootimgPartitionPlugin(SourcePlugin):
 
         logger.debug('Boot files: %s', boot_files)
 
-        # list of tuples (src_name, dst_name)
-        deploy_files = []
-        for src_entry in re.findall(r'[\w;\-\./\*]+', boot_files):
-            if ';' in src_entry:
-                dst_entry = tuple(src_entry.split(';'))
-                if not dst_entry[0] or not dst_entry[1]:
-                    raise WicError('Malformed boot file entry: %s' % src_entry)
-            else:
-                dst_entry = (src_entry, src_entry)
-
-            logger.debug('Destination entry: %r', dst_entry)
-            deploy_files.append(dst_entry)
-
-        cls.install_task = [];
-        for deploy_entry in deploy_files:
-            src, dst = deploy_entry
-            if '*' in src:
-                # by default install files under their basename
-                entry_name_fn = os.path.basename
-                if dst != src:
-                    # unless a target name was given, then treat name
-                    # as a directory and append a basename
-                    entry_name_fn = lambda name: \
-                                    os.path.join(dst,
-                                                 os.path.basename(name))
-
-                srcs = glob(os.path.join(kernel_dir, src))
-
-                logger.debug('Globbed sources: %s', ', '.join(srcs))
-                for entry in srcs:
-                    src = os.path.relpath(entry, kernel_dir)
-                    entry_dst_name = entry_name_fn(entry)
-                    cls.install_task.append((src, entry_dst_name))
-            else:
-                cls.install_task.append((src, dst))
-
+        cls.install_task = get_boot_files(kernel_dir, boot_files)
         if source_params.get('loader') != "u-boot":
             return
 
-- 
2.45.1

[PATCH v4 2/2] image-bootfiles.bbclass: new class, copy boot files to root filesystem

[Marcus Folkesson]

image-bootfiles class copy files listed in IMAGE_BOOT_FILES
to the IMAGE_BOOT_FILES_DIR directory of the root filesystem.

This is useful when there is no explicit boot partition but all boot
files should instead reside inside the root filesystem.

Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>
---
 meta/classes/image-bootfiles.bbclass | 38 ++++++++++++++++++++++++++++
 1 file changed, 38 insertions(+)
 create mode 100644 meta/classes/image-bootfiles.bbclass

diff --git a/meta/classes/image-bootfiles.bbclass b/meta/classes/image-bootfiles.bbclass
new file mode 100644
index 0000000000..0b95d47e06
--- /dev/null
+++ b/meta/classes/image-bootfiles.bbclass
@@ -0,0 +1,38 @@
+#
+# SPDX-License-Identifier: MIT
+#
+# Copyright (C) 2024 Marcus Folkesson
+# Author: Marcus Folkesson <marcus.folkesson@gmail.com>
+#
+# Writes IMAGE_BOOT_FILES to the IMAGE_BOOT_FILES_DIR directory.
+#
+# Usage: add "inherit image-bootfiles" to your image.
+#
+
+IMAGE_BOOT_FILES_DIR ?= "boot"
+
+python bootfiles_populate() {
+    import shutil
+    from oe.bootfiles import get_boot_files
+
+    deploy_image_dir = d.getVar("DEPLOY_DIR_IMAGE")
+    boot_dir = os.path.join(d.getVar("IMAGE_ROOTFS"), d.getVar("IMAGE_BOOT_FILES_DIR"))
+
+    boot_files = d.getVar("IMAGE_BOOT_FILES")
+    if boot_files is None:
+        return
+
+    install_files = get_boot_files(deploy_image_dir, boot_files)
+    if install_files is None:
+        bb.warn("Could not find any boot files to install even though IMAGE_BOOT_FILES is not empty")
+        return
+
+    os.makedirs(boot_dir, exist_ok=True)
+    for src, dst  in install_files:
+        image_src = os.path.join(deploy_image_dir, src)
+        image_dst = os.path.join(boot_dir, dst)
+        os.makedirs(os.path.dirname(image_dst), exist_ok=True)
+        shutil.copyfile(image_src, image_dst)
+}
+
+IMAGE_PREPROCESS_COMMAND += "bootfiles_populate;"
-- 
2.45.1

Re: [PATCH v4 2/2] image-bootfiles.bbclass: new class, copy boot files to root filesystem

[Quentin Schulz]

Hi Marcus,

On 5/30/24 11:53 AM, Marcus Folkesson wrote:
> image-bootfiles class copy files listed in IMAGE_BOOT_FILES
> to the IMAGE_BOOT_FILES_DIR directory of the root filesystem.
> 
> This is useful when there is no explicit boot partition but all boot
> files should instead reside inside the root filesystem.
> 
> Signed-off-by: Marcus Folkesson <marcus.folkesson@gmail.com>

Reviewed-by: Quentin Schulz <quentin.schulz@cherry.de>

Thanks!
Quentin

Re: [PATCH v4 2/2] image-bootfiles.bbclass: new class, copy boot files to root filesystem

[Konrad Weihmann]

[-- Attachment #1: Type: text/plain, Size: 887 bytes --]

Looking at the combination of the patches and the following

+ boot_files = d.getVar("IMAGE_BOOT_FILES")
+ if boot_files is None:
+ return
+
+ install_files = get_boot_files(deploy_image_dir, boot_files)
+ if install_files is None:
+ bb.warn("Could not find any boot files to install even though IMAGE_BOOT_FILES is not empty")
+ return

get_boot_files only returns None if boot_files is None, which is caught already earlier on here, so the bb.warn clause likely will not be reached

Personally a check like `if install_files:` would make more sense

IMAGE_PREPROCESS_COMMAND += "bootfiles_populate;"

I'm not sure if that wouldn't raise issues if any of the packages would install a file by the same path/name as also set here.

In my opinion moving it to a POSTPROCESS function with some checks that it won't overwrite any files/folders might be the better option

[-- Attachment #2: Type: text/html, Size: 16938 bytes --]

Re: [OE-core] [PATCH v4 2/2] image-bootfiles.bbclass: new class, copy boot files to root filesystem

[Marcus Folkesson]

[-- Attachment #1: Type: text/plain, Size: 1699 bytes --]


Hi Konrad,

Thanks for your comments!

On Sat, Jun 15, 2024 at 09:26:49PM -0700, Konrad Weihmann via lists.openembedded.org wrote:
> Looking at the combination of the patches and the following
> 
> + boot_files = d.getVar("IMAGE_BOOT_FILES")
> + if boot_files is None:
> + return
> +
> + install_files = get_boot_files(deploy_image_dir, boot_files)
> + if install_files is None:
> + bb.warn("Could not find any boot files to install even though IMAGE_BOOT_FILES is not empty")
> + return
> 
> get_boot_files only returns None if boot_files is None, which is caught already earlier on here, so the bb.warn clause likely will not be reached

You are right. The case i wanted to cover was if install_files was an
empty list.
This could occour if using asterix, e.g. IMAGE_BOOT_FILES = "bootfile.*" and there is
no "bootfile.*" in the DEPLOY_DIR.

> 
> Personally a check like `if install_files:` would make more sense

I think I will keep the check but change it to
if not install_files:

Does that make sense?
> 
> IMAGE_PREPROCESS_COMMAND += "bootfiles_populate;"
> 
> I'm not sure if that wouldn't raise issues if any of the packages would install a file by the same path/name as also set here.
> 
> In my opinion moving it to a POSTPROCESS function with some checks that it won't overwrite any files/folders might be the better option


Looking through the documentation, I think ROOTFS_POSTPROCESS_COMMAND
could be a candidate.
I'm not sure which one I prefer though.

Check that it won't overwrite any files/folder could be a good thing. I
will implement that and raise an error if the file/folder already
exists.

Best regards,
Marcus Folkesson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [OE-core] [PATCH v4 0/2] image-bootfiles: new class

[Ross Burton]

On 30 May 2024, at 10:53, Marcus Folkesson via lists.openembedded.org <marcus.folkesson=gmail.com@lists.openembedded.org> wrote:
> The image-bootfiles class is used to put all files listed in
> IMAGE_BOOT_FILES into the root filesystem.
> 
> IMAGE_BOOT_FILES is used by the bootimg-partition wic plugin to put the
> files into a boot partition. 
> Be able to list files as "boot files" in e.g. your .conf or image files
> instead of install those in every recipe is a good thing.
> 
> It is not always desired to have a separate boot partition for boot
> files. Sometimes it could be good to have them as a part of the root
> filesystem.
> 
> For example, if a double copy strategy is used for update the system,
> then you probably want to update both the boot files and root filesystem
> at the same time as there may be dependencies.

In my mind, IMAGE_BOOT_FILES is a workaround for the fact that some /boot partitions (such as ones generated by wic) are not managed by bitbake directly.  If you have a setup where you just have a / that contains /boot isn’t adding eg grub to IMAGE_INSTALL sufficient to get it in the right place in the rootfs?

Ross

Re: [OE-core] [PATCH v4 0/2] image-bootfiles: new class

[Marcus Folkesson]

[-- Attachment #1: Type: text/plain, Size: 2241 bytes --]

Hi Ross,

On Tue, Jun 11, 2024 at 10:37:06AM +0000, Ross Burton wrote:
> On 30 May 2024, at 10:53, Marcus Folkesson via lists.openembedded.org <marcus.folkesson=gmail.com@lists.openembedded.org> wrote:
> > The image-bootfiles class is used to put all files listed in
> > IMAGE_BOOT_FILES into the root filesystem.
> > 
> > IMAGE_BOOT_FILES is used by the bootimg-partition wic plugin to put the
> > files into a boot partition. 
> > Be able to list files as "boot files" in e.g. your .conf or image files
> > instead of install those in every recipe is a good thing.
> > 
> > It is not always desired to have a separate boot partition for boot
> > files. Sometimes it could be good to have them as a part of the root
> > filesystem.
> > 
> > For example, if a double copy strategy is used for update the system,
> > then you probably want to update both the boot files and root filesystem
> > at the same time as there may be dependencies.
> 
> In my mind, IMAGE_BOOT_FILES is a workaround for the fact that some /boot partitions (such as ones generated by wic) are not managed by bitbake directly.  If you have a setup where you just have a / that contains /boot isn’t adding eg grub to IMAGE_INSTALL sufficient to get it in the right place in the rootfs?

I don't know about the workaround, but it would'nt surprise me as it is
not handled by bitbake as it is now.

For some packages yes, but not for all. If you, for instance, have an embedded
system where you depend on other files that are critical for the boot
process, there is no uniform way to specify that for those files.

IMAGE_BOOT_FILES is good as it let you include e.g. ramdisks and such
that does not have installation scripts to the boot partition.

The use case that I had was that I was first using a separate boot
partition using the bootimg-partition wic plugin. Everything was good.
Then I wanted to include those files into the root filesystem instead to
be able to do an atomic update on everything, but there is no good way
to achive that.

This image-class make the swap from bootimg-partition to rootfs
seamless as it uses the same mechanics for both implementations.

> 
> Ross
> 

Best regards,
Marcus Folkesson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [OE-core] [PATCH v4 0/2] image-bootfiles: new class

[Martin Hundebøll]

On Tue, 2024-06-11 at 15:22 +0200, Marcus Folkesson wrote:
> Hi Ross,
> 
> On Tue, Jun 11, 2024 at 10:37:06AM +0000, Ross Burton wrote:
> > On 30 May 2024, at 10:53, Marcus Folkesson via
> > lists.openembedded.org
> > <marcus.folkesson=gmail.com@lists.openembedded.org> wrote:
> > > The image-bootfiles class is used to put all files listed in
> > > IMAGE_BOOT_FILES into the root filesystem.
> > > 
> > > IMAGE_BOOT_FILES is used by the bootimg-partition wic plugin to
> > > put the
> > > files into a boot partition. 
> > > Be able to list files as "boot files" in e.g. your .conf or image
> > > files
> > > instead of install those in every recipe is a good thing.
> > > 
> > > It is not always desired to have a separate boot partition for
> > > boot
> > > files. Sometimes it could be good to have them as a part of the
> > > root
> > > filesystem.
> > > 
> > > For example, if a double copy strategy is used for update the
> > > system,
> > > then you probably want to update both the boot files and root
> > > filesystem
> > > at the same time as there may be dependencies.
> > 
> > In my mind, IMAGE_BOOT_FILES is a workaround for the fact that some
> > /boot partitions (such as ones generated by wic) are not managed by
> > bitbake directly.  If you have a setup where you just have a / that
> > contains /boot isn’t adding eg grub to IMAGE_INSTALL sufficient to
> > get it in the right place in the rootfs?
> 
> I don't know about the workaround, but it would'nt surprise me as it
> is
> not handled by bitbake as it is now.
> 
> For some packages yes, but not for all. If you, for instance, have an
> embedded
> system where you depend on other files that are critical for the boot
> process, there is no uniform way to specify that for those files.
> 
> IMAGE_BOOT_FILES is good as it let you include e.g. ramdisks and such
> that does not have installation scripts to the boot partition.
> 
> The use case that I had was that I was first using a separate boot
> partition using the bootimg-partition wic plugin. Everything was
> good.
> Then I wanted to include those files into the root filesystem instead
> to
> be able to do an atomic update on everything, but there is no good
> way
> to achive that.

I'd suggest updating the recipe providing those boot files, so it
installs them into /boot (maybe in addition to deploying them to
$DEPLOYDIR).

> This image-class make the swap from bootimg-partition to rootfs
> seamless as it uses the same mechanics for both implementations.

I have always disliked the concept of recipes pulling files from
${DEPLOY_DIR_IMAGE}. That folder is an output folder, and should not be
used as input for other packages.

Instead, a proper do_populate_deploy task similar to
do_populate_sysroot would make it possible to DEPEND += on deployed
files from other recipes.

But this is probably a discussion for another mail thread. Sorry for
raising my opinion so late in the process.

// Martin

Re: [OE-core] [PATCH v4 0/2] image-bootfiles: new class

[Marcus Folkesson]

[-- Attachment #1: Type: text/plain, Size: 3570 bytes --]

Hello Martin,

On Tue, Jun 11, 2024 at 04:44:04PM +0200, Martin Hundebøll wrote:
> On Tue, 2024-06-11 at 15:22 +0200, Marcus Folkesson wrote:
> > Hi Ross,
> > 
> > On Tue, Jun 11, 2024 at 10:37:06AM +0000, Ross Burton wrote:
> > > On 30 May 2024, at 10:53, Marcus Folkesson via
> > > lists.openembedded.org
> > > <marcus.folkesson=gmail.com@lists.openembedded.org> wrote:
> > > > The image-bootfiles class is used to put all files listed in
> > > > IMAGE_BOOT_FILES into the root filesystem.
> > > > 
> > > > IMAGE_BOOT_FILES is used by the bootimg-partition wic plugin to
> > > > put the
> > > > files into a boot partition. 
> > > > Be able to list files as "boot files" in e.g. your .conf or image
> > > > files
> > > > instead of install those in every recipe is a good thing.
> > > > 
> > > > It is not always desired to have a separate boot partition for
> > > > boot
> > > > files. Sometimes it could be good to have them as a part of the
> > > > root
> > > > filesystem.
> > > > 
> > > > For example, if a double copy strategy is used for update the
> > > > system,
> > > > then you probably want to update both the boot files and root
> > > > filesystem
> > > > at the same time as there may be dependencies.
> > > 
> > > In my mind, IMAGE_BOOT_FILES is a workaround for the fact that some
> > > /boot partitions (such as ones generated by wic) are not managed by
> > > bitbake directly.  If you have a setup where you just have a / that
> > > contains /boot isn’t adding eg grub to IMAGE_INSTALL sufficient to
> > > get it in the right place in the rootfs?
> > 
> > I don't know about the workaround, but it would'nt surprise me as it
> > is
> > not handled by bitbake as it is now.
> > 
> > For some packages yes, but not for all. If you, for instance, have an
> > embedded
> > system where you depend on other files that are critical for the boot
> > process, there is no uniform way to specify that for those files.
> > 
> > IMAGE_BOOT_FILES is good as it let you include e.g. ramdisks and such
> > that does not have installation scripts to the boot partition.
> > 
> > The use case that I had was that I was first using a separate boot
> > partition using the bootimg-partition wic plugin. Everything was
> > good.
> > Then I wanted to include those files into the root filesystem instead
> > to
> > be able to do an atomic update on everything, but there is no good
> > way
> > to achive that.
> 
> I'd suggest updating the recipe providing those boot files, so it
> installs them into /boot (maybe in addition to deploying them to
> $DEPLOYDIR).

Yes, that is the option.

> 
> > This image-class make the swap from bootimg-partition to rootfs
> > seamless as it uses the same mechanics for both implementations.
> 
> I have always disliked the concept of recipes pulling files from
> ${DEPLOY_DIR_IMAGE}. That folder is an output folder, and should not be
> used as input for other packages.
> 
> Instead, a proper do_populate_deploy task similar to
> do_populate_sysroot would make it possible to DEPEND += on deployed
> files from other recipes.

I would probably have implemented it differently if we did not already
had the bootimg-partition plugin that works this way.

I think there is a point in having them work in a similar a way.

> 
> But this is probably a discussion for another mail thread. Sorry for
> raising my opinion so late in the process.

No problem, all feedback is good. Thank you

> 
> // Martin


Best regards,
Marcus Folkesson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [OE-core] [PATCH v4 0/2] image-bootfiles: new class

[Ross Burton]

> On 11 Jun 2024, at 14:22, Marcus Folkesson <marcus.folkesson@gmail.com> wrote:
> 
> Hi Ross,
> 
> On Tue, Jun 11, 2024 at 10:37:06AM +0000, Ross Burton wrote:
>> On 30 May 2024, at 10:53, Marcus Folkesson via lists.openembedded.org <marcus.folkesson=gmail.com@lists.openembedded.org> wrote:
>>> The image-bootfiles class is used to put all files listed in
>>> IMAGE_BOOT_FILES into the root filesystem.
>>> 
>>> IMAGE_BOOT_FILES is used by the bootimg-partition wic plugin to put the
>>> files into a boot partition. 
>>> Be able to list files as "boot files" in e.g. your .conf or image files
>>> instead of install those in every recipe is a good thing.
>>> 
>>> It is not always desired to have a separate boot partition for boot
>>> files. Sometimes it could be good to have them as a part of the root
>>> filesystem.
>>> 
>>> For example, if a double copy strategy is used for update the system,
>>> then you probably want to update both the boot files and root filesystem
>>> at the same time as there may be dependencies.
>> 
>> In my mind, IMAGE_BOOT_FILES is a workaround for the fact that some /boot partitions (such as ones generated by wic) are not managed by bitbake directly.  If you have a setup where you just have a / that contains /boot isn’t adding eg grub to IMAGE_INSTALL sufficient to get it in the right place in the rootfs?
> 
> I don't know about the workaround, but it would'nt surprise me as it is
> not handled by bitbake as it is now.
> 
> For some packages yes, but not for all. If you, for instance, have an embedded
> system where you depend on other files that are critical for the boot
> process, there is no uniform way to specify that for those files.
> 
> IMAGE_BOOT_FILES is good as it let you include e.g. ramdisks and such
> that does not have installation scripts to the boot partition.
> 
> The use case that I had was that I was first using a separate boot
> partition using the bootimg-partition wic plugin. Everything was good.
> Then I wanted to include those files into the root filesystem instead to
> be able to do an atomic update on everything, but there is no good way
> to achive that.
> 
> This image-class make the swap from bootimg-partition to rootfs
> seamless as it uses the same mechanics for both implementations.

The uniform way to put files into the file system is to package them up, surely.

For example, the u-boot recipe produces a u-boot package:

$ oe-pkgdata-util list-pkg-files u-boot
u-boot:
	/boot/u-boot-qemuarm64-2024.07-r0.bin
	/boot/u-boot.bin

If I add u-boot to the image then it contains /boot/u-boot.bin.

So sorry, I still don’t understand what problem this is solving.

Cheers,
Ross

Re: [OE-core] [PATCH v4 0/2] image-bootfiles: new class

[Marcus Folkesson]

[-- Attachment #1: Type: text/plain, Size: 3278 bytes --]

Hi Ross,

On Tue, Jul 16, 2024 at 11:28:46AM +0000, Ross Burton wrote:
> 
> 
> > On 11 Jun 2024, at 14:22, Marcus Folkesson <marcus.folkesson@gmail.com> wrote:
> > 
> > Hi Ross,
> > 
> > On Tue, Jun 11, 2024 at 10:37:06AM +0000, Ross Burton wrote:
> >> On 30 May 2024, at 10:53, Marcus Folkesson via lists.openembedded.org <marcus.folkesson=gmail.com@lists.openembedded.org> wrote:
> >>> The image-bootfiles class is used to put all files listed in
> >>> IMAGE_BOOT_FILES into the root filesystem.
> >>> 
> >>> IMAGE_BOOT_FILES is used by the bootimg-partition wic plugin to put the
> >>> files into a boot partition. 
> >>> Be able to list files as "boot files" in e.g. your .conf or image files
> >>> instead of install those in every recipe is a good thing.
> >>> 
> >>> It is not always desired to have a separate boot partition for boot
> >>> files. Sometimes it could be good to have them as a part of the root
> >>> filesystem.
> >>> 
> >>> For example, if a double copy strategy is used for update the system,
> >>> then you probably want to update both the boot files and root filesystem
> >>> at the same time as there may be dependencies.
> >> 
> >> In my mind, IMAGE_BOOT_FILES is a workaround for the fact that some /boot partitions (such as ones generated by wic) are not managed by bitbake directly.  If you have a setup where you just have a / that contains /boot isn’t adding eg grub to IMAGE_INSTALL sufficient to get it in the right place in the rootfs?
> > 
> > I don't know about the workaround, but it would'nt surprise me as it is
> > not handled by bitbake as it is now.
> > 
> > For some packages yes, but not for all. If you, for instance, have an embedded
> > system where you depend on other files that are critical for the boot
> > process, there is no uniform way to specify that for those files.
> > 
> > IMAGE_BOOT_FILES is good as it let you include e.g. ramdisks and such
> > that does not have installation scripts to the boot partition.
> > 
> > The use case that I had was that I was first using a separate boot
> > partition using the bootimg-partition wic plugin. Everything was good.
> > Then I wanted to include those files into the root filesystem instead to
> > be able to do an atomic update on everything, but there is no good way
> > to achive that.
> > 
> > This image-class make the swap from bootimg-partition to rootfs
> > seamless as it uses the same mechanics for both implementations.
> 
> The uniform way to put files into the file system is to package them up, surely.
> 
> For example, the u-boot recipe produces a u-boot package:
> 
> $ oe-pkgdata-util list-pkg-files u-boot
> u-boot:
> 	/boot/u-boot-qemuarm64-2024.07-r0.bin
> 	/boot/u-boot.bin
> 
> If I add u-boot to the image then it contains /boot/u-boot.bin.
> 
> So sorry, I still don’t understand what problem this is solving.

The biggest benefit is that the bootimg-partition and image-bootfiles
becomes interchangeable, you may go from a separate boot partition with
all your boot files to whatever directory in rootfs you want. The
content will be the same, all renamed files will be the same and
everything works as before.


> 
> Cheers,
> Ross


Best regards,
Marcus Folkesson

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH v4 0/2] image-bootfiles: new class

[Marcus Folkesson]

[-- Attachment #1: Type: text/plain, Size: 1243 bytes --]

Hello

On Thu, May 30, 2024 at 11:53:12AM +0200, Marcus Folkesson wrote:
> The image-bootfiles class is used to put all files listed in
> IMAGE_BOOT_FILES into the root filesystem.
> 
> IMAGE_BOOT_FILES is used by the bootimg-partition wic plugin to put the
> files into a boot partition. 
> Be able to list files as "boot files" in e.g. your .conf or image files
> instead of install those in every recipe is a good thing.
> 
> It is not always desired to have a separate boot partition for boot
> files. Sometimes it could be good to have them as a part of the root
> filesystem.
> 
> For example, if a double copy strategy is used for update the system,
> then you probably want to update both the boot files and root filesystem
> at the same time as there may be dependencies.

I have received some feedback on this, but I would appreciate it if more
people could share their thought about it.

I still think this is a feature we should add, even though the way
this (and bootimg-partition) works has some drawbacks in terms of design.

Since the codebase is now shared between image-bootfiles
and bootimg-partition, I imagine that both are redone in a potential
re-design.

Best regards,
Marcus Folkesson


[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]