[PATCH v2 0/1] wic: extra partition plugin

[PATCH v2 0/1] wic: extra partition plugin

[pierre-loup.gosse]

From: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>

Some files are not part of the root filesystem and must be installed in a
separate partition. This is already handled for boot files by the
bootimg_partition plugin and its IMAGE_BOOT_FILES BitBake variable.

This patch introduces a similar approach for generic "extra" files. A new
extra_partition plugin is added, which populates a partition with files
listed in the new IMAGE_EXTRA_FILES BitBake variable. Like the
bootimg_partition plugin, files should be deployed to the DEPLOY_DIR_IMAGE
directory.

The plugin supports:
 - Glob pattern matching for file selection.
 - File renaming.
 - Suffixes to specify the target partition (by label, UUID, or partname),
   enabling multiple extra partitions to coexist, similar to the
   bootimg_partition plugin.

For example, with the following (simplified) wks file:
  part / --source extra_partition --fstype=ext4 --label foo --size 5M
  part / --source extra_partition --fstype=ext4 --uuid e7d0824e-cda3-4bed-9f54-9ef5312d105d --size 5M
  part / --source extra_partition --fstype=ext4 --label bar --size 5M

And this configuration:
  IMAGE_EXTRA_FILES_label-foo = "bar.conf;foo.conf"
  IMAGE_EXTRA_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d = "bar.conf;foobar.conf"
  IMAGE_EXTRA_FILES = "foo/*"
  WICVARS:append = "\
      IMAGE_EXTRA_FILES_label-foo \
      IMAGE_EXTRA_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d \
  "

The file bar.conf is installed as foo.conf in the first partition, as
foobar.conf in the second partition, and the last partition installs all
files from the foo directory.

Pierre-Loup GOSSE (1):
  wic: extra partition plugin

 meta/classes-recipe/image_types_wic.bbclass   |   1 +
 meta/lib/oeqa/selftest/cases/wic.py           |  41 ++++++
 .../lib/wic/plugins/source/extra_partition.py | 134 ++++++++++++++++++
 3 files changed, 176 insertions(+)
 create mode 100644 scripts/lib/wic/plugins/source/extra_partition.py

-- 
2.34.1

[PATCH v2 1/1] wic: extra partition plugin

[pierre-loup.gosse]

From: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>

The extra_partition plugin allows populating an extra partition with
files listed in the new IMAGE_EXTRA_FILES variable. The implementation
is similar to the bootimg_partition plugin.

This plugin provides an easy way to install files that are not part of
the rootfs.

Signed-off-by: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
Reviewed-by: Yoann CONGAL <yoann.congal@smile.fr>
---
changes in v2:
- missing signed-off-by
- typo
- assertion message
---
 meta/classes-recipe/image_types_wic.bbclass   |   1 +
 meta/lib/oeqa/selftest/cases/wic.py           |  41 ++++++
 .../lib/wic/plugins/source/extra_partition.py | 134 ++++++++++++++++++
 3 files changed, 176 insertions(+)
 create mode 100644 scripts/lib/wic/plugins/source/extra_partition.py

diff --git a/meta/classes-recipe/image_types_wic.bbclass b/meta/classes-recipe/image_types_wic.bbclass
index 6180874a4c..549a69db60 100644
--- a/meta/classes-recipe/image_types_wic.bbclass
+++ b/meta/classes-recipe/image_types_wic.bbclass
@@ -17,6 +17,7 @@ WICVARS ?= "\
 	IMAGE_BOOT_FILES \
 	IMAGE_CLASSES \
 	IMAGE_EFI_BOOT_FILES \
+	IMAGE_EXTRA_FILES \
 	IMAGE_LINK_NAME \
 	IMAGE_ROOTFS \
 	IMGDEPLOYDIR \
diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py
index b1c318bd4e..acbece4279 100644
--- a/meta/lib/oeqa/selftest/cases/wic.py
+++ b/meta/lib/oeqa/selftest/cases/wic.py
@@ -18,6 +18,7 @@ from glob import glob
 from shutil import rmtree, copy
 from tempfile import NamedTemporaryFile
 from tempfile import TemporaryDirectory
+from textwrap import dedent
 
 from oeqa.selftest.case import OESelftestTestCase
 from oeqa.core.decorator import OETestTag
@@ -1647,6 +1648,46 @@ INITRAMFS_IMAGE = "core-image-initramfs-boot"
             status, output = qemu.run_serial(cmd)
             self.assertEqual(1, status, 'Failed to run command "%s": %s' % (cmd, output))
 
+    def test_extra_partition_plugin(self):
+        """Test extra partition plugin"""
+        config = dedent("""\
+        IMAGE_EXTRA_FILES_label-foo = "bar.conf;foo.conf"
+        IMAGE_EXTRA_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d = "bar.conf;foobar.conf"
+        IMAGE_EXTRA_FILES = "foo/*"
+        WICVARS:append = "\
+            IMAGE_EXTRA_FILES_label-foo \
+            IMAGE_EXTRA_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d \
+        "
+        """)
+        self.append_config(config)
+
+        deploy_dir = get_bb_var('DEPLOY_DIR_IMAGE')
+
+        testfile = open(os.path.join(deploy_dir, "bar.conf"), "w")
+        testfile.write("test")
+        testfile.close()
+
+        os.mkdir(os.path.join(deploy_dir, "foo"))
+        testfile = open(os.path.join(deploy_dir, "foo", "bar.conf"), "w")
+        testfile.write("test")
+        testfile.close()
+
+        with NamedTemporaryFile("w", suffix=".wks") as wks:
+            wks.writelines(['part / --source extra_partition --ondisk sda --fstype=ext4 --label foo --align 4 --size 5M\n',
+                            'part / --source extra_partition --ondisk sda --fstype=ext4 --uuid e7d0824e-cda3-4bed-9f54-9ef5312d105d --align 4 --size 5M\n',
+                            'part / --source extra_partition --ondisk sda --fstype=ext4 --label bar --align 4 --size 5M\n'])
+            wks.flush()
+            _, wicimg = self._get_wic(wks.name)
+
+            result = runCmd("wic ls %s | wc -l" % wicimg)
+            self.assertEqual('4', result.output, msg="Expect 3 partitions, not %s" % result.output)
+
+            for part, file in enumerate(["foo.conf", "foobar.conf", "bar.conf"]):
+                result = runCmd("wic ls %s:%d | grep -q \"%s\"" % (wicimg, part + 1, file))
+                self.assertEqual(0, result.status, msg="File '%s' not found in the partition #%d" % (file, part))
+
+        self.remove_config(config)
+
     def test_fs_types(self):
         """Test filesystem types for empty and not empty partitions"""
         img = 'core-image-minimal'
diff --git a/scripts/lib/wic/plugins/source/extra_partition.py b/scripts/lib/wic/plugins/source/extra_partition.py
new file mode 100644
index 0000000000..499bede280
--- /dev/null
+++ b/scripts/lib/wic/plugins/source/extra_partition.py
@@ -0,0 +1,134 @@
+import logging
+import os
+import re
+
+from glob import glob
+
+from wic import WicError
+from wic.pluginbase import SourcePlugin
+from wic.misc import exec_cmd, get_bitbake_var
+
+logger = logging.getLogger('wic')
+
+class ExtraPartitionPlugin(SourcePlugin):
+    """
+    Populates an extra partition with files listed in the IMAGE_EXTRA_FILES
+    BitBake variable. Files should be deployed to the DEPLOY_DIR_IMAGE directory.
+
+    The plugin supports:
+    - Glob pattern matching for file selection.
+    - File renaming.
+    - Suffixes to specify the target partition (by label, UUID, or partname),
+      enabling multiple extra partitions to coexist.
+
+    For example:
+
+        IMAGE_EXTRA_FILES_label-foo = "bar.conf;foo.conf"
+        IMAGE_EXTRA_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d = "bar.conf;foobar.conf"
+        IMAGE_EXTRA_FILES = "foo/*"
+        WICVARS:append = "\
+            IMAGE_EXTRA_FILES_label-foo \
+            IMAGE_EXTRA_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d \
+        "
+
+    """
+
+    name = 'extra_partition'
+    image_extra_files_var_name = 'IMAGE_EXTRA_FILES'
+
+    @classmethod
+    def do_configure_partition(cls, part, source_params, cr, cr_workdir,
+                             oe_builddir, bootimg_dir, kernel_dir,
+                             native_sysroot):
+        """
+        Called before do_prepare_partition(), list the files to copy
+        """
+        extradir = "%s/extra.%d" % (cr_workdir, part.lineno)
+        install_cmd = "install -d %s" % extradir
+        exec_cmd(install_cmd)
+
+        if not kernel_dir:
+            kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
+            if not kernel_dir:
+                raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting")
+
+        extra_files = None
+        for (fmt, id) in (("_uuid-%s", part.uuid), ("_label-%s", part.label), ("_part-name-%s", part.part_name), (None, None)):
+            if fmt:
+                var = fmt % id
+            else:
+                var = ""
+            extra_files = get_bitbake_var(cls.image_extra_files_var_name + var)
+            if extra_files is not None:
+                break
+
+        if extra_files is None:
+            raise WicError('No extra files defined, %s unset for entry #%d' % (cls.image_extra_files_var_name, part.lineno))
+
+        logger.info('Extra files: %s', extra_files)
+
+        # list of tuples (src_name, dst_name)
+        deploy_files = []
+        for src_entry in re.findall(r'[\w;\-\./\*]+', extra_files):
+            if ';' in src_entry:
+                dst_entry = tuple(src_entry.split(';'))
+                if not dst_entry[0] or not dst_entry[1]:
+                    raise WicError('Malformed extra file entry: %s' % src_entry)
+            else:
+                dst_entry = (src_entry, src_entry)
+
+            logger.debug('Destination entry: %r', dst_entry)
+            deploy_files.append(dst_entry)
+
+        cls.install_task = [];
+        for deploy_entry in deploy_files:
+            src, dst = deploy_entry
+            if '*' in src:
+                # by default install files under their basename
+                entry_name_fn = os.path.basename
+                if dst != src:
+                    # unless a target name was given, then treat name
+                    # as a directory and append a basename
+                    entry_name_fn = lambda name: \
+                                    os.path.join(dst,
+                                                 os.path.basename(name))
+
+                srcs = glob(os.path.join(kernel_dir, src))
+
+                logger.debug('Globbed sources: %s', ', '.join(srcs))
+                for entry in srcs:
+                    src = os.path.relpath(entry, kernel_dir)
+                    entry_dst_name = entry_name_fn(entry)
+                    cls.install_task.append((src, entry_dst_name))
+            else:
+                cls.install_task.append((src, dst))
+
+
+    @classmethod
+    def do_prepare_partition(cls, part, source_params, cr, cr_workdir,
+                             oe_builddir, bootimg_dir, kernel_dir,
+                             rootfs_dir, native_sysroot):
+        """
+        Called to do the actual content population for a partition i.e. it
+        'prepares' the partition to be incorporated into the image.
+        In this case, we copies all files listed in IMAGE_EXTRA_FILES variable.
+        """
+        extradir = "%s/extra.%d" % (cr_workdir, part.lineno)
+
+        if not kernel_dir:
+            kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
+            if not kernel_dir:
+                raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting")
+
+        for task in cls.install_task:
+            src_path, dst_path = task
+            logger.debug('Install %s as %s', src_path, dst_path)
+            install_cmd = "install -m 0644 -D %s %s" \
+                          % (os.path.join(kernel_dir, src_path),
+                             os.path.join(extradir, dst_path))
+            exec_cmd(install_cmd)
+
+        logger.debug('Prepare extra partition using rootfs in %s', extradir)
+        part.prepare_rootfs(cr_workdir, oe_builddir, extradir,
+                            native_sysroot, False)
+
-- 
2.34.1

Re: [PATCH v2 1/1] wic: extra partition plugin

[Yoann Congal]

[-- Attachment #1: Type: text/plain, Size: 10501 bytes --]

Le jeu. 25 sept. 2025 à 09:23, <pierre-loup.gosse@smile.fr> a écrit :

> From: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
>
> The extra_partition plugin allows populating an extra partition with
> files listed in the new IMAGE_EXTRA_FILES variable. The implementation
> is similar to the bootimg_partition plugin.
>
> This plugin provides an easy way to install files that are not part of
> the rootfs.
>
> Signed-off-by: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
> Reviewed-by: Yoann CONGAL <yoann.congal@smile.fr>
>

To confirm, I indeed did review it internally.


> ---
> changes in v2:
> - missing signed-off-by
> - typo
> - assertion message
> ---
>  meta/classes-recipe/image_types_wic.bbclass   |   1 +
>  meta/lib/oeqa/selftest/cases/wic.py           |  41 ++++++
>  .../lib/wic/plugins/source/extra_partition.py | 134 ++++++++++++++++++
>  3 files changed, 176 insertions(+)
>  create mode 100644 scripts/lib/wic/plugins/source/extra_partition.py
>
> diff --git a/meta/classes-recipe/image_types_wic.bbclass
> b/meta/classes-recipe/image_types_wic.bbclass
> index 6180874a4c..549a69db60 100644
> --- a/meta/classes-recipe/image_types_wic.bbclass
> +++ b/meta/classes-recipe/image_types_wic.bbclass
> @@ -17,6 +17,7 @@ WICVARS ?= "\
>         IMAGE_BOOT_FILES \
>         IMAGE_CLASSES \
>         IMAGE_EFI_BOOT_FILES \
> +       IMAGE_EXTRA_FILES \
>         IMAGE_LINK_NAME \
>         IMAGE_ROOTFS \
>         IMGDEPLOYDIR \
> diff --git a/meta/lib/oeqa/selftest/cases/wic.py
> b/meta/lib/oeqa/selftest/cases/wic.py
> index b1c318bd4e..acbece4279 100644
> --- a/meta/lib/oeqa/selftest/cases/wic.py
> +++ b/meta/lib/oeqa/selftest/cases/wic.py
> @@ -18,6 +18,7 @@ from glob import glob
>  from shutil import rmtree, copy
>  from tempfile import NamedTemporaryFile
>  from tempfile import TemporaryDirectory
> +from textwrap import dedent
>
>  from oeqa.selftest.case import OESelftestTestCase
>  from oeqa.core.decorator import OETestTag
> @@ -1647,6 +1648,46 @@ INITRAMFS_IMAGE = "core-image-initramfs-boot"
>              status, output = qemu.run_serial(cmd)
>              self.assertEqual(1, status, 'Failed to run command "%s": %s'
> % (cmd, output))
>
> +    def test_extra_partition_plugin(self):
> +        """Test extra partition plugin"""
> +        config = dedent("""\
> +        IMAGE_EXTRA_FILES_label-foo = "bar.conf;foo.conf"
> +        IMAGE_EXTRA_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d =
> "bar.conf;foobar.conf"
> +        IMAGE_EXTRA_FILES = "foo/*"
> +        WICVARS:append = "\
> +            IMAGE_EXTRA_FILES_label-foo \
> +            IMAGE_EXTRA_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d \
> +        "
> +        """)
> +        self.append_config(config)
> +
> +        deploy_dir = get_bb_var('DEPLOY_DIR_IMAGE')
> +
> +        testfile = open(os.path.join(deploy_dir, "bar.conf"), "w")
> +        testfile.write("test")
> +        testfile.close()
> +
> +        os.mkdir(os.path.join(deploy_dir, "foo"))
> +        testfile = open(os.path.join(deploy_dir, "foo", "bar.conf"), "w")
> +        testfile.write("test")
> +        testfile.close()
> +
> +        with NamedTemporaryFile("w", suffix=".wks") as wks:
> +            wks.writelines(['part / --source extra_partition --ondisk sda
> --fstype=ext4 --label foo --align 4 --size 5M\n',
> +                            'part / --source extra_partition --ondisk sda
> --fstype=ext4 --uuid e7d0824e-cda3-4bed-9f54-9ef5312d105d --align 4 --size
> 5M\n',
> +                            'part / --source extra_partition --ondisk sda
> --fstype=ext4 --label bar --align 4 --size 5M\n'])
> +            wks.flush()
> +            _, wicimg = self._get_wic(wks.name)
> +
> +            result = runCmd("wic ls %s | wc -l" % wicimg)
> +            self.assertEqual('4', result.output, msg="Expect 3
> partitions, not %s" % result.output)
> +
> +            for part, file in enumerate(["foo.conf", "foobar.conf",
> "bar.conf"]):
> +                result = runCmd("wic ls %s:%d | grep -q \"%s\"" %
> (wicimg, part + 1, file))
> +                self.assertEqual(0, result.status, msg="File '%s' not
> found in the partition #%d" % (file, part))
> +
> +        self.remove_config(config)
> +
>      def test_fs_types(self):
>          """Test filesystem types for empty and not empty partitions"""
>          img = 'core-image-minimal'
> diff --git a/scripts/lib/wic/plugins/source/extra_partition.py
> b/scripts/lib/wic/plugins/source/extra_partition.py
> new file mode 100644
> index 0000000000..499bede280
> --- /dev/null
> +++ b/scripts/lib/wic/plugins/source/extra_partition.py
> @@ -0,0 +1,134 @@
> +import logging
> +import os
> +import re
> +
> +from glob import glob
> +
> +from wic import WicError
> +from wic.pluginbase import SourcePlugin
> +from wic.misc import exec_cmd, get_bitbake_var
> +
> +logger = logging.getLogger('wic')
> +
> +class ExtraPartitionPlugin(SourcePlugin):
> +    """
> +    Populates an extra partition with files listed in the
> IMAGE_EXTRA_FILES
> +    BitBake variable. Files should be deployed to the DEPLOY_DIR_IMAGE
> directory.
> +
> +    The plugin supports:
> +    - Glob pattern matching for file selection.
> +    - File renaming.
> +    - Suffixes to specify the target partition (by label, UUID, or
> partname),
> +      enabling multiple extra partitions to coexist.
> +
> +    For example:
> +
> +        IMAGE_EXTRA_FILES_label-foo = "bar.conf;foo.conf"
> +        IMAGE_EXTRA_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d =
> "bar.conf;foobar.conf"
> +        IMAGE_EXTRA_FILES = "foo/*"
> +        WICVARS:append = "\
> +            IMAGE_EXTRA_FILES_label-foo \
> +            IMAGE_EXTRA_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d \
> +        "
> +
> +    """
> +
> +    name = 'extra_partition'
> +    image_extra_files_var_name = 'IMAGE_EXTRA_FILES'
> +
> +    @classmethod
> +    def do_configure_partition(cls, part, source_params, cr, cr_workdir,
> +                             oe_builddir, bootimg_dir, kernel_dir,
> +                             native_sysroot):
> +        """
> +        Called before do_prepare_partition(), list the files to copy
> +        """
> +        extradir = "%s/extra.%d" % (cr_workdir, part.lineno)
> +        install_cmd = "install -d %s" % extradir
> +        exec_cmd(install_cmd)
> +
> +        if not kernel_dir:
> +            kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
> +            if not kernel_dir:
> +                raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting")
> +
> +        extra_files = None
> +        for (fmt, id) in (("_uuid-%s", part.uuid), ("_label-%s",
> part.label), ("_part-name-%s", part.part_name), (None, None)):
> +            if fmt:
> +                var = fmt % id
> +            else:
> +                var = ""
> +            extra_files = get_bitbake_var(cls.image_extra_files_var_name
> + var)
> +            if extra_files is not None:
> +                break
> +
> +        if extra_files is None:
> +            raise WicError('No extra files defined, %s unset for entry
> #%d' % (cls.image_extra_files_var_name, part.lineno))
> +
> +        logger.info('Extra files: %s', extra_files)
> +
> +        # list of tuples (src_name, dst_name)
> +        deploy_files = []
> +        for src_entry in re.findall(r'[\w;\-\./\*]+', extra_files):
> +            if ';' in src_entry:
> +                dst_entry = tuple(src_entry.split(';'))
> +                if not dst_entry[0] or not dst_entry[1]:
> +                    raise WicError('Malformed extra file entry: %s' %
> src_entry)
> +            else:
> +                dst_entry = (src_entry, src_entry)
> +
> +            logger.debug('Destination entry: %r', dst_entry)
> +            deploy_files.append(dst_entry)
> +
> +        cls.install_task = [];
> +        for deploy_entry in deploy_files:
> +            src, dst = deploy_entry
> +            if '*' in src:
> +                # by default install files under their basename
> +                entry_name_fn = os.path.basename
> +                if dst != src:
> +                    # unless a target name was given, then treat name
> +                    # as a directory and append a basename
> +                    entry_name_fn = lambda name: \
> +                                    os.path.join(dst,
> +                                                 os.path.basename(name))
> +
> +                srcs = glob(os.path.join(kernel_dir, src))
> +
> +                logger.debug('Globbed sources: %s', ', '.join(srcs))
> +                for entry in srcs:
> +                    src = os.path.relpath(entry, kernel_dir)
> +                    entry_dst_name = entry_name_fn(entry)
> +                    cls.install_task.append((src, entry_dst_name))
> +            else:
> +                cls.install_task.append((src, dst))
> +
> +
> +    @classmethod
> +    def do_prepare_partition(cls, part, source_params, cr, cr_workdir,
> +                             oe_builddir, bootimg_dir, kernel_dir,
> +                             rootfs_dir, native_sysroot):
> +        """
> +        Called to do the actual content population for a partition i.e. it
> +        'prepares' the partition to be incorporated into the image.
> +        In this case, we copies all files listed in IMAGE_EXTRA_FILES
> variable.
> +        """
> +        extradir = "%s/extra.%d" % (cr_workdir, part.lineno)
> +
> +        if not kernel_dir:
> +            kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
> +            if not kernel_dir:
> +                raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting")
> +
> +        for task in cls.install_task:
> +            src_path, dst_path = task
> +            logger.debug('Install %s as %s', src_path, dst_path)
> +            install_cmd = "install -m 0644 -D %s %s" \
> +                          % (os.path.join(kernel_dir, src_path),
> +                             os.path.join(extradir, dst_path))
> +            exec_cmd(install_cmd)
> +
> +        logger.debug('Prepare extra partition using rootfs in %s',
> extradir)
> +        part.prepare_rootfs(cr_workdir, oe_builddir, extradir,
> +                            native_sysroot, False)
> +
> --
> 2.34.1
>
>

-- 
Yoann Congal
Smile ECS

[-- Attachment #2: Type: text/html, Size: 13568 bytes --]

Re: [OE-core] [PATCH v2 1/1] wic: extra partition plugin

[Alexander Kanavin]

On Thu, 25 Sept 2025 at 09:56, Yoann Congal via lists.openembedded.org
<yoann.congal=smile.fr@lists.openembedded.org> wrote:

>> The extra_partition plugin allows populating an extra partition with
>> files listed in the new IMAGE_EXTRA_FILES variable. The implementation
>> is similar to the bootimg_partition plugin.
>>
>> This plugin provides an easy way to install files that are not part of
>> the rootfs.
>>
>> Signed-off-by: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
>> Reviewed-by: Yoann CONGAL <yoann.congal@smile.fr>
>
>
> To confirm, I indeed did review it internally.

I don't object to the plugin, but this looks like a very common use
case. How have people been handling it up to now? I'm sure I saw
builds and products with such 'extra' partitions, but I can't for the
life of me remember how those were configured. Did they all write
custom wic plugins for themselves?

Alex

Re: [OE-core] [PATCH v2 1/1] wic: extra partition plugin

[Pierre-loup GOSSE]

[-- Attachment #1: Type: text/plain, Size: 1374 bytes --]

Maybe these 'extra' partitions are populated at runtime (first boot) from
the rootfs?
I wrote this plugin for a project because I couldn’t find an easy way like
bootimg_partition to install 'extra' files at build time.
If a similar solution already exists in Yocto, I’d be interested to know.

Pierre-Loup,

On Thu, Sep 25, 2025 at 10:23 AM Alexander Kanavin <alex.kanavin@gmail.com>
wrote:

> On Thu, 25 Sept 2025 at 09:56, Yoann Congal via lists.openembedded.org
> <yoann.congal=smile.fr@lists.openembedded.org> wrote:
>
> >> The extra_partition plugin allows populating an extra partition with
> >> files listed in the new IMAGE_EXTRA_FILES variable. The implementation
> >> is similar to the bootimg_partition plugin.
> >>
> >> This plugin provides an easy way to install files that are not part of
> >> the rootfs.
> >>
> >> Signed-off-by: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
> >> Reviewed-by: Yoann CONGAL <yoann.congal@smile.fr>
> >
> >
> > To confirm, I indeed did review it internally.
>
> I don't object to the plugin, but this looks like a very common use
> case. How have people been handling it up to now? I'm sure I saw
> builds and products with such 'extra' partitions, but I can't for the
> life of me remember how those were configured. Did they all write
> custom wic plugins for themselves?
>
> Alex
>

[-- Attachment #2: Type: text/html, Size: 2327 bytes --]

Re: [OE-core] [PATCH v2 1/1] wic: extra partition plugin

[Peter Bergin]

[-- Attachment #1: Type: text/plain, Size: 872 bytes --]

Hi,

On 9/25/25 11:13, Pierre-loup GOSSE via lists.openembedded.org wrote:
> Maybe these 'extra' partitions are populated at runtime (first boot) 
> from the rootfs?
> I wrote this plugin for a project because I couldn’t find an easy way 
> like |bootimg_partition to install 'extra' files at build time.|
> If a similar solution already exists in Yocto, I’d be interested to know.
>
in the past I have used to rootfs plugin with '--exclude-path' [1] and 
'--change-directory' [2] assuming that I have isolated all files going 
in to the extra partition in to one directory of rootfs.

   part / --source rootfs --exclude-path=config/ ...
   part /config --source rootfs --change-directory=config

[1] https://git.yoctoproject.org/poky/tree/scripts/lib/wic/help.py#n987

[2] https://git.yoctoproject.org/poky/tree/scripts/lib/wic/help.py#n1007

Best regards,
/Peter

[-- Attachment #2: Type: text/html, Size: 1929 bytes --]

Re: [OE-core] [PATCH v2 1/1] wic: extra partition plugin

[Pierre-loup GOSSE]

[-- Attachment #1: Type: text/plain, Size: 1662 bytes --]

>
> in the past I have used to rootfs plugin with '--exclude-path' [1] and
> '--change-directory' [2] assuming that I have isolated all files going in
> to the extra partition in to one directory of rootfs.

This is a solution when files are installed in the rootfs but need to be
split across multiple partitions.

However, some files aren't or shouldn’t be in the rootfs. For instance,
I’ve worked on boards requiring a dedicated VFAT partition just for the
U-Boot script. Other use cases include keeping business software separate
from the rootfs, especially when they are installed and updated
independently (with a SWU package).
Pierre-Loup,

On Thu, Sep 25, 2025 at 12:58 PM Peter Bergin <peter@berginkonsult.se>
wrote:

> Hi,
> On 9/25/25 11:13, Pierre-loup GOSSE via lists.openembedded.org wrote:
>
> Maybe these 'extra' partitions are populated at runtime (first boot) from
> the rootfs?
> I wrote this plugin for a project because I couldn’t find an easy way like
> bootimg_partition to install 'extra' files at build time.
> If a similar solution already exists in Yocto, I’d be interested to know.
>
> in the past I have used to rootfs plugin with '--exclude-path' [1] and
> '--change-directory' [2] assuming that I have isolated all files going in
> to the extra partition in to one directory of rootfs.
>
>   part / --source rootfs --exclude-path=config/ ...
>   part /config --source rootfs --change-directory=config
>
> [1] https://git.yoctoproject.org/poky/tree/scripts/lib/wic/help.py#n987
>
> [2] https://git.yoctoproject.org/poky/tree/scripts/lib/wic/help.py#n1007
> Best regards,
> /Peter
>

[-- Attachment #2: Type: text/html, Size: 2936 bytes --]

Re: [OE-core] [PATCH v2 1/1] wic: extra partition plugin

[Antonin Godard]

On Thu Sep 25, 2025 at 2:43 PM CEST, Pierre-loup GOSSE via lists.openembedded.org wrote:
>>
>> in the past I have used to rootfs plugin with '--exclude-path' [1] and
>> '--change-directory' [2] assuming that I have isolated all files going in
>> to the extra partition in to one directory of rootfs.
>
> This is a solution when files are installed in the rootfs but need to be
> split across multiple partitions.
>
> However, some files aren't or shouldn’t be in the rootfs. For instance,
> I’ve worked on boards requiring a dedicated VFAT partition just for the
> U-Boot script. Other use cases include keeping business software separate
> from the rootfs, especially when they are installed and updated
> independently (with a SWU package).
> Pierre-Loup,

Why not install all the files in IMAGE_ROOTFS in a single image, then do the
filtering in the WKS?

What I do is:

  part rootfs --source rootfs --fstype=ext4 --exclude-path=data/ ...
  part data --source rootfs --rootfs-dir=${IMAGE_ROOTFS}/data ...

In this case the rootfs partition will not include the files in /data, and the
data partition will include them.

Antonin

-- 
Antonin Godard, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH v2 0/1] wic: extra partition plugin

[Mathieu Dubois-Briand]

On Thu Sep 25, 2025 at 9:22 AM CEST, Pierre-loup GOSSE via lists.openembedded.org wrote:
> From: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
>
> Some files are not part of the root filesystem and must be installed in a
> separate partition. This is already handled for boot files by the
> bootimg_partition plugin and its IMAGE_BOOT_FILES BitBake variable.
>
> This patch introduces a similar approach for generic "extra" files. A new
> extra_partition plugin is added, which populates a partition with files
> listed in the new IMAGE_EXTRA_FILES BitBake variable. Like the
> bootimg_partition plugin, files should be deployed to the DEPLOY_DIR_IMAGE
> directory.
>
> The plugin supports:
>  - Glob pattern matching for file selection.
>  - File renaming.
>  - Suffixes to specify the target partition (by label, UUID, or partname),
>    enabling multiple extra partitions to coexist, similar to the
>    bootimg_partition plugin.
>
> For example, with the following (simplified) wks file:
>   part / --source extra_partition --fstype=ext4 --label foo --size 5M
>   part / --source extra_partition --fstype=ext4 --uuid e7d0824e-cda3-4bed-9f54-9ef5312d105d --size 5M
>   part / --source extra_partition --fstype=ext4 --label bar --size 5M
>
> And this configuration:
>   IMAGE_EXTRA_FILES_label-foo = "bar.conf;foo.conf"
>   IMAGE_EXTRA_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d = "bar.conf;foobar.conf"
>   IMAGE_EXTRA_FILES = "foo/*"
>   WICVARS:append = "\
>       IMAGE_EXTRA_FILES_label-foo \
>       IMAGE_EXTRA_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d \
>   "
>
> The file bar.conf is installed as foo.conf in the first partition, as
> foobar.conf in the second partition, and the last partition installs all
> files from the foo directory.
>
> Pierre-Loup GOSSE (1):
>   wic: extra partition plugin
>
>  meta/classes-recipe/image_types_wic.bbclass   |   1 +
>  meta/lib/oeqa/selftest/cases/wic.py           |  41 ++++++
>  .../lib/wic/plugins/source/extra_partition.py | 134 ++++++++++++++++++
>  3 files changed, 176 insertions(+)
>  create mode 100644 scripts/lib/wic/plugins/source/extra_partition.py

Hi Pierre-Loup,

Thanks for your patch.

It looks like this is breaking the wic.Wic2.test_image_env selftest:

2025-09-25 18:08:01,857 - oe-selftest - INFO - 9: 80/93 623/638 (54.59s) (0 failed) (wic.Wic2.test_image_env)
2025-09-25 18:08:01,858 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
  File "/srv/pokybuild/yocto-worker/oe-selftest-armhost/build/meta/lib/oeqa/selftest/cases/wic.py", line 1030, in test_image_env
    self.assertTrue(var in content, "%s is not in .env file" % var)
    ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3.13/unittest/case.py", line 744, in assertTrue
    raise self.failureException(msg)
AssertionError: False is not true : IMAGE_EXTRA_FILES is not in .env file
...
2025-09-25 18:18:17,328 - oe-selftest - INFO - RESULTS - wic.Wic2.test_image_env: FAILED (54.59s)

Can you make sure the test passes?

Thanks,
Mathieu

-- 
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com

Re: [OE-core] [PATCH v2 1/1] wic: extra partition plugin

[Pierre-loup GOSSE]

[-- Attachment #1: Type: text/plain, Size: 2265 bytes --]

>
> Why not install all the files in IMAGE_ROOTFS in a single image, then do
> the
> filtering in the WKS?


This isn’t a suitable solution if you don’t want the files included in the
rootfs image initially. Perhaps my earlier description wasn’t clear, let me
clarify with a practical example.
I use SWUpdate to update the rootfs and business software separately. The
business software shouldn't be part of the rootfs, as it would otherwise be
bundled into the rootfs SWU package. However, I still need to install it in
the output WIC image, specifically in an 'app' partition, and update it
later via the app SWU package.

Based on the bootimg_partition plugin, I wrote this plugin to handle these
extra files, that aren't part of the rootfs but are fetched and deployed in
the DEPLOY_DIR_IMAGE.

Pierre-Loup,



On Thu, Sep 25, 2025 at 5:20 PM Antonin Godard <antonin.godard@bootlin.com>
wrote:

> On Thu Sep 25, 2025 at 2:43 PM CEST, Pierre-loup GOSSE via
> lists.openembedded.org wrote:
> >>
> >> in the past I have used to rootfs plugin with '--exclude-path' [1] and
> >> '--change-directory' [2] assuming that I have isolated all files going
> in
> >> to the extra partition in to one directory of rootfs.
> >
> > This is a solution when files are installed in the rootfs but need to be
> > split across multiple partitions.
> >
> > However, some files aren't or shouldn’t be in the rootfs. For instance,
> > I’ve worked on boards requiring a dedicated VFAT partition just for the
> > U-Boot script. Other use cases include keeping business software separate
> > from the rootfs, especially when they are installed and updated
> > independently (with a SWU package).
> > Pierre-Loup,
>
> Why not install all the files in IMAGE_ROOTFS in a single image, then do
> the
> filtering in the WKS?
>
> What I do is:
>
>   part rootfs --source rootfs --fstype=ext4 --exclude-path=data/ ...
>   part data --source rootfs --rootfs-dir=${IMAGE_ROOTFS}/data ...
>
> In this case the rootfs partition will not include the files in /data, and
> the
> data partition will include them.
>
> Antonin
>
> --
> Antonin Godard, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>
>

[-- Attachment #2: Type: text/html, Size: 3112 bytes --]