* [PATCH v2 0/2] wic: extra-partitions: new variable to be combined with sourceparams
@ 2026-01-12 8:29 Louis Rannou via B4 Relay
2026-01-12 8:29 ` [PATCH v2 1/2] oeqa/selftest: wic: improve extra-partition plugin tests Louis Rannou via B4 Relay
2026-01-12 8:29 ` [PATCH v2 2/2] wic: extra-partition: introduce variable matching sourceparams Louis Rannou via B4 Relay
0 siblings, 2 replies; 4+ messages in thread
From: Louis Rannou via B4 Relay @ 2026-01-12 8:29 UTC (permalink / raw)
To: openembedded-core; +Cc: Louis Rannou, pierre-loup.gosse
The current extra_partition plugin uses predictable variable names (the
configuration must predict the image partition label or uuid). This has two
inconveniences:
- an extra partition that has a specific label (or uuid) is doomed with the
associated variable. The partition cannot be configured unless the label
(or uuid) is changed.
- two partitions with same label from distincts .wks cannot have different
content.
The first commit increases the tests. It now tests several files can be included
and that the file system type can be specified. It also simplifies the 'wic ls'
command by removing unneeded pipes.
The second commit allows the new suffix _name-%s to be appended to
IMAGE_EXTRA_PARTITION_FILES. This variable will be preferred whenever the
.wks specifies a name in the --sourceparams option.
For example:
an image may define:
IMAGE_EXTRA_PARTITION_FILES_name-foo = "bar.conf"
and the .wks line have line:
part --source extra_partition --sourceparams "name=foo"
Signed-off-by: Louis Rannou <louis.rannou@non.se.com>
---
Changes in v2:
- change suuffix _sourceparam-%s to _name-%s
- Link to v1: https://lore.kernel.org/r/20260107-extrafiles-v1-0-a4ddd07df231@non.se.com
---
Louis Rannou (2):
oeqa/selftest: wic: improve extra-partition plugin tests
wic: extra-partition: introduce variable matching sourceparams
meta/lib/oeqa/selftest/cases/wic.py | 55 ++++++++++++++++-------
scripts/lib/wic/plugins/source/extra_partition.py | 24 +++++++---
2 files changed, 57 insertions(+), 22 deletions(-)
---
base-commit: 0ed4776533145c901611d7f4f942490e33481335
change-id: 20260107-extrafiles-ad9a25529b20
Best regards,
--
Louis Rannou <louis.rannou@non.se.com>
^ permalink raw reply [flat|nested] 4+ messages in thread* [PATCH v2 1/2] oeqa/selftest: wic: improve extra-partition plugin tests 2026-01-12 8:29 [PATCH v2 0/2] wic: extra-partitions: new variable to be combined with sourceparams Louis Rannou via B4 Relay @ 2026-01-12 8:29 ` Louis Rannou via B4 Relay 2026-01-12 8:29 ` [PATCH v2 2/2] wic: extra-partition: introduce variable matching sourceparams Louis Rannou via B4 Relay 1 sibling, 0 replies; 4+ messages in thread From: Louis Rannou via B4 Relay @ 2026-01-12 8:29 UTC (permalink / raw) To: openembedded-core; +Cc: Louis Rannou, pierre-loup.gosse From: Louis Rannou <louis.rannou@non.se.com> Increase the number of inputs to check several files can be copied. Change the partition type to GPT in order to avoid logical partitions. Test various file systems and check the wic image partitions match. Simplify 'wic ls' commands (remove pipes) and use '-n' to specify wic-tools sysroot and avoid host contamination. Also dedent the code just after the wic image is built to close the .wks temporary file when its goal is achieved. Signed-off-by: Louis Rannou <louis.rannou@non.se.com> --- meta/lib/oeqa/selftest/cases/wic.py | 47 +++++++++++++++++++++++++------------ 1 file changed, 32 insertions(+), 15 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py index d7a9b14658..55fb8dfd1b 100644 --- a/meta/lib/oeqa/selftest/cases/wic.py +++ b/meta/lib/oeqa/selftest/cases/wic.py @@ -1658,7 +1658,7 @@ INITRAMFS_IMAGE = "core-image-initramfs-boot" """Test extra partition plugin""" config = dedent("""\ IMAGE_EXTRA_PARTITION_FILES_label-foo = "bar.conf;foo.conf" - IMAGE_EXTRA_PARTITION_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d = "bar.conf;foobar.conf" + IMAGE_EXTRA_PARTITION_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d = "bar.conf;foobar.conf bar2.conf;foobar2.conf bar3.conf bar4.conf" IMAGE_EXTRA_PARTITION_FILES = "foo/*" WICVARS:append = "\ IMAGE_EXTRA_PARTITION_FILES_label-foo \ @@ -1668,36 +1668,53 @@ INITRAMFS_IMAGE = "core-image-initramfs-boot" self.append_config(config) deploy_dir = get_bb_var('DEPLOY_DIR_IMAGE') + sysroot = get_bb_var('RECIPE_SYSROOT_NATIVE', 'wic-tools') - testfile = open(os.path.join(deploy_dir, "bar.conf"), "w") - testfile.write("test") - testfile.close() + # Write test files + for testfilename in ["bar.conf", "bar2.conf", "bar3.conf", "bar4.conf"]: + testfile = open(os.path.join(deploy_dir, testfilename), "w") + testfile.write("test %s" % testfilename) + testfile.close() + # Create directory foo/ and testfiles within os.mkdir(os.path.join(deploy_dir, "foo")) - testfile = open(os.path.join(deploy_dir, "foo", "bar.conf"), "w") - testfile.write("test") - testfile.close() + for testfilename in ["bar.conf", "bar2.conf"]: + testfile = open(os.path.join(deploy_dir, "foo", testfilename), "w") + testfile.write("test %s" % testfilename) + testfile.close() oldpath = os.environ['PATH'] os.environ['PATH'] = get_bb_var("PATH", "wic-tools") try: with NamedTemporaryFile("w", suffix=".wks") as wks: - wks.writelines(['part / --source extra_partition --ondisk sda --fstype=ext4 --label foo --align 4 --size 5M\n', - 'part / --source extra_partition --ondisk sda --fstype=ext4 --uuid e7d0824e-cda3-4bed-9f54-9ef5312d105d --align 4 --size 5M\n', + wks.writelines(['part / --source extra_partition --ondisk sda --label foo --align 4 --size 5M\n', + 'part / --source extra_partition --ondisk sda --fstype=vfat --uuid e7d0824e-cda3-4bed-9f54-9ef5312d105d --align 4 --size 5M\n', 'part / --source extra_partition --ondisk sda --fstype=ext4 --label bar --align 4 --size 5M\n']) wks.flush() _, wicimg = self._get_wic(wks.name) - result = runCmd("wic ls %s | wc -l" % wicimg) - self.assertEqual('4', result.output, msg="Expect 3 partitions, not %s" % result.output) + result = runCmd("wic ls %s -n %s" % (wicimg, sysroot)) + partls = result.output.split('\n')[1:] - for part, file in enumerate(["foo.conf", "foobar.conf", "bar.conf"]): - result = runCmd("wic ls %s:%d | grep -q \"%s\"" % (wicimg, part + 1, file)) - self.assertEqual(0, result.status, msg="File '%s' not found in the partition #%d" % (file, part)) + # Assert the number of partitions is correct + self.assertEqual(3, len(partls), msg="Expect 3 partitions, not %s" % result.output) - self.remove_config(config) + # Fstype column from 'wic ls' should be fstype as given in the part command + for part_id, part_fs in enumerate(["fat16", "fat16", "ext4"]): + self.assertIn(part_fs, partls[part_id]) + # For each partition, assert expected files exist + for part, part_glob in enumerate([ + ["foo.conf"], + ["foobar.conf", "foobar2.conf", "bar3.conf", "bar4.conf"], + ["bar.conf", "bar2.conf"], + ]): + for part_file in part_glob: + result = runCmd("wic ls %s:%d/%s -n %s" % (wicimg, part + 1, part_file, sysroot)) + self.assertEqual(0, result.status, msg="File '%s' not found in the partition #%d" % (part_file, part)) + + self.remove_config(config) finally: os.environ['PATH'] = oldpath -- 2.43.0 ^ permalink raw reply related [flat|nested] 4+ messages in thread
* [PATCH v2 2/2] wic: extra-partition: introduce variable matching sourceparams 2026-01-12 8:29 [PATCH v2 0/2] wic: extra-partitions: new variable to be combined with sourceparams Louis Rannou via B4 Relay 2026-01-12 8:29 ` [PATCH v2 1/2] oeqa/selftest: wic: improve extra-partition plugin tests Louis Rannou via B4 Relay @ 2026-01-12 8:29 ` Louis Rannou via B4 Relay 2026-01-12 16:20 ` Pierre-loup GOSSE 1 sibling, 1 reply; 4+ messages in thread From: Louis Rannou via B4 Relay @ 2026-01-12 8:29 UTC (permalink / raw) To: openembedded-core; +Cc: Louis Rannou, pierre-loup.gosse From: Louis Rannou <louis.rannou@non.se.com> The current extra_partition plugin uses predictable variable names (the configuration must predict the image partition label or uuid). This has two inconveniences: - an extra partition that has a specific label (or uuid) is doomed with the associated variable. The partition cannot be configured unless the label (or uuid) is changed. - two partitions with same label from distincts .wks cannot have different content. This allows the new suffix _name_%s to be appended to IMAGE_EXTRA_PARTITION_FILES. This variable will be preferred whenever the .wks specifies a name in the --sourceparams option. For example: an image may define: IMAGE_EXTRA_PARTITION_FILES_name_foo = "bar.conf" and the .wks line have line: part --source extra_partition --sourceparams "name=foo" Signed-off-by: Louis Rannou <louis.rannou@non.se.com> --- meta/lib/oeqa/selftest/cases/wic.py | 16 ++++++++++----- scripts/lib/wic/plugins/source/extra_partition.py | 24 +++++++++++++++++------ 2 files changed, 29 insertions(+), 11 deletions(-) diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py index 55fb8dfd1b..409437a5c0 100644 --- a/meta/lib/oeqa/selftest/cases/wic.py +++ b/meta/lib/oeqa/selftest/cases/wic.py @@ -1657,6 +1657,7 @@ INITRAMFS_IMAGE = "core-image-initramfs-boot" def test_extra_partition_plugin(self): """Test extra partition plugin""" config = dedent("""\ + IMAGE_EXTRA_PARTITION_FILES_name-foo = "bar.conf" IMAGE_EXTRA_PARTITION_FILES_label-foo = "bar.conf;foo.conf" IMAGE_EXTRA_PARTITION_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d = "bar.conf;foobar.conf bar2.conf;foobar2.conf bar3.conf bar4.conf" IMAGE_EXTRA_PARTITION_FILES = "foo/*" @@ -1688,9 +1689,13 @@ INITRAMFS_IMAGE = "core-image-initramfs-boot" try: with NamedTemporaryFile("w", suffix=".wks") as wks: - wks.writelines(['part / --source extra_partition --ondisk sda --label foo --align 4 --size 5M\n', - 'part / --source extra_partition --ondisk sda --fstype=vfat --uuid e7d0824e-cda3-4bed-9f54-9ef5312d105d --align 4 --size 5M\n', - 'part / --source extra_partition --ondisk sda --fstype=ext4 --label bar --align 4 --size 5M\n']) + wks.writelines([ + 'part / --source extra_partition --ondisk sda --sourceparams "name=foo" --align 4 --size 5M\n', + 'part / --source extra_partition --ondisk sda --label foo --align 4 --size 5M\n', + 'part / --source extra_partition --ondisk sda --fstype=vfat --uuid e7d0824e-cda3-4bed-9f54-9ef5312d105d --align 4 --size 5M\n', + 'part / --source extra_partition --ondisk sda --fstype=ext4 --label bar --align 4 --size 5M\n', + 'bootloader --ptable gpt\n', + ]) wks.flush() _, wicimg = self._get_wic(wks.name) @@ -1698,14 +1703,15 @@ INITRAMFS_IMAGE = "core-image-initramfs-boot" partls = result.output.split('\n')[1:] # Assert the number of partitions is correct - self.assertEqual(3, len(partls), msg="Expect 3 partitions, not %s" % result.output) + self.assertEqual(4, len(partls), msg="Expect 4 partitions, not %s" % result.output) # Fstype column from 'wic ls' should be fstype as given in the part command - for part_id, part_fs in enumerate(["fat16", "fat16", "ext4"]): + for part_id, part_fs in enumerate(["fat16", "fat16", "fat16", "ext4"]): self.assertIn(part_fs, partls[part_id]) # For each partition, assert expected files exist for part, part_glob in enumerate([ + ["bar.conf"], ["foo.conf"], ["foobar.conf", "foobar2.conf", "bar3.conf", "bar4.conf"], ["bar.conf", "bar2.conf"], diff --git a/scripts/lib/wic/plugins/source/extra_partition.py b/scripts/lib/wic/plugins/source/extra_partition.py index d370b0107e..0d49e2bc7c 100644 --- a/scripts/lib/wic/plugins/source/extra_partition.py +++ b/scripts/lib/wic/plugins/source/extra_partition.py @@ -18,15 +18,17 @@ class ExtraPartitionPlugin(SourcePlugin): The plugin supports: - Glob pattern matching for file selection. - File renaming. - - Suffixes to specify the target partition (by label, UUID, or partname), - enabling multiple extra partitions to coexist. + - Suffixes to specify the target partition (by params-name, label, UUID, + or partname), enabling multiple extra partitions to coexist. For example: + IMAGE_EXTRA_PARTITION_FILES_name-randomname = "bar.conf;foo.conf" IMAGE_EXTRA_PARTITION_FILES_label-foo = "bar.conf;foo.conf" IMAGE_EXTRA_PARTITION_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d = "bar.conf;foobar.conf" IMAGE_EXTRA_PARTITION_FILES = "foo/*" WICVARS:append = "\ + IMAGE_EXTRA_PARTITION_FILES_name-randomname \ IMAGE_EXTRA_PARTITION_FILES_label-foo \ IMAGE_EXTRA_PARTITION_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d \ " @@ -53,11 +55,21 @@ class ExtraPartitionPlugin(SourcePlugin): raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting") extra_files = None - for (fmt, id) in (("_uuid-%s", part.uuid), ("_label-%s", part.label), ("_part-name-%s", part.part_name), (None, None)): - if fmt: - var = fmt % id - else: + for (fmt, part_id) in ( + ("_name-%s", source_params.get("name")), + ("_uuid-%s", part.uuid), + ("_label-%s", part.label), + ("_part-name-%s", part.part_name), + (None, None) + ): + if fmt is None: var = "" + elif part_id is not None: + var = fmt % part_id + else: + continue + + logger.debug("Looking for extra files in %s" % cls.image_extra_partition_files_var_name + var) extra_files = get_bitbake_var(cls.image_extra_partition_files_var_name + var) if extra_files is not None: break -- 2.43.0 ^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH v2 2/2] wic: extra-partition: introduce variable matching sourceparams 2026-01-12 8:29 ` [PATCH v2 2/2] wic: extra-partition: introduce variable matching sourceparams Louis Rannou via B4 Relay @ 2026-01-12 16:20 ` Pierre-loup GOSSE 0 siblings, 0 replies; 4+ messages in thread From: Pierre-loup GOSSE @ 2026-01-12 16:20 UTC (permalink / raw) To: louis.rannou; +Cc: openembedded-core [-- Attachment #1: Type: text/plain, Size: 7138 bytes --] Reviewed-by: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr> Thanks, Pierre-Loup PS: You forgot to add me in the Cc of your last answer <https://lore.kernel.org/all/530964.1767862855686685645@lists.openembedded.org/> . On Mon, Jan 12, 2026 at 9:29 AM Louis Rannou via B4 Relay < devnull+louis.rannou.non.se.com@kernel.org> wrote: > From: Louis Rannou <louis.rannou@non.se.com> > > The current extra_partition plugin uses predictable variable names (the > configuration must predict the image partition label or uuid). This has two > inconveniences: > > - an extra partition that has a specific label (or uuid) is doomed with the > associated variable. The partition cannot be configured unless the label > (or uuid) is changed. > - two partitions with same label from distincts .wks cannot have different > content. > > This allows the new suffix _name_%s to be appended to > IMAGE_EXTRA_PARTITION_FILES. This variable will be preferred whenever the > .wks specifies a name in the --sourceparams option. > > For example: > an image may define: > IMAGE_EXTRA_PARTITION_FILES_name_foo = "bar.conf" > and the .wks line have line: > part --source extra_partition --sourceparams "name=foo" > > Signed-off-by: Louis Rannou <louis.rannou@non.se.com> > --- > meta/lib/oeqa/selftest/cases/wic.py | 16 ++++++++++----- > scripts/lib/wic/plugins/source/extra_partition.py | 24 > +++++++++++++++++------ > 2 files changed, 29 insertions(+), 11 deletions(-) > > diff --git a/meta/lib/oeqa/selftest/cases/wic.py > b/meta/lib/oeqa/selftest/cases/wic.py > index 55fb8dfd1b..409437a5c0 100644 > --- a/meta/lib/oeqa/selftest/cases/wic.py > +++ b/meta/lib/oeqa/selftest/cases/wic.py > @@ -1657,6 +1657,7 @@ INITRAMFS_IMAGE = "core-image-initramfs-boot" > def test_extra_partition_plugin(self): > """Test extra partition plugin""" > config = dedent("""\ > + IMAGE_EXTRA_PARTITION_FILES_name-foo = "bar.conf" > IMAGE_EXTRA_PARTITION_FILES_label-foo = "bar.conf;foo.conf" > > IMAGE_EXTRA_PARTITION_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d = > "bar.conf;foobar.conf bar2.conf;foobar2.conf bar3.conf bar4.conf" > IMAGE_EXTRA_PARTITION_FILES = "foo/*" > @@ -1688,9 +1689,13 @@ INITRAMFS_IMAGE = "core-image-initramfs-boot" > > try: > with NamedTemporaryFile("w", suffix=".wks") as wks: > - wks.writelines(['part / --source extra_partition --ondisk > sda --label foo --align 4 --size 5M\n', > - 'part / --source extra_partition --ondisk > sda --fstype=vfat --uuid e7d0824e-cda3-4bed-9f54-9ef5312d105d --align 4 > --size 5M\n', > - 'part / --source extra_partition --ondisk > sda --fstype=ext4 --label bar --align 4 --size 5M\n']) > + wks.writelines([ > + 'part / --source extra_partition --ondisk sda > --sourceparams "name=foo" --align 4 --size 5M\n', > + 'part / --source extra_partition --ondisk sda --label > foo --align 4 --size 5M\n', > + 'part / --source extra_partition --ondisk sda > --fstype=vfat --uuid e7d0824e-cda3-4bed-9f54-9ef5312d105d --align 4 --size > 5M\n', > + 'part / --source extra_partition --ondisk sda > --fstype=ext4 --label bar --align 4 --size 5M\n', > + 'bootloader --ptable gpt\n', > + ]) > wks.flush() > _, wicimg = self._get_wic(wks.name) > > @@ -1698,14 +1703,15 @@ INITRAMFS_IMAGE = "core-image-initramfs-boot" > partls = result.output.split('\n')[1:] > > # Assert the number of partitions is correct > - self.assertEqual(3, len(partls), msg="Expect 3 partitions, > not %s" % result.output) > + self.assertEqual(4, len(partls), msg="Expect 4 partitions, > not %s" % result.output) > > # Fstype column from 'wic ls' should be fstype as given in > the part command > - for part_id, part_fs in enumerate(["fat16", "fat16", "ext4"]): > + for part_id, part_fs in enumerate(["fat16", "fat16", "fat16", > "ext4"]): > self.assertIn(part_fs, partls[part_id]) > > # For each partition, assert expected files exist > for part, part_glob in enumerate([ > + ["bar.conf"], > ["foo.conf"], > ["foobar.conf", "foobar2.conf", "bar3.conf", "bar4.conf"], > ["bar.conf", "bar2.conf"], > diff --git a/scripts/lib/wic/plugins/source/extra_partition.py > b/scripts/lib/wic/plugins/source/extra_partition.py > index d370b0107e..0d49e2bc7c 100644 > --- a/scripts/lib/wic/plugins/source/extra_partition.py > +++ b/scripts/lib/wic/plugins/source/extra_partition.py > @@ -18,15 +18,17 @@ class ExtraPartitionPlugin(SourcePlugin): > The plugin supports: > - Glob pattern matching for file selection. > - File renaming. > - - Suffixes to specify the target partition (by label, UUID, or > partname), > - enabling multiple extra partitions to coexist. > + - Suffixes to specify the target partition (by params-name, label, > UUID, > + or partname), enabling multiple extra partitions to coexist. > > For example: > > + IMAGE_EXTRA_PARTITION_FILES_name-randomname = "bar.conf;foo.conf" > IMAGE_EXTRA_PARTITION_FILES_label-foo = "bar.conf;foo.conf" > > IMAGE_EXTRA_PARTITION_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d = > "bar.conf;foobar.conf" > IMAGE_EXTRA_PARTITION_FILES = "foo/*" > WICVARS:append = "\ > + IMAGE_EXTRA_PARTITION_FILES_name-randomname \ > IMAGE_EXTRA_PARTITION_FILES_label-foo \ > > IMAGE_EXTRA_PARTITION_FILES_uuid-e7d0824e-cda3-4bed-9f54-9ef5312d105d \ > " > @@ -53,11 +55,21 @@ class ExtraPartitionPlugin(SourcePlugin): > raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting") > > extra_files = None > - for (fmt, id) in (("_uuid-%s", part.uuid), ("_label-%s", > part.label), ("_part-name-%s", part.part_name), (None, None)): > - if fmt: > - var = fmt % id > - else: > + for (fmt, part_id) in ( > + ("_name-%s", source_params.get("name")), > + ("_uuid-%s", part.uuid), > + ("_label-%s", part.label), > + ("_part-name-%s", part.part_name), > + (None, None) > + ): > + if fmt is None: > var = "" > + elif part_id is not None: > + var = fmt % part_id > + else: > + continue > + > + logger.debug("Looking for extra files in %s" % > cls.image_extra_partition_files_var_name + var) > extra_files = > get_bitbake_var(cls.image_extra_partition_files_var_name + var) > if extra_files is not None: > break > > -- > 2.43.0 > > > [-- Attachment #2: Type: text/html, Size: 9152 bytes --] ^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2026-01-12 16:21 UTC | newest] Thread overview: 4+ messages (download: mbox.gz follow: Atom feed -- links below jump to the message on this page -- 2026-01-12 8:29 [PATCH v2 0/2] wic: extra-partitions: new variable to be combined with sourceparams Louis Rannou via B4 Relay 2026-01-12 8:29 ` [PATCH v2 1/2] oeqa/selftest: wic: improve extra-partition plugin tests Louis Rannou via B4 Relay 2026-01-12 8:29 ` [PATCH v2 2/2] wic: extra-partition: introduce variable matching sourceparams Louis Rannou via B4 Relay 2026-01-12 16:20 ` Pierre-loup GOSSE
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox