* [PATCH v5 1/2] wic: add --extra-partition-space option to set unused space
@ 2025-09-03 14:45 pierre-loup.gosse
2025-09-03 14:45 ` [PATCH v5 2/2] wic: rename wks flag --extra-space to --extra-filesystem-space pierre-loup.gosse
2025-10-03 15:15 ` [PATCH v5 1/2] wic: add --extra-partition-space option to set unused space Mathieu Dubois-Briand
0 siblings, 2 replies; 9+ messages in thread
From: pierre-loup.gosse @ 2025-09-03 14:45 UTC (permalink / raw)
To: openembedded-core
Cc: Pierre-Loup GOSSE, Alexander Kanavin, Mathieu Dubois-Briand
From: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
By default, the content of the partition is filled by the filesystem
without leaving any unused free space. The --extra-space flag adds
extra space to the filesystem size, not to the partition.
Unused free space after the filesystem can be useful for some cases,
such as encrypting a partition at runtime.
With --extra-partition-space 32M, we ensure that the last 32M of the
partition is unused: this space does not contain filesystem data and
can store the LUKS2 header.
The implementation sets a difference between the partition and
filesystem size:
- With --fixed-size, the extra part space is removed from the
filesystem size.
- Otherwise (with or without --size flag), the extra part space is
added to the partition size.
Signed-off-by: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
CC: Alexander Kanavin <alex.kanavin@gmail.com>
CC: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
---
changes in v2:
- renaming the option to --extra-part-space
- adding tests
changes in v3:
- renaming the option to --extra-partition-space
- fixing the tests
- adding --extra-partition-space help description
changes in v4:
- removing an irrelevant test
changes in v5:
- no change
---
meta/lib/oeqa/selftest/cases/wic.py | 49 +++++++++++++++++++++++++++--
scripts/lib/wic/help.py | 8 +++++
scripts/lib/wic/ksparser.py | 3 ++
scripts/lib/wic/partition.py | 34 ++++++++++++--------
4 files changed, 79 insertions(+), 15 deletions(-)
diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py
index 44442e402d..c244c9f188 100644
--- a/meta/lib/oeqa/selftest/cases/wic.py
+++ b/meta/lib/oeqa/selftest/cases/wic.py
@@ -1166,7 +1166,7 @@ run_wic_cmd() {
return wkspath
- def _get_wic_partitions(self, wkspath, native_sysroot=None, ignore_status=False):
+ def _get_wic(self, wkspath, ignore_status=False):
p = runCmd("wic create %s -e core-image-minimal -o %s" % (wkspath, self.resultdir),
ignore_status=ignore_status)
@@ -1180,7 +1180,13 @@ run_wic_cmd() {
if not wicout:
return (p, None)
- wicimg = wicout[0]
+ return (p, wicout[0])
+
+ def _get_wic_partitions(self, wkspath, native_sysroot=None, ignore_status=False):
+ p, wicimg = self._get_wic(wkspath, ignore_status)
+
+ if wicimg is None:
+ return (p, None)
if not native_sysroot:
native_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "wic-tools")
@@ -1320,6 +1326,45 @@ run_wic_cmd() {
size = int(size[:-3])
self.assertGreaterEqual(size, 204800)
+ def test_extra_partition_space(self):
+ native_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "wic-tools")
+
+ with NamedTemporaryFile("w", suffix=".wks") as tempf:
+ tempf.write("bootloader --ptable gpt\n" \
+ "part --ondisk hda --size 10M --extra-partition-space 10M --fstype=ext4\n" \
+ "part --ondisk hda --fixed-size 20M --extra-partition-space 10M --fstype=ext4\n" \
+ "part --source rootfs --ondisk hda --extra-partition-space 10M --fstype=ext4\n" \
+ "part --source rootfs --ondisk hda --fixed-size 200M --extra-partition-space 10M --fstype=ext4\n")
+ tempf.flush()
+
+ _, wicimg = self._get_wic(tempf.name)
+
+ res = runCmd("parted -m %s unit b p" % wicimg,
+ native_sysroot=native_sysroot, stderr=subprocess.PIPE)
+
+ # parse parted output which looks like this:
+ # BYT;\n
+ # /var/tmp/wic/build/tmpfwvjjkf_-201611101222-hda.direct:200MiB:file:512:512:msdos::;\n
+ # 1:0.00MiB:200MiB:200MiB:ext4::;\n
+ partlns = res.output.splitlines()[2:]
+
+ self.assertEqual(4, len(partlns))
+
+ # Test for each partitions that the extra part space exists
+ for part in range(0, len(partlns)):
+ part_file = os.path.join(self.resultdir, "selftest_img.part%d" % (part + 1))
+ partln = partlns[part].split(":")
+ self.assertEqual(7, len(partln))
+ self.assertRegex(partln[3], r'^[0-9]+B$')
+ part_size = int(partln[3].rstrip("B"))
+ start = int(partln[1].rstrip("B")) / 512
+ length = part_size / 512
+ runCmd("dd if=%s of=%s skip=%d count=%d" %
+ (wicimg, part_file, start, length))
+ res = runCmd("dumpe2fs %s -h | grep \"^Block count\"" % part_file)
+ fs_size = int(res.output.split(":")[1].strip()) * 1024
+ self.assertLessEqual(fs_size + 10485760, part_size, "part file: %s" % part_file)
+
# TODO this test could also work on aarch64
@skipIfNotArch(['i586', 'i686', 'x86_64'])
@OETestTag("runqemu")
diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
index 2e3061f343..800c0abf0f 100644
--- a/scripts/lib/wic/help.py
+++ b/scripts/lib/wic/help.py
@@ -1020,6 +1020,14 @@ DESCRIPTION
By default, 10MB. This option cannot be used
with --fixed-size option.
+ --extra-partition-space: This option is specific to wic. It adds extra
+ empty space after the space filled by the
+ filesystem. With --fixed-size, the extra
+ partition space is removed from the filesystem
+ size. Otherwise (with or without --size flag),
+ the extra partition space is added to the final
+ paritition size. The default value is 0MB.
+
--overhead-factor: This option is specific to wic. The
size of the partition is multiplied by
this factor. It has to be greater than or
diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
index 596b6e8e7e..a1aaf1b4b3 100644
--- a/scripts/lib/wic/ksparser.py
+++ b/scripts/lib/wic/ksparser.py
@@ -154,6 +154,7 @@ class KickStart():
part.add_argument('--include-path', nargs='+', action='append')
part.add_argument('--change-directory')
part.add_argument("--extra-space", type=sizetype("M"))
+ part.add_argument('--extra-partition-space', type=sizetype("M"))
part.add_argument('--fsoptions', dest='fsopts')
part.add_argument('--fspassno', dest='fspassno')
part.add_argument('--fstype', default='vfat',
@@ -259,6 +260,8 @@ class KickStart():
err = "%s:%d: Must set the label with --label" \
% (confpath, lineno)
raise KickStartError(err)
+ if not parsed.extra_partition_space:
+ parsed.extra_partition_space = 0
# using ArgumentParser one cannot easily tell if option
# was passed as argument, if said option has a default
# value; --overhead-factor/--extra-space cannot be used
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index b34691d313..d358aabbd6 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -29,6 +29,7 @@ class Partition():
self.disk = args.disk
self.device = None
self.extra_space = args.extra_space
+ self.extra_partition_space = args.extra_partition_space
self.exclude_path = args.exclude_path
self.include_path = args.include_path
self.change_directory = args.change_directory
@@ -91,13 +92,12 @@ class Partition():
def get_rootfs_size(self, actual_rootfs_size=0):
"""
Calculate the required size of rootfs taking into consideration
- --size/--fixed-size flags as well as overhead and extra space, as
- specified in kickstart file. Raises an error if the
- `actual_rootfs_size` is larger than fixed-size rootfs.
-
+ --size/--fixed-size and --extra-partition-space flags as well as overhead
+ and extra space, as specified in kickstart file. Raises an error
+ if the `actual_rootfs_size` is larger than fixed-size rootfs.
"""
if self.fixed_size:
- rootfs_size = self.fixed_size
+ rootfs_size = self.fixed_size - self.extra_partition_space
if actual_rootfs_size > rootfs_size:
raise WicError("Actual rootfs size (%d kB) is larger than "
"allowed size %d kB" %
@@ -119,10 +119,18 @@ class Partition():
def disk_size(self):
"""
Obtain on-disk size of partition taking into consideration
- --size/--fixed-size options.
+ --size/--fixed-size and --extra-partition-space options.
+
+ """
+ return self.fixed_size if self.fixed_size else self.size + self.extra_partition_space
+ @property
+ def fs_size(self):
+ """
+ Obtain on-disk size of filesystem inside the partition taking into
+ consideration --size/--fixed-size and --extra-partition-space options.
"""
- return self.fixed_size if self.fixed_size else self.size
+ return self.fixed_size - self.extra_partition_space if self.fixed_size else self.size
def prepare(self, creator, cr_workdir, oe_builddir, rootfs_dir,
bootimg_dir, kernel_dir, native_sysroot, updated_fstab_path):
@@ -202,10 +210,10 @@ class Partition():
"This a bug in source plugin %s and needs to be fixed." %
(self.mountpoint, self.source))
- if self.fixed_size and self.size > self.fixed_size:
+ if self.fixed_size and self.size + self.extra_partition_space > self.fixed_size:
raise WicError("File system image of partition %s is "
- "larger (%d kB) than its allowed size %d kB" %
- (self.mountpoint, self.size, self.fixed_size))
+ "larger (%d kB + %d kB extra part space) than its allowed size %d kB" %
+ (self.mountpoint, self.size, self.extra_partition_space, self.fixed_size))
def prepare_rootfs(self, cr_workdir, oe_builddir, rootfs_dir,
native_sysroot, real_rootfs = True, pseudo_dir = None):
@@ -440,7 +448,7 @@ class Partition():
"""
Prepare an empty ext2/3/4 partition.
"""
- size = self.disk_size
+ size = self.fs_size
with open(rootfs, 'w') as sparse:
os.ftruncate(sparse.fileno(), size * 1024)
@@ -464,7 +472,7 @@ class Partition():
"""
Prepare an empty btrfs partition.
"""
- size = self.disk_size
+ size = self.fs_size
with open(rootfs, 'w') as sparse:
os.ftruncate(sparse.fileno(), size * 1024)
@@ -482,7 +490,7 @@ class Partition():
"""
Prepare an empty vfat partition.
"""
- blocks = self.disk_size
+ blocks = self.fs_size
label_str = "-n boot"
if self.label:
--
2.34.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [PATCH v5 2/2] wic: rename wks flag --extra-space to --extra-filesystem-space
2025-09-03 14:45 [PATCH v5 1/2] wic: add --extra-partition-space option to set unused space pierre-loup.gosse
@ 2025-09-03 14:45 ` pierre-loup.gosse
2025-09-09 14:30 ` [OE-core] " Ross Burton
2025-10-03 15:15 ` [PATCH v5 1/2] wic: add --extra-partition-space option to set unused space Mathieu Dubois-Briand
1 sibling, 1 reply; 9+ messages in thread
From: pierre-loup.gosse @ 2025-09-03 14:45 UTC (permalink / raw)
To: openembedded-core
Cc: Pierre-Loup GOSSE, Alexander Kanavin, Mathieu Dubois-Briand
From: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
Makes a clear distinction with --extra-partition-space flag.
Signed-off-by: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
CC: Alexander Kanavin <alex.kanavin@gmail.com>
CC: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
---
changes in v3:
- adding this patch
changes in v4:
- no change
changes in v5:
- fix extra_filesystem_space check
---
meta/lib/oeqa/selftest/cases/wic.py | 4 ++--
scripts/lib/wic/help.py | 12 ++++++------
scripts/lib/wic/ksparser.py | 25 +++++++++++++------------
scripts/lib/wic/partition.py | 6 +++---
4 files changed, 24 insertions(+), 23 deletions(-)
diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py
index c244c9f188..b1c318bd4e 100644
--- a/meta/lib/oeqa/selftest/cases/wic.py
+++ b/meta/lib/oeqa/selftest/cases/wic.py
@@ -1311,12 +1311,12 @@ run_wic_cmd() {
p, _ = self._get_wic_partitions(tempf.name, ignore_status=True)
self.assertNotEqual(p.status, 0, "wic exited successfully when an error was expected:\n%s" % p.output)
- def test_extra_space(self):
+ def test_extra_filesystem_space(self):
native_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "wic-tools")
with NamedTemporaryFile("w", suffix=".wks") as tempf:
tempf.write("bootloader --ptable gpt\n" \
- "part / --source rootfs --ondisk hda --extra-space 200M --fstype=ext4\n")
+ "part / --source rootfs --ondisk hda --extra-filesystem-space 200M --fstype=ext4\n")
tempf.flush()
_, partlns = self._get_wic_partitions(tempf.name, native_sysroot)
diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
index 800c0abf0f..6b49a67de9 100644
--- a/scripts/lib/wic/help.py
+++ b/scripts/lib/wic/help.py
@@ -1013,12 +1013,12 @@ DESCRIPTION
--no-fstab-update: This option is specific to wic. It does not update the
'/etc/fstab' stock file for the given partition.
- --extra-space: This option is specific to wic. It adds extra
- space after the space filled by the content
- of the partition. The final size can go
- beyond the size specified by --size.
- By default, 10MB. This option cannot be used
- with --fixed-size option.
+ --extra-filesystem-space: This option is specific to wic. It adds extra
+ space after the space filled by the content
+ of the partition. The final size can go
+ beyond the size specified by --size.
+ By default, 10MB. This option cannot be used
+ with --fixed-size option.
--extra-partition-space: This option is specific to wic. It adds extra
empty space after the space filled by the
diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
index a1aaf1b4b3..705f989750 100644
--- a/scripts/lib/wic/ksparser.py
+++ b/scripts/lib/wic/ksparser.py
@@ -132,7 +132,7 @@ def systemidtype(arg):
class KickStart():
"""Kickstart parser implementation."""
- DEFAULT_EXTRA_SPACE = 10*1024
+ DEFAULT_EXTRA_FILESYSTEM_SPACE = 10*1024
DEFAULT_OVERHEAD_FACTOR = 1.3
def __init__(self, confpath):
@@ -153,7 +153,7 @@ class KickStart():
part.add_argument('--exclude-path', nargs='+')
part.add_argument('--include-path', nargs='+', action='append')
part.add_argument('--change-directory')
- part.add_argument("--extra-space", type=sizetype("M"))
+ part.add_argument('--extra-filesystem-space', type=sizetype("M"))
part.add_argument('--extra-partition-space', type=sizetype("M"))
part.add_argument('--fsoptions', dest='fsopts')
part.add_argument('--fspassno', dest='fspassno')
@@ -175,9 +175,9 @@ class KickStart():
part.add_argument('--hidden', action='store_true')
# --size and --fixed-size cannot be specified together; options
- # ----extra-space and --overhead-factor should also raise a parser
- # --error, but since nesting mutually exclusive groups does not work,
- # ----extra-space/--overhead-factor are handled later
+ # ----extra-filesystem-space and --overhead-factor should also raise a
+ # parser error, but since nesting mutually exclusive groups does not work,
+ # ----extra-filesystem-space/--overhead-factor are handled later
sizeexcl = part.add_mutually_exclusive_group()
sizeexcl.add_argument('--size', type=sizetype("M"), default=0)
sizeexcl.add_argument('--fixed-size', type=sizetype("M"), default=0)
@@ -264,12 +264,13 @@ class KickStart():
parsed.extra_partition_space = 0
# using ArgumentParser one cannot easily tell if option
# was passed as argument, if said option has a default
- # value; --overhead-factor/--extra-space cannot be used
- # with --fixed-size, so at least detect when these were
- # passed with non-0 values ...
+ # value; --overhead-factor/--extra-filesystem-space
+ # cannot be used with --fixed-size, so at least detect
+ # when these were passed with non-0 values ...
if parsed.fixed_size:
- if parsed.overhead_factor or parsed.extra_space:
- err = "%s:%d: arguments --overhead-factor and --extra-space not "\
+ if parsed.overhead_factor or parsed.extra_filesystem_space:
+ err = "%s:%d: arguments --overhead-factor and "\
+ "--extra-filesystem-space not "\
"allowed with argument --fixed-size" \
% (confpath, lineno)
raise KickStartError(err)
@@ -280,8 +281,8 @@ class KickStart():
# with value equal to 0)
if not parsed.overhead_factor:
parsed.overhead_factor = self.DEFAULT_OVERHEAD_FACTOR
- if not parsed.extra_space:
- parsed.extra_space = self.DEFAULT_EXTRA_SPACE
+ if not parsed.extra_filesystem_space:
+ parsed.extra_filesystem_space = self.DEFAULT_EXTRA_FILESYSTEM_SPACE
self.partnum += 1
self.partitions.append(Partition(parsed, self.partnum))
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index d358aabbd6..0c9b1a5b96 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -28,7 +28,7 @@ class Partition():
self.align = args.align
self.disk = args.disk
self.device = None
- self.extra_space = args.extra_space
+ self.extra_filesystem_space = args.extra_filesystem_space
self.extra_partition_space = args.extra_partition_space
self.exclude_path = args.exclude_path
self.include_path = args.include_path
@@ -104,8 +104,8 @@ class Partition():
(actual_rootfs_size, rootfs_size))
else:
extra_blocks = self.get_extra_block_count(actual_rootfs_size)
- if extra_blocks < self.extra_space:
- extra_blocks = self.extra_space
+ if extra_blocks < self.extra_filesystem_space:
+ extra_blocks = self.extra_filesystem_space
rootfs_size = actual_rootfs_size + extra_blocks
rootfs_size = int(rootfs_size * self.overhead_factor)
--
2.34.1
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [OE-core] [PATCH v5 2/2] wic: rename wks flag --extra-space to --extra-filesystem-space
2025-09-03 14:45 ` [PATCH v5 2/2] wic: rename wks flag --extra-space to --extra-filesystem-space pierre-loup.gosse
@ 2025-09-09 14:30 ` Ross Burton
2025-09-09 14:42 ` Pierre-loup GOSSE
0 siblings, 1 reply; 9+ messages in thread
From: Ross Burton @ 2025-09-09 14:30 UTC (permalink / raw)
To: pierre-loup.gosse@smile.fr
Cc: openembedded-core@lists.openembedded.org, Antonin Godard
These changes need to be documented, so would you be able to send a patch for the documentation (yocto-docs repository)?
CCing Antonin so he’s aware.
Ross
> On 3 Sep 2025, at 15:45, pierre-loup.gosse via lists.openembedded.org <pierre-loup.gosse=smile.fr@lists.openembedded.org> wrote:
>
> From: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
>
> Makes a clear distinction with --extra-partition-space flag.
>
> Signed-off-by: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
>
> CC: Alexander Kanavin <alex.kanavin@gmail.com>
> CC: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
> ---
> changes in v3:
> - adding this patch
>
> changes in v4:
> - no change
>
> changes in v5:
> - fix extra_filesystem_space check
> ---
> meta/lib/oeqa/selftest/cases/wic.py | 4 ++--
> scripts/lib/wic/help.py | 12 ++++++------
> scripts/lib/wic/ksparser.py | 25 +++++++++++++------------
> scripts/lib/wic/partition.py | 6 +++---
> 4 files changed, 24 insertions(+), 23 deletions(-)
>
> diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py
> index c244c9f188..b1c318bd4e 100644
> --- a/meta/lib/oeqa/selftest/cases/wic.py
> +++ b/meta/lib/oeqa/selftest/cases/wic.py
> @@ -1311,12 +1311,12 @@ run_wic_cmd() {
> p, _ = self._get_wic_partitions(tempf.name, ignore_status=True)
> self.assertNotEqual(p.status, 0, "wic exited successfully when an error was expected:\n%s" % p.output)
>
> - def test_extra_space(self):
> + def test_extra_filesystem_space(self):
> native_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "wic-tools")
>
> with NamedTemporaryFile("w", suffix=".wks") as tempf:
> tempf.write("bootloader --ptable gpt\n" \
> - "part / --source rootfs --ondisk hda --extra-space 200M --fstype=ext4\n")
> + "part / --source rootfs --ondisk hda --extra-filesystem-space 200M --fstype=ext4\n")
> tempf.flush()
>
> _, partlns = self._get_wic_partitions(tempf.name, native_sysroot)
> diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
> index 800c0abf0f..6b49a67de9 100644
> --- a/scripts/lib/wic/help.py
> +++ b/scripts/lib/wic/help.py
> @@ -1013,12 +1013,12 @@ DESCRIPTION
> --no-fstab-update: This option is specific to wic. It does not update the
> '/etc/fstab' stock file for the given partition.
>
> - --extra-space: This option is specific to wic. It adds extra
> - space after the space filled by the content
> - of the partition. The final size can go
> - beyond the size specified by --size.
> - By default, 10MB. This option cannot be used
> - with --fixed-size option.
> + --extra-filesystem-space: This option is specific to wic. It adds extra
> + space after the space filled by the content
> + of the partition. The final size can go
> + beyond the size specified by --size.
> + By default, 10MB. This option cannot be used
> + with --fixed-size option.
>
> --extra-partition-space: This option is specific to wic. It adds extra
> empty space after the space filled by the
> diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
> index a1aaf1b4b3..705f989750 100644
> --- a/scripts/lib/wic/ksparser.py
> +++ b/scripts/lib/wic/ksparser.py
> @@ -132,7 +132,7 @@ def systemidtype(arg):
> class KickStart():
> """Kickstart parser implementation."""
>
> - DEFAULT_EXTRA_SPACE = 10*1024
> + DEFAULT_EXTRA_FILESYSTEM_SPACE = 10*1024
> DEFAULT_OVERHEAD_FACTOR = 1.3
>
> def __init__(self, confpath):
> @@ -153,7 +153,7 @@ class KickStart():
> part.add_argument('--exclude-path', nargs='+')
> part.add_argument('--include-path', nargs='+', action='append')
> part.add_argument('--change-directory')
> - part.add_argument("--extra-space", type=sizetype("M"))
> + part.add_argument('--extra-filesystem-space', type=sizetype("M"))
> part.add_argument('--extra-partition-space', type=sizetype("M"))
> part.add_argument('--fsoptions', dest='fsopts')
> part.add_argument('--fspassno', dest='fspassno')
> @@ -175,9 +175,9 @@ class KickStart():
> part.add_argument('--hidden', action='store_true')
>
> # --size and --fixed-size cannot be specified together; options
> - # ----extra-space and --overhead-factor should also raise a parser
> - # --error, but since nesting mutually exclusive groups does not work,
> - # ----extra-space/--overhead-factor are handled later
> + # ----extra-filesystem-space and --overhead-factor should also raise a
> + # parser error, but since nesting mutually exclusive groups does not work,
> + # ----extra-filesystem-space/--overhead-factor are handled later
> sizeexcl = part.add_mutually_exclusive_group()
> sizeexcl.add_argument('--size', type=sizetype("M"), default=0)
> sizeexcl.add_argument('--fixed-size', type=sizetype("M"), default=0)
> @@ -264,12 +264,13 @@ class KickStart():
> parsed.extra_partition_space = 0
> # using ArgumentParser one cannot easily tell if option
> # was passed as argument, if said option has a default
> - # value; --overhead-factor/--extra-space cannot be used
> - # with --fixed-size, so at least detect when these were
> - # passed with non-0 values ...
> + # value; --overhead-factor/--extra-filesystem-space
> + # cannot be used with --fixed-size, so at least detect
> + # when these were passed with non-0 values ...
> if parsed.fixed_size:
> - if parsed.overhead_factor or parsed.extra_space:
> - err = "%s:%d: arguments --overhead-factor and --extra-space not "\
> + if parsed.overhead_factor or parsed.extra_filesystem_space:
> + err = "%s:%d: arguments --overhead-factor and "\
> + "--extra-filesystem-space not "\
> "allowed with argument --fixed-size" \
> % (confpath, lineno)
> raise KickStartError(err)
> @@ -280,8 +281,8 @@ class KickStart():
> # with value equal to 0)
> if not parsed.overhead_factor:
> parsed.overhead_factor = self.DEFAULT_OVERHEAD_FACTOR
> - if not parsed.extra_space:
> - parsed.extra_space = self.DEFAULT_EXTRA_SPACE
> + if not parsed.extra_filesystem_space:
> + parsed.extra_filesystem_space = self.DEFAULT_EXTRA_FILESYSTEM_SPACE
>
> self.partnum += 1
> self.partitions.append(Partition(parsed, self.partnum))
> diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
> index d358aabbd6..0c9b1a5b96 100644
> --- a/scripts/lib/wic/partition.py
> +++ b/scripts/lib/wic/partition.py
> @@ -28,7 +28,7 @@ class Partition():
> self.align = args.align
> self.disk = args.disk
> self.device = None
> - self.extra_space = args.extra_space
> + self.extra_filesystem_space = args.extra_filesystem_space
> self.extra_partition_space = args.extra_partition_space
> self.exclude_path = args.exclude_path
> self.include_path = args.include_path
> @@ -104,8 +104,8 @@ class Partition():
> (actual_rootfs_size, rootfs_size))
> else:
> extra_blocks = self.get_extra_block_count(actual_rootfs_size)
> - if extra_blocks < self.extra_space:
> - extra_blocks = self.extra_space
> + if extra_blocks < self.extra_filesystem_space:
> + extra_blocks = self.extra_filesystem_space
>
> rootfs_size = actual_rootfs_size + extra_blocks
> rootfs_size = int(rootfs_size * self.overhead_factor)
> --
> 2.34.1
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#222864): https://lists.openembedded.org/g/openembedded-core/message/222864
> Mute This Topic: https://lists.openembedded.org/mt/115046316/6875888
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [ross.burton@arm.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [OE-core] [PATCH v5 2/2] wic: rename wks flag --extra-space to --extra-filesystem-space
2025-09-09 14:30 ` [OE-core] " Ross Burton
@ 2025-09-09 14:42 ` Pierre-loup GOSSE
0 siblings, 0 replies; 9+ messages in thread
From: Pierre-loup GOSSE @ 2025-09-09 14:42 UTC (permalink / raw)
To: Ross Burton; +Cc: openembedded-core@lists.openembedded.org, Antonin Godard
[-- Attachment #1: Type: text/plain, Size: 9862 bytes --]
I have already sent patches for documentation to the yocto-docs mailing
list.
Pierre-Loup,
On Tue, Sep 9, 2025 at 4:31 PM Ross Burton <Ross.Burton@arm.com> wrote:
> These changes need to be documented, so would you be able to send a patch
> for the documentation (yocto-docs repository)?
>
> CCing Antonin so he’s aware.
>
> Ross
>
> > On 3 Sep 2025, at 15:45, pierre-loup.gosse via lists.openembedded.org
> <pierre-loup.gosse=smile.fr@lists.openembedded.org> wrote:
> >
> > From: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
> >
> > Makes a clear distinction with --extra-partition-space flag.
> >
> > Signed-off-by: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
> >
> > CC: Alexander Kanavin <alex.kanavin@gmail.com>
> > CC: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
> > ---
> > changes in v3:
> > - adding this patch
> >
> > changes in v4:
> > - no change
> >
> > changes in v5:
> > - fix extra_filesystem_space check
> > ---
> > meta/lib/oeqa/selftest/cases/wic.py | 4 ++--
> > scripts/lib/wic/help.py | 12 ++++++------
> > scripts/lib/wic/ksparser.py | 25 +++++++++++++------------
> > scripts/lib/wic/partition.py | 6 +++---
> > 4 files changed, 24 insertions(+), 23 deletions(-)
> >
> > diff --git a/meta/lib/oeqa/selftest/cases/wic.py
> b/meta/lib/oeqa/selftest/cases/wic.py
> > index c244c9f188..b1c318bd4e 100644
> > --- a/meta/lib/oeqa/selftest/cases/wic.py
> > +++ b/meta/lib/oeqa/selftest/cases/wic.py
> > @@ -1311,12 +1311,12 @@ run_wic_cmd() {
> > p, _ = self._get_wic_partitions(tempf.name,
> ignore_status=True)
> > self.assertNotEqual(p.status, 0, "wic exited successfully
> when an error was expected:\n%s" % p.output)
> >
> > - def test_extra_space(self):
> > + def test_extra_filesystem_space(self):
> > native_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "wic-tools")
> >
> > with NamedTemporaryFile("w", suffix=".wks") as tempf:
> > tempf.write("bootloader --ptable gpt\n" \
> > - "part / --source rootfs --ondisk hda
> --extra-space 200M --fstype=ext4\n")
> > + "part / --source rootfs --ondisk hda
> --extra-filesystem-space 200M --fstype=ext4\n")
> > tempf.flush()
> >
> > _, partlns = self._get_wic_partitions(tempf.name,
> native_sysroot)
> > diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
> > index 800c0abf0f..6b49a67de9 100644
> > --- a/scripts/lib/wic/help.py
> > +++ b/scripts/lib/wic/help.py
> > @@ -1013,12 +1013,12 @@ DESCRIPTION
> > --no-fstab-update: This option is specific to wic. It does not
> update the
> > '/etc/fstab' stock file for the given
> partition.
> >
> > - --extra-space: This option is specific to wic. It adds extra
> > - space after the space filled by the content
> > - of the partition. The final size can go
> > - beyond the size specified by --size.
> > - By default, 10MB. This option cannot be used
> > - with --fixed-size option.
> > + --extra-filesystem-space: This option is specific to wic. It
> adds extra
> > + space after the space filled by the
> content
> > + of the partition. The final size can
> go
> > + beyond the size specified by --size.
> > + By default, 10MB. This option cannot
> be used
> > + with --fixed-size option.
> >
> > --extra-partition-space: This option is specific to wic. It
> adds extra
> > empty space after the space filled by
> the
> > diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
> > index a1aaf1b4b3..705f989750 100644
> > --- a/scripts/lib/wic/ksparser.py
> > +++ b/scripts/lib/wic/ksparser.py
> > @@ -132,7 +132,7 @@ def systemidtype(arg):
> > class KickStart():
> > """Kickstart parser implementation."""
> >
> > - DEFAULT_EXTRA_SPACE = 10*1024
> > + DEFAULT_EXTRA_FILESYSTEM_SPACE = 10*1024
> > DEFAULT_OVERHEAD_FACTOR = 1.3
> >
> > def __init__(self, confpath):
> > @@ -153,7 +153,7 @@ class KickStart():
> > part.add_argument('--exclude-path', nargs='+')
> > part.add_argument('--include-path', nargs='+', action='append')
> > part.add_argument('--change-directory')
> > - part.add_argument("--extra-space", type=sizetype("M"))
> > + part.add_argument('--extra-filesystem-space',
> type=sizetype("M"))
> > part.add_argument('--extra-partition-space', type=sizetype("M"))
> > part.add_argument('--fsoptions', dest='fsopts')
> > part.add_argument('--fspassno', dest='fspassno')
> > @@ -175,9 +175,9 @@ class KickStart():
> > part.add_argument('--hidden', action='store_true')
> >
> > # --size and --fixed-size cannot be specified together; options
> > - # ----extra-space and --overhead-factor should also raise a
> parser
> > - # --error, but since nesting mutually exclusive groups does not
> work,
> > - # ----extra-space/--overhead-factor are handled later
> > + # ----extra-filesystem-space and --overhead-factor should also
> raise a
> > + # parser error, but since nesting mutually exclusive groups
> does not work,
> > + # ----extra-filesystem-space/--overhead-factor are handled later
> > sizeexcl = part.add_mutually_exclusive_group()
> > sizeexcl.add_argument('--size', type=sizetype("M"), default=0)
> > sizeexcl.add_argument('--fixed-size', type=sizetype("M"),
> default=0)
> > @@ -264,12 +264,13 @@ class KickStart():
> > parsed.extra_partition_space = 0
> > # using ArgumentParser one cannot easily tell if
> option
> > # was passed as argument, if said option has a
> default
> > - # value; --overhead-factor/--extra-space cannot
> be used
> > - # with --fixed-size, so at least detect when
> these were
> > - # passed with non-0 values ...
> > + # value;
> --overhead-factor/--extra-filesystem-space
> > + # cannot be used with --fixed-size, so at least
> detect
> > + # when these were passed with non-0 values ...
> > if parsed.fixed_size:
> > - if parsed.overhead_factor or
> parsed.extra_space:
> > - err = "%s:%d: arguments
> --overhead-factor and --extra-space not "\
> > + if parsed.overhead_factor or
> parsed.extra_filesystem_space:
> > + err = "%s:%d: arguments
> --overhead-factor and "\
> > + "--extra-filesystem-space not "\
> > "allowed with argument
> --fixed-size" \
> > % (confpath, lineno)
> > raise KickStartError(err)
> > @@ -280,8 +281,8 @@ class KickStart():
> > # with value equal to 0)
> > if not parsed.overhead_factor:
> > parsed.overhead_factor =
> self.DEFAULT_OVERHEAD_FACTOR
> > - if not parsed.extra_space:
> > - parsed.extra_space =
> self.DEFAULT_EXTRA_SPACE
> > + if not parsed.extra_filesystem_space:
> > + parsed.extra_filesystem_space =
> self.DEFAULT_EXTRA_FILESYSTEM_SPACE
> >
> > self.partnum += 1
> > self.partitions.append(Partition(parsed,
> self.partnum))
> > diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
> > index d358aabbd6..0c9b1a5b96 100644
> > --- a/scripts/lib/wic/partition.py
> > +++ b/scripts/lib/wic/partition.py
> > @@ -28,7 +28,7 @@ class Partition():
> > self.align = args.align
> > self.disk = args.disk
> > self.device = None
> > - self.extra_space = args.extra_space
> > + self.extra_filesystem_space = args.extra_filesystem_space
> > self.extra_partition_space = args.extra_partition_space
> > self.exclude_path = args.exclude_path
> > self.include_path = args.include_path
> > @@ -104,8 +104,8 @@ class Partition():
> > (actual_rootfs_size, rootfs_size))
> > else:
> > extra_blocks = self.get_extra_block_count(actual_rootfs_size)
> > - if extra_blocks < self.extra_space:
> > - extra_blocks = self.extra_space
> > + if extra_blocks < self.extra_filesystem_space:
> > + extra_blocks = self.extra_filesystem_space
> >
> > rootfs_size = actual_rootfs_size + extra_blocks
> > rootfs_size = int(rootfs_size * self.overhead_factor)
> > --
> > 2.34.1
> >
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#222864):
> https://lists.openembedded.org/g/openembedded-core/message/222864
> > Mute This Topic: https://lists.openembedded.org/mt/115046316/6875888
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> ross.burton@arm.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
>
>
[-- Attachment #2: Type: text/html, Size: 13244 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v5 1/2] wic: add --extra-partition-space option to set unused space
2025-09-03 14:45 [PATCH v5 1/2] wic: add --extra-partition-space option to set unused space pierre-loup.gosse
2025-09-03 14:45 ` [PATCH v5 2/2] wic: rename wks flag --extra-space to --extra-filesystem-space pierre-loup.gosse
@ 2025-10-03 15:15 ` Mathieu Dubois-Briand
2025-10-03 15:20 ` Mathieu Dubois-Briand
1 sibling, 1 reply; 9+ messages in thread
From: Mathieu Dubois-Briand @ 2025-10-03 15:15 UTC (permalink / raw)
To: pierre-loup.gosse, openembedded-core; +Cc: Alexander Kanavin
On Wed Sep 3, 2025 at 4:45 PM CEST, pierre-loup.gosse wrote:
> From: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
>
> By default, the content of the partition is filled by the filesystem
> without leaving any unused free space. The --extra-space flag adds
> extra space to the filesystem size, not to the partition.
>
> Unused free space after the filesystem can be useful for some cases,
> such as encrypting a partition at runtime.
> With --extra-partition-space 32M, we ensure that the last 32M of the
> partition is unused: this space does not contain filesystem data and
> can store the LUKS2 header.
>
> The implementation sets a difference between the partition and
> filesystem size:
> - With --fixed-size, the extra part space is removed from the
> filesystem size.
> - Otherwise (with or without --size flag), the extra part space is
> added to the partition size.
>
> Signed-off-by: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
>
> CC: Alexander Kanavin <alex.kanavin@gmail.com>
> CC: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
> ---
Hi Pierre-Loup,
Thanks for the new version.
> + def test_extra_partition_space(self):
> + native_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "wic-tools")
> +
> + with NamedTemporaryFile("w", suffix=".wks") as tempf:
> + tempf.write("bootloader --ptable gpt\n" \
> + "part --ondisk hda --size 10M --extra-partition-space 10M --fstype=ext4\n" \
> + "part --ondisk hda --fixed-size 20M --extra-partition-space 10M --fstype=ext4\n" \
> + "part --source rootfs --ondisk hda --extra-partition-space 10M --fstype=ext4\n" \
> + "part --source rootfs --ondisk hda --fixed-size 200M --extra-partition-space 10M --fstype=ext4\n")
> + tempf.flush()
> +
> + _, wicimg = self._get_wic(tempf.name)
> +
> + res = runCmd("parted -m %s unit b p" % wicimg,
> + native_sysroot=native_sysroot, stderr=subprocess.PIPE)
> +
This fails when "parted" is not installed on the host machine:
2025-10-03 13:42:20,053 - oe-selftest - INFO - 9: 73/93 615/639 (37.32s) (0 failed) (wic.Wic2.test_extra_partition_plugin)
2025-10-03 13:42:20,053 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/meta/lib/oeqa/selftest/cases/wic.py", line 1683, in test_extra_partition_plugin
self.assertEqual('4', result.output, msg="Expect 3 partitions, not %s" % result.output)
File "/usr/lib/python3.11/unittest/case.py", line 873, in assertEqual
assertion_func(first, second, msg=msg)
File "/usr/lib/python3.11/unittest/case.py", line 1253, in assertMultiLineEqual
self.fail(self._formatMessage(msg, standardMsg))
File "/usr/lib/python3.11/unittest/case.py", line 703, in fail
raise self.failureException(msg)
AssertionError: '4' != "ERROR: Can't find executable parted\n1"
- 4
+ ERROR: Can't find executable parted
1
: Expect 3 partitions, not ERROR: Can't find executable parted
1
https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2456
You can easily reproduce this on a test machine by removing parted from
the host. Also, test_wic_sector_size use parted but does not show the
same issue:
oe-selftest -r wic.Wic.test_wic_sector_size
...
2025-10-03 15:05:58,051 - oe-selftest - INFO - RESULTS - wic.Wic.test_wic_sector_size: PASSED (94.37s)
I believe this comes from the following line and the associated
try/finally block to revert to the default PATH variable:
os.environ['PATH'] = get_bb_var("PATH", "wic-tools")
Can you use something similar?
Thanks,
Mathieu
--
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [PATCH v5 1/2] wic: add --extra-partition-space option to set unused space
2025-10-03 15:15 ` [PATCH v5 1/2] wic: add --extra-partition-space option to set unused space Mathieu Dubois-Briand
@ 2025-10-03 15:20 ` Mathieu Dubois-Briand
2025-10-03 21:22 ` [OE-core] " Yoann Congal
0 siblings, 1 reply; 9+ messages in thread
From: Mathieu Dubois-Briand @ 2025-10-03 15:20 UTC (permalink / raw)
To: pierre-loup.gosse, openembedded-core; +Cc: Alexander Kanavin
On Fri Oct 3, 2025 at 5:15 PM CEST, Mathieu Dubois-Briand wrote:
> On Wed Sep 3, 2025 at 4:45 PM CEST, pierre-loup.gosse wrote:
>> From: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
>>
>> By default, the content of the partition is filled by the filesystem
>> without leaving any unused free space. The --extra-space flag adds
>> extra space to the filesystem size, not to the partition.
>>
>> Unused free space after the filesystem can be useful for some cases,
>> such as encrypting a partition at runtime.
>> With --extra-partition-space 32M, we ensure that the last 32M of the
>> partition is unused: this space does not contain filesystem data and
>> can store the LUKS2 header.
>>
>> The implementation sets a difference between the partition and
>> filesystem size:
>> - With --fixed-size, the extra part space is removed from the
>> filesystem size.
>> - Otherwise (with or without --size flag), the extra part space is
>> added to the partition size.
>>
>> Signed-off-by: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
>>
>> CC: Alexander Kanavin <alex.kanavin@gmail.com>
>> CC: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
>> ---
>
> Hi Pierre-Loup,
>
> Thanks for the new version.
>
>> + def test_extra_partition_space(self):
>> + native_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE", "wic-tools")
>> +
>> + with NamedTemporaryFile("w", suffix=".wks") as tempf:
>> + tempf.write("bootloader --ptable gpt\n" \
>> + "part --ondisk hda --size 10M --extra-partition-space 10M --fstype=ext4\n" \
>> + "part --ondisk hda --fixed-size 20M --extra-partition-space 10M --fstype=ext4\n" \
>> + "part --source rootfs --ondisk hda --extra-partition-space 10M --fstype=ext4\n" \
>> + "part --source rootfs --ondisk hda --fixed-size 200M --extra-partition-space 10M --fstype=ext4\n")
>> + tempf.flush()
>> +
>> + _, wicimg = self._get_wic(tempf.name)
>> +
>> + res = runCmd("parted -m %s unit b p" % wicimg,
>> + native_sysroot=native_sysroot, stderr=subprocess.PIPE)
>> +
>
> This fails when "parted" is not installed on the host machine:
>
> 2025-10-03 13:42:20,053 - oe-selftest - INFO - 9: 73/93 615/639 (37.32s) (0 failed) (wic.Wic2.test_extra_partition_plugin)
> 2025-10-03 13:42:20,053 - oe-selftest - INFO - testtools.testresult.real._StringException: Traceback (most recent call last):
> File "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/meta/lib/oeqa/selftest/cases/wic.py", line 1683, in test_extra_partition_plugin
> self.assertEqual('4', result.output, msg="Expect 3 partitions, not %s" % result.output)
> File "/usr/lib/python3.11/unittest/case.py", line 873, in assertEqual
> assertion_func(first, second, msg=msg)
> File "/usr/lib/python3.11/unittest/case.py", line 1253, in assertMultiLineEqual
> self.fail(self._formatMessage(msg, standardMsg))
> File "/usr/lib/python3.11/unittest/case.py", line 703, in fail
> raise self.failureException(msg)
> AssertionError: '4' != "ERROR: Can't find executable parted\n1"
> - 4
> + ERROR: Can't find executable parted
> 1
> : Expect 3 partitions, not ERROR: Can't find executable parted
> 1
>
> https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2456
>
> You can easily reproduce this on a test machine by removing parted from
> the host. Also, test_wic_sector_size use parted but does not show the
> same issue:
>
> oe-selftest -r wic.Wic.test_wic_sector_size
> ...
> 2025-10-03 15:05:58,051 - oe-selftest - INFO - RESULTS - wic.Wic.test_wic_sector_size: PASSED (94.37s)
>
> I believe this comes from the following line and the associated
> try/finally block to revert to the default PATH variable:
>
> os.environ['PATH'] = get_bb_var("PATH", "wic-tools")
>
> Can you use something similar?
>
> Thanks,
> Mathieu
And now I see that this patch is merged and I'm replying to a mail sent
a month ago... Sorry.
Still, this will lead to intermittent failures, can you take a bit of
time to send a fix?
Thanks,
Mathieu
--
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [OE-core] [PATCH v5 1/2] wic: add --extra-partition-space option to set unused space
2025-10-03 15:20 ` Mathieu Dubois-Briand
@ 2025-10-03 21:22 ` Yoann Congal
2025-10-06 7:18 ` Pierre-loup GOSSE
0 siblings, 1 reply; 9+ messages in thread
From: Yoann Congal @ 2025-10-03 21:22 UTC (permalink / raw)
To: mathieu.dubois-briand
Cc: pierre-loup.gosse, openembedded-core, Alexander Kanavin
[-- Attachment #1: Type: text/plain, Size: 5841 bytes --]
Le ven. 3 oct. 2025 à 17:20, Mathieu Dubois-Briand via
lists.openembedded.org <mathieu.dubois-briand=
bootlin.com@lists.openembedded.org> a écrit :
> On Fri Oct 3, 2025 at 5:15 PM CEST, Mathieu Dubois-Briand wrote:
> > On Wed Sep 3, 2025 at 4:45 PM CEST, pierre-loup.gosse wrote:
> >> From: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
> >>
> >> By default, the content of the partition is filled by the filesystem
> >> without leaving any unused free space. The --extra-space flag adds
> >> extra space to the filesystem size, not to the partition.
> >>
> >> Unused free space after the filesystem can be useful for some cases,
> >> such as encrypting a partition at runtime.
> >> With --extra-partition-space 32M, we ensure that the last 32M of the
> >> partition is unused: this space does not contain filesystem data and
> >> can store the LUKS2 header.
> >>
> >> The implementation sets a difference between the partition and
> >> filesystem size:
> >> - With --fixed-size, the extra part space is removed from the
> >> filesystem size.
> >> - Otherwise (with or without --size flag), the extra part space is
> >> added to the partition size.
> >>
> >> Signed-off-by: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
> >>
> >> CC: Alexander Kanavin <alex.kanavin@gmail.com>
> >> CC: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
> >> ---
> >
> > Hi Pierre-Loup,
> >
> > Thanks for the new version.
> >
> >> + def test_extra_partition_space(self):
> >> + native_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE",
> "wic-tools")
> >> +
> >> + with NamedTemporaryFile("w", suffix=".wks") as tempf:
> >> + tempf.write("bootloader --ptable gpt\n" \
> >> + "part --ondisk hda --size 10M
> --extra-partition-space 10M --fstype=ext4\n" \
> >> + "part --ondisk hda
> --fixed-size 20M --extra-partition-space 10M --fstype=ext4\n" \
> >> + "part --source rootfs --ondisk hda
> --extra-partition-space 10M --fstype=ext4\n" \
> >> + "part --source rootfs --ondisk hda
> --fixed-size 200M --extra-partition-space 10M --fstype=ext4\n")
> >> + tempf.flush()
> >> +
> >> + _, wicimg = self._get_wic(tempf.name)
> >> +
> >> + res = runCmd("parted -m %s unit b p" % wicimg,
> >> + native_sysroot=native_sysroot,
> stderr=subprocess.PIPE)
> >> +
> >
> > This fails when "parted" is not installed on the host machine:
> >
> > 2025-10-03 13:42:20,053 - oe-selftest - INFO - 9: 73/93 615/639 (37.32s)
> (0 failed) (wic.Wic2.test_extra_partition_plugin)
> > 2025-10-03 13:42:20,053 - oe-selftest - INFO -
> testtools.testresult.real._StringException: Traceback (most recent call
> last):
> > File
> "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/meta/lib/oeqa/selftest/cases/wic.py",
> line 1683, in test_extra_partition_plugin
> > self.assertEqual('4', result.output, msg="Expect 3 partitions, not
> %s" % result.output)
> > File "/usr/lib/python3.11/unittest/case.py", line 873, in assertEqual
> > assertion_func(first, second, msg=msg)
> > File "/usr/lib/python3.11/unittest/case.py", line 1253, in
> assertMultiLineEqual
> > self.fail(self._formatMessage(msg, standardMsg))
> > File "/usr/lib/python3.11/unittest/case.py", line 703, in fail
> > raise self.failureException(msg)
> > AssertionError: '4' != "ERROR: Can't find executable parted\n1"
> > - 4
> > + ERROR: Can't find executable parted
> > 1
> > : Expect 3 partitions, not ERROR: Can't find executable parted
> > 1
> >
> > https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2456
> >
> > You can easily reproduce this on a test machine by removing parted from
> > the host. Also, test_wic_sector_size use parted but does not show the
> > same issue:
> >
> > oe-selftest -r wic.Wic.test_wic_sector_size
> > ...
> > 2025-10-03 15:05:58,051 - oe-selftest - INFO - RESULTS -
> wic.Wic.test_wic_sector_size: PASSED (94.37s)
> >
> > I believe this comes from the following line and the associated
> > try/finally block to revert to the default PATH variable:
> >
> > os.environ['PATH'] = get_bb_var("PATH", "wic-tools")
> >
> > Can you use something similar?
> >
> > Thanks,
> > Mathieu
>
> And now I see that this patch is merged and I'm replying to a mail sent
> a month ago... Sorry.
>
> Still, this will lead to intermittent failures, can you take a bit of
> time to send a fix?
>
Hi Mathieu,
Here are the fixes: (Since that was triggered on AB, I prefered to do it
quickly)
There are 2 patches.
First one, for master:
[PATCH 1/2] oeqa/selftest/wic: fix PATH for
wic.Wic2.test_extra_partition_space
https://lists.openembedded.org/g/openembedded-core/topic/patch_1_2/115578268
Second one for master-next: (but it could be squashed with Pierre-Loup
patch)
[PATCH 2/2 master-next] oeqa/selftest/wic: fix PATH for
wic.Wic2.test_extra_partition_plugin
https://lists.openembedded.org/g/openembedded-core/topic/patch_2_2_master_next/115578271
Regards,
> Thanks,
> Mathieu
>
> --
> Mathieu Dubois-Briand, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#224397):
> https://lists.openembedded.org/g/openembedded-core/message/224397
> Mute This Topic: https://lists.openembedded.org/mt/115046314/4316185
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> yoann.congal@smile.fr]
> -=-=-=-=-=-=-=-=-=-=-=-
>
>
--
Yoann Congal
Smile ECS
[-- Attachment #2: Type: text/html, Size: 8866 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [OE-core] [PATCH v5 1/2] wic: add --extra-partition-space option to set unused space
2025-10-03 21:22 ` [OE-core] " Yoann Congal
@ 2025-10-06 7:18 ` Pierre-loup GOSSE
2025-10-06 8:09 ` Mathieu Dubois-Briand
0 siblings, 1 reply; 9+ messages in thread
From: Pierre-loup GOSSE @ 2025-10-06 7:18 UTC (permalink / raw)
To: mathieu.dubois-briand; +Cc: openembedded-core, Alexander Kanavin, Yoann Congal
[-- Attachment #1: Type: text/plain, Size: 6246 bytes --]
Hi Mathieu,
Sorry for the inconvenience, I’ll pay more attention next time. Thanks
Yoann for the quick fix.
Pierre-Loup,
On Fri, Oct 3, 2025 at 11:22 PM Yoann Congal <yoann.congal@smile.fr> wrote:
>
>
> Le ven. 3 oct. 2025 à 17:20, Mathieu Dubois-Briand via
> lists.openembedded.org <mathieu.dubois-briand=
> bootlin.com@lists.openembedded.org> a écrit :
>
>> On Fri Oct 3, 2025 at 5:15 PM CEST, Mathieu Dubois-Briand wrote:
>> > On Wed Sep 3, 2025 at 4:45 PM CEST, pierre-loup.gosse wrote:
>> >> From: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
>> >>
>> >> By default, the content of the partition is filled by the filesystem
>> >> without leaving any unused free space. The --extra-space flag adds
>> >> extra space to the filesystem size, not to the partition.
>> >>
>> >> Unused free space after the filesystem can be useful for some cases,
>> >> such as encrypting a partition at runtime.
>> >> With --extra-partition-space 32M, we ensure that the last 32M of the
>> >> partition is unused: this space does not contain filesystem data and
>> >> can store the LUKS2 header.
>> >>
>> >> The implementation sets a difference between the partition and
>> >> filesystem size:
>> >> - With --fixed-size, the extra part space is removed from the
>> >> filesystem size.
>> >> - Otherwise (with or without --size flag), the extra part space is
>> >> added to the partition size.
>> >>
>> >> Signed-off-by: Pierre-Loup GOSSE <pierre-loup.gosse@smile.fr>
>> >>
>> >> CC: Alexander Kanavin <alex.kanavin@gmail.com>
>> >> CC: Mathieu Dubois-Briand <mathieu.dubois-briand@bootlin.com>
>> >> ---
>> >
>> > Hi Pierre-Loup,
>> >
>> > Thanks for the new version.
>> >
>> >> + def test_extra_partition_space(self):
>> >> + native_sysroot = get_bb_var("RECIPE_SYSROOT_NATIVE",
>> "wic-tools")
>> >> +
>> >> + with NamedTemporaryFile("w", suffix=".wks") as tempf:
>> >> + tempf.write("bootloader --ptable gpt\n" \
>> >> + "part --ondisk hda --size
>> 10M --extra-partition-space 10M --fstype=ext4\n" \
>> >> + "part --ondisk hda
>> --fixed-size 20M --extra-partition-space 10M --fstype=ext4\n" \
>> >> + "part --source rootfs --ondisk hda
>> --extra-partition-space 10M --fstype=ext4\n" \
>> >> + "part --source rootfs --ondisk hda
>> --fixed-size 200M --extra-partition-space 10M --fstype=ext4\n")
>> >> + tempf.flush()
>> >> +
>> >> + _, wicimg = self._get_wic(tempf.name)
>> >> +
>> >> + res = runCmd("parted -m %s unit b p" % wicimg,
>> >> + native_sysroot=native_sysroot,
>> stderr=subprocess.PIPE)
>> >> +
>> >
>> > This fails when "parted" is not installed on the host machine:
>> >
>> > 2025-10-03 13:42:20,053 - oe-selftest - INFO - 9: 73/93 615/639
>> (37.32s) (0 failed) (wic.Wic2.test_extra_partition_plugin)
>> > 2025-10-03 13:42:20,053 - oe-selftest - INFO -
>> testtools.testresult.real._StringException: Traceback (most recent call
>> last):
>> > File
>> "/srv/pokybuild/yocto-worker/oe-selftest-debian/build/meta/lib/oeqa/selftest/cases/wic.py",
>> line 1683, in test_extra_partition_plugin
>> > self.assertEqual('4', result.output, msg="Expect 3 partitions, not
>> %s" % result.output)
>> > File "/usr/lib/python3.11/unittest/case.py", line 873, in assertEqual
>> > assertion_func(first, second, msg=msg)
>> > File "/usr/lib/python3.11/unittest/case.py", line 1253, in
>> assertMultiLineEqual
>> > self.fail(self._formatMessage(msg, standardMsg))
>> > File "/usr/lib/python3.11/unittest/case.py", line 703, in fail
>> > raise self.failureException(msg)
>> > AssertionError: '4' != "ERROR: Can't find executable parted\n1"
>> > - 4
>> > + ERROR: Can't find executable parted
>> > 1
>> > : Expect 3 partitions, not ERROR: Can't find executable parted
>> > 1
>> >
>> > https://autobuilder.yoctoproject.org/valkyrie/#/builders/35/builds/2456
>> >
>> > You can easily reproduce this on a test machine by removing parted from
>> > the host. Also, test_wic_sector_size use parted but does not show the
>> > same issue:
>> >
>> > oe-selftest -r wic.Wic.test_wic_sector_size
>> > ...
>> > 2025-10-03 15:05:58,051 - oe-selftest - INFO - RESULTS -
>> wic.Wic.test_wic_sector_size: PASSED (94.37s)
>> >
>> > I believe this comes from the following line and the associated
>> > try/finally block to revert to the default PATH variable:
>> >
>> > os.environ['PATH'] = get_bb_var("PATH", "wic-tools")
>> >
>> > Can you use something similar?
>> >
>> > Thanks,
>> > Mathieu
>>
>> And now I see that this patch is merged and I'm replying to a mail sent
>> a month ago... Sorry.
>>
>> Still, this will lead to intermittent failures, can you take a bit of
>> time to send a fix?
>>
>
> Hi Mathieu,
>
> Here are the fixes: (Since that was triggered on AB, I prefered to do it
> quickly)
> There are 2 patches.
>
> First one, for master:
> [PATCH 1/2] oeqa/selftest/wic: fix PATH for
> wic.Wic2.test_extra_partition_space
>
> https://lists.openembedded.org/g/openembedded-core/topic/patch_1_2/115578268
>
> Second one for master-next: (but it could be squashed with Pierre-Loup
> patch)
> [PATCH 2/2 master-next] oeqa/selftest/wic: fix PATH for
> wic.Wic2.test_extra_partition_plugin
>
> https://lists.openembedded.org/g/openembedded-core/topic/patch_2_2_master_next/115578271
>
> Regards,
>
>
>> Thanks,
>> Mathieu
>>
>> --
>> Mathieu Dubois-Briand, Bootlin
>> Embedded Linux and Kernel engineering
>> https://bootlin.com
>>
>>
>> -=-=-=-=-=-=-=-=-=-=-=-
>> Links: You receive all messages sent to this group.
>> View/Reply Online (#224397):
>> https://lists.openembedded.org/g/openembedded-core/message/224397
>> Mute This Topic: https://lists.openembedded.org/mt/115046314/4316185
>> Group Owner: openembedded-core+owner@lists.openembedded.org
>> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
>> yoann.congal@smile.fr]
>> -=-=-=-=-=-=-=-=-=-=-=-
>>
>>
>
> --
> Yoann Congal
> Smile ECS
>
[-- Attachment #2: Type: text/html, Size: 9475 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [OE-core] [PATCH v5 1/2] wic: add --extra-partition-space option to set unused space
2025-10-06 7:18 ` Pierre-loup GOSSE
@ 2025-10-06 8:09 ` Mathieu Dubois-Briand
0 siblings, 0 replies; 9+ messages in thread
From: Mathieu Dubois-Briand @ 2025-10-06 8:09 UTC (permalink / raw)
To: pierre-loup.gosse; +Cc: openembedded-core, Alexander Kanavin, Yoann Congal
On Mon Oct 6, 2025 at 9:18 AM CEST, Pierre-loup GOSSE via lists.openembedded.org wrote:
> Hi Mathieu,
>
> Sorry for the inconvenience, I’ll pay more attention next time. Thanks
> Yoann for the quick fix.
>
> Pierre-Loup,
>
> On Fri, Oct 3, 2025 at 11:22 PM Yoann Congal <yoann.congal@smile.fr> wrote:
>
Hi Pierre-Loup,
No problem, these errors are not so easy to catch, as they depend on
what is installed on the host. That's also why it took a bit of time
before it was seen on the autobuilder.
I confirm the fix from Yoann seems to fix it.
--
Mathieu Dubois-Briand, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2025-10-06 8:09 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2025-09-03 14:45 [PATCH v5 1/2] wic: add --extra-partition-space option to set unused space pierre-loup.gosse
2025-09-03 14:45 ` [PATCH v5 2/2] wic: rename wks flag --extra-space to --extra-filesystem-space pierre-loup.gosse
2025-09-09 14:30 ` [OE-core] " Ross Burton
2025-09-09 14:42 ` Pierre-loup GOSSE
2025-10-03 15:15 ` [PATCH v5 1/2] wic: add --extra-partition-space option to set unused space Mathieu Dubois-Briand
2025-10-03 15:20 ` Mathieu Dubois-Briand
2025-10-03 21:22 ` [OE-core] " Yoann Congal
2025-10-06 7:18 ` Pierre-loup GOSSE
2025-10-06 8:09 ` Mathieu Dubois-Briand
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox