* [oe-core][PATCH 0/1] wic: Add support for F2FS in WIC rootfs plugin
@ 2023-02-28 15:00 Anna-Lena Marx
2023-02-28 15:00 ` [oe-core][PATCH 1/1] " Anna-Lena Marx
0 siblings, 1 reply; 8+ messages in thread
From: Anna-Lena Marx @ 2023-02-28 15:00 UTC (permalink / raw)
To: openembedded-core; +Cc: Anna-Lena Marx
Hi there,
Yocto/OpenEmbedded supports generating F2FS rootfs in general
for years but WIC does not, yet.
The patch adds support to do so in the "rootfs" WIC plugin.
Generating the F2FS rootfs follows mostly the way it is implemented
in meta/classes-recipe/image_types.bbclass.
Thereby, F2FS needs a minimum size in order to work correctly
as it uses an overprovision functionality.
-- Anna
Anna-Lena Marx (1):
wic: Add support for F2FS in WIC rootfs plugin
meta/classes-recipe/image_types_wic.bbclass | 2 +-
meta/lib/oeqa/selftest/cases/wic.py | 2 ++
meta/recipes-core/meta/wic-tools.bb | 2 +-
scripts/lib/wic/help.py | 1 +
scripts/lib/wic/ksparser.py | 2 +-
scripts/lib/wic/misc.py | 1 +
scripts/lib/wic/partition.py | 32 ++++++++++++++++++++-
7 files changed, 38 insertions(+), 4 deletions(-)
--
2.39.2
^ permalink raw reply [flat|nested] 8+ messages in thread
* [oe-core][PATCH 1/1] wic: Add support for F2FS in WIC rootfs plugin
2023-02-28 15:00 [oe-core][PATCH 0/1] wic: Add support for F2FS in WIC rootfs plugin Anna-Lena Marx
@ 2023-02-28 15:00 ` Anna-Lena Marx
2023-02-28 15:20 ` Alexandre Belloni
0 siblings, 1 reply; 8+ messages in thread
From: Anna-Lena Marx @ 2023-02-28 15:00 UTC (permalink / raw)
To: openembedded-core; +Cc: Anna-Lena Marx
Building F2FS rootfs is supported in Yocto/OpenEmbedded -
see image_types.bbclass, but not in WIC.
This adds it as a nice convenience feature.
F2FS needs a minimum size of ~500M to operate correctly due to
an overprovision functionality.
Signed-off-by: Anna-Lena Marx <anna-lena.marx@inovex.de>
---
meta/classes-recipe/image_types_wic.bbclass | 2 +-
meta/lib/oeqa/selftest/cases/wic.py | 2 ++
meta/recipes-core/meta/wic-tools.bb | 2 +-
scripts/lib/wic/help.py | 1 +
scripts/lib/wic/ksparser.py | 2 +-
scripts/lib/wic/misc.py | 1 +
scripts/lib/wic/partition.py | 32 ++++++++++++++++++++-
7 files changed, 38 insertions(+), 4 deletions(-)
diff --git a/meta/classes-recipe/image_types_wic.bbclass b/meta/classes-recipe/image_types_wic.bbclass
index c339b9bdfb..8ae5ecca4d 100644
--- a/meta/classes-recipe/image_types_wic.bbclass
+++ b/meta/classes-recipe/image_types_wic.bbclass
@@ -89,7 +89,7 @@ do_image_wic[recrdeptask] += "do_deploy"
do_image_wic[deptask] += "do_image_complete"
WKS_FILE_DEPENDS_DEFAULT = '${@bb.utils.contains_any("BUILD_ARCH", [ 'x86_64', 'i686' ], "syslinux-native", "",d)}'
-WKS_FILE_DEPENDS_DEFAULT += "bmap-tools-native cdrtools-native btrfs-tools-native squashfs-tools-native e2fsprogs-native erofs-utils-native"
+WKS_FILE_DEPENDS_DEFAULT += "bmap-tools-native cdrtools-native btrfs-tools-native squashfs-tools-native e2fsprogs-native erofs-utils-native f2fs-tools-native"
# Unified kernel images need objcopy
WKS_FILE_DEPENDS_DEFAULT += "virtual/${MLPREFIX}${TARGET_PREFIX}binutils"
WKS_FILE_DEPENDS_BOOTLOADERS = ""
diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py
index ca1abb970a..5e255e35c7 100644
--- a/meta/lib/oeqa/selftest/cases/wic.py
+++ b/meta/lib/oeqa/selftest/cases/wic.py
@@ -1188,6 +1188,7 @@ class Wic2(WicTestCase):
with NamedTemporaryFile("w", suffix=".wks") as wks:
wks.writelines(['part ext2 --fstype ext2 --source rootfs\n',
'part btrfs --fstype btrfs --source rootfs --size 40M\n',
+ 'part f2fs --fstype f2fs --source rootfs --size 50M\n',
'part squash --fstype squashfs --source rootfs\n',
'part swap --fstype swap --size 1M\n',
'part emptyvfat --fstype vfat --size 1M\n',
@@ -1258,6 +1259,7 @@ class Wic2(WicTestCase):
wks.writelines(
['part ext2 --fstype ext2 --source rootfs --mkfs-extraopts "-D -F -i 8192"\n',
"part btrfs --fstype btrfs --source rootfs --size 40M --mkfs-extraopts='--quiet'\n",
+ "part f2fs --fstype f2fs --source rootfs --size 50M --mkfs-extraopts='-O extra_attr -o 20'\n",
'part squash --fstype squashfs --source rootfs --mkfs-extraopts "-no-sparse -b 4096"\n',
'part emptyvfat --fstype vfat --size 1M --mkfs-extraopts "-S 1024 -s 64"\n',
'part emptymsdos --fstype msdos --size 1M --mkfs-extraopts "-S 1024 -s 64"\n',
diff --git a/meta/recipes-core/meta/wic-tools.bb b/meta/recipes-core/meta/wic-tools.bb
index 9282d36a4d..d30ce66b56 100644
--- a/meta/recipes-core/meta/wic-tools.bb
+++ b/meta/recipes-core/meta/wic-tools.bb
@@ -7,7 +7,7 @@ DEPENDS = "\
mtools-native bmap-tools-native grub-native cdrtools-native \
btrfs-tools-native squashfs-tools-native pseudo-native \
e2fsprogs-native util-linux-native tar-native erofs-utils-native \
- virtual/${TARGET_PREFIX}binutils \
+ f2fs-tools-native virtual/${TARGET_PREFIX}binutils \
"
DEPENDS:append:x86 = " syslinux-native syslinux grub-efi systemd-boot"
DEPENDS:append:x86-64 = " syslinux-native syslinux grub-efi systemd-boot"
diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
index 73e3380cde..0094065931 100644
--- a/scripts/lib/wic/help.py
+++ b/scripts/lib/wic/help.py
@@ -931,6 +931,7 @@ DESCRIPTION
btrfs
squashfs
erofs
+ f2fs
swap
--fsoptions: Specifies a free-form string of options to be
diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
index d1e546b12d..e10ce6581d 100644
--- a/scripts/lib/wic/ksparser.py
+++ b/scripts/lib/wic/ksparser.py
@@ -159,7 +159,7 @@ class KickStart():
part.add_argument('--fstype', default='vfat',
choices=('ext2', 'ext3', 'ext4', 'btrfs',
'squashfs', 'vfat', 'msdos', 'erofs',
- 'swap', 'none'))
+ 'f2fs', 'swap', 'none'))
part.add_argument('--mkfs-extraopts', default='')
part.add_argument('--label')
part.add_argument('--use-label', action='store_true')
diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py
index 2b90821b30..dc84a37423 100644
--- a/scripts/lib/wic/misc.py
+++ b/scripts/lib/wic/misc.py
@@ -41,6 +41,7 @@ NATIVE_RECIPES = {"bmaptool": "bmap-tools",
"mkfs.ext3": "e2fsprogs",
"mkfs.ext4": "e2fsprogs",
"mkfs.vfat": "dosfstools",
+ "mkfs.f2fs": "f2fs-tools",
"mksquashfs": "squashfs-tools",
"mkswap": "util-linux",
"mmd": "mtools",
diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
index 382afa44bc..6adcaa041f 100644
--- a/scripts/lib/wic/partition.py
+++ b/scripts/lib/wic/partition.py
@@ -208,7 +208,7 @@ class Partition():
Prepare content for a rootfs partition i.e. create a partition
and fill it from a /rootfs dir.
- Currently handles ext2/3/4, btrfs, vfat and squashfs.
+ Currently handles ext2/3/4, btrfs, vfat, squashfs and f2fs.
"""
rootfs = "%s/rootfs_%s.%s.%s" % (cr_workdir, self.label,
@@ -351,6 +351,36 @@ class Partition():
self.mkfs_extraopts, self.fsuuid, rootfs)
exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
+ def prepare_rootfs_f2fs(self, rootfs, cr_workdir, oe_builddir, rootfs_dir,
+ native_sysroot, pseudo):
+ """
+ Prepare content for a f2fs rootfs partition.
+ """
+ du_cmd = "du -ks %s" % rootfs_dir
+ out = exec_cmd(du_cmd)
+ actual_rootfs_size = int(out.split()[0])
+
+ min_f2fs_size = 524288
+ rootfs_size = self.get_rootfs_size(actual_rootfs_size)
+ if rootfs_size < min_f2fs_size:
+ rootfs_size = min_f2fs_size
+
+ with open(rootfs, 'w') as sparse:
+ os.ftruncate(sparse.fileno(), rootfs_size * 1024)
+
+ extraopts = self.mkfs_extraopts or "-O extra_attr,compression"
+
+ label_str = ""
+ if self.label:
+ label_str = "-l %s" % self.label
+
+ mkfs_f2fs_cmd = "mkfs.f2fs %s %s %s" % \
+ (label_str, extraopts, rootfs)
+ exec_native_cmd(mkfs_f2fs_cmd, native_sysroot, pseudo=pseudo)
+
+ sload_f2fs_cmd = "sload.f2fs -f %s %s" % (rootfs_dir, rootfs)
+ exec_native_cmd(sload_f2fs_cmd, native_sysroot, pseudo=pseudo)
+
def prepare_rootfs_msdos(self, rootfs, cr_workdir, oe_builddir, rootfs_dir,
native_sysroot, pseudo):
"""
--
2.39.2
^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [oe-core][PATCH 1/1] wic: Add support for F2FS in WIC rootfs plugin
2023-02-28 15:00 ` [oe-core][PATCH 1/1] " Anna-Lena Marx
@ 2023-02-28 15:20 ` Alexandre Belloni
2023-03-01 14:36 ` Anna-Lena Marx
0 siblings, 1 reply; 8+ messages in thread
From: Alexandre Belloni @ 2023-02-28 15:20 UTC (permalink / raw)
To: anna-lena.marx; +Cc: openembedded-core
Hello,
this failed on the autobuilders:
ERROR: Nothing PROVIDES 'f2fs-tools-native' (but /home/pokybuild/yocto-worker/beaglebone/build/meta/recipes-sato/images/core-image-sato.bb, /home/pokybuild/yocto-worker/beaglebone/build/meta/recipes-sato/images/core-image-sato-sdk.bb, /home/pokybuild/yocto-worker/beaglebone/build/meta/recipes-core/images/core-image-minimal.bb, /home/pokybuild/yocto-worker/beaglebone/build/meta/recipes-core/images/core-image-minimal-dev.bb, /home/pokybuild/yocto-worker/beaglebone/build/meta/recipes-core/images/core-image-ptest-all.bb DEPENDS on or otherwise requires it). Close matches:
btrfs-tools-native
dosfstools-native
spirv-tools-native
https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/6790/steps/11/logs/stdio
On 28/02/2023 16:00:58+0100, Anna-Lena Marx via lists.openembedded.org wrote:
> Building F2FS rootfs is supported in Yocto/OpenEmbedded -
> see image_types.bbclass, but not in WIC.
> This adds it as a nice convenience feature.
>
> F2FS needs a minimum size of ~500M to operate correctly due to
> an overprovision functionality.
>
> Signed-off-by: Anna-Lena Marx <anna-lena.marx@inovex.de>
> ---
> meta/classes-recipe/image_types_wic.bbclass | 2 +-
> meta/lib/oeqa/selftest/cases/wic.py | 2 ++
> meta/recipes-core/meta/wic-tools.bb | 2 +-
> scripts/lib/wic/help.py | 1 +
> scripts/lib/wic/ksparser.py | 2 +-
> scripts/lib/wic/misc.py | 1 +
> scripts/lib/wic/partition.py | 32 ++++++++++++++++++++-
> 7 files changed, 38 insertions(+), 4 deletions(-)
>
> diff --git a/meta/classes-recipe/image_types_wic.bbclass b/meta/classes-recipe/image_types_wic.bbclass
> index c339b9bdfb..8ae5ecca4d 100644
> --- a/meta/classes-recipe/image_types_wic.bbclass
> +++ b/meta/classes-recipe/image_types_wic.bbclass
> @@ -89,7 +89,7 @@ do_image_wic[recrdeptask] += "do_deploy"
> do_image_wic[deptask] += "do_image_complete"
>
> WKS_FILE_DEPENDS_DEFAULT = '${@bb.utils.contains_any("BUILD_ARCH", [ 'x86_64', 'i686' ], "syslinux-native", "",d)}'
> -WKS_FILE_DEPENDS_DEFAULT += "bmap-tools-native cdrtools-native btrfs-tools-native squashfs-tools-native e2fsprogs-native erofs-utils-native"
> +WKS_FILE_DEPENDS_DEFAULT += "bmap-tools-native cdrtools-native btrfs-tools-native squashfs-tools-native e2fsprogs-native erofs-utils-native f2fs-tools-native"
> # Unified kernel images need objcopy
> WKS_FILE_DEPENDS_DEFAULT += "virtual/${MLPREFIX}${TARGET_PREFIX}binutils"
> WKS_FILE_DEPENDS_BOOTLOADERS = ""
> diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py
> index ca1abb970a..5e255e35c7 100644
> --- a/meta/lib/oeqa/selftest/cases/wic.py
> +++ b/meta/lib/oeqa/selftest/cases/wic.py
> @@ -1188,6 +1188,7 @@ class Wic2(WicTestCase):
> with NamedTemporaryFile("w", suffix=".wks") as wks:
> wks.writelines(['part ext2 --fstype ext2 --source rootfs\n',
> 'part btrfs --fstype btrfs --source rootfs --size 40M\n',
> + 'part f2fs --fstype f2fs --source rootfs --size 50M\n',
> 'part squash --fstype squashfs --source rootfs\n',
> 'part swap --fstype swap --size 1M\n',
> 'part emptyvfat --fstype vfat --size 1M\n',
> @@ -1258,6 +1259,7 @@ class Wic2(WicTestCase):
> wks.writelines(
> ['part ext2 --fstype ext2 --source rootfs --mkfs-extraopts "-D -F -i 8192"\n',
> "part btrfs --fstype btrfs --source rootfs --size 40M --mkfs-extraopts='--quiet'\n",
> + "part f2fs --fstype f2fs --source rootfs --size 50M --mkfs-extraopts='-O extra_attr -o 20'\n",
> 'part squash --fstype squashfs --source rootfs --mkfs-extraopts "-no-sparse -b 4096"\n',
> 'part emptyvfat --fstype vfat --size 1M --mkfs-extraopts "-S 1024 -s 64"\n',
> 'part emptymsdos --fstype msdos --size 1M --mkfs-extraopts "-S 1024 -s 64"\n',
> diff --git a/meta/recipes-core/meta/wic-tools.bb b/meta/recipes-core/meta/wic-tools.bb
> index 9282d36a4d..d30ce66b56 100644
> --- a/meta/recipes-core/meta/wic-tools.bb
> +++ b/meta/recipes-core/meta/wic-tools.bb
> @@ -7,7 +7,7 @@ DEPENDS = "\
> mtools-native bmap-tools-native grub-native cdrtools-native \
> btrfs-tools-native squashfs-tools-native pseudo-native \
> e2fsprogs-native util-linux-native tar-native erofs-utils-native \
> - virtual/${TARGET_PREFIX}binutils \
> + f2fs-tools-native virtual/${TARGET_PREFIX}binutils \
> "
> DEPENDS:append:x86 = " syslinux-native syslinux grub-efi systemd-boot"
> DEPENDS:append:x86-64 = " syslinux-native syslinux grub-efi systemd-boot"
> diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
> index 73e3380cde..0094065931 100644
> --- a/scripts/lib/wic/help.py
> +++ b/scripts/lib/wic/help.py
> @@ -931,6 +931,7 @@ DESCRIPTION
> btrfs
> squashfs
> erofs
> + f2fs
> swap
>
> --fsoptions: Specifies a free-form string of options to be
> diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
> index d1e546b12d..e10ce6581d 100644
> --- a/scripts/lib/wic/ksparser.py
> +++ b/scripts/lib/wic/ksparser.py
> @@ -159,7 +159,7 @@ class KickStart():
> part.add_argument('--fstype', default='vfat',
> choices=('ext2', 'ext3', 'ext4', 'btrfs',
> 'squashfs', 'vfat', 'msdos', 'erofs',
> - 'swap', 'none'))
> + 'f2fs', 'swap', 'none'))
> part.add_argument('--mkfs-extraopts', default='')
> part.add_argument('--label')
> part.add_argument('--use-label', action='store_true')
> diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py
> index 2b90821b30..dc84a37423 100644
> --- a/scripts/lib/wic/misc.py
> +++ b/scripts/lib/wic/misc.py
> @@ -41,6 +41,7 @@ NATIVE_RECIPES = {"bmaptool": "bmap-tools",
> "mkfs.ext3": "e2fsprogs",
> "mkfs.ext4": "e2fsprogs",
> "mkfs.vfat": "dosfstools",
> + "mkfs.f2fs": "f2fs-tools",
> "mksquashfs": "squashfs-tools",
> "mkswap": "util-linux",
> "mmd": "mtools",
> diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
> index 382afa44bc..6adcaa041f 100644
> --- a/scripts/lib/wic/partition.py
> +++ b/scripts/lib/wic/partition.py
> @@ -208,7 +208,7 @@ class Partition():
> Prepare content for a rootfs partition i.e. create a partition
> and fill it from a /rootfs dir.
>
> - Currently handles ext2/3/4, btrfs, vfat and squashfs.
> + Currently handles ext2/3/4, btrfs, vfat, squashfs and f2fs.
> """
>
> rootfs = "%s/rootfs_%s.%s.%s" % (cr_workdir, self.label,
> @@ -351,6 +351,36 @@ class Partition():
> self.mkfs_extraopts, self.fsuuid, rootfs)
> exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
>
> + def prepare_rootfs_f2fs(self, rootfs, cr_workdir, oe_builddir, rootfs_dir,
> + native_sysroot, pseudo):
> + """
> + Prepare content for a f2fs rootfs partition.
> + """
> + du_cmd = "du -ks %s" % rootfs_dir
> + out = exec_cmd(du_cmd)
> + actual_rootfs_size = int(out.split()[0])
> +
> + min_f2fs_size = 524288
> + rootfs_size = self.get_rootfs_size(actual_rootfs_size)
> + if rootfs_size < min_f2fs_size:
> + rootfs_size = min_f2fs_size
> +
> + with open(rootfs, 'w') as sparse:
> + os.ftruncate(sparse.fileno(), rootfs_size * 1024)
> +
> + extraopts = self.mkfs_extraopts or "-O extra_attr,compression"
> +
> + label_str = ""
> + if self.label:
> + label_str = "-l %s" % self.label
> +
> + mkfs_f2fs_cmd = "mkfs.f2fs %s %s %s" % \
> + (label_str, extraopts, rootfs)
> + exec_native_cmd(mkfs_f2fs_cmd, native_sysroot, pseudo=pseudo)
> +
> + sload_f2fs_cmd = "sload.f2fs -f %s %s" % (rootfs_dir, rootfs)
> + exec_native_cmd(sload_f2fs_cmd, native_sysroot, pseudo=pseudo)
> +
> def prepare_rootfs_msdos(self, rootfs, cr_workdir, oe_builddir, rootfs_dir,
> native_sysroot, pseudo):
> """
> --
> 2.39.2
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#177839): https://lists.openembedded.org/g/openembedded-core/message/177839
> Mute This Topic: https://lists.openembedded.org/mt/97291696/3617179
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alexandre.belloni@bootlin.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
--
Alexandre Belloni, co-owner and COO, Bootlin
Embedded Linux and Kernel engineering
https://bootlin.com
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [oe-core][PATCH 1/1] wic: Add support for F2FS in WIC rootfs plugin
2023-02-28 15:20 ` Alexandre Belloni
@ 2023-03-01 14:36 ` Anna-Lena Marx
2023-03-01 16:24 ` Alexander Kanavin
0 siblings, 1 reply; 8+ messages in thread
From: Anna-Lena Marx @ 2023-03-01 14:36 UTC (permalink / raw)
To: Alexandre Belloni; +Cc: openembedded-core
[-- Attachment #1: Type: text/plain, Size: 10282 bytes --]
Hi,
took me a while to reproduce the failure locally.
It's because meta-oe and meta-filesystems are not as default included in
the bblayers.conf used for oe-selftest.
I don't want to touch bblayers and I think this is not intended as it would
be the same for e.g. erofs for which no test case was added.
Alexander Kanavin pointed me on the poky mailing list to these tests and in
fact, it's good to have them in general.
So what's the preferred way for oe-core?
Leave the tests and add the dependencies (for the specific tests) or just
remove the test cases?
-- Anna
Am Di., 28. Feb. 2023 um 16:20 Uhr schrieb Alexandre Belloni <
alexandre.belloni@bootlin.com>:
> Hello,
>
> this failed on the autobuilders:
>
> ERROR: Nothing PROVIDES 'f2fs-tools-native' (but
> /home/pokybuild/yocto-worker/beaglebone/build/meta/recipes-sato/images/
> core-image-sato.bb,
> /home/pokybuild/yocto-worker/beaglebone/build/meta/recipes-sato/images/
> core-image-sato-sdk.bb,
> /home/pokybuild/yocto-worker/beaglebone/build/meta/recipes-core/images/
> core-image-minimal.bb,
> /home/pokybuild/yocto-worker/beaglebone/build/meta/recipes-core/images/
> core-image-minimal-dev.bb,
> /home/pokybuild/yocto-worker/beaglebone/build/meta/recipes-core/images/
> core-image-ptest-all.bb DEPENDS on or otherwise requires it). Close
> matches:
> btrfs-tools-native
> dosfstools-native
> spirv-tools-native
>
>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/6790/steps/11/logs/stdio
>
> On 28/02/2023 16:00:58+0100, Anna-Lena Marx via lists.openembedded.org
> wrote:
> > Building F2FS rootfs is supported in Yocto/OpenEmbedded -
> > see image_types.bbclass, but not in WIC.
> > This adds it as a nice convenience feature.
> >
> > F2FS needs a minimum size of ~500M to operate correctly due to
> > an overprovision functionality.
> >
> > Signed-off-by: Anna-Lena Marx <anna-lena.marx@inovex.de>
> > ---
> > meta/classes-recipe/image_types_wic.bbclass | 2 +-
> > meta/lib/oeqa/selftest/cases/wic.py | 2 ++
> > meta/recipes-core/meta/wic-tools.bb | 2 +-
> > scripts/lib/wic/help.py | 1 +
> > scripts/lib/wic/ksparser.py | 2 +-
> > scripts/lib/wic/misc.py | 1 +
> > scripts/lib/wic/partition.py | 32 ++++++++++++++++++++-
> > 7 files changed, 38 insertions(+), 4 deletions(-)
> >
> > diff --git a/meta/classes-recipe/image_types_wic.bbclass
> b/meta/classes-recipe/image_types_wic.bbclass
> > index c339b9bdfb..8ae5ecca4d 100644
> > --- a/meta/classes-recipe/image_types_wic.bbclass
> > +++ b/meta/classes-recipe/image_types_wic.bbclass
> > @@ -89,7 +89,7 @@ do_image_wic[recrdeptask] += "do_deploy"
> > do_image_wic[deptask] += "do_image_complete"
> >
> > WKS_FILE_DEPENDS_DEFAULT = '${@bb.utils.contains_any("BUILD_ARCH", [
> 'x86_64', 'i686' ], "syslinux-native", "",d)}'
> > -WKS_FILE_DEPENDS_DEFAULT += "bmap-tools-native cdrtools-native
> btrfs-tools-native squashfs-tools-native e2fsprogs-native
> erofs-utils-native"
> > +WKS_FILE_DEPENDS_DEFAULT += "bmap-tools-native cdrtools-native
> btrfs-tools-native squashfs-tools-native e2fsprogs-native
> erofs-utils-native f2fs-tools-native"
> > # Unified kernel images need objcopy
> > WKS_FILE_DEPENDS_DEFAULT +=
> "virtual/${MLPREFIX}${TARGET_PREFIX}binutils"
> > WKS_FILE_DEPENDS_BOOTLOADERS = ""
> > diff --git a/meta/lib/oeqa/selftest/cases/wic.py
> b/meta/lib/oeqa/selftest/cases/wic.py
> > index ca1abb970a..5e255e35c7 100644
> > --- a/meta/lib/oeqa/selftest/cases/wic.py
> > +++ b/meta/lib/oeqa/selftest/cases/wic.py
> > @@ -1188,6 +1188,7 @@ class Wic2(WicTestCase):
> > with NamedTemporaryFile("w", suffix=".wks") as wks:
> > wks.writelines(['part ext2 --fstype ext2 --source
> rootfs\n',
> > 'part btrfs --fstype btrfs --source
> rootfs --size 40M\n',
> > + 'part f2fs --fstype f2fs --source
> rootfs --size 50M\n',
> > 'part squash --fstype squashfs --source
> rootfs\n',
> > 'part swap --fstype swap --size 1M\n',
> > 'part emptyvfat --fstype vfat --size
> 1M\n',
> > @@ -1258,6 +1259,7 @@ class Wic2(WicTestCase):
> > wks.writelines(
> > ['part ext2 --fstype ext2 --source rootfs
> --mkfs-extraopts "-D -F -i 8192"\n',
> > "part btrfs --fstype btrfs --source rootfs --size
> 40M --mkfs-extraopts='--quiet'\n",
> > + "part f2fs --fstype f2fs --source rootfs --size
> 50M --mkfs-extraopts='-O extra_attr -o 20'\n",
> > 'part squash --fstype squashfs --source rootfs
> --mkfs-extraopts "-no-sparse -b 4096"\n',
> > 'part emptyvfat --fstype vfat --size 1M
> --mkfs-extraopts "-S 1024 -s 64"\n',
> > 'part emptymsdos --fstype msdos --size 1M
> --mkfs-extraopts "-S 1024 -s 64"\n',
> > diff --git a/meta/recipes-core/meta/wic-tools.bb
> b/meta/recipes-core/meta/wic-tools.bb
> > index 9282d36a4d..d30ce66b56 100644
> > --- a/meta/recipes-core/meta/wic-tools.bb
> > +++ b/meta/recipes-core/meta/wic-tools.bb
> > @@ -7,7 +7,7 @@ DEPENDS = "\
> > mtools-native bmap-tools-native grub-native cdrtools-native \
> > btrfs-tools-native squashfs-tools-native pseudo-native \
> > e2fsprogs-native util-linux-native tar-native
> erofs-utils-native \
> > - virtual/${TARGET_PREFIX}binutils \
> > + f2fs-tools-native virtual/${TARGET_PREFIX}binutils \
> > "
> > DEPENDS:append:x86 = " syslinux-native syslinux grub-efi systemd-boot"
> > DEPENDS:append:x86-64 = " syslinux-native syslinux grub-efi
> systemd-boot"
> > diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
> > index 73e3380cde..0094065931 100644
> > --- a/scripts/lib/wic/help.py
> > +++ b/scripts/lib/wic/help.py
> > @@ -931,6 +931,7 @@ DESCRIPTION
> > btrfs
> > squashfs
> > erofs
> > + f2fs
> > swap
> >
> > --fsoptions: Specifies a free-form string of options to be
> > diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
> > index d1e546b12d..e10ce6581d 100644
> > --- a/scripts/lib/wic/ksparser.py
> > +++ b/scripts/lib/wic/ksparser.py
> > @@ -159,7 +159,7 @@ class KickStart():
> > part.add_argument('--fstype', default='vfat',
> > choices=('ext2', 'ext3', 'ext4', 'btrfs',
> > 'squashfs', 'vfat', 'msdos', 'erofs',
> > - 'swap', 'none'))
> > + 'f2fs', 'swap', 'none'))
> > part.add_argument('--mkfs-extraopts', default='')
> > part.add_argument('--label')
> > part.add_argument('--use-label', action='store_true')
> > diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py
> > index 2b90821b30..dc84a37423 100644
> > --- a/scripts/lib/wic/misc.py
> > +++ b/scripts/lib/wic/misc.py
> > @@ -41,6 +41,7 @@ NATIVE_RECIPES = {"bmaptool": "bmap-tools",
> > "mkfs.ext3": "e2fsprogs",
> > "mkfs.ext4": "e2fsprogs",
> > "mkfs.vfat": "dosfstools",
> > + "mkfs.f2fs": "f2fs-tools",
> > "mksquashfs": "squashfs-tools",
> > "mkswap": "util-linux",
> > "mmd": "mtools",
> > diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
> > index 382afa44bc..6adcaa041f 100644
> > --- a/scripts/lib/wic/partition.py
> > +++ b/scripts/lib/wic/partition.py
> > @@ -208,7 +208,7 @@ class Partition():
> > Prepare content for a rootfs partition i.e. create a partition
> > and fill it from a /rootfs dir.
> >
> > - Currently handles ext2/3/4, btrfs, vfat and squashfs.
> > + Currently handles ext2/3/4, btrfs, vfat, squashfs and f2fs.
> > """
> >
> > rootfs = "%s/rootfs_%s.%s.%s" % (cr_workdir, self.label,
> > @@ -351,6 +351,36 @@ class Partition():
> > self.mkfs_extraopts, self.fsuuid, rootfs)
> > exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
> >
> > + def prepare_rootfs_f2fs(self, rootfs, cr_workdir, oe_builddir,
> rootfs_dir,
> > + native_sysroot, pseudo):
> > + """
> > + Prepare content for a f2fs rootfs partition.
> > + """
> > + du_cmd = "du -ks %s" % rootfs_dir
> > + out = exec_cmd(du_cmd)
> > + actual_rootfs_size = int(out.split()[0])
> > +
> > + min_f2fs_size = 524288
> > + rootfs_size = self.get_rootfs_size(actual_rootfs_size)
> > + if rootfs_size < min_f2fs_size:
> > + rootfs_size = min_f2fs_size
> > +
> > + with open(rootfs, 'w') as sparse:
> > + os.ftruncate(sparse.fileno(), rootfs_size * 1024)
> > +
> > + extraopts = self.mkfs_extraopts or "-O extra_attr,compression"
> > +
> > + label_str = ""
> > + if self.label:
> > + label_str = "-l %s" % self.label
> > +
> > + mkfs_f2fs_cmd = "mkfs.f2fs %s %s %s" % \
> > + (label_str, extraopts, rootfs)
> > + exec_native_cmd(mkfs_f2fs_cmd, native_sysroot, pseudo=pseudo)
> > +
> > + sload_f2fs_cmd = "sload.f2fs -f %s %s" % (rootfs_dir, rootfs)
> > + exec_native_cmd(sload_f2fs_cmd, native_sysroot, pseudo=pseudo)
> > +
> > def prepare_rootfs_msdos(self, rootfs, cr_workdir, oe_builddir,
> rootfs_dir,
> > native_sysroot, pseudo):
> > """
> > --
> > 2.39.2
> >
>
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#177839):
> https://lists.openembedded.org/g/openembedded-core/message/177839
> > Mute This Topic: https://lists.openembedded.org/mt/97291696/3617179
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> alexandre.belloni@bootlin.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
>
>
> --
> Alexandre Belloni, co-owner and COO, Bootlin
> Embedded Linux and Kernel engineering
> https://bootlin.com
>
[-- Attachment #2: Type: text/html, Size: 14837 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [oe-core][PATCH 1/1] wic: Add support for F2FS in WIC rootfs plugin
2023-03-01 14:36 ` Anna-Lena Marx
@ 2023-03-01 16:24 ` Alexander Kanavin
2023-03-06 15:10 ` Anna-Lena Marx
0 siblings, 1 reply; 8+ messages in thread
From: Alexander Kanavin @ 2023-03-01 16:24 UTC (permalink / raw)
To: anna-lena.marx; +Cc: Alexandre Belloni, openembedded-core
If f2fs support requires recipes that are not available in oe-core, is
it possible to add that support (and tests) to the layer that has the
needed recipes, e.g. meta-filesystems? In general, we try to avoid
having things in oe-core that cannot be tested without adding other
layers.
Alex
On Wed, 1 Mar 2023 at 15:37, Anna-Lena Marx via lists.openembedded.org
<anna-lena.marx=inovex.de@lists.openembedded.org> wrote:
>
> Hi,
>
> took me a while to reproduce the failure locally.
> It's because meta-oe and meta-filesystems are not as default included in the bblayers.conf used for oe-selftest.
>
> I don't want to touch bblayers and I think this is not intended as it would be the same for e.g. erofs for which no test case was added.
> Alexander Kanavin pointed me on the poky mailing list to these tests and in fact, it's good to have them in general.
> So what's the preferred way for oe-core?
> Leave the tests and add the dependencies (for the specific tests) or just remove the test cases?
>
> -- Anna
>
>
>
>
> Am Di., 28. Feb. 2023 um 16:20 Uhr schrieb Alexandre Belloni <alexandre.belloni@bootlin.com>:
>>
>> Hello,
>>
>> this failed on the autobuilders:
>>
>> ERROR: Nothing PROVIDES 'f2fs-tools-native' (but /home/pokybuild/yocto-worker/beaglebone/build/meta/recipes-sato/images/core-image-sato.bb, /home/pokybuild/yocto-worker/beaglebone/build/meta/recipes-sato/images/core-image-sato-sdk.bb, /home/pokybuild/yocto-worker/beaglebone/build/meta/recipes-core/images/core-image-minimal.bb, /home/pokybuild/yocto-worker/beaglebone/build/meta/recipes-core/images/core-image-minimal-dev.bb, /home/pokybuild/yocto-worker/beaglebone/build/meta/recipes-core/images/core-image-ptest-all.bb DEPENDS on or otherwise requires it). Close matches:
>> btrfs-tools-native
>> dosfstools-native
>> spirv-tools-native
>>
>> https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/6790/steps/11/logs/stdio
>>
>> On 28/02/2023 16:00:58+0100, Anna-Lena Marx via lists.openembedded.org wrote:
>> > Building F2FS rootfs is supported in Yocto/OpenEmbedded -
>> > see image_types.bbclass, but not in WIC.
>> > This adds it as a nice convenience feature.
>> >
>> > F2FS needs a minimum size of ~500M to operate correctly due to
>> > an overprovision functionality.
>> >
>> > Signed-off-by: Anna-Lena Marx <anna-lena.marx@inovex.de>
>> > ---
>> > meta/classes-recipe/image_types_wic.bbclass | 2 +-
>> > meta/lib/oeqa/selftest/cases/wic.py | 2 ++
>> > meta/recipes-core/meta/wic-tools.bb | 2 +-
>> > scripts/lib/wic/help.py | 1 +
>> > scripts/lib/wic/ksparser.py | 2 +-
>> > scripts/lib/wic/misc.py | 1 +
>> > scripts/lib/wic/partition.py | 32 ++++++++++++++++++++-
>> > 7 files changed, 38 insertions(+), 4 deletions(-)
>> >
>> > diff --git a/meta/classes-recipe/image_types_wic.bbclass b/meta/classes-recipe/image_types_wic.bbclass
>> > index c339b9bdfb..8ae5ecca4d 100644
>> > --- a/meta/classes-recipe/image_types_wic.bbclass
>> > +++ b/meta/classes-recipe/image_types_wic.bbclass
>> > @@ -89,7 +89,7 @@ do_image_wic[recrdeptask] += "do_deploy"
>> > do_image_wic[deptask] += "do_image_complete"
>> >
>> > WKS_FILE_DEPENDS_DEFAULT = '${@bb.utils.contains_any("BUILD_ARCH", [ 'x86_64', 'i686' ], "syslinux-native", "",d)}'
>> > -WKS_FILE_DEPENDS_DEFAULT += "bmap-tools-native cdrtools-native btrfs-tools-native squashfs-tools-native e2fsprogs-native erofs-utils-native"
>> > +WKS_FILE_DEPENDS_DEFAULT += "bmap-tools-native cdrtools-native btrfs-tools-native squashfs-tools-native e2fsprogs-native erofs-utils-native f2fs-tools-native"
>> > # Unified kernel images need objcopy
>> > WKS_FILE_DEPENDS_DEFAULT += "virtual/${MLPREFIX}${TARGET_PREFIX}binutils"
>> > WKS_FILE_DEPENDS_BOOTLOADERS = ""
>> > diff --git a/meta/lib/oeqa/selftest/cases/wic.py b/meta/lib/oeqa/selftest/cases/wic.py
>> > index ca1abb970a..5e255e35c7 100644
>> > --- a/meta/lib/oeqa/selftest/cases/wic.py
>> > +++ b/meta/lib/oeqa/selftest/cases/wic.py
>> > @@ -1188,6 +1188,7 @@ class Wic2(WicTestCase):
>> > with NamedTemporaryFile("w", suffix=".wks") as wks:
>> > wks.writelines(['part ext2 --fstype ext2 --source rootfs\n',
>> > 'part btrfs --fstype btrfs --source rootfs --size 40M\n',
>> > + 'part f2fs --fstype f2fs --source rootfs --size 50M\n',
>> > 'part squash --fstype squashfs --source rootfs\n',
>> > 'part swap --fstype swap --size 1M\n',
>> > 'part emptyvfat --fstype vfat --size 1M\n',
>> > @@ -1258,6 +1259,7 @@ class Wic2(WicTestCase):
>> > wks.writelines(
>> > ['part ext2 --fstype ext2 --source rootfs --mkfs-extraopts "-D -F -i 8192"\n',
>> > "part btrfs --fstype btrfs --source rootfs --size 40M --mkfs-extraopts='--quiet'\n",
>> > + "part f2fs --fstype f2fs --source rootfs --size 50M --mkfs-extraopts='-O extra_attr -o 20'\n",
>> > 'part squash --fstype squashfs --source rootfs --mkfs-extraopts "-no-sparse -b 4096"\n',
>> > 'part emptyvfat --fstype vfat --size 1M --mkfs-extraopts "-S 1024 -s 64"\n',
>> > 'part emptymsdos --fstype msdos --size 1M --mkfs-extraopts "-S 1024 -s 64"\n',
>> > diff --git a/meta/recipes-core/meta/wic-tools.bb b/meta/recipes-core/meta/wic-tools.bb
>> > index 9282d36a4d..d30ce66b56 100644
>> > --- a/meta/recipes-core/meta/wic-tools.bb
>> > +++ b/meta/recipes-core/meta/wic-tools.bb
>> > @@ -7,7 +7,7 @@ DEPENDS = "\
>> > mtools-native bmap-tools-native grub-native cdrtools-native \
>> > btrfs-tools-native squashfs-tools-native pseudo-native \
>> > e2fsprogs-native util-linux-native tar-native erofs-utils-native \
>> > - virtual/${TARGET_PREFIX}binutils \
>> > + f2fs-tools-native virtual/${TARGET_PREFIX}binutils \
>> > "
>> > DEPENDS:append:x86 = " syslinux-native syslinux grub-efi systemd-boot"
>> > DEPENDS:append:x86-64 = " syslinux-native syslinux grub-efi systemd-boot"
>> > diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
>> > index 73e3380cde..0094065931 100644
>> > --- a/scripts/lib/wic/help.py
>> > +++ b/scripts/lib/wic/help.py
>> > @@ -931,6 +931,7 @@ DESCRIPTION
>> > btrfs
>> > squashfs
>> > erofs
>> > + f2fs
>> > swap
>> >
>> > --fsoptions: Specifies a free-form string of options to be
>> > diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
>> > index d1e546b12d..e10ce6581d 100644
>> > --- a/scripts/lib/wic/ksparser.py
>> > +++ b/scripts/lib/wic/ksparser.py
>> > @@ -159,7 +159,7 @@ class KickStart():
>> > part.add_argument('--fstype', default='vfat',
>> > choices=('ext2', 'ext3', 'ext4', 'btrfs',
>> > 'squashfs', 'vfat', 'msdos', 'erofs',
>> > - 'swap', 'none'))
>> > + 'f2fs', 'swap', 'none'))
>> > part.add_argument('--mkfs-extraopts', default='')
>> > part.add_argument('--label')
>> > part.add_argument('--use-label', action='store_true')
>> > diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py
>> > index 2b90821b30..dc84a37423 100644
>> > --- a/scripts/lib/wic/misc.py
>> > +++ b/scripts/lib/wic/misc.py
>> > @@ -41,6 +41,7 @@ NATIVE_RECIPES = {"bmaptool": "bmap-tools",
>> > "mkfs.ext3": "e2fsprogs",
>> > "mkfs.ext4": "e2fsprogs",
>> > "mkfs.vfat": "dosfstools",
>> > + "mkfs.f2fs": "f2fs-tools",
>> > "mksquashfs": "squashfs-tools",
>> > "mkswap": "util-linux",
>> > "mmd": "mtools",
>> > diff --git a/scripts/lib/wic/partition.py b/scripts/lib/wic/partition.py
>> > index 382afa44bc..6adcaa041f 100644
>> > --- a/scripts/lib/wic/partition.py
>> > +++ b/scripts/lib/wic/partition.py
>> > @@ -208,7 +208,7 @@ class Partition():
>> > Prepare content for a rootfs partition i.e. create a partition
>> > and fill it from a /rootfs dir.
>> >
>> > - Currently handles ext2/3/4, btrfs, vfat and squashfs.
>> > + Currently handles ext2/3/4, btrfs, vfat, squashfs and f2fs.
>> > """
>> >
>> > rootfs = "%s/rootfs_%s.%s.%s" % (cr_workdir, self.label,
>> > @@ -351,6 +351,36 @@ class Partition():
>> > self.mkfs_extraopts, self.fsuuid, rootfs)
>> > exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
>> >
>> > + def prepare_rootfs_f2fs(self, rootfs, cr_workdir, oe_builddir, rootfs_dir,
>> > + native_sysroot, pseudo):
>> > + """
>> > + Prepare content for a f2fs rootfs partition.
>> > + """
>> > + du_cmd = "du -ks %s" % rootfs_dir
>> > + out = exec_cmd(du_cmd)
>> > + actual_rootfs_size = int(out.split()[0])
>> > +
>> > + min_f2fs_size = 524288
>> > + rootfs_size = self.get_rootfs_size(actual_rootfs_size)
>> > + if rootfs_size < min_f2fs_size:
>> > + rootfs_size = min_f2fs_size
>> > +
>> > + with open(rootfs, 'w') as sparse:
>> > + os.ftruncate(sparse.fileno(), rootfs_size * 1024)
>> > +
>> > + extraopts = self.mkfs_extraopts or "-O extra_attr,compression"
>> > +
>> > + label_str = ""
>> > + if self.label:
>> > + label_str = "-l %s" % self.label
>> > +
>> > + mkfs_f2fs_cmd = "mkfs.f2fs %s %s %s" % \
>> > + (label_str, extraopts, rootfs)
>> > + exec_native_cmd(mkfs_f2fs_cmd, native_sysroot, pseudo=pseudo)
>> > +
>> > + sload_f2fs_cmd = "sload.f2fs -f %s %s" % (rootfs_dir, rootfs)
>> > + exec_native_cmd(sload_f2fs_cmd, native_sysroot, pseudo=pseudo)
>> > +
>> > def prepare_rootfs_msdos(self, rootfs, cr_workdir, oe_builddir, rootfs_dir,
>> > native_sysroot, pseudo):
>> > """
>> > --
>> > 2.39.2
>> >
>>
>> >
>> >
>> >
>>
>>
>> --
>> Alexandre Belloni, co-owner and COO, Bootlin
>> Embedded Linux and Kernel engineering
>> https://bootlin.com
>
>
> -=-=-=-=-=-=-=-=-=-=-=-
> Links: You receive all messages sent to this group.
> View/Reply Online (#177888): https://lists.openembedded.org/g/openembedded-core/message/177888
> Mute This Topic: https://lists.openembedded.org/mt/97291696/1686489
> Group Owner: openembedded-core+owner@lists.openembedded.org
> Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [alex.kanavin@gmail.com]
> -=-=-=-=-=-=-=-=-=-=-=-
>
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [oe-core][PATCH 1/1] wic: Add support for F2FS in WIC rootfs plugin
2023-03-01 16:24 ` Alexander Kanavin
@ 2023-03-06 15:10 ` Anna-Lena Marx
2023-03-08 13:32 ` Ross Burton
0 siblings, 1 reply; 8+ messages in thread
From: Anna-Lena Marx @ 2023-03-06 15:10 UTC (permalink / raw)
To: Alexander Kanavin; +Cc: Alexandre Belloni, openembedded-core
[-- Attachment #1: Type: text/plain, Size: 12240 bytes --]
Hi,
moving the support or the tests to meta-filesystems does not make sense to
me.
Wic is located in oe-core, move support and tests would not really work at
all and create another dependency.
It may be an option to move f2fs from meta-filesystems to
meta/recipes-devtools where e.g. btrfs, dosfs tools and erofs are located.
Otherwise, allowing it without tests would be an option, but that's not
ideal either.
On the other hand is there merged support for some filesystems in wic
without matching tests.
-- Anna
Am Mi., 1. März 2023 um 17:25 Uhr schrieb Alexander Kanavin <
alex.kanavin@gmail.com>:
> If f2fs support requires recipes that are not available in oe-core, is
> it possible to add that support (and tests) to the layer that has the
> needed recipes, e.g. meta-filesystems? In general, we try to avoid
> having things in oe-core that cannot be tested without adding other
> layers.
>
> Alex
>
> On Wed, 1 Mar 2023 at 15:37, Anna-Lena Marx via lists.openembedded.org
> <anna-lena.marx=inovex.de@lists.openembedded.org> wrote:
> >
> > Hi,
> >
> > took me a while to reproduce the failure locally.
> > It's because meta-oe and meta-filesystems are not as default included in
> the bblayers.conf used for oe-selftest.
> >
> > I don't want to touch bblayers and I think this is not intended as it
> would be the same for e.g. erofs for which no test case was added.
> > Alexander Kanavin pointed me on the poky mailing list to these tests and
> in fact, it's good to have them in general.
> > So what's the preferred way for oe-core?
> > Leave the tests and add the dependencies (for the specific tests) or
> just remove the test cases?
> >
> > -- Anna
> >
> >
> >
> >
> > Am Di., 28. Feb. 2023 um 16:20 Uhr schrieb Alexandre Belloni <
> alexandre.belloni@bootlin.com>:
> >>
> >> Hello,
> >>
> >> this failed on the autobuilders:
> >>
> >> ERROR: Nothing PROVIDES 'f2fs-tools-native' (but
> /home/pokybuild/yocto-worker/beaglebone/build/meta/recipes-sato/images/
> core-image-sato.bb,
> /home/pokybuild/yocto-worker/beaglebone/build/meta/recipes-sato/images/
> core-image-sato-sdk.bb,
> /home/pokybuild/yocto-worker/beaglebone/build/meta/recipes-core/images/
> core-image-minimal.bb,
> /home/pokybuild/yocto-worker/beaglebone/build/meta/recipes-core/images/
> core-image-minimal-dev.bb,
> /home/pokybuild/yocto-worker/beaglebone/build/meta/recipes-core/images/
> core-image-ptest-all.bb DEPENDS on or otherwise requires it). Close
> matches:
> >> btrfs-tools-native
> >> dosfstools-native
> >> spirv-tools-native
> >>
> >>
> https://autobuilder.yoctoproject.org/typhoon/#/builders/65/builds/6790/steps/11/logs/stdio
> >>
> >> On 28/02/2023 16:00:58+0100, Anna-Lena Marx via lists.openembedded.org
> wrote:
> >> > Building F2FS rootfs is supported in Yocto/OpenEmbedded -
> >> > see image_types.bbclass, but not in WIC.
> >> > This adds it as a nice convenience feature.
> >> >
> >> > F2FS needs a minimum size of ~500M to operate correctly due to
> >> > an overprovision functionality.
> >> >
> >> > Signed-off-by: Anna-Lena Marx <anna-lena.marx@inovex.de>
> >> > ---
> >> > meta/classes-recipe/image_types_wic.bbclass | 2 +-
> >> > meta/lib/oeqa/selftest/cases/wic.py | 2 ++
> >> > meta/recipes-core/meta/wic-tools.bb | 2 +-
> >> > scripts/lib/wic/help.py | 1 +
> >> > scripts/lib/wic/ksparser.py | 2 +-
> >> > scripts/lib/wic/misc.py | 1 +
> >> > scripts/lib/wic/partition.py | 32
> ++++++++++++++++++++-
> >> > 7 files changed, 38 insertions(+), 4 deletions(-)
> >> >
> >> > diff --git a/meta/classes-recipe/image_types_wic.bbclass
> b/meta/classes-recipe/image_types_wic.bbclass
> >> > index c339b9bdfb..8ae5ecca4d 100644
> >> > --- a/meta/classes-recipe/image_types_wic.bbclass
> >> > +++ b/meta/classes-recipe/image_types_wic.bbclass
> >> > @@ -89,7 +89,7 @@ do_image_wic[recrdeptask] += "do_deploy"
> >> > do_image_wic[deptask] += "do_image_complete"
> >> >
> >> > WKS_FILE_DEPENDS_DEFAULT = '${@bb.utils.contains_any("BUILD_ARCH", [
> 'x86_64', 'i686' ], "syslinux-native", "",d)}'
> >> > -WKS_FILE_DEPENDS_DEFAULT += "bmap-tools-native cdrtools-native
> btrfs-tools-native squashfs-tools-native e2fsprogs-native
> erofs-utils-native"
> >> > +WKS_FILE_DEPENDS_DEFAULT += "bmap-tools-native cdrtools-native
> btrfs-tools-native squashfs-tools-native e2fsprogs-native
> erofs-utils-native f2fs-tools-native"
> >> > # Unified kernel images need objcopy
> >> > WKS_FILE_DEPENDS_DEFAULT +=
> "virtual/${MLPREFIX}${TARGET_PREFIX}binutils"
> >> > WKS_FILE_DEPENDS_BOOTLOADERS = ""
> >> > diff --git a/meta/lib/oeqa/selftest/cases/wic.py
> b/meta/lib/oeqa/selftest/cases/wic.py
> >> > index ca1abb970a..5e255e35c7 100644
> >> > --- a/meta/lib/oeqa/selftest/cases/wic.py
> >> > +++ b/meta/lib/oeqa/selftest/cases/wic.py
> >> > @@ -1188,6 +1188,7 @@ class Wic2(WicTestCase):
> >> > with NamedTemporaryFile("w", suffix=".wks") as wks:
> >> > wks.writelines(['part ext2 --fstype ext2 --source
> rootfs\n',
> >> > 'part btrfs --fstype btrfs --source
> rootfs --size 40M\n',
> >> > + 'part f2fs --fstype f2fs --source
> rootfs --size 50M\n',
> >> > 'part squash --fstype squashfs --source
> rootfs\n',
> >> > 'part swap --fstype swap --size 1M\n',
> >> > 'part emptyvfat --fstype vfat --size
> 1M\n',
> >> > @@ -1258,6 +1259,7 @@ class Wic2(WicTestCase):
> >> > wks.writelines(
> >> > ['part ext2 --fstype ext2 --source rootfs
> --mkfs-extraopts "-D -F -i 8192"\n',
> >> > "part btrfs --fstype btrfs --source rootfs
> --size 40M --mkfs-extraopts='--quiet'\n",
> >> > + "part f2fs --fstype f2fs --source rootfs
> --size 50M --mkfs-extraopts='-O extra_attr -o 20'\n",
> >> > 'part squash --fstype squashfs --source rootfs
> --mkfs-extraopts "-no-sparse -b 4096"\n',
> >> > 'part emptyvfat --fstype vfat --size 1M
> --mkfs-extraopts "-S 1024 -s 64"\n',
> >> > 'part emptymsdos --fstype msdos --size 1M
> --mkfs-extraopts "-S 1024 -s 64"\n',
> >> > diff --git a/meta/recipes-core/meta/wic-tools.bb
> b/meta/recipes-core/meta/wic-tools.bb
> >> > index 9282d36a4d..d30ce66b56 100644
> >> > --- a/meta/recipes-core/meta/wic-tools.bb
> >> > +++ b/meta/recipes-core/meta/wic-tools.bb
> >> > @@ -7,7 +7,7 @@ DEPENDS = "\
> >> > mtools-native bmap-tools-native grub-native
> cdrtools-native \
> >> > btrfs-tools-native squashfs-tools-native pseudo-native \
> >> > e2fsprogs-native util-linux-native tar-native
> erofs-utils-native \
> >> > - virtual/${TARGET_PREFIX}binutils \
> >> > + f2fs-tools-native virtual/${TARGET_PREFIX}binutils \
> >> > "
> >> > DEPENDS:append:x86 = " syslinux-native syslinux grub-efi
> systemd-boot"
> >> > DEPENDS:append:x86-64 = " syslinux-native syslinux grub-efi
> systemd-boot"
> >> > diff --git a/scripts/lib/wic/help.py b/scripts/lib/wic/help.py
> >> > index 73e3380cde..0094065931 100644
> >> > --- a/scripts/lib/wic/help.py
> >> > +++ b/scripts/lib/wic/help.py
> >> > @@ -931,6 +931,7 @@ DESCRIPTION
> >> > btrfs
> >> > squashfs
> >> > erofs
> >> > + f2fs
> >> > swap
> >> >
> >> > --fsoptions: Specifies a free-form string of options to be
> >> > diff --git a/scripts/lib/wic/ksparser.py b/scripts/lib/wic/ksparser.py
> >> > index d1e546b12d..e10ce6581d 100644
> >> > --- a/scripts/lib/wic/ksparser.py
> >> > +++ b/scripts/lib/wic/ksparser.py
> >> > @@ -159,7 +159,7 @@ class KickStart():
> >> > part.add_argument('--fstype', default='vfat',
> >> > choices=('ext2', 'ext3', 'ext4', 'btrfs',
> >> > 'squashfs', 'vfat', 'msdos',
> 'erofs',
> >> > - 'swap', 'none'))
> >> > + 'f2fs', 'swap', 'none'))
> >> > part.add_argument('--mkfs-extraopts', default='')
> >> > part.add_argument('--label')
> >> > part.add_argument('--use-label', action='store_true')
> >> > diff --git a/scripts/lib/wic/misc.py b/scripts/lib/wic/misc.py
> >> > index 2b90821b30..dc84a37423 100644
> >> > --- a/scripts/lib/wic/misc.py
> >> > +++ b/scripts/lib/wic/misc.py
> >> > @@ -41,6 +41,7 @@ NATIVE_RECIPES = {"bmaptool": "bmap-tools",
> >> > "mkfs.ext3": "e2fsprogs",
> >> > "mkfs.ext4": "e2fsprogs",
> >> > "mkfs.vfat": "dosfstools",
> >> > + "mkfs.f2fs": "f2fs-tools",
> >> > "mksquashfs": "squashfs-tools",
> >> > "mkswap": "util-linux",
> >> > "mmd": "mtools",
> >> > diff --git a/scripts/lib/wic/partition.py
> b/scripts/lib/wic/partition.py
> >> > index 382afa44bc..6adcaa041f 100644
> >> > --- a/scripts/lib/wic/partition.py
> >> > +++ b/scripts/lib/wic/partition.py
> >> > @@ -208,7 +208,7 @@ class Partition():
> >> > Prepare content for a rootfs partition i.e. create a
> partition
> >> > and fill it from a /rootfs dir.
> >> >
> >> > - Currently handles ext2/3/4, btrfs, vfat and squashfs.
> >> > + Currently handles ext2/3/4, btrfs, vfat, squashfs and f2fs.
> >> > """
> >> >
> >> > rootfs = "%s/rootfs_%s.%s.%s" % (cr_workdir, self.label,
> >> > @@ -351,6 +351,36 @@ class Partition():
> >> > self.mkfs_extraopts, self.fsuuid, rootfs)
> >> > exec_native_cmd(mkfs_cmd, native_sysroot, pseudo=pseudo)
> >> >
> >> > + def prepare_rootfs_f2fs(self, rootfs, cr_workdir, oe_builddir,
> rootfs_dir,
> >> > + native_sysroot, pseudo):
> >> > + """
> >> > + Prepare content for a f2fs rootfs partition.
> >> > + """
> >> > + du_cmd = "du -ks %s" % rootfs_dir
> >> > + out = exec_cmd(du_cmd)
> >> > + actual_rootfs_size = int(out.split()[0])
> >> > +
> >> > + min_f2fs_size = 524288
> >> > + rootfs_size = self.get_rootfs_size(actual_rootfs_size)
> >> > + if rootfs_size < min_f2fs_size:
> >> > + rootfs_size = min_f2fs_size
> >> > +
> >> > + with open(rootfs, 'w') as sparse:
> >> > + os.ftruncate(sparse.fileno(), rootfs_size * 1024)
> >> > +
> >> > + extraopts = self.mkfs_extraopts or "-O
> extra_attr,compression"
> >> > +
> >> > + label_str = ""
> >> > + if self.label:
> >> > + label_str = "-l %s" % self.label
> >> > +
> >> > + mkfs_f2fs_cmd = "mkfs.f2fs %s %s %s" % \
> >> > + (label_str, extraopts, rootfs)
> >> > + exec_native_cmd(mkfs_f2fs_cmd, native_sysroot, pseudo=pseudo)
> >> > +
> >> > + sload_f2fs_cmd = "sload.f2fs -f %s %s" % (rootfs_dir, rootfs)
> >> > + exec_native_cmd(sload_f2fs_cmd, native_sysroot,
> pseudo=pseudo)
> >> > +
> >> > def prepare_rootfs_msdos(self, rootfs, cr_workdir, oe_builddir,
> rootfs_dir,
> >> > native_sysroot, pseudo):
> >> > """
> >> > --
> >> > 2.39.2
> >> >
> >>
> >> >
> >> >
> >> >
> >>
> >>
> >> --
> >> Alexandre Belloni, co-owner and COO, Bootlin
> >> Embedded Linux and Kernel engineering
> >> https://bootlin.com
> >
> >
> > -=-=-=-=-=-=-=-=-=-=-=-
> > Links: You receive all messages sent to this group.
> > View/Reply Online (#177888):
> https://lists.openembedded.org/g/openembedded-core/message/177888
> > Mute This Topic: https://lists.openembedded.org/mt/97291696/1686489
> > Group Owner: openembedded-core+owner@lists.openembedded.org
> > Unsubscribe: https://lists.openembedded.org/g/openembedded-core/unsub [
> alex.kanavin@gmail.com]
> > -=-=-=-=-=-=-=-=-=-=-=-
> >
>
[-- Attachment #2: Type: text/html, Size: 17917 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [oe-core][PATCH 1/1] wic: Add support for F2FS in WIC rootfs plugin
2023-03-06 15:10 ` Anna-Lena Marx
@ 2023-03-08 13:32 ` Ross Burton
2023-03-13 15:41 ` Anna-Lena Marx
0 siblings, 1 reply; 8+ messages in thread
From: Ross Burton @ 2023-03-08 13:32 UTC (permalink / raw)
To: anna-lena.marx@inovex.de; +Cc: OE-core
On 6 Mar 2023, at 15:10, Anna-Lena Marx via lists.openembedded.org <anna-lena.marx=inovex.de@lists.openembedded.org> wrote:
>
> Hi,
> moving the support or the tests to meta-filesystems does not make sense to me.
> Wic is located in oe-core, move support and tests would not really work at all and create another dependency.
> It may be an option to move f2fs from meta-filesystems to meta/recipes-devtools where e.g. btrfs, dosfs tools and erofs are located.
> Otherwise, allowing it without tests would be an option, but that's not ideal either.
> On the other hand is there merged support for some filesystems in wic without matching tests.
The other option would be for the test to determine if the recipes are available, so adding meta-filesystems would enable the tests.
Ross
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [oe-core][PATCH 1/1] wic: Add support for F2FS in WIC rootfs plugin
2023-03-08 13:32 ` Ross Burton
@ 2023-03-13 15:41 ` Anna-Lena Marx
0 siblings, 0 replies; 8+ messages in thread
From: Anna-Lena Marx @ 2023-03-13 15:41 UTC (permalink / raw)
To: openembedded-core, Ross Burton
[-- Attachment #1: Type: text/plain, Size: 2049 bytes --]
Hi Ross,
> The other option would be for the test to determine if the recipes are
available, so adding meta-filesystems would enable the tests.
I'm not sure if I understand your proposal correctly.
If the idea is to add meta-filesystems as a kind of before class/test
setup and remove it afterwards (like tested in
meta/lib/oeqa/selftest/cases/bblayers.py)
- yes, I think this would be possible, if its an acceptable way to do it.
If not, please give me a hint about what you meant.
-- Anna
*Anna-Lena Marx*
M.Sc. Embedded Systems
Embedded Software Development
inovex GmbH · Ludwig-Erhard-Allee 6 · 76131 Karlsruhe
015233181260 · anna-lena.marx@inovex.de · www.inovex.de
Sitz der Gesellschaft: Pforzheim · AG Mannheim, HRB 502126
Geschäftsführer: Stephan Müller, Tobias Joch
Wir nutzen Technologien, um unsere Kunden glücklich zu machen. Und uns
selbst.
blog.inovex.de – Deep Dives in digitale Innovationen.
culture.inovex.de – Werte & Worklife bei inovex.
newsletter.inovex.de – 1 x pro Monat News & Events.
design.inovex.de – Digital Product Interfaces von inovex.
Am Mi., 8. März 2023 um 14:33 Uhr schrieb Ross Burton <Ross.Burton@arm.com>:
> On 6 Mar 2023, at 15:10, Anna-Lena Marx via lists.openembedded.org
> <anna-lena.marx=inovex.de@lists.openembedded.org> wrote:
> >
> > Hi,
> > moving the support or the tests to meta-filesystems does not make sense
> to me.
> > Wic is located in oe-core, move support and tests would not really work
> at all and create another dependency.
> > It may be an option to move f2fs from meta-filesystems to
> meta/recipes-devtools where e.g. btrfs, dosfs tools and erofs are located.
> > Otherwise, allowing it without tests would be an option, but that's not
> ideal either.
> > On the other hand is there merged support for some filesystems in wic
> without matching tests.
>
> The other option would be for the test to determine if the recipes are
> available, so adding meta-filesystems would enable the tests.
>
> Ross
[-- Attachment #2: Type: text/html, Size: 4296 bytes --]
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2023-03-13 15:41 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2023-02-28 15:00 [oe-core][PATCH 0/1] wic: Add support for F2FS in WIC rootfs plugin Anna-Lena Marx
2023-02-28 15:00 ` [oe-core][PATCH 1/1] " Anna-Lena Marx
2023-02-28 15:20 ` Alexandre Belloni
2023-03-01 14:36 ` Anna-Lena Marx
2023-03-01 16:24 ` Alexander Kanavin
2023-03-06 15:10 ` Anna-Lena Marx
2023-03-08 13:32 ` Ross Burton
2023-03-13 15:41 ` Anna-Lena Marx
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox