Re: Errors on newly created file system

[Ferry Toth <fntoth@gmail.com>]

[-- Attachment #1.1: Type: text/plain, Size: 6007 bytes --]

Op woensdag 23 april 2025 00:00:36 CEST schreef Qu Wenruo:
> 
> 在 2025/4/23 07:02, Ferry Toth 写道:
> > Hi,
> > 
> > Op 21-04-2025 om 00:00 schreef Qu Wenruo:
> >>
> >>
> >> 在 2025/4/21 07:15, Ferry Toth 写道:
> >>> The following is originally done by Yocto's bitbake, but when I try 
> >>> manually it reproduces.
> >>>
> >>> I create a new fs on  a file using -r as ordinary user, then btrfs 
> >>> check the file (before or after mounting makes no difference), also 
> >>> as an ordinary user.
> >>>
> >>> The fs has 1000's of errors, I cut most because it seems the same 
> >>> type of errors. The files system is unrepaired bootable, but can be 
> >>> repaired using --repair, in which 1000's of files are moved to 
> >>> lost+found.
> >>>
> >>> The below was mkfs on a non-existing file, but writing to 16GB dduped 
> >>> file (rootfs is 1.4GB) makes no difference. Neither does dropping -- 
> >>> shrink, -m or -n.
> >>>
> >>> Also, writing the file to an actual disk and then check the disk 
> >>> gives the same errors.
> >>>
> >>> What could this be?
> >>>
> >>> ferry@delfion:~/tmp/edison/edison-scarthgap$ mkfs.btrfs -n 4096 -- 
> >>> shrink -M -v -r /home/ferry/tmp/edison-intel/my/edison-morty/out/ 
> >>> linux64/build/ tmp/work/edison-poky-linux/edison-image/1.0/rootfs 
> >>> edison-image- edison.rootfs.btrfs
> >>> btrfs-progs v6.6.3
> >>> See https://btrfs.readthedocs.io for more information.
> >>>
> >>> ERROR: zoned: unable to stat edison-image-edison.rootfs.btrfs
> >>> NOTE: several default settings have changed in version 5.15, please 
> >>> make sure
> >>>        this does not affect your deployments:
> >>>        - DUP for metadata (-m dup)
> >>>        - enabled no-holes (-O no-holes)
> >>>        - enabled free-space-tree (-R free-space-tree)
> >>>
> >>> Rootdir from: /home/ferry/tmp/edison-intel/my/edison-morty/out/ 
> >>> linux64/ build/tmp/work/edison-poky-linux/edison-image/1.0/rootfs
> >>>    Shrink:           yes
> >>> Label:              (null)
> >>> UUID:               c2ecfaca-168a-401b-a12a-e73694d7485a
> >>> Node size:          4096
> >>> Sector size:        4096
> >>> Filesystem size:    1.43GiB
> >>> Block group profiles:
> >>>    Data+Metadata:    single            1.42GiB
> >>>    System:           single            4.00MiB
> >>> SSD detected:       no
> >>> Zoned device:       no
> >>> Incompat features:  mixed-bg, extref, skinny-metadata, no-holes, 
> >>> free- space-tree
> >>> Runtime features:   free-space-tree
> >>> Checksum:           crc32c
> >>> Number of devices:  1
> >>> Devices:
> >>>     ID        SIZE  PATH
> >>>      1     1.43GiB  edison-image-edison.rootfs.btrfs
> >>>
> >>> ferry@delfion:~/tmp/edison/edison-scarthgap$ btrfs check edison- 
> >>> image- edison.rootfs.btrfs
> >>> Opening filesystem to check...
> >>> Checking filesystem on edison-image-edison.rootfs.btrfs
> >>> UUID: c2ecfaca-168a-401b-a12a-e73694d7485a
> >>> [1/7] checking root items
> >>> [2/7] checking extents
> >>> [3/7] checking free space tree
> >>> [4/7] checking fs roots
> >>> root 5 inode 252551099 errors 2000, link count wrong
> >>>          unresolved ref dir 260778488 index 2 namelen 11 name 
> >>> COPYING.MIT filetype 1 errors 0
> >>
> >> Looks like exactly the nlink bugs related to --rootdir option.
> >>
> >> And that's fixed in v6.10 first, by the commit c6464d3f99ed ("btrfs- 
> >> progs: mkfs: rework how we traverse rootdir"), then further improved 
> >> in v6.12 with the commit ef1157473372 ("btrfs-progs: mkfs: add hard 
> >> link support for --rootdir").
> >>
> >>
> >> So in short, if the directory contains hardlinks out of the directory, 
> >> then you have to use btrfs-progs newer than v6.12.
> > 
> > Hi Qu I am confirming v6.12 resolves this issue. Afaik Yocto uses 
> > reflinks. I'm guessing that generates the same issue?
> 
> Reflink should not cause the problem, shared extents (reflinks) are not 
> longer shared inside the new btrfs.
> 
> E.g. if two inodes shares the same data extent, it will be created as 
> two data extents, one for each inode.
> Thus it will cause extra space usage.
> 
> 
> It's only the hard links causing problems, as older progs directly uses 
> the st_nlink reported from the host fs, but it's very possible that, 
> some hard links are not inside the rootdir, thus causing missing nlinks 
> inside the created btrfs.
> 
> > 
> > As a work around for people on Ubuntu Noble (20.04 LTS) with btrfs-progs 
> > version 6.6.3-1.1build2, installing the package from Plucky (no other 
> > dependencies) with version 6.12-1build1 solves this issue.
> 
> Sorry we didn't notice the bug early enough, thus the fixes are only 
> landed into v6.12, and we do not maintain backports for older progs.

I never noticed either, while I was in a very good position to do so.

Times when I noticed filesystem errors I attributed these to unstable kernel crash. Which was probably true, never noticed errors at T0 until now.

> Thus I'm afraid Yocto or similar projects have to require a much newer 
> progs instead.

Yes that is the workaround below. You can copy the newer ”recipe” and that overrides the older one. Of course goes all the way and brings in new headers which can break dependencies (btrfs-compsize in this case).

> Thanks,
> Qu
> 
> > 
> > Yocto users on Scarthgap (5.0 LTS) with version 6.7.1 may copy the 
> > recipe meta/recipes-devtools/btrfs-tools/btrfs-tools_6.13.bb from 
> > walnascar or 6.14 from master. If they are building additional tools 
> > that use headers from this package like btrfs-compsize these may break.
> >> Thanks,
> >> Qu

While here, am I right that we can not generate the rootfs with compression on?

Reason I ask is, Yocto of course builds the rootfs and than has mkfs.btrfs create the image. But it runs as unprivileged user, so can not do mount.
And then can not do defrag.

> > 
> > 
> 
> 



[-- Attachment #1.2: Type: text/html, Size: 17587 bytes --]

[-- Attachment #2: This is a digitally signed message part. --]
[-- Type: application/pgp-signature, Size: 488 bytes --]