Re: [PATCH e2fsprogs] e2fsck: preen inline data no attr

[Daniel Tang <danielzgtg.opensource@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 5658 bytes --]

> Does the problem go away if you disable fast_commit?

Disabling fast_commit on all 5 ext4 partitions successfully brought me
from 4 unpreenable unclean shutdowns to a streak of 2 preenable unclean
shutdowns. `/`, /boot, /var, /tmp are clean without needs_recovery in
dumpe2fs, and `fsck.ext4 -f` found no needed fixes. /home was
"Filesystem state: clean" but had needs_recovery and orphan_present.
`fsck.ext4 -p` removed 1–2 orphaned inodes.

Testing was performed on the tablet with Firefox open to new tab
without internet, and the same USB-C–HDMI adapter. I might leave
fast_commit off for all partition except `/`, which I mount readonly
except when updating while I sit in a stable environment.

```console
root@ubuntu:~# cryptsetup luksOpen /dev/nvme0n1p9 homecrypt
Enter passphrase for /dev/nvme0n1p9: 
root@ubuntu:~# dumpe2fs -h /dev/mapper/homecrypt
dumpe2fs 1.47.0 (5-Feb-2023)
Filesystem volume name:   <none>
Last mounted on:          /home
Filesystem UUID:          b4dee0c9-72e8-4ad8-a277-d14ab505732b
Filesystem magic number:  0xEF53
Filesystem revision #:    1 (dynamic)
Filesystem features:      has_journal ext_attr dir_index sparse_super2 orphan_file filetype needs_recovery extent flex_bg inline_data encrypt sparse_super large_file metadata_csum orphan_present
Filesystem flags:         signed_directory_hash 
Default mount options:    user_xattr acl
Filesystem state:         clean
Errors behavior:          Remount read-only
Filesystem OS type:       Linux
Inode count:              9756672
Block count:              78022481
Reserved block count:     0
Overhead clusters:        880816
Free blocks:              72084435
Free inodes:              9277324
First block:              0
Block size:               4096
Fragment size:            4096
Blocks per group:         32768
Fragments per group:      32768
Inodes per group:         4096
Inode blocks per group:   256
Flex block group size:    8192
Filesystem created:       Mon Feb  9 17:26:30 2026
Last mount time:          Sun Mar  8 00:22:42 2026
Last write time:          Sun Mar  8 00:22:42 2026
Mount count:              1
Maximum mount count:      -1
Last checked:             Sun Mar  8 00:22:40 2026
Check interval:           0 (<none>)
Lifetime writes:          38 GB
Reserved blocks uid:      0 (user root)
Reserved blocks gid:      0 (group root)
First inode:              11
Inode size:               256
Required extra isize:     32
Desired extra isize:      32
Journal inode:            8
Default directory hash:   half_md4
Directory Hash Seed:      3d04ee5a-9500-4206-8eaf-dac860feeb2c
Journal backup:           inode blocks
Checksum type:            crc32c
Checksum:                 0x81ad4fc4
Orphan file inode:        12
Journal features:         journal_incompat_revoke journal_checksum_v3
Total journal size:       1040M
Total journal blocks:     266240
Max transaction length:   266240
Fast commit length:       0
Journal sequence:         0x00001cf8
Journal start:            205738
Journal checksum type:    crc32c
Journal checksum:         0xfe1d1be0

root@ubuntu:~# fsck.ext4 -p /dev/mapper/homecrypt
/dev/mapper/homecrypt: recovering journal
/dev/mapper/homecrypt: Clearing orphaned inode 430228 (uid=1000, gid=1000, mode=0100664, size=8028)
/dev/mapper/homecrypt: clean, 479363/9756672 files, 5937874/78022481 blocks
```

> What would be really interesting is
> to see the logs from the fsck.ext4 or kernel run to see what the
> initial complete was

See the attachment in my previous email. Ubuntu and/or systemd will
always fsck before mounting, even on clean reboots. This time I booted
into a Ubuntu 24.04 LiveUSB after the unclean shutdowns to run
dumpe2fs and fsck.ext4.

> More importantly, information about the source of the inconsistency
> report would be written to the superblock

What could have the opportunity to write anything to the superblock?
Before a panic, there's no inconsistency. After a panic, Linux would
say "not syncing", or after a panic, hardware stops before new writes
can reach the disk. systemd, as shown by `systemd-analyze plot` runs
fsck before attempting any `.mount`. initramfs mounts `/` readonly,
and any readwrite setting in `/etc/fstab` is only applied later. In
LiveUSB, I too always fsck first before mounting. Basically, the
filesystem would need to be readwrite or at least mounted and have a
corrupt file be accessed, for the inconsistency report to be written.
Since the fsck always precedes the mount, I don't think any report
will be written to the superblock.

> with the intersection of fast_commit and
> inline_data.
> sign that something isn't quite right,

`/proc/fs/ext4/*/fc-info`'s Ineligible reasons' "Extended attributes
changed" suggests that inline data will requests that fast commit be
disabled each time a small file is written. But the workload that
benefits from each is different. Inline data is for mostly-reading
30,000 mostly-small Javascript files totalling 100 MiB. Fast commit is
for monthly-apt-upgrading 250,000-max (TeX Live) 300-average
(google-chrome-stable) files totaling 64 GiB-max 2 GiB-average.

> is there a reason why you enabled inline_data and fast_commit in the
> first place?  Was there something specific about how you expect the
> file systems will be used that led you to believe they would be
> helpful?

The reason is performance, especially on my desktop with a HDD. Raw
data is in the attached file. In summary:

* apt-get is 7% (48.018 s) faster with fast_commit writes
* npm is 5% (0.9154 s) faster with inline_data reads

[-- Attachment #2: desktop_20260308_ext4_inlinedata_fastcommit_benchmarks.txt --]
[-- Type: text/plain, Size: 10589 bytes --]

root@daniel-desktop3:~# uname -a
Linux daniel-desktop3 6.17.0-14-generic #14-Ubuntu SMP PREEMPT_DYNAMIC Fri Jan  9 17:01:16 UTC 2026 x86_64 GNU/Linux
root@daniel-desktop3:~# cat /sys/block/sdb/device/model
WDC WD80EAZZ-00B
root@daniel-desktop3:~# fdisk -l /dev/sdb
Disk /dev/sdb: 7.28 TiB, 8001563222016 bytes, 15628053168 sectors
Disk model: WDC WD80EAZZ-00B
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 5C03C302-C74E-6A42-8293-CFFDB44076B9

Device           Start         End     Sectors  Size Type
/dev/sdb1         2048 15611275263 15611273216  7.3T Linux filesystem
/dev/sdb2  15611275264 15628052479    16777216    8G Linux filesystem
root@daniel-desktop3:~# cat /etc/mke2fs.conf
[defaults]
        creator_os = Linux
        base_features = ""
        fs_type = ext4
        enable_periodic_fsck = 0
        errors = remount-ro
        force_undo = 0
        auto_64-bit_support = 0
        default_mntopts = acl,user_xattr
        blocksize = 4096
        lazy_itable_init = 1
        lazy_journal_init = 0
        num_backup_sb = 0
        packed_meta_blocks = 1
        inode_ratio = 32768
        inode_size = 256
        reserved_ratio = 0
        hash_alg = half_md4
        flex_bg_size = 8196
        discard = 1
        encoding = utf8-12.1
        encoding_flags = strict

[fs_types]
        ext4 = {
                features = dir_index,ext_attr,extent,fast_commit,filetype,flex_bg,has_journal,inline_data,large_file,metadata_csum,orphan_file,sparse_super,sparse_super2
        }
        tmp = {
                discard = 0
                features = ^has_journal # Note that I didn't disable journaling on my tablet
        }
        home = {
                features = casefold,encrypt
        }
        boot = {
                features = ^inline_data
                inode_ratio = 65536
        }
        small = {
                inode_ratio = 4096
        }
        floppy = {
                inode_ratio = 8192
        }
        big = {
                inode_ratio = 32768
        }
        huge = {
                inode_ratio = 65536
        }
        news = {
                inode_ratio = 4096
        }
        largefile = {
                inode_ratio = 1048576
                blocksize = -1
        }
        largefile4 = {
                inode_ratio = 4194304
                blocksize = -1
        }
        hurd = {
                blocksize = 4096
                inode_size = 128
                warn_y2038_dates = 0
        }
root@daniel-desktop3:~# systemctl stop containerd docker
root@daniel-desktop3:~# mkdir -p /run/downloadtmp /run/archives
root@daniel-desktop3:~# mount -o remount,size=8G /run
root@daniel-desktop3:~# mount --bind /run/downloadtmp /var/lib/containerd
root@daniel-desktop3:~# systemctl start containerd docker
root@daniel-desktop3:~# docker run --rm -itv /run/archives:/var/cache/apt/archives ubuntu:24.04 bash -c 'apt update && apt install -dy texlive-full'
[... 4417 MB and 755 packages ...]
root@daniel-desktop3:~# systemctl stop containerd docker
root@daniel-desktop3:~# umount /var/lib/containerd
root@daniel-desktop3:~# wipefs -a /dev/sdb2
root@daniel-desktop3:~# mkfs.ext4 /dev/sdb2
mke2fs 1.47.2 (1-Jan-2025)
64-bit filesystem support is not enabled.  The larger fields afforded by this feature enable full-strength checksumming.  Pass -O 64bit to rectify.
Creating filesystem with 2097152 4k blocks and 262144 inodes
Filesystem UUID: d370114c-abe5-4b16-adba-e6b626b76f1d
Superblock backups stored on blocks:

Allocating group tables: done
Writing inode tables: done
Creating journal (16640 blocks): done
Writing superblocks and filesystem accounting information: done
root@daniel-desktop3:~# mount /dev/sdb2 /var/lib/containerd
root@daniel-desktop3:~# rm -rf /var/lib/containerd/* /run/containerd
root@daniel-desktop3:~# systemctl start containerd docker
root@daniel-desktop3:~# docker pull ubuntu:24.04
24.04: Pulling from library/ubuntu
01d7766a2e4a: Pull complete
fd8cda969ed2: Download complete
Digest: sha256:d1e2e92c075e5ca139d51a140fff46f84315c0fdce203eab2807c7e495eff4f9
Status: Downloaded newer image for ubuntu:24.04
docker.io/library/ubuntu:24.04
root@daniel-desktop3:~# sync; sysctl -w vm.drop_caches=3
vm.drop_caches = 3
root@daniel-desktop3:~# time docker run --rm -v /run/archives:/mnt ubuntu:24.04 bash -c 'dpkg --force-all -i /mnt/*.deb ; sync'
[...]
real    10m11.195s
user    0m0.102s
sys     0m0.135s
root@daniel-desktop3:~# systemctl stop containerd docker
root@daniel-desktop3:~# umount /var/lib/containerd
root@daniel-desktop3:~# wipefs -a /dev/sdb2
root@daniel-desktop3:~# mkfs.ext4 /dev/sdb2 -O ^fast_commit
[... similar ...]
root@daniel-desktop3:~# mount /dev/sdb2 /var/lib/containerd
root@daniel-desktop3:~# rm -rf /var/lib/containerd/* /run/containerd
root@daniel-desktop3:~# systemctl start containerd docker
root@daniel-desktop3:~# docker pull ubuntu:24.04
[... same ...]
root@daniel-desktop3:~# sync; sysctl -w vm.drop_caches=3
vm.drop_caches = 3
root@daniel-desktop3:~# time docker run --rm -v /run/archives:/mnt ubuntu:24.04 bash -c 'dpkg --force-all -i /mnt/*.deb ; sync'
[...]
real    10m59.213s
user    0m0.117s
sys     0m0.128s
root@daniel-desktop3:~# systemctl stop containerd docker
root@daniel-desktop3:~# umount /var/lib/containerd
root@daniel-desktop3:~# mount -t tmpfs tmpfs /var/lib/containerd
root@daniel-desktop3:~# systemctl start containerd docker
root@daniel-desktop3:~# docker pull ubuntu:24.04
[... same ...]
root@daniel-desktop3:~# runuser home -c 'git -C /home/home/.nvm rev-parse HEAD'
4c556a19b08728989267a89728152e4b6765000b
root@daniel-desktop3:~# runuser home -c 'mkdir /home/home/.nvm2'
root@daniel-desktop3:~# wipefs -a /dev/sdb2
root@daniel-desktop3:~# mkfs.ext4 /dev/sdb2 -E root_owner=1000:1000
[... similar ...]
root@daniel-desktop3:~# mount /dev/sdb2 /home/home/.nvm2
root@daniel-desktop3:~# rm -rf /home/home/.nvm2/lost+found
root@daniel-desktop3:~# runuser home -c 'git clone /home/home/.nvm /home/home/.nvm2'
Cloning into '/home/home/.nvm2'...
done.
root@daniel-desktop3:~# docker run --rm -v /home/home/.nvm2:/.nvm -v /usr/lib/x86_64-linux-gnu/libatomic.so.1.2.0:/usr/lib/x86_64-linux-gnu/libatomic.so.1.2.0:ro ubuntu:24.04 bash -c 'apt update && apt install -y bzip2 wget && . .nvm/nvm.sh && nvm install 25.6.0 && npm i -g @angular/cli@21.1.3 apollo@2.34.0 asar@3.2.0 bun-repl@2.1.1 bun@1.3.9 clipboard-cli@5.0.0 create-react-app@5.1.0 detox-cli@20.47.0 elasticdump@6.124.2 expo-cli@6.3.12 express-generator@4.16.1 firebase-tools@15.5.1 google-closure-compiler@20260128.0.0 graphql@16.12.0 har-extractor@1.1.2 http-server-upload@3.0.0 json@11.0.0 lighthouse@13.0.2 markdown-pdf@11.0.0 npm-check-updates@19.3.2 npm@11.9.0 prettier@3.8.1 qrcode@1.5.4 showdown@2.1.0 svgo@4.0.0 typescript@5.9.3 uglify-js@3.19.3 web-ext@9.2.0'
[...]
added 3892 packages, removed 1 package, and changed 24 packages in [...]s
[...]
root@daniel-desktop3:~# time docker run --rm -v /home/home/.nvm2:/.nvm -v /usr/lib/x86_64-linux-gnu/libatomic.so.1.2.0:/usr/lib/x86_64-linux-gnu/libatomic.so.1.2.0:ro ubuntu:24.04 bash -c 'ln -s /usr/lib/x86_64-linux-gnu/libatomic.so.1.2.0 /usr/lib/x86_64-linux-gnu/libatomic.so.1 && . .nvm/nvm.sh && npm ls -g --depth=0'
[...]
real    0m1.785s
user    0m0.007s
sys     0m0.012s
root@daniel-desktop3:~# sync; sysctl -w vm.drop_caches=3
root@daniel-desktop3:~# time docker run --rm -v /home/home/.nvm2:/.nvm -v /usr/lib/x86_64-linux-gnu/libatomic.so.1.2.0:/usr/lib/x86_64-linux-gnu/libatomic.so.1.2.0:ro ubuntu:24.04 bash -c 'ln -s /usr/lib/x86_64-linux-gnu/libatomic.so.1.2.0 /usr/lib/x86_64-linux-gnu/libatomic.so.1 && . .nvm/nvm.sh && npm ls -g --depth=0'
[...]
real    0m15.520s
user    0m0.007s
sys     0m0.022s
[rerunning, remembering drop_caches:]
real    0m16.480s
user    0m0.007s
sys     0m0.024s
real    0m16.997s
user    0m0.011s
sys     0m0.020s
real    0m16.651s
user    0m0.009s
sys     0m0.020s
real    0m17.035s
user    0m0.008s
sys     0m0.023s
[mean = 16.5366]
[real/user is meaningless with `time docker`]
root@daniel-desktop3:~# umount /home/home/.nvm2
root@daniel-desktop3:~# wipefs -a /dev/sdb2
root@daniel-desktop3:~# mkfs.ext4 /dev/sdb2 -E root_owner=1000:1000 -O ^inline_data
[... similar ...]
root@daniel-desktop3:~# mount /dev/sdb2 /home/home/.nvm2
root@daniel-desktop3:~# rm -rf /home/home/.nvm2/lost+found
root@daniel-desktop3:~# runuser home -c 'git clone /home/home/.nvm /home/home/.nvm2'
Cloning into '/home/home/.nvm2'...
done.
root@daniel-desktop3:~# docker run --rm -v /home/home/.nvm2:/.nvm -v /usr/lib/x86_64-linux-gnu/libatomic.so.1.2.0:/usr/lib/x86_64-linux-gnu/libatomic.so.1.2.0:ro ubuntu:24.04 bash -c 'apt update && apt install -y bzip2 wget && . .nvm/nvm.sh && nvm install 25.6.0 && npm i -g @angular/cli@21.1.3 apollo@2.34.0 asar@3.2.0 bun-repl@2.1.1 bun@1.3.9 clipboard-cli@5.0.0 create-react-app@5.1.0 detox-cli@20.47.0 elasticdump@6.124.2 expo-cli@6.3.12 express-generator@4.16.1 firebase-tools@15.5.1 google-closure-compiler@20260128.0.0 graphql@16.12.0 har-extractor@1.1.2 http-server-upload@3.0.0 json@11.0.0 lighthouse@13.0.2 markdown-pdf@11.0.0 npm-check-updates@19.3.2 npm@11.9.0 prettier@3.8.1 qrcode@1.5.4 showdown@2.1.0 svgo@4.0.0 typescript@5.9.3 uglify-js@3.19.3 web-ext@9.2.0'
[...]
added 3892 packages, removed 1 package, and changed 24 packages in [...]s
[...]
root@daniel-desktop3:~# time docker run --rm -v /home/home/.nvm2:/.nvm -v /usr/lib/x86_64-linux-gnu/libatomic.so.1.2.0:/usr/lib/x86_64-linux-gnu/libatomic.so.1.2.0:ro ubuntu:24.04 bash -c 'ln -s /usr/lib/x86_64-linux-gnu/libatomic.so.1.2.0 /usr/lib/x86_64-linux-gnu/libatomic.so.1 && . .nvm/nvm.sh && npm ls -g --depth=0'
[...]
real    0m1.812s
user    0m0.007s
sys     0m0.012s
root@daniel-desktop3:~# sync; sysctl -w vm.drop_caches=3
root@daniel-desktop3:~# time docker run --rm -v /home/home/.nvm2:/.nvm -v /usr/lib/x86_64-linux-gnu/libatomic.so.1.2.0:/usr/lib/x86_64-linux-gnu/libatomic.so.1.2.0:ro ubuntu:24.04 bash -c 'ln -s /usr/lib/x86_64-linux-gnu/libatomic.so.1.2.0 /usr/lib/x86_64-linux-gnu/libatomic.so.1 && . .nvm/nvm.sh && npm ls -g --depth=0'
[...]
real    0m16.651s
user    0m0.010s
sys     0m0.021s
[rerunning, remembering drop_caches:]
real    0m17.615s
user    0m0.004s
sys     0m0.025s
real    0m17.930s
user    0m0.007s
sys     0m0.025s
real    0m17.495s
user    0m0.008s
sys     0m0.023s
real    0m17.574s
user    0m0.011s
sys     0m0.018s
[mean = 17.453]
[I didn't have time to repeat mkfs.ext4, but I assumed inode placement differences' effects small]