created ext4 disk image differs depending on the underlying filesystem

[Johannes Schauer Marin Rodrigues <josch@mister-muffin.de>]

[-- Attachment #1: Type: text/plain, Size: 4642 bytes --]

Hi,

I originally observed this issue when creating ext4 disk images on a 9p
filesystem which differed from the images I created on a tmpfs. I observed that
the difference also exists when the underlying file system is fat32, so I'm
using this as an example here. For what it's worth, the ext4 filesystem images
created on a tmpfs are identical to those created on an ext4 fs. To demonstrate
the issue, please see the script at the end of this mail (it requires sudo to
mount and unmount the fat32 disk image). As you can see from the printed
hashes, the disk images produced outside the fat32 disk are always identical as
expected. The diff between the reproducible images and those stored on fat32 is
also very short but I don't know what data is stored at those points:

@@ -85,7 +85,7 @@
 00000540: 0000 0000 0000 0000 0000 0000 0000 1000  ................
 00000550: 0000 0000 0000 0000 0000 0000 2000 2000  ............ . .
 00000560: 0200 0000 0000 0000 0000 0000 0000 0000  ................
-00000570: 0000 0000 0401 0000 8c04 0000 0000 0000  ................
+00000570: 0000 0000 0401 0000 4900 0000 0000 0000  ........I.......
 00000580: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00000590: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 000005a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
@@ -125,9 +125,9 @@
 000007c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 000007d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 000007e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
-000007f0: 0000 0000 0000 0000 0000 0000 264c 0251  ............&L.Q
+000007f0: 0000 0000 0000 0000 0000 0000 64ca bba5  ............d...
 00000800: 1200 0000 2200 0000 3200 0000 9d03 7300  ...."...2.....s.
-00000810: 0200 0000 0000 0000 babb 8a41 7300 2004  ...........As. .
+00000810: 0200 0400 0000 0000 babb 8a41 7300 69f5  ...........As.i.
 00000820: 0000 0000 0000 0000 0000 0000 0000 0000  ................
 00000830: 0000 0000 0000 0000 bc7a 6e31 0000 0000  .........zn1....
 00000840: 0000 0000 0000 0000 0000 0000 0000 0000  ................

Any idea what is going on? Is there a better way to diff two ext4 disk images
than diffing the xxd output? If I try diffing the dumpe2fs output I get these
differences:

@@ -32,7 +32,7 @@
 Maximum mount count:      -1
 Last checked:             Fri May  3 16:14:49 2024
 Check interval:           0 (<none>)
-Lifetime writes:          1164 kB
+Lifetime writes:          73 kB
 Reserved blocks uid:      0 (user root)
 Reserved blocks gid:      0 (group root)
 First inode:              11
@@ -44,7 +44,7 @@
 Directory Hash Seed:      0b7f9cfd-0113-486c-a453-4f5483bd486b
 Journal backup:           inode blocks
 Checksum type:            crc32c
-Checksum:                 0x51024c26
+Checksum:                 0xa5bbca64
 Checksum seed:            0xf81d767d
 Orphan file inode:        12
 Journal features:         (none)
@@ -56,7 +56,7 @@
 Journal start:            0
 
 
-Group 0: (Blocks 1-2047) csum 0x0420
+Group 0: (Blocks 1-2047) csum 0xf569 [ITABLE_ZEROED]
   Primary superblock at 1, Group descriptors at 2-2
   Reserved GDT blocks at 3-17
   Block bitmap at 18 (+17), csum 0x7abcbbba

Why would these bits differ depending on the filesystem on which the disk image
is stored? Is there a way to equalize this information so that the disk image
looks the same independent on the underlying filesystem?

Thanks!

cheers, josch

#!/bin/sh
set -eu
mkfs() {
	imgpath="$1"
	rm -f "$imgpath"
	dd if=/dev/zero of="$imgpath" bs=1024 count=2048 2>/dev/null
	echo H4sIAAAAAAAAA+3OQQrCMBCF4Vl7ihwho9PkPKVEtJgU2rjo7a240JXSRSnC/20ew5vFy/P5ekulzUk24xfB7JkaG/+ZL3oUtaCnYE2IUZZbTcX57Sa93afajs5JP0zd5cvfr/5P5bkbSk2lHvZeAgAAAAAAAAAAAAAAAABY4wEWZDwwACgAAA== \
	| base64 -d \
	| env LC_ALL=C.UTF-8 SOURCE_DATE_EPOCH=1714745689 /sbin/mke2fs -d - \
		-q -F -o Linux -T ext4 -O metadata_csum,64bit \
		-U 0b7f9cfd-0113-486c-a453-4f5483bd486b \
		-E hash_seed=0b7f9cfd-0113-486c-a453-4f5483bd486b \
		-b 1024 "$imgpath"
	md5sum "$imgpath"
}

mkfs "/dev/shm/disk.ext4"
mkfs disk.ext4

rm -f fat32.img
mkdir -p mnt
dd if=/dev/zero of=fat32.img bs=1024 count=65536 2>/dev/null
/sbin/mkfs.vfat -F 32 fat32.img
sudo mount -o rw,umask=0000 fat32.img mnt
mkfs mnt/disk.ext4
bash -c 'diff -u <(xxd mnt/disk.ext4) <(xxd disk.ext4) || true'
bash -c 'diff -u <(/sbin/dumpe2fs mnt/disk.ext4) <(/sbin/dumpe2fs disk.ext4) || true'
sudo umount mnt
mkfs disk.ext4
mkfs "/dev/shm/disk.ext4"
rm "/dev/shm/disk.ext4" disk.ext4 fat32.img
rmdir mnt

[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

-----BEGIN PGP SIGNATURE-----

iQIzBAABCgAdFiEElFhU6KL81LF4wVq58sulx4+9g+EFAmY2Rw8ACgkQ8sulx4+9
g+EgwxAAixKFTjRPvk1WI+G5pcpCJUdXM7DP4pbwBaEN5qtvBl2mQlfqT18akynH
c/DCtIgFwV0TU0cqwSO6HegAUWwo87PiPVBY7RiZT9xHz/n2PdSgR4vv5eua8oH4
s9jNzvy51WFFp3HUHAlYQVWzjttYmf900Vcsf3sGGeoJjnSi9CZjceS7E/1qCtgW
+XEFB3OeTOMl6gw/Y7tP91tqHztG8OMq0HeArgcKgSr1qIUpJSQ3wentBWglOncD
5GeZMFick3ETG3rT3n+UE8Z+ETQNbbo2i7QR/kRWsdcP5vcUUiMynjrxEXPLdSBY
opTVWTmfhmRLqFZu5Jb2KYgt6IBmeQblB64LzHU0Ud7oW/rKEY2c2DyQjnN8S9Bv
nLM9zA+K19N3fdYiwQCOX1p4SvgUVzywdtyVg/XcNhAQqt3rX5QHoeDYMc0BcF2o
e8PKWBwJE0jH7F5r5qCj4TRdLTL6a8ZTDz8Ijb7hpnbfXqJjv8e/s35upQ55X/Nm
miYViEOe9Ee+vxFm4/g5dvWfMWTPWv807Uy22/XltiGFT2E912whRCcAXaW+uWB/
2p1rlIlYdMD0PemA+q+YNIGj43BeW5VCi1EuQGyx4uftFM6j9M6Tq1ydFdbpFkeL
ld66pOh8xfk4TWBCb81GpT0HsgurmzMPOOTilWoa8lDCF4obBII=
=9rT8
-----END PGP SIGNATURE-----