From: Johannes Schauer Marin Rodrigues <josch@mister-muffin.de>
To: linux-ext4@vger.kernel.org
Subject: Re: created ext4 disk image differs depending on the underlying filesystem
Date: Sat, 04 May 2024 19:53:29 +0200 [thread overview]
Message-ID: <171484520952.2626447.2160419274451668597@localhost> (raw)
In-Reply-To: <171483317081.2626447.5951155062757257572@localhost>
[-- Attachment #1: Type: text/plain, Size: 5829 bytes --]
Quoting Johannes Schauer Marin Rodrigues (2024-05-04 16:32:50)
> I originally observed this issue when creating ext4 disk images on a 9p
> filesystem which differed from the images I created on a tmpfs. I observed
> that the difference also exists when the underlying file system is fat32, so
> I'm using this as an example here. For what it's worth, the ext4 filesystem
> images created on a tmpfs are identical to those created on an ext4 fs. To
> demonstrate the issue, please see the script at the end of this mail (it
> requires sudo to mount and unmount the fat32 disk image). As you can see from
> the printed hashes, the disk images produced outside the fat32 disk are
> always identical as expected. The diff between the reproducible images and
> those stored on fat32 is also very short but I don't know what data is stored
> at those points:
>
> @@ -85,7 +85,7 @@
> 00000540: 0000 0000 0000 0000 0000 0000 0000 1000 ................
> 00000550: 0000 0000 0000 0000 0000 0000 2000 2000 ............ . .
> 00000560: 0200 0000 0000 0000 0000 0000 0000 0000 ................
> -00000570: 0000 0000 0401 0000 8c04 0000 0000 0000 ................
> +00000570: 0000 0000 0401 0000 4900 0000 0000 0000 ........I.......
> 00000580: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 00000590: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 000005a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> @@ -125,9 +125,9 @@
> 000007c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 000007d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 000007e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> -000007f0: 0000 0000 0000 0000 0000 0000 264c 0251 ............&L.Q
> +000007f0: 0000 0000 0000 0000 0000 0000 64ca bba5 ............d...
> 00000800: 1200 0000 2200 0000 3200 0000 9d03 7300 ...."...2.....s.
> -00000810: 0200 0000 0000 0000 babb 8a41 7300 2004 ...........As. .
> +00000810: 0200 0400 0000 0000 babb 8a41 7300 69f5 ...........As.i.
> 00000820: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 00000830: 0000 0000 0000 0000 bc7a 6e31 0000 0000 .........zn1....
> 00000840: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>
> Any idea what is going on? Is there a better way to diff two ext4 disk images
> than diffing the xxd output? If I try diffing the dumpe2fs output I get these
> differences:
>
> @@ -32,7 +32,7 @@
> Maximum mount count: -1
> Last checked: Fri May 3 16:14:49 2024
> Check interval: 0 (<none>)
> -Lifetime writes: 1164 kB
> +Lifetime writes: 73 kB
> Reserved blocks uid: 0 (user root)
> Reserved blocks gid: 0 (group root)
> First inode: 11
> @@ -44,7 +44,7 @@
> Directory Hash Seed: 0b7f9cfd-0113-486c-a453-4f5483bd486b
> Journal backup: inode blocks
> Checksum type: crc32c
> -Checksum: 0x51024c26
> +Checksum: 0xa5bbca64
> Checksum seed: 0xf81d767d
> Orphan file inode: 12
> Journal features: (none)
> @@ -56,7 +56,7 @@
> Journal start: 0
>
>
> -Group 0: (Blocks 1-2047) csum 0x0420
> +Group 0: (Blocks 1-2047) csum 0xf569 [ITABLE_ZEROED]
> Primary superblock at 1, Group descriptors at 2-2
> Reserved GDT blocks at 3-17
> Block bitmap at 18 (+17), csum 0x7abcbbba
>
> Why would these bits differ depending on the filesystem on which the disk image
> is stored? Is there a way to equalize this information so that the disk image
> looks the same independent on the underlying filesystem?
The diff becomes a bit smaller when using
-E lazy_itable_init=0,assume_storage_prezeroed=0,nodiscard
@@ -85,7 +85,7 @@
00000540: 0000 0000 0000 0000 0000 0000 0000 1000 ................
00000550: 0000 0000 0000 0000 0000 0000 2000 2000 ............ . .
00000560: 0200 0000 0000 0000 0000 0000 0000 0000 ................
-00000570: 0000 0000 0401 0000 ac04 0000 0000 0000 ................
+00000570: 0000 0000 0401 0000 4900 0000 0000 0000 ........I.......
00000580: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000590: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000005a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
@@ -125,7 +125,7 @@
000007c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000007d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000007e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
-000007f0: 0000 0000 0000 0000 0000 0000 fb8d a90f ................
+000007f0: 0000 0000 0000 0000 0000 0000 64ca bba5 ............d...
00000800: 1200 0000 2200 0000 3200 0000 9d03 7300 ...."...2.....s.
00000810: 0200 0400 0000 0000 babb 8a41 7300 69f5 ...........As.i.
00000820: 0000 0000 0000 0000 0000 0000 0000 0000 ................
@@ -32,7 +32,7 @@
Maximum mount count: -1
Last checked: Fri May 3 16:14:49 2024
Check interval: 0 (<none>)
-Lifetime writes: 1196 kB
+Lifetime writes: 73 kB
Reserved blocks uid: 0 (user root)
Reserved blocks gid: 0 (group root)
First inode: 11
@@ -44,7 +44,7 @@
Directory Hash Seed: 0b7f9cfd-0113-486c-a453-4f5483bd486b
Journal backup: inode blocks
Checksum type: crc32c
-Checksum: 0x0fa98dfb
+Checksum: 0xa5bbca64
Checksum seed: 0xf81d767d
Orphan file inode: 12
Journal features: (none)
The "Lifetime writes" being much higher on fat32 suggests that despite
"nodiscard", less zeroes were written out when ext4 or tmpfs are the underlying
FS?
Thanks!
cheers, josch
[-- Attachment #2: signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
-----BEGIN PGP SIGNATURE-----
iQIzBAABCgAdFiEElFhU6KL81LF4wVq58sulx4+9g+EFAmY2dhYACgkQ8sulx4+9
g+HkOBAAjgHV4v0L4BXR03VadrNQJw8welBnJfWNrpASPF9eXMA8zCGbD2lIC3tk
aVHWyk/AoYOSvZp7ED74pr8W8SSTmBNrqdEwsJLg18b2Ix09X+07j8nW5pjw56B5
6ERMQjl07U21PHkV7SJE9R6ZGXHhI6grxf1wy5vDRjRP4l7R6wVyxfFuN4VLk4FD
Ur3bEUQaqYj620g/jeZwJTKPjtCYEYeEMtAZuKq8ZHVyZcjZpwbjkihVwBkr5Mjj
SFp1H7/RsvbL08u3LWUD54N2qrt21TMuY1NMtdPUVCmbv29xRnKu5jyWjBNRPfHn
Cs+aaHjEC6RUA6AfwPRC07DGQCnlv1PVZeiHFP21OTpv7d0I746pS3qSeS6OlQpa
7EBwUNboZZECJhiRhxumaZ1BfLE63zNUQruAgL6R/HFN1Xy0CYRbOu3zOhizGxJ6
GmiyNpB/19dfr9L+M7gSACchOONXtFtH9JnM3w0Ef8MuFg1Y0Io+vuwBNswGUsoU
YodRqeUqHKUxE0iyd90k3oXoKK9BrRVWLO8WrbkpCDQT/LH486f0d5C0Aus+biwE
5gOZwydl11M4lyZsAdcoORlPK/rvxaGoC/uKyD3Sl2y6wgpvbXNyoC6w+M1szulf
iFbg2RXhHJdRJReC/sFg4vaSSazGCZxHcU3X7gvlLccjBtpEFmY=
=rVim
-----END PGP SIGNATURE-----
next prev parent reply other threads:[~2024-05-04 17:53 UTC|newest]
Thread overview: 6+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-05-04 14:32 created ext4 disk image differs depending on the underlying filesystem Johannes Schauer Marin Rodrigues
2024-05-04 17:53 ` Johannes Schauer Marin Rodrigues [this message]
2024-05-05 0:10 ` Theodore Ts'o
2024-05-11 5:34 ` Johannes Schauer Marin Rodrigues
2024-05-11 21:11 ` Theodore Ts'o
2024-05-16 6:56 ` Johannes Schauer Marin Rodrigues
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=171484520952.2626447.2160419274451668597@localhost \
--to=josch@mister-muffin.de \
--cc=linux-ext4@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox