btrfs-convert on 24Gb image corrupts files.

btrfs-convert on 24Gb image corrupts files.

[jordan.j]

[-- Attachment #1.1: Type: text/plain, Size: 8717 bytes --]




cat /dev/sda3 >sda3.img
fsck.ext4 -fyv sda3.img 


e2fsck 1.47.0 (5-Feb-2023)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information


 246589 inodes used (15.68%, out of 1572864)
 191 non-contiguous files (0.1%)
 116 non-contiguous directories (0.0%)
 # of inodes with ind/dind/tind blocks: 0/0/0
 Extent depth histogram: 235622/51/1
 2219268 blocks used (35.27%, out of 6291456)
 0 bad blocks
 1 large file


 220831 regular files
 14688 directories
 174 character device files
 97 block device files
 0 fifos
 1641 links
 10790 symbolic links (10636 fast symbolic links)
 0 sockets
------------
 248221 files


mkdir k
mount -o ro sda3.img k/
cd k
find -type f -print0 | xargs -0 md5sum >sda3_files.md5
cd ..
umount k
btrfs-convert sda3.img 


btrfs-convert from btrfs-progs v6.7.1


Source filesystem:
 Type: ext2
 Label: 
 Blocksize: 4096
 UUID: b3a78a9f-37e7-4ccb-bedb-1f800a6a5a19
Target filesystem:
 Label: 
 Blocksize: 4096
 Nodesize: 16384
 UUID: 8e9b57b4-92cd-466f-9c02-c081a768bf40
 Checksum: crc32c
 Features: extref, skinny-metadata, no-holes, free-space-tree (default)
 Data csum: yes
 Inline data: yes
 Copy xattr: yes
Reported stats:
 Total space: 25769803776
 Free space: 8862961664 (34.39%)
 Inode count: 1572864
 Free inodes: 1326275
 Block count: 6291456
Create initial btrfs filesystem
Create ext2 image file
Create btrfs metadata
Copy inodes [.] [ 386128/ 246589]
Free space cache cleared
Conversion complete


mount -o ro sda3.img k/
cd k
md5sum -c sda3_files.md5 | grep -v OK
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite:
FAILED
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite:
FAILED
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite:
FAILED
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite:
FAILED
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite:
FAILED
./root/.mozilla/firefox/s554srh9.default-release/favicons.sqlite: FAILED
./root/.mozilla/firefox/s554srh9.default-release/places.sqlite: FAILED
md5sum: WARNING: 7 computed checksums did NOT match


cd ..
umount k
btrfs-convert -r sda3.img 
btrfs-convert from btrfs-progs v6.7.1


Open filesystem for rollback:
 Label: 
 UUID: 8e9b57b4-92cd-466f-9c02-c081a768bf40
 Restoring from: ext2_saved/image
Rollback succeeded


mount -o ro sda3.img k/


cd k
md5sum -c sda3_files.md5 | grep -v OK


*******
Few times did the same with exactly the same corruption on exactly the same
7 files.
Then transferred sda3.img to another machine, and did the same with exactly
the same result, to be sure it is not hardware related.
Corrupted files on both machines are exactly the same, with exactly the
same contents - their md5sums match.
Corrupt part of the filles is at the end of all 7 files - piece from
another file each of them - linux kernel source particularly, except the
last - some binary data.


The size of sda3.img file is 24G, 8G used, ext4 , md5sum of sda3.img and
/dev/sda3 matched until the conversion, but not after the rollback.


*******


On the first machine:


Linux livecd 6.6.21-gentoo-x86_64 #1 SMP PREEMPT_DYNAMIC Sun Apr 28
18:15:06 UTC 2024 x86_64 Intel(R) Xeon(R) CPU X5670 @ 2.93GHz GenuineIntel
GNU/Linux


btrfs-progs v6.7.1 (tried with 6.8.1 - all the same)


btrfs fi show
Label: none uuid: 62c086a7-e96e-414d-9c21-e5d1e310fae1
 Total devices 1 FS bytes used 8.81GiB
 devid 1 size 24.00GiB used 16.25GiB path /dev/loop1


btrfs fi df k
Data, single: total=15.74GiB, used=8.47GiB
System, single: total=4.00MiB, used=16.00KiB
Metadata, single: total=512.00MiB, used=352.00MiB
GlobalReserve, single: total=18.88MiB, used=0.00B


second machine:


Linux le 6.8.8-gentoo #2 SMP PREEMPT_DYNAMIC Sat May 4 19:39:02 EEST 2024
x86_64 Intel(R) Core(TM) i5 CPU M 520 @ 2.40GHz GenuineIntel GNU/Linux


btrfs-progs v6.7.1


btrfs fi show
Label: none uuid: ac30b8a1-a671-4743-aa98-b9dd7b0fd9fc
 Total devices 1 FS bytes used 9.17GiB
 devid 1 size 24.00GiB used 16.44GiB path /dev/loop0


btrfs fi df m
Data, single: total=15.94GiB, used=8.82GiB
System, single: total=4.00MiB, used=16.00KiB
Metadata, single: total=512.00MiB, used=354.08MiB
GlobalReserve, single: total=19.30MiB, used=0.00B


*******


no errors in dmesgs


*******


on ext4:


ilefrag -v
m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
Filesystem type is: ef53
File size of
m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
is 49152 (12 blocks of 4096 bytes)
 ext: logical_offset: physical_offset: length: expected: flags:
 0: 0.. 1: 2217003.. 2217004: 2: 
 1: 2.. 10: 2217019.. 2217027: 9: 2217005:
 2: 11.. 11: 2217028.. 2217028: 1: last,unwritten,eof
m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite:
2 extents found


on btrfs:


filefrag -v
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite

Filesystem type is: 9123683e
File size of
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
is 49152 (12 blocks of 4096 bytes)
 ext: logical_offset: physical_offset: length: expected: flags:
 0: 0.. 1: 2217003.. 2217004: 2: shared
 1: 2.. 11: 2217019.. 2217028: 10: 2217005: last,shared,eof
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite:
2 extents found


*********


Tried the four patches from:
https://lore.kernel.org/linux-btrfs/6d2a19ced4551bfcf2a5d841921af7f84c4ea950.1714722726.git.anand.jain@oracle.com/


 on the https://github.com/kdave/btrfs-progs.git tree - no change.


**********


Attached file 1st is the original file from the ext4 image, 2nd is the
corrupted file(with some linux source code "injected"), from btrfs.


**********


How looks a corrupted file on the boundary between the non-corrupted(1st
part) and corrupted(2nd part) of the file, seen in less:


^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@*vdev,
 struct virtio_shm_region *region, u8 id);
};


/* If driver didn't advertise the feature, it will never appear. */
void virtio_check_driver_offered_feature(const struct virtio_device *vdev,
 unsigned int fbit);


/**
 * __virtio_test_bit - helper to test feature bits. For use by transports.
 * Devices should normally use virtio_has_feature,
 * which includes more checks.
 * @vdev: the device
 * @fbit: the feature bit
 */
static inline bool __virtio_test_bit(const struct virtio_device *vdev,
 unsigned int fbit)
{
 /* Did you forget to fix assumptions on max features? */
 if (__builtin_constant_p(fbit))
 BUILD_BUG_ON(fbit >= 64);
 else
 BUG_ON(fbit >= 64);


 return vdev->features 
}


/**
 * __virtio_set_bit - helper to set feature bits. For use by transports.
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
lines 10-38/162 93%


Regards.


Jordan.



[-- Attachment #1.2: Type: text/html, Size: 16624 bytes --]

[-- Attachment #2: 3561288849sdhlie.sqlite --]
[-- Type: application/octet-stream, Size: 49152 bytes --]

[-- Attachment #3: 3561288849sdhlie.sqlite --]
[-- Type: application/octet-stream, Size: 49152 bytes --]

Re: btrfs-convert on 24Gb image corrupts files.

[Qu Wenruo]

在 2024/5/5 10:04, jordan.j@mail.bg 写道:
[...]
> *******
>
> on ext4:
>
> ilefrag -v
> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
> Filesystem type is: ef53
> File size of
> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite is 49152 (12 blocks of 4096 bytes)
>   ext:     logical_offset:        physical_offset: length:   expected:
> flags:
>     0:        0..       1:    2217003..   2217004:      2:
>     1:        2..      10:    2217019..   2217027:      9:    2217005:
>     2:       11..      11:    2217028..   2217028:      1:
> last,unwritten,eof
> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite: 2 extents found
>
> on btrfs:
>
> filefrag -v
> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
> Filesystem type is: 9123683e
> File size of
> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite is 49152 (12 blocks of 4096 bytes)
>   ext:     logical_offset:        physical_offset: length:   expected:
> flags:
>     0:        0..       1:    2217003..   2217004:      2:
> shared
>     1:        2..      11:    2217019..   2217028:     10:    2217005:
> last,shared,eof
> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite: 2 extents found
>

This is exactly the problem of unwritten (preallocated) extents.

Considering all corrupted files are from sqlite, it's highly possible
related to that.

> *********
>
> Tried the four patches from:
> https://lore.kernel.org/linux-btrfs/6d2a19ced4551bfcf2a5d841921af7f84c4ea950.1714722726.git.anand.jain@oracle.com/
>
> on the https://github.com/kdave/btrfs-progs.git tree - no change.

Unfortunately this means the fix is not working.

Mind to provide the "filefrag -v" output after conversion using Anand's
fixes?

I'm afraid there is something wrong for the fix.

For the worst case, I can go a simpler solution by just converting
preallocated extents to holes as a hot fix.

To me, there is no chance we can utilize the preallocated space anyway,
since the extent is shared with the ext4 image, thus we have to do COW.
So focusing on creating the same preallocated data extents in conversion
makes no sense.

Thanks,
Qu

Re: btrfs-convert on 24Gb image corrupts files.

[Anand Jain]

> on ext4:
> 
> ilefrag -v 
> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
> Filesystem type is: ef53
> File size of 
> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite is 49152 (12 blocks of 4096 bytes)
>   ext:     logical_offset:        physical_offset: length:   expected: 
> flags:
>     0:        0..       1:    2217003..   2217004:      2:
>     1:        2..      10:    2217019..   2217027:      9:    2217005:
>     2:       11..      11:    2217028..   2217028:      1:             
> last,unwritten,eof
> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite: 2 extents found
> 


btrfs-convert doesn't support ext4's "unwritten" extents. V2 of the fix 
is in progress.

   [PATCH 0/4] btrfs-progs: add support ext4 unwritten file extent


Thanks, Anand

Re: btrfs-convert on 24Gb image corrupts files.

[Qu Wenruo]

在 2024/5/5 10:26, Anand Jain 写道:
>
>> on ext4:
>>
>> ilefrag -v
>> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
>> Filesystem type is: ef53
>> File size of
>> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite is 49152 (12 blocks of 4096 bytes)
>>   ext:     logical_offset:        physical_offset: length:   expected:
>> flags:
>>     0:        0..       1:    2217003..   2217004:      2:
>>     1:        2..      10:    2217019..   2217027:      9:    2217005:
>>     2:       11..      11:    2217028..   2217028:      1:
>> last,unwritten,eof
>> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite: 2 extents found
>>
>
>
> btrfs-convert doesn't support ext4's "unwritten" extents. V2 of the fix
> is in progress.
>
>    [PATCH 0/4] btrfs-progs: add support ext4 unwritten file extent

But the reporter also tried your patchset already:

 > *********
 >
 > Tried the four patches from:
https://lore.kernel.org/linux-btrfs/6d2a19ced4551bfcf2a5d841921af7f84c4ea950.1714722726.git.anand.jain@oracle.com/

 > on the https://github.com/kdave/btrfs-progs.git tree - no change.
 >
 > **********

Any clue why it doesn't work?

Anyway, for now I'd prefer to change those unwritten extents just to holes.

Remember every file extent is shared between the inode and the ext4
image, meaning even if we created the preallocated extents correctly,
any newer write would be COWed anyway.

Thus for now, I believe changing the unwritten extents to holes would be
much easier.

Thanks,
Qu
>
>
> Thanks, Anand
>

Re: btrfs-convert on 24Gb image corrupts files.

[Yordan]

Hi, it's Jordan - the reporter, I'm switching the email server as the
first one presumably did dirty things and was rejected by the "mlmmj
program managing the mailing list".

>Mind to provide the "filefrag -v" output after conversion using Anand's
fixes?

>Mind to provide the "filefrag -v" output after conversion using Anand's

fixes?

After applying the 4 patches and compiling:

(chroot) livecd / # B/btrfs-progs/btrfs-convert sda3.img
btrfs-convert from btrfs-progs v6.8.1

Source filesystem:
Type: ext2
Label:
Blocksize: 4096
UUID: b3a78a9f-37e7-4ccb-bedb-1f800a6a5a19
Target filesystem:
Label:
Blocksize: 4096
Nodesize: 16384
UUID: 981d2a1c-c1a9-47b7-acf2-9724ebc96ed0
Checksum: crc32c
Features: extref, skinny-metadata, no-holes, free-space-tree (default)
Data csum: yes
Inline data: yes
Copy xattr: yes
Reported stats:
Total space: 25769803776
Free space: 8862961664 (34.39%)
Inode count: 1572864
Free inodes: 1326275
Block count: 6291456
Create initial btrfs filesystem
Create ext2 image file
Create btrfs metadata
Copy inodes [o] [ 388838/ 246589]
Free space cache cleared
Conversion complete

(chroot) livecd / # mount -o ro sda3.img k
(chroot) livecd / # cd k
(chroot) livecd /k # md5sum -c ../files.md5 | grep -v OK
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite:
FAILED
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite:
FAILED
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite:
FAILED
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite:
FAILED
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite:
FAILED
./root/.mozilla/firefox/s554srh9.default-release/favicons.sqlite: FAILED
./root/.mozilla/firefox/s554srh9.default-release/places.sqlite: FAILED
md5sum: WARNING: 7 computed checksums did NOT match

(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
Filesystem type is: 9123683e
File size of ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
is 49152 (12 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 1: 2217003.. 2217004: 2: shared
1: 2.. 11: 2217019.. 2217028: 10: 2217005: last,shared,eof
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite:
2 extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite
Filesystem type is: 9123683e
File size of ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite
is 49152 (12 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 1: 2217085.. 2217086: 2: shared
1: 2.. 11: 2217101.. 2217110: 10: 2217087: last,shared,eof
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite:
2 extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite
Filesystem type is: 9123683e
File size of ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite
is 49152 (12 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 1: 2222643.. 2222644: 2: shared
1: 2.. 11: 2244137.. 2244146: 10: 2222645: last,shared,eof
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite:
2 extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite
Filesystem type is: 9123683e
File size of ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite
is 49152 (12 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 1: 2244148.. 2244149: 2: shared
1: 2.. 11: 2244164.. 2244173: 10: 2244150: last,shared,eof
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite:
2 extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite
Filesystem type is: 9123683e
File size of ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite
is 49152 (12 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 1: 2217030.. 2217031: 2: shared
1: 2.. 11: 2217046.. 2217055: 10: 2217032: last,shared,eof
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite:
2 extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/favicons.sqlite
Filesystem type is: 9123683e
File size of ./root/.mozilla/firefox/s554srh9.default-release/favicons.sqlite
is 5242880 (1280 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 7: 2244106.. 2244113: 8: shared
1: 8.. 1279: 1449984.. 1451255: 1272: 2244114: last,shared,eof
./root/.mozilla/firefox/s554srh9.default-release/favicons.sqlite: 2
extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/places.sqlite
Filesystem type is: 9123683e
File size of ./root/.mozilla/firefox/s554srh9.default-release/places.sqlite
is 5242880 (1280 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 7: 2244097.. 2244104: 8: shared
1: 8.. 1279: 1485312.. 1486583: 1272: 2244105: last,shared,eof
./root/.mozilla/firefox/s554srh9.default-release/places.sqlite: 2 extents found
(chroot) livecd /k # cd ..
(chroot) livecd / # umount k
(chroot) livecd / # B/btrfs-progs/btrfs-convert -r sda3.img
btrfs-convert from btrfs-progs v6.8.1

Open filesystem for rollback:
Label:
UUID: 981d2a1c-c1a9-47b7-acf2-9724ebc96ed0
Restoring from: ext2_saved/image
Rollback succeeded

(chroot) livecd / # mount -o ro sda3.img k
(chroot) livecd / # cd k
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
Filesystem type is: ef53
File size of ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
is 49152 (12 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 1: 2217003.. 2217004: 2:
1: 2.. 10: 2217019.. 2217027: 9: 2217005:
2: 11.. 11: 2217028.. 2217028: 1: last,unwritten,eof
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite:
2 extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite
Filesystem type is: ef53
File size of ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite
is 49152 (12 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 1: 2217085.. 2217086: 2:
1: 2.. 10: 2217101.. 2217109: 9: 2217087:
2: 11.. 11: 2217110.. 2217110: 1: last,unwritten,eof
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite:
2 extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite
Filesystem type is: ef53
File size of ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite
is 49152 (12 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 1: 2222643.. 2222644: 2:
1: 2.. 10: 2244137.. 2244145: 9: 2222645:
2: 11.. 11: 2244146.. 2244146: 1: last,unwritten,eof
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite:
2 extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite
Filesystem type is: ef53
File size of ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite
is 49152 (12 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 1: 2244148.. 2244149: 2:
1: 2.. 10: 2244164.. 2244172: 9: 2244150:
2: 11.. 11: 2244173.. 2244173: 1: last,unwritten,eof
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite:
2 extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite
Filesystem type is: ef53
File size of ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite
is 49152 (12 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 1: 2217030.. 2217031: 2:
1: 2.. 10: 2217046.. 2217054: 9: 2217032:
2: 11.. 11: 2217055.. 2217055: 1: last,unwritten,eof
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite:
2 extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/favicons.sqlite
Filesystem type is: ef53
File size of ./root/.mozilla/firefox/s554srh9.default-release/favicons.sqlite
is 5242880 (1280 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 7: 2244106.. 2244113: 8:
1: 8.. 79: 1449984.. 1450055: 72: 2244114:
2: 80.. 1279: 1450056.. 1451255: 1200: last,unwritten,eof
./root/.mozilla/firefox/s554srh9.default-release/favicons.sqlite: 2
extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/places.sqlite
Filesystem type is: ef53
File size of ./root/.mozilla/firefox/s554srh9.default-release/places.sqlite
is 5242880 (1280 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 7: 2244097.. 2244104: 8:
1: 8.. 415: 1485312.. 1485719: 408: 2244105:
2: 416.. 1279: 1485720.. 1486583: 864: last,unwritten,eof
./root/.mozilla/firefox/s554srh9.default-release/places.sqlite: 2 extents found
(chroot) livecd /k #

Regards, Jordan.

Re: btrfs-convert on 24Gb image corrupts files.

[Qu Wenruo]

在 2024/5/6 00:35, Yordan 写道:
> Hi, it's Jordan - the reporter, I'm switching the email server as the
> first one presumably did dirty things and was rejected by the "mlmmj
> program managing the mailing list".
>
>> Mind to provide the "filefrag -v" output after conversion using Anand's
> fixes?
>
>> Mind to provide the "filefrag -v" output after conversion using Anand's
>
> fixes?
>
> After applying the 4 patches and compiling:
>
[...]
> (chroot) livecd /k # filefrag -v
> ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
> Filesystem type is: 9123683e
> File size of ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
> is 49152 (12 blocks of 4096 bytes)
> ext: logical_offset: physical_offset: length: expected: flags:
> 0: 0.. 1: 2217003.. 2217004: 2: shared
> 1: 2.. 11: 2217019.. 2217028: 10: 2217005: last,shared,eof

OK, that's the problem, compared to the older values from ext4:

 > 0: 0.. 1: 2217003.. 2217004: 2:
 > 1: 2.. 10: 2217019.. 2217027: 9: 2217005:
 > 2: 11.. 11: 2217028.. 2217028: 1: last,unwritten,eof

The last extent of btrfs still doesn't properly have the preallocated
flag, thus we read the garbage instead of filling it with 0.

I'll give more comprehensive review on Anand's fix.
And for the worst case I can change btrfs-convert to create a hole to
properly fix it.

Thanks for the detailed report!
Qu

Re: btrfs-convert on 24Gb image corrupts files.

[Anand Jain]

On 5/5/24 13:54, Qu Wenruo wrote:
> 
> 
> 在 2024/5/5 10:26, Anand Jain 写道:
>>
>>> on ext4:
>>>
>>> ilefrag -v
>>> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
>>> Filesystem type is: ef53
>>> File size of
>>> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite is 49152 (12 blocks of 4096 bytes)
>>>   ext:     logical_offset:        physical_offset: length:   expected:
>>> flags:
>>>     0:        0..       1:    2217003..   2217004:      2:
>>>     1:        2..      10:    2217019..   2217027:      9:    2217005:
>>>     2:       11..      11:    2217028..   2217028:      1:
>>> last,unwritten,eof
>>> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite: 2 extents found
>>>
>>
>>
>> btrfs-convert doesn't support ext4's "unwritten" extents. V2 of the fix
>> is in progress.
>>
>>    [PATCH 0/4] btrfs-progs: add support ext4 unwritten file extent
> 
> But the reporter also tried your patchset already:
> 
>  > *********
>  >
>  > Tried the four patches from:
> https://lore.kernel.org/linux-btrfs/6d2a19ced4551bfcf2a5d841921af7f84c4ea950.1714722726.git.anand.jain@oracle.com/
> 
>  > on the https://github.com/kdave/btrfs-progs.git tree - no change.
>  >
>  > **********
> 
> Any clue why it doesn't work?
> 
> Anyway, for now I'd prefer to change those unwritten extents just to holes.
> 
> Remember every file extent is shared between the inode and the ext4
> image, meaning even if we created the preallocated extents correctly,
> any newer write would be COWed anyway.
> 
> Thus for now, I believe changing the unwritten extents to holes would be
> much easier.
> 

Ah. I missed that part. Thanks for highlighting.

The data size and extent size might not align. In v2, I'll detect
this and handle it. I'm still figuring out how to split merged
blocks into separate extents, so that prealloc flag/as a hole
can be set on only appropriate extent size.

Thanks, Anand

> Thanks,
> Qu
>>
>>
>> Thanks, Anand
>>

Re: btrfs-convert on 24Gb image corrupts files.

[Yordan]

[-- Attachment #1: Type: text/plain, Size: 2341 bytes --]

The attached file which is a reduced version of the problematic image,
produced by removing all files and directories, except 5 of the
problematic files and their path. Then its filesystem filled with zero
file, shrinked with "resize2fs" to 512M and compressed to 7M.

md5sum sda3.img.zst
9eec41fee47e3db555edeaba5d8d2e9a  sda3.img.zst

(chroot) livecd / # zstd -d sda3.img.zst -o sda3.img
sda3.img.zst        : 536870912 bytes
(chroot) livecd / # mount -o ro sda3.img k
(chroot) livecd / # find k/ -type f | xargs md5sum >files.md5
(chroot) livecd / # md5sum -c files.md5 | grep -v OK

(chroot) livecd / # umount k
(chroot) livecd / # B/btrfs-progs/btrfs-convert sda3.img
btrfs-convert from btrfs-progs v6.8.1

Source filesystem:
  Type:           ext2
  Label:
  Blocksize:      4096
  UUID:           b3a78a9f-37e7-4ccb-bedb-1f800a6a5a19
Target filesystem:
  Label:
  Blocksize:      4096
  Nodesize:       16384
  UUID:           d7c77d2f-d470-450a-ba0e-b6567ad3f4b3
  Checksum:       crc32c
  Features:       extref, skinny-metadata, no-holes, free-space-tree (default)
    Data csum:    yes
    Inline data:  yes
    Copy xattr:   yes
Reported stats:
  Total space:       536870912
  Free space:        326238208 (60.77%)
  Inode count:           32768
  Free inodes:           32743
  Block count:          131072
Create initial btrfs filesystem
Create ext2 image file
Create btrfs metadata
Copy inodes [o] [         1/        25]
Free space cache cleared
Conversion complete

(chroot) livecd / # mount -o ro sda3.img k
(chroot) livecd / # md5sum -c files.md5
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite:
FAILED
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite:
FAILED
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite:
FAILED
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite:
FAILED
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3870112724rsegmnoittet-es.sqlite:
OK
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite:
FAILED
md5sum: WARNING: 5 computed checksums did NOT match

Regards, Jordan.

[-- Attachment #2: sda3.img.zst --]
[-- Type: application/octet-stream, Size: 7314513 bytes --]

[-- Attachment #3: sda3.img.zst.md5 --]
[-- Type: application/octet-stream, Size: 47 bytes --]

9eec41fee47e3db555edeaba5d8d2e9a  sda3.img.zst

Re: btrfs-convert on 24Gb image corrupts files.

[Anand Jain]

On 5/6/24 18:53, Yordan wrote:
> The attached file which is a reduced version of the problematic image,
> produced by removing all files and directories, except 5 of the
> problematic files and their path. Then its filesystem filled with zero
> file, shrinked with "resize2fs" to 512M and compressed to 7M.
> 
> md5sum sda3.img.zst
> 9eec41fee47e3db555edeaba5d8d2e9a  sda3.img.zst
> 
> (chroot) livecd / # zstd -d sda3.img.zst -o sda3.img
> sda3.img.zst        : 536870912 bytes
> (chroot) livecd / # mount -o ro sda3.img k
> (chroot) livecd / # find k/ -type f | xargs md5sum >files.md5
> (chroot) livecd / # md5sum -c files.md5 | grep -v OK
> 
> (chroot) livecd / # umount k
> (chroot) livecd / # B/btrfs-progs/btrfs-convert sda3.img
> btrfs-convert from btrfs-progs v6.8.1
>  > Source filesystem:
>    Type:           ext2
>    Label:
>    Blocksize:      4096
>    UUID:           b3a78a9f-37e7-4ccb-bedb-1f800a6a5a19
> Target filesystem:
>    Label:
>    Blocksize:      4096
>    Nodesize:       16384
>    UUID:           d7c77d2f-d470-450a-ba0e-b6567ad3f4b3
>    Checksum:       crc32c
>    Features:       extref, skinny-metadata, no-holes, free-space-tree (default)
>      Data csum:    yes
>      Inline data:  yes
>      Copy xattr:   yes
> Reported stats:
>    Total space:       536870912
>    Free space:        326238208 (60.77%)
>    Inode count:           32768
>    Free inodes:           32743
>    Block count:          131072
> Create initial btrfs filesystem
> Create ext2 image file
> Create btrfs metadata
> Copy inodes [o] [         1/        25]
> Free space cache cleared
> Conversion complete
> 
> (chroot) livecd / # mount -o ro sda3.img k
> (chroot) livecd / # md5sum -c files.md5
> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite:
> FAILED
> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite:
> FAILED
> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite:
> FAILED
> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite:
> FAILED
> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3870112724rsegmnoittet-es.sqlite:
> OK
> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite:
> FAILED
> md5sum: WARNING: 5 computed checksums did NOT match
> 


Are these test results with the v2 patchset? Thanks, Anand.


> Regards, Jordan.

Re: btrfs-convert on 24Gb image corrupts files.

[Yordan]

NO, I have problems applying patch4, so its unpatchet

error: patch failed: convert/source-fs.c:316
error: convert/source-fs.c: patch does not apply

I just sent you a small problematic image so you can test it yourself.

On Mon, May 6, 2024 at 4:17 PM Anand Jain <anand.jain@oracle.com> wrote:
>
>
>
> On 5/6/24 18:53, Yordan wrote:
> > The attached file which is a reduced version of the problematic image,
> > produced by removing all files and directories, except 5 of the
> > problematic files and their path. Then its filesystem filled with zero
> > file, shrinked with "resize2fs" to 512M and compressed to 7M.
> >
> > md5sum sda3.img.zst
> > 9eec41fee47e3db555edeaba5d8d2e9a  sda3.img.zst
> >
> > (chroot) livecd / # zstd -d sda3.img.zst -o sda3.img
> > sda3.img.zst        : 536870912 bytes
> > (chroot) livecd / # mount -o ro sda3.img k
> > (chroot) livecd / # find k/ -type f | xargs md5sum >files.md5
> > (chroot) livecd / # md5sum -c files.md5 | grep -v OK
> >
> > (chroot) livecd / # umount k
> > (chroot) livecd / # B/btrfs-progs/btrfs-convert sda3.img
> > btrfs-convert from btrfs-progs v6.8.1
> >  > Source filesystem:
> >    Type:           ext2
> >    Label:
> >    Blocksize:      4096
> >    UUID:           b3a78a9f-37e7-4ccb-bedb-1f800a6a5a19
> > Target filesystem:
> >    Label:
> >    Blocksize:      4096
> >    Nodesize:       16384
> >    UUID:           d7c77d2f-d470-450a-ba0e-b6567ad3f4b3
> >    Checksum:       crc32c
> >    Features:       extref, skinny-metadata, no-holes, free-space-tree (default)
> >      Data csum:    yes
> >      Inline data:  yes
> >      Copy xattr:   yes
> > Reported stats:
> >    Total space:       536870912
> >    Free space:        326238208 (60.77%)
> >    Inode count:           32768
> >    Free inodes:           32743
> >    Block count:          131072
> > Create initial btrfs filesystem
> > Create ext2 image file
> > Create btrfs metadata
> > Copy inodes [o] [         1/        25]
> > Free space cache cleared
> > Conversion complete
> >
> > (chroot) livecd / # mount -o ro sda3.img k
> > (chroot) livecd / # md5sum -c files.md5
> > k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite:
> > FAILED
> > k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite:
> > FAILED
> > k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite:
> > FAILED
> > k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite:
> > FAILED
> > k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3870112724rsegmnoittet-es.sqlite:
> > OK
> > k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite:
> > FAILED
> > md5sum: WARNING: 5 computed checksums did NOT match
> >
>
>
> Are these test results with the v2 patchset? Thanks, Anand.
>
>
> > Regards, Jordan.

Re: btrfs-convert on 24Gb image corrupts files.

[Anand Jain]

It is meant to be applied on latest devel branch.


With the v2 patches:

$ ./btrfs-convert ~/sda3.img
btrfs-convert from btrfs-progs v6.8

Source filesystem:
   Type:           ext2
   Label:
   Blocksize:      4096
   UUID:           b3a78a9f-37e7-4ccb-bedb-1f800a6a5a19
Target filesystem:
   Label:
   Blocksize:      4096
   Nodesize:       16384
   UUID:           d8c61c42-1b2c-478d-8ae6-bf45213e8df4
   Checksum:       crc32c
   Features:       extref, skinny-metadata, no-holes, free-space-tree 
(default)
     Data csum:    yes
     Inline data:  yes
     Copy xattr:   yes
Reported stats:
   Total space:       536870912
   Free space:        326238208 (60.77%)
   Inode count:           32768
   Free inodes:           32743
   Block count:          131072
Create initial btrfs filesystem
Create ext2 image file
Create btrfs metadata
ERROR: inode 20 index 0: identified unsupported merged block length 1 
wanted 12
ERROR: failed to copy ext2 inode 20: -22
ERROR: error during copy_inodes -22
WARNING: error during conversion, the original filesystem is not modified





On 5/6/24 21:35, Yordan wrote:
> NO, I have problems applying patch4, so its unpatchet
> 
> error: patch failed: convert/source-fs.c:316
> error: convert/source-fs.c: patch does not apply
> 
> I just sent you a small problematic image so you can test it yourself.
> 
> On Mon, May 6, 2024 at 4:17 PM Anand Jain <anand.jain@oracle.com> wrote:
>>
>>
>>
>> On 5/6/24 18:53, Yordan wrote:
>>> The attached file which is a reduced version of the problematic image,
>>> produced by removing all files and directories, except 5 of the
>>> problematic files and their path. Then its filesystem filled with zero
>>> file, shrinked with "resize2fs" to 512M and compressed to 7M.
>>>
>>> md5sum sda3.img.zst
>>> 9eec41fee47e3db555edeaba5d8d2e9a  sda3.img.zst
>>>
>>> (chroot) livecd / # zstd -d sda3.img.zst -o sda3.img
>>> sda3.img.zst        : 536870912 bytes
>>> (chroot) livecd / # mount -o ro sda3.img k
>>> (chroot) livecd / # find k/ -type f | xargs md5sum >files.md5
>>> (chroot) livecd / # md5sum -c files.md5 | grep -v OK
>>>
>>> (chroot) livecd / # umount k
>>> (chroot) livecd / # B/btrfs-progs/btrfs-convert sda3.img
>>> btrfs-convert from btrfs-progs v6.8.1
>>>   > Source filesystem:
>>>     Type:           ext2
>>>     Label:
>>>     Blocksize:      4096
>>>     UUID:           b3a78a9f-37e7-4ccb-bedb-1f800a6a5a19
>>> Target filesystem:
>>>     Label:
>>>     Blocksize:      4096
>>>     Nodesize:       16384
>>>     UUID:           d7c77d2f-d470-450a-ba0e-b6567ad3f4b3
>>>     Checksum:       crc32c
>>>     Features:       extref, skinny-metadata, no-holes, free-space-tree (default)
>>>       Data csum:    yes
>>>       Inline data:  yes
>>>       Copy xattr:   yes
>>> Reported stats:
>>>     Total space:       536870912
>>>     Free space:        326238208 (60.77%)
>>>     Inode count:           32768
>>>     Free inodes:           32743
>>>     Block count:          131072
>>> Create initial btrfs filesystem
>>> Create ext2 image file
>>> Create btrfs metadata
>>> Copy inodes [o] [         1/        25]
>>> Free space cache cleared
>>> Conversion complete
>>>
>>> (chroot) livecd / # mount -o ro sda3.img k
>>> (chroot) livecd / # md5sum -c files.md5
>>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite:
>>> FAILED
>>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite:
>>> FAILED
>>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite:
>>> FAILED
>>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite:
>>> FAILED
>>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3870112724rsegmnoittet-es.sqlite:
>>> OK
>>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite:
>>> FAILED
>>> md5sum: WARNING: 5 computed checksums did NOT match
>>>
>>
>>
>> Are these test results with the v2 patchset? Thanks, Anand.
>>
>>
>>> Regards, Jordan.

Re: btrfs-convert on 24Gb image corrupts files.

[Yordan]

Yes, the same error after patching, and the same error on the original
24G image:

B/btrfs-progs/btrfs-convert sda3.img
btrfs-convert from btrfs-progs v6.8.1

Source filesystem:
  Type:           ext2
  Label:
  Blocksize:      4096
  UUID:           b3a78a9f-37e7-4ccb-bedb-1f800a6a5a19
Target filesystem:
  Label:
  Blocksize:      4096
  Nodesize:       16384
  UUID:           6595aa9c-415d-4ab0-bd9f-9a7f2ee8e3a9
  Checksum:       crc32c
  Features:       extref, skinny-metadata, no-holes, free-space-tree (default)
    Data csum:    yes
    Inline data:  yes
    Copy xattr:   yes
Reported stats:
  Total space:     25769803776
  Free space:       8655269888 (33.59%)
  Inode count:         1572864
  Free inodes:         1325061
  Block count:         6291456
Create initial btrfs filesystem
Create ext2 image file
Create btrfs metadata
ERROR: inode 527126 index 2: identified unsupported merged block
length 9 wanted 10
ERROR: failed to copy ext2 inode 527126: -22
ERROR: error during copy_inodes -22
WARNING: error during conversion, the original filesystem is not modified

On Mon, May 6, 2024 at 4:43 PM Anand Jain <anand.jain@oracle.com> wrote:
>
>
> It is meant to be applied on latest devel branch.
>
>
> With the v2 patches:
>
> $ ./btrfs-convert ~/sda3.img
> btrfs-convert from btrfs-progs v6.8
>
> Source filesystem:
>    Type:           ext2
>    Label:
>    Blocksize:      4096
>    UUID:           b3a78a9f-37e7-4ccb-bedb-1f800a6a5a19
> Target filesystem:
>    Label:
>    Blocksize:      4096
>    Nodesize:       16384
>    UUID:           d8c61c42-1b2c-478d-8ae6-bf45213e8df4
>    Checksum:       crc32c
>    Features:       extref, skinny-metadata, no-holes, free-space-tree
> (default)
>      Data csum:    yes
>      Inline data:  yes
>      Copy xattr:   yes
> Reported stats:
>    Total space:       536870912
>    Free space:        326238208 (60.77%)
>    Inode count:           32768
>    Free inodes:           32743
>    Block count:          131072
> Create initial btrfs filesystem
> Create ext2 image file
> Create btrfs metadata
> ERROR: inode 20 index 0: identified unsupported merged block length 1
> wanted 12
> ERROR: failed to copy ext2 inode 20: -22
> ERROR: error during copy_inodes -22
> WARNING: error during conversion, the original filesystem is not modified
>
>
>
>
>
> On 5/6/24 21:35, Yordan wrote:
> > NO, I have problems applying patch4, so its unpatchet
> >
> > error: patch failed: convert/source-fs.c:316
> > error: convert/source-fs.c: patch does not apply
> >
> > I just sent you a small problematic image so you can test it yourself.
> >
> > On Mon, May 6, 2024 at 4:17 PM Anand Jain <anand.jain@oracle.com> wrote:
> >>
> >>
> >>
> >> On 5/6/24 18:53, Yordan wrote:
> >>> The attached file which is a reduced version of the problematic image,
> >>> produced by removing all files and directories, except 5 of the
> >>> problematic files and their path. Then its filesystem filled with zero
> >>> file, shrinked with "resize2fs" to 512M and compressed to 7M.
> >>>
> >>> md5sum sda3.img.zst
> >>> 9eec41fee47e3db555edeaba5d8d2e9a  sda3.img.zst
> >>>
> >>> (chroot) livecd / # zstd -d sda3.img.zst -o sda3.img
> >>> sda3.img.zst        : 536870912 bytes
> >>> (chroot) livecd / # mount -o ro sda3.img k
> >>> (chroot) livecd / # find k/ -type f | xargs md5sum >files.md5
> >>> (chroot) livecd / # md5sum -c files.md5 | grep -v OK
> >>>
> >>> (chroot) livecd / # umount k
> >>> (chroot) livecd / # B/btrfs-progs/btrfs-convert sda3.img
> >>> btrfs-convert from btrfs-progs v6.8.1
> >>>   > Source filesystem:
> >>>     Type:           ext2
> >>>     Label:
> >>>     Blocksize:      4096
> >>>     UUID:           b3a78a9f-37e7-4ccb-bedb-1f800a6a5a19
> >>> Target filesystem:
> >>>     Label:
> >>>     Blocksize:      4096
> >>>     Nodesize:       16384
> >>>     UUID:           d7c77d2f-d470-450a-ba0e-b6567ad3f4b3
> >>>     Checksum:       crc32c
> >>>     Features:       extref, skinny-metadata, no-holes, free-space-tree (default)
> >>>       Data csum:    yes
> >>>       Inline data:  yes
> >>>       Copy xattr:   yes
> >>> Reported stats:
> >>>     Total space:       536870912
> >>>     Free space:        326238208 (60.77%)
> >>>     Inode count:           32768
> >>>     Free inodes:           32743
> >>>     Block count:          131072
> >>> Create initial btrfs filesystem
> >>> Create ext2 image file
> >>> Create btrfs metadata
> >>> Copy inodes [o] [         1/        25]
> >>> Free space cache cleared
> >>> Conversion complete
> >>>
> >>> (chroot) livecd / # mount -o ro sda3.img k
> >>> (chroot) livecd / # md5sum -c files.md5
> >>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite:
> >>> FAILED
> >>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite:
> >>> FAILED
> >>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite:
> >>> FAILED
> >>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite:
> >>> FAILED
> >>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3870112724rsegmnoittet-es.sqlite:
> >>> OK
> >>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite:
> >>> FAILED
> >>> md5sum: WARNING: 5 computed checksums did NOT match
> >>>
> >>
> >>
> >> Are these test results with the v2 patchset? Thanks, Anand.
> >>
> >>
> >>> Regards, Jordan.