* You can reproduce it yourself now...
@ 2024-05-05 22:30 Yordan
2024-05-06 10:53 ` btrfs-convert on 24Gb image corrupts files Yordan
0 siblings, 1 reply; 13+ messages in thread
From: Yordan @ 2024-05-05 22:30 UTC (permalink / raw)
To: quwenruo.btrfs, linux-btrfs, anand.jain
[-- Attachment #1: Type: text/plain, Size: 2341 bytes --]
The attached file which is a reduced version of the problematic image,
produced by removing all files and directories, except 5 of the
problematic files and their path. Then its filesystem filled with zero
file, shrinked with "resize2fs" to 512M and compressed to 7M.
md5sum sda3.img.zst
9eec41fee47e3db555edeaba5d8d2e9a sda3.img.zst
(chroot) livecd / # zstd -d sda3.img.zst -o sda3.img
sda3.img.zst : 536870912 bytes
(chroot) livecd / # mount -o ro sda3.img k
(chroot) livecd / # find k/ -type f | xargs md5sum >files.md5
(chroot) livecd / # md5sum -c files.md5 | grep -v OK
(chroot) livecd / # umount k
(chroot) livecd / # B/btrfs-progs/btrfs-convert sda3.img
btrfs-convert from btrfs-progs v6.8.1
Source filesystem:
Type: ext2
Label:
Blocksize: 4096
UUID: b3a78a9f-37e7-4ccb-bedb-1f800a6a5a19
Target filesystem:
Label:
Blocksize: 4096
Nodesize: 16384
UUID: d7c77d2f-d470-450a-ba0e-b6567ad3f4b3
Checksum: crc32c
Features: extref, skinny-metadata, no-holes, free-space-tree (default)
Data csum: yes
Inline data: yes
Copy xattr: yes
Reported stats:
Total space: 536870912
Free space: 326238208 (60.77%)
Inode count: 32768
Free inodes: 32743
Block count: 131072
Create initial btrfs filesystem
Create ext2 image file
Create btrfs metadata
Copy inodes [o] [ 1/ 25]
Free space cache cleared
Conversion complete
(chroot) livecd / # mount -o ro sda3.img k
(chroot) livecd / # md5sum -c files.md5
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite:
FAILED
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite:
FAILED
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite:
FAILED
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite:
FAILED
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3870112724rsegmnoittet-es.sqlite:
OK
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite:
FAILED
md5sum: WARNING: 5 computed checksums did NOT match
Regards, Jordan.
[-- Attachment #2: sda3.img.zst --]
[-- Type: application/zstd, Size: 7314513 bytes --]
[-- Attachment #3: sda3.img.zst.md5 --]
[-- Type: application/octet-stream, Size: 47 bytes --]
9eec41fee47e3db555edeaba5d8d2e9a sda3.img.zst
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: btrfs-convert on 24Gb image corrupts files.
2024-05-05 22:30 You can reproduce it yourself now Yordan
@ 2024-05-06 10:53 ` Yordan
2024-05-06 13:17 ` Anand Jain
0 siblings, 1 reply; 13+ messages in thread
From: Yordan @ 2024-05-06 10:53 UTC (permalink / raw)
To: quwenruo.btrfs, linux-btrfs, anand.jain
[-- Attachment #1: Type: text/plain, Size: 2341 bytes --]
The attached file which is a reduced version of the problematic image,
produced by removing all files and directories, except 5 of the
problematic files and their path. Then its filesystem filled with zero
file, shrinked with "resize2fs" to 512M and compressed to 7M.
md5sum sda3.img.zst
9eec41fee47e3db555edeaba5d8d2e9a sda3.img.zst
(chroot) livecd / # zstd -d sda3.img.zst -o sda3.img
sda3.img.zst : 536870912 bytes
(chroot) livecd / # mount -o ro sda3.img k
(chroot) livecd / # find k/ -type f | xargs md5sum >files.md5
(chroot) livecd / # md5sum -c files.md5 | grep -v OK
(chroot) livecd / # umount k
(chroot) livecd / # B/btrfs-progs/btrfs-convert sda3.img
btrfs-convert from btrfs-progs v6.8.1
Source filesystem:
Type: ext2
Label:
Blocksize: 4096
UUID: b3a78a9f-37e7-4ccb-bedb-1f800a6a5a19
Target filesystem:
Label:
Blocksize: 4096
Nodesize: 16384
UUID: d7c77d2f-d470-450a-ba0e-b6567ad3f4b3
Checksum: crc32c
Features: extref, skinny-metadata, no-holes, free-space-tree (default)
Data csum: yes
Inline data: yes
Copy xattr: yes
Reported stats:
Total space: 536870912
Free space: 326238208 (60.77%)
Inode count: 32768
Free inodes: 32743
Block count: 131072
Create initial btrfs filesystem
Create ext2 image file
Create btrfs metadata
Copy inodes [o] [ 1/ 25]
Free space cache cleared
Conversion complete
(chroot) livecd / # mount -o ro sda3.img k
(chroot) livecd / # md5sum -c files.md5
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite:
FAILED
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite:
FAILED
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite:
FAILED
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite:
FAILED
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3870112724rsegmnoittet-es.sqlite:
OK
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite:
FAILED
md5sum: WARNING: 5 computed checksums did NOT match
Regards, Jordan.
[-- Attachment #2: sda3.img.zst --]
[-- Type: application/octet-stream, Size: 7314513 bytes --]
[-- Attachment #3: sda3.img.zst.md5 --]
[-- Type: application/octet-stream, Size: 47 bytes --]
9eec41fee47e3db555edeaba5d8d2e9a sda3.img.zst
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: btrfs-convert on 24Gb image corrupts files.
2024-05-06 10:53 ` btrfs-convert on 24Gb image corrupts files Yordan
@ 2024-05-06 13:17 ` Anand Jain
2024-05-06 13:35 ` Yordan
0 siblings, 1 reply; 13+ messages in thread
From: Anand Jain @ 2024-05-06 13:17 UTC (permalink / raw)
To: Yordan; +Cc: quwenruo.btrfs, linux-btrfs
On 5/6/24 18:53, Yordan wrote:
> The attached file which is a reduced version of the problematic image,
> produced by removing all files and directories, except 5 of the
> problematic files and their path. Then its filesystem filled with zero
> file, shrinked with "resize2fs" to 512M and compressed to 7M.
>
> md5sum sda3.img.zst
> 9eec41fee47e3db555edeaba5d8d2e9a sda3.img.zst
>
> (chroot) livecd / # zstd -d sda3.img.zst -o sda3.img
> sda3.img.zst : 536870912 bytes
> (chroot) livecd / # mount -o ro sda3.img k
> (chroot) livecd / # find k/ -type f | xargs md5sum >files.md5
> (chroot) livecd / # md5sum -c files.md5 | grep -v OK
>
> (chroot) livecd / # umount k
> (chroot) livecd / # B/btrfs-progs/btrfs-convert sda3.img
> btrfs-convert from btrfs-progs v6.8.1
> > Source filesystem:
> Type: ext2
> Label:
> Blocksize: 4096
> UUID: b3a78a9f-37e7-4ccb-bedb-1f800a6a5a19
> Target filesystem:
> Label:
> Blocksize: 4096
> Nodesize: 16384
> UUID: d7c77d2f-d470-450a-ba0e-b6567ad3f4b3
> Checksum: crc32c
> Features: extref, skinny-metadata, no-holes, free-space-tree (default)
> Data csum: yes
> Inline data: yes
> Copy xattr: yes
> Reported stats:
> Total space: 536870912
> Free space: 326238208 (60.77%)
> Inode count: 32768
> Free inodes: 32743
> Block count: 131072
> Create initial btrfs filesystem
> Create ext2 image file
> Create btrfs metadata
> Copy inodes [o] [ 1/ 25]
> Free space cache cleared
> Conversion complete
>
> (chroot) livecd / # mount -o ro sda3.img k
> (chroot) livecd / # md5sum -c files.md5
> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite:
> FAILED
> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite:
> FAILED
> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite:
> FAILED
> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite:
> FAILED
> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3870112724rsegmnoittet-es.sqlite:
> OK
> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite:
> FAILED
> md5sum: WARNING: 5 computed checksums did NOT match
>
Are these test results with the v2 patchset? Thanks, Anand.
> Regards, Jordan.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: btrfs-convert on 24Gb image corrupts files.
2024-05-06 13:17 ` Anand Jain
@ 2024-05-06 13:35 ` Yordan
2024-05-06 13:43 ` Anand Jain
0 siblings, 1 reply; 13+ messages in thread
From: Yordan @ 2024-05-06 13:35 UTC (permalink / raw)
To: Anand Jain; +Cc: quwenruo.btrfs, linux-btrfs
NO, I have problems applying patch4, so its unpatchet
error: patch failed: convert/source-fs.c:316
error: convert/source-fs.c: patch does not apply
I just sent you a small problematic image so you can test it yourself.
On Mon, May 6, 2024 at 4:17 PM Anand Jain <anand.jain@oracle.com> wrote:
>
>
>
> On 5/6/24 18:53, Yordan wrote:
> > The attached file which is a reduced version of the problematic image,
> > produced by removing all files and directories, except 5 of the
> > problematic files and their path. Then its filesystem filled with zero
> > file, shrinked with "resize2fs" to 512M and compressed to 7M.
> >
> > md5sum sda3.img.zst
> > 9eec41fee47e3db555edeaba5d8d2e9a sda3.img.zst
> >
> > (chroot) livecd / # zstd -d sda3.img.zst -o sda3.img
> > sda3.img.zst : 536870912 bytes
> > (chroot) livecd / # mount -o ro sda3.img k
> > (chroot) livecd / # find k/ -type f | xargs md5sum >files.md5
> > (chroot) livecd / # md5sum -c files.md5 | grep -v OK
> >
> > (chroot) livecd / # umount k
> > (chroot) livecd / # B/btrfs-progs/btrfs-convert sda3.img
> > btrfs-convert from btrfs-progs v6.8.1
> > > Source filesystem:
> > Type: ext2
> > Label:
> > Blocksize: 4096
> > UUID: b3a78a9f-37e7-4ccb-bedb-1f800a6a5a19
> > Target filesystem:
> > Label:
> > Blocksize: 4096
> > Nodesize: 16384
> > UUID: d7c77d2f-d470-450a-ba0e-b6567ad3f4b3
> > Checksum: crc32c
> > Features: extref, skinny-metadata, no-holes, free-space-tree (default)
> > Data csum: yes
> > Inline data: yes
> > Copy xattr: yes
> > Reported stats:
> > Total space: 536870912
> > Free space: 326238208 (60.77%)
> > Inode count: 32768
> > Free inodes: 32743
> > Block count: 131072
> > Create initial btrfs filesystem
> > Create ext2 image file
> > Create btrfs metadata
> > Copy inodes [o] [ 1/ 25]
> > Free space cache cleared
> > Conversion complete
> >
> > (chroot) livecd / # mount -o ro sda3.img k
> > (chroot) livecd / # md5sum -c files.md5
> > k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite:
> > FAILED
> > k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite:
> > FAILED
> > k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite:
> > FAILED
> > k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite:
> > FAILED
> > k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3870112724rsegmnoittet-es.sqlite:
> > OK
> > k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite:
> > FAILED
> > md5sum: WARNING: 5 computed checksums did NOT match
> >
>
>
> Are these test results with the v2 patchset? Thanks, Anand.
>
>
> > Regards, Jordan.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: btrfs-convert on 24Gb image corrupts files.
2024-05-06 13:35 ` Yordan
@ 2024-05-06 13:43 ` Anand Jain
2024-05-06 14:57 ` Yordan
0 siblings, 1 reply; 13+ messages in thread
From: Anand Jain @ 2024-05-06 13:43 UTC (permalink / raw)
To: Yordan; +Cc: quwenruo.btrfs, linux-btrfs
It is meant to be applied on latest devel branch.
With the v2 patches:
$ ./btrfs-convert ~/sda3.img
btrfs-convert from btrfs-progs v6.8
Source filesystem:
Type: ext2
Label:
Blocksize: 4096
UUID: b3a78a9f-37e7-4ccb-bedb-1f800a6a5a19
Target filesystem:
Label:
Blocksize: 4096
Nodesize: 16384
UUID: d8c61c42-1b2c-478d-8ae6-bf45213e8df4
Checksum: crc32c
Features: extref, skinny-metadata, no-holes, free-space-tree
(default)
Data csum: yes
Inline data: yes
Copy xattr: yes
Reported stats:
Total space: 536870912
Free space: 326238208 (60.77%)
Inode count: 32768
Free inodes: 32743
Block count: 131072
Create initial btrfs filesystem
Create ext2 image file
Create btrfs metadata
ERROR: inode 20 index 0: identified unsupported merged block length 1
wanted 12
ERROR: failed to copy ext2 inode 20: -22
ERROR: error during copy_inodes -22
WARNING: error during conversion, the original filesystem is not modified
On 5/6/24 21:35, Yordan wrote:
> NO, I have problems applying patch4, so its unpatchet
>
> error: patch failed: convert/source-fs.c:316
> error: convert/source-fs.c: patch does not apply
>
> I just sent you a small problematic image so you can test it yourself.
>
> On Mon, May 6, 2024 at 4:17 PM Anand Jain <anand.jain@oracle.com> wrote:
>>
>>
>>
>> On 5/6/24 18:53, Yordan wrote:
>>> The attached file which is a reduced version of the problematic image,
>>> produced by removing all files and directories, except 5 of the
>>> problematic files and their path. Then its filesystem filled with zero
>>> file, shrinked with "resize2fs" to 512M and compressed to 7M.
>>>
>>> md5sum sda3.img.zst
>>> 9eec41fee47e3db555edeaba5d8d2e9a sda3.img.zst
>>>
>>> (chroot) livecd / # zstd -d sda3.img.zst -o sda3.img
>>> sda3.img.zst : 536870912 bytes
>>> (chroot) livecd / # mount -o ro sda3.img k
>>> (chroot) livecd / # find k/ -type f | xargs md5sum >files.md5
>>> (chroot) livecd / # md5sum -c files.md5 | grep -v OK
>>>
>>> (chroot) livecd / # umount k
>>> (chroot) livecd / # B/btrfs-progs/btrfs-convert sda3.img
>>> btrfs-convert from btrfs-progs v6.8.1
>>> > Source filesystem:
>>> Type: ext2
>>> Label:
>>> Blocksize: 4096
>>> UUID: b3a78a9f-37e7-4ccb-bedb-1f800a6a5a19
>>> Target filesystem:
>>> Label:
>>> Blocksize: 4096
>>> Nodesize: 16384
>>> UUID: d7c77d2f-d470-450a-ba0e-b6567ad3f4b3
>>> Checksum: crc32c
>>> Features: extref, skinny-metadata, no-holes, free-space-tree (default)
>>> Data csum: yes
>>> Inline data: yes
>>> Copy xattr: yes
>>> Reported stats:
>>> Total space: 536870912
>>> Free space: 326238208 (60.77%)
>>> Inode count: 32768
>>> Free inodes: 32743
>>> Block count: 131072
>>> Create initial btrfs filesystem
>>> Create ext2 image file
>>> Create btrfs metadata
>>> Copy inodes [o] [ 1/ 25]
>>> Free space cache cleared
>>> Conversion complete
>>>
>>> (chroot) livecd / # mount -o ro sda3.img k
>>> (chroot) livecd / # md5sum -c files.md5
>>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite:
>>> FAILED
>>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite:
>>> FAILED
>>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite:
>>> FAILED
>>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite:
>>> FAILED
>>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3870112724rsegmnoittet-es.sqlite:
>>> OK
>>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite:
>>> FAILED
>>> md5sum: WARNING: 5 computed checksums did NOT match
>>>
>>
>>
>> Are these test results with the v2 patchset? Thanks, Anand.
>>
>>
>>> Regards, Jordan.
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: btrfs-convert on 24Gb image corrupts files.
2024-05-06 13:43 ` Anand Jain
@ 2024-05-06 14:57 ` Yordan
0 siblings, 0 replies; 13+ messages in thread
From: Yordan @ 2024-05-06 14:57 UTC (permalink / raw)
To: Anand Jain; +Cc: quwenruo.btrfs, linux-btrfs
Yes, the same error after patching, and the same error on the original
24G image:
B/btrfs-progs/btrfs-convert sda3.img
btrfs-convert from btrfs-progs v6.8.1
Source filesystem:
Type: ext2
Label:
Blocksize: 4096
UUID: b3a78a9f-37e7-4ccb-bedb-1f800a6a5a19
Target filesystem:
Label:
Blocksize: 4096
Nodesize: 16384
UUID: 6595aa9c-415d-4ab0-bd9f-9a7f2ee8e3a9
Checksum: crc32c
Features: extref, skinny-metadata, no-holes, free-space-tree (default)
Data csum: yes
Inline data: yes
Copy xattr: yes
Reported stats:
Total space: 25769803776
Free space: 8655269888 (33.59%)
Inode count: 1572864
Free inodes: 1325061
Block count: 6291456
Create initial btrfs filesystem
Create ext2 image file
Create btrfs metadata
ERROR: inode 527126 index 2: identified unsupported merged block
length 9 wanted 10
ERROR: failed to copy ext2 inode 527126: -22
ERROR: error during copy_inodes -22
WARNING: error during conversion, the original filesystem is not modified
On Mon, May 6, 2024 at 4:43 PM Anand Jain <anand.jain@oracle.com> wrote:
>
>
> It is meant to be applied on latest devel branch.
>
>
> With the v2 patches:
>
> $ ./btrfs-convert ~/sda3.img
> btrfs-convert from btrfs-progs v6.8
>
> Source filesystem:
> Type: ext2
> Label:
> Blocksize: 4096
> UUID: b3a78a9f-37e7-4ccb-bedb-1f800a6a5a19
> Target filesystem:
> Label:
> Blocksize: 4096
> Nodesize: 16384
> UUID: d8c61c42-1b2c-478d-8ae6-bf45213e8df4
> Checksum: crc32c
> Features: extref, skinny-metadata, no-holes, free-space-tree
> (default)
> Data csum: yes
> Inline data: yes
> Copy xattr: yes
> Reported stats:
> Total space: 536870912
> Free space: 326238208 (60.77%)
> Inode count: 32768
> Free inodes: 32743
> Block count: 131072
> Create initial btrfs filesystem
> Create ext2 image file
> Create btrfs metadata
> ERROR: inode 20 index 0: identified unsupported merged block length 1
> wanted 12
> ERROR: failed to copy ext2 inode 20: -22
> ERROR: error during copy_inodes -22
> WARNING: error during conversion, the original filesystem is not modified
>
>
>
>
>
> On 5/6/24 21:35, Yordan wrote:
> > NO, I have problems applying patch4, so its unpatchet
> >
> > error: patch failed: convert/source-fs.c:316
> > error: convert/source-fs.c: patch does not apply
> >
> > I just sent you a small problematic image so you can test it yourself.
> >
> > On Mon, May 6, 2024 at 4:17 PM Anand Jain <anand.jain@oracle.com> wrote:
> >>
> >>
> >>
> >> On 5/6/24 18:53, Yordan wrote:
> >>> The attached file which is a reduced version of the problematic image,
> >>> produced by removing all files and directories, except 5 of the
> >>> problematic files and their path. Then its filesystem filled with zero
> >>> file, shrinked with "resize2fs" to 512M and compressed to 7M.
> >>>
> >>> md5sum sda3.img.zst
> >>> 9eec41fee47e3db555edeaba5d8d2e9a sda3.img.zst
> >>>
> >>> (chroot) livecd / # zstd -d sda3.img.zst -o sda3.img
> >>> sda3.img.zst : 536870912 bytes
> >>> (chroot) livecd / # mount -o ro sda3.img k
> >>> (chroot) livecd / # find k/ -type f | xargs md5sum >files.md5
> >>> (chroot) livecd / # md5sum -c files.md5 | grep -v OK
> >>>
> >>> (chroot) livecd / # umount k
> >>> (chroot) livecd / # B/btrfs-progs/btrfs-convert sda3.img
> >>> btrfs-convert from btrfs-progs v6.8.1
> >>> > Source filesystem:
> >>> Type: ext2
> >>> Label:
> >>> Blocksize: 4096
> >>> UUID: b3a78a9f-37e7-4ccb-bedb-1f800a6a5a19
> >>> Target filesystem:
> >>> Label:
> >>> Blocksize: 4096
> >>> Nodesize: 16384
> >>> UUID: d7c77d2f-d470-450a-ba0e-b6567ad3f4b3
> >>> Checksum: crc32c
> >>> Features: extref, skinny-metadata, no-holes, free-space-tree (default)
> >>> Data csum: yes
> >>> Inline data: yes
> >>> Copy xattr: yes
> >>> Reported stats:
> >>> Total space: 536870912
> >>> Free space: 326238208 (60.77%)
> >>> Inode count: 32768
> >>> Free inodes: 32743
> >>> Block count: 131072
> >>> Create initial btrfs filesystem
> >>> Create ext2 image file
> >>> Create btrfs metadata
> >>> Copy inodes [o] [ 1/ 25]
> >>> Free space cache cleared
> >>> Conversion complete
> >>>
> >>> (chroot) livecd / # mount -o ro sda3.img k
> >>> (chroot) livecd / # md5sum -c files.md5
> >>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite:
> >>> FAILED
> >>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite:
> >>> FAILED
> >>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite:
> >>> FAILED
> >>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite:
> >>> FAILED
> >>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3870112724rsegmnoittet-es.sqlite:
> >>> OK
> >>> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite:
> >>> FAILED
> >>> md5sum: WARNING: 5 computed checksums did NOT match
> >>>
> >>
> >>
> >> Are these test results with the v2 patchset? Thanks, Anand.
> >>
> >>
> >>> Regards, Jordan.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: btrfs-convert on 24Gb image corrupts files.
@ 2024-05-05 15:05 Yordan
2024-05-05 22:31 ` Qu Wenruo
0 siblings, 1 reply; 13+ messages in thread
From: Yordan @ 2024-05-05 15:05 UTC (permalink / raw)
To: quwenruo.btrfs, linux-btrfs
Hi, it's Jordan - the reporter, I'm switching the email server as the
first one presumably did dirty things and was rejected by the "mlmmj
program managing the mailing list".
>Mind to provide the "filefrag -v" output after conversion using Anand's
fixes?
>Mind to provide the "filefrag -v" output after conversion using Anand's
fixes?
After applying the 4 patches and compiling:
(chroot) livecd / # B/btrfs-progs/btrfs-convert sda3.img
btrfs-convert from btrfs-progs v6.8.1
Source filesystem:
Type: ext2
Label:
Blocksize: 4096
UUID: b3a78a9f-37e7-4ccb-bedb-1f800a6a5a19
Target filesystem:
Label:
Blocksize: 4096
Nodesize: 16384
UUID: 981d2a1c-c1a9-47b7-acf2-9724ebc96ed0
Checksum: crc32c
Features: extref, skinny-metadata, no-holes, free-space-tree (default)
Data csum: yes
Inline data: yes
Copy xattr: yes
Reported stats:
Total space: 25769803776
Free space: 8862961664 (34.39%)
Inode count: 1572864
Free inodes: 1326275
Block count: 6291456
Create initial btrfs filesystem
Create ext2 image file
Create btrfs metadata
Copy inodes [o] [ 388838/ 246589]
Free space cache cleared
Conversion complete
(chroot) livecd / # mount -o ro sda3.img k
(chroot) livecd / # cd k
(chroot) livecd /k # md5sum -c ../files.md5 | grep -v OK
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite:
FAILED
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite:
FAILED
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite:
FAILED
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite:
FAILED
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite:
FAILED
./root/.mozilla/firefox/s554srh9.default-release/favicons.sqlite: FAILED
./root/.mozilla/firefox/s554srh9.default-release/places.sqlite: FAILED
md5sum: WARNING: 7 computed checksums did NOT match
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
Filesystem type is: 9123683e
File size of ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
is 49152 (12 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 1: 2217003.. 2217004: 2: shared
1: 2.. 11: 2217019.. 2217028: 10: 2217005: last,shared,eof
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite:
2 extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite
Filesystem type is: 9123683e
File size of ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite
is 49152 (12 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 1: 2217085.. 2217086: 2: shared
1: 2.. 11: 2217101.. 2217110: 10: 2217087: last,shared,eof
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite:
2 extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite
Filesystem type is: 9123683e
File size of ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite
is 49152 (12 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 1: 2222643.. 2222644: 2: shared
1: 2.. 11: 2244137.. 2244146: 10: 2222645: last,shared,eof
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite:
2 extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite
Filesystem type is: 9123683e
File size of ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite
is 49152 (12 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 1: 2244148.. 2244149: 2: shared
1: 2.. 11: 2244164.. 2244173: 10: 2244150: last,shared,eof
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite:
2 extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite
Filesystem type is: 9123683e
File size of ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite
is 49152 (12 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 1: 2217030.. 2217031: 2: shared
1: 2.. 11: 2217046.. 2217055: 10: 2217032: last,shared,eof
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite:
2 extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/favicons.sqlite
Filesystem type is: 9123683e
File size of ./root/.mozilla/firefox/s554srh9.default-release/favicons.sqlite
is 5242880 (1280 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 7: 2244106.. 2244113: 8: shared
1: 8.. 1279: 1449984.. 1451255: 1272: 2244114: last,shared,eof
./root/.mozilla/firefox/s554srh9.default-release/favicons.sqlite: 2
extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/places.sqlite
Filesystem type is: 9123683e
File size of ./root/.mozilla/firefox/s554srh9.default-release/places.sqlite
is 5242880 (1280 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 7: 2244097.. 2244104: 8: shared
1: 8.. 1279: 1485312.. 1486583: 1272: 2244105: last,shared,eof
./root/.mozilla/firefox/s554srh9.default-release/places.sqlite: 2 extents found
(chroot) livecd /k # cd ..
(chroot) livecd / # umount k
(chroot) livecd / # B/btrfs-progs/btrfs-convert -r sda3.img
btrfs-convert from btrfs-progs v6.8.1
Open filesystem for rollback:
Label:
UUID: 981d2a1c-c1a9-47b7-acf2-9724ebc96ed0
Restoring from: ext2_saved/image
Rollback succeeded
(chroot) livecd / # mount -o ro sda3.img k
(chroot) livecd / # cd k
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
Filesystem type is: ef53
File size of ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
is 49152 (12 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 1: 2217003.. 2217004: 2:
1: 2.. 10: 2217019.. 2217027: 9: 2217005:
2: 11.. 11: 2217028.. 2217028: 1: last,unwritten,eof
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite:
2 extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite
Filesystem type is: ef53
File size of ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite
is 49152 (12 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 1: 2217085.. 2217086: 2:
1: 2.. 10: 2217101.. 2217109: 9: 2217087:
2: 11.. 11: 2217110.. 2217110: 1: last,unwritten,eof
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite:
2 extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite
Filesystem type is: ef53
File size of ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite
is 49152 (12 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 1: 2222643.. 2222644: 2:
1: 2.. 10: 2244137.. 2244145: 9: 2222645:
2: 11.. 11: 2244146.. 2244146: 1: last,unwritten,eof
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite:
2 extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite
Filesystem type is: ef53
File size of ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite
is 49152 (12 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 1: 2244148.. 2244149: 2:
1: 2.. 10: 2244164.. 2244172: 9: 2244150:
2: 11.. 11: 2244173.. 2244173: 1: last,unwritten,eof
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite:
2 extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite
Filesystem type is: ef53
File size of ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite
is 49152 (12 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 1: 2217030.. 2217031: 2:
1: 2.. 10: 2217046.. 2217054: 9: 2217032:
2: 11.. 11: 2217055.. 2217055: 1: last,unwritten,eof
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite:
2 extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/favicons.sqlite
Filesystem type is: ef53
File size of ./root/.mozilla/firefox/s554srh9.default-release/favicons.sqlite
is 5242880 (1280 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 7: 2244106.. 2244113: 8:
1: 8.. 79: 1449984.. 1450055: 72: 2244114:
2: 80.. 1279: 1450056.. 1451255: 1200: last,unwritten,eof
./root/.mozilla/firefox/s554srh9.default-release/favicons.sqlite: 2
extents found
(chroot) livecd /k # filefrag -v
./root/.mozilla/firefox/s554srh9.default-release/places.sqlite
Filesystem type is: ef53
File size of ./root/.mozilla/firefox/s554srh9.default-release/places.sqlite
is 5242880 (1280 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 7: 2244097.. 2244104: 8:
1: 8.. 415: 1485312.. 1485719: 408: 2244105:
2: 416.. 1279: 1485720.. 1486583: 864: last,unwritten,eof
./root/.mozilla/firefox/s554srh9.default-release/places.sqlite: 2 extents found
(chroot) livecd /k #
Regards, Jordan.
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: btrfs-convert on 24Gb image corrupts files.
2024-05-05 15:05 Yordan
@ 2024-05-05 22:31 ` Qu Wenruo
0 siblings, 0 replies; 13+ messages in thread
From: Qu Wenruo @ 2024-05-05 22:31 UTC (permalink / raw)
To: Yordan, linux-btrfs
在 2024/5/6 00:35, Yordan 写道:
> Hi, it's Jordan - the reporter, I'm switching the email server as the
> first one presumably did dirty things and was rejected by the "mlmmj
> program managing the mailing list".
>
>> Mind to provide the "filefrag -v" output after conversion using Anand's
> fixes?
>
>> Mind to provide the "filefrag -v" output after conversion using Anand's
>
> fixes?
>
> After applying the 4 patches and compiling:
>
[...]
> (chroot) livecd /k # filefrag -v
> ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
> Filesystem type is: 9123683e
> File size of ./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
> is 49152 (12 blocks of 4096 bytes)
> ext: logical_offset: physical_offset: length: expected: flags:
> 0: 0.. 1: 2217003.. 2217004: 2: shared
> 1: 2.. 11: 2217019.. 2217028: 10: 2217005: last,shared,eof
OK, that's the problem, compared to the older values from ext4:
> 0: 0.. 1: 2217003.. 2217004: 2:
> 1: 2.. 10: 2217019.. 2217027: 9: 2217005:
> 2: 11.. 11: 2217028.. 2217028: 1: last,unwritten,eof
The last extent of btrfs still doesn't properly have the preallocated
flag, thus we read the garbage instead of filling it with 0.
I'll give more comprehensive review on Anand's fix.
And for the worst case I can change btrfs-convert to create a hole to
properly fix it.
Thanks for the detailed report!
Qu
^ permalink raw reply [flat|nested] 13+ messages in thread
* btrfs-convert on 24Gb image corrupts files.
@ 2024-05-05 0:34 jordan.j
2024-05-05 0:50 ` Qu Wenruo
2024-05-05 0:56 ` Anand Jain
0 siblings, 2 replies; 13+ messages in thread
From: jordan.j @ 2024-05-05 0:34 UTC (permalink / raw)
To: linux-btrfs
[-- Attachment #1.1: Type: text/plain, Size: 8717 bytes --]
cat /dev/sda3 >sda3.img
fsck.ext4 -fyv sda3.img
e2fsck 1.47.0 (5-Feb-2023)
Pass 1: Checking inodes, blocks, and sizes
Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
246589 inodes used (15.68%, out of 1572864)
191 non-contiguous files (0.1%)
116 non-contiguous directories (0.0%)
# of inodes with ind/dind/tind blocks: 0/0/0
Extent depth histogram: 235622/51/1
2219268 blocks used (35.27%, out of 6291456)
0 bad blocks
1 large file
220831 regular files
14688 directories
174 character device files
97 block device files
0 fifos
1641 links
10790 symbolic links (10636 fast symbolic links)
0 sockets
------------
248221 files
mkdir k
mount -o ro sda3.img k/
cd k
find -type f -print0 | xargs -0 md5sum >sda3_files.md5
cd ..
umount k
btrfs-convert sda3.img
btrfs-convert from btrfs-progs v6.7.1
Source filesystem:
Type: ext2
Label:
Blocksize: 4096
UUID: b3a78a9f-37e7-4ccb-bedb-1f800a6a5a19
Target filesystem:
Label:
Blocksize: 4096
Nodesize: 16384
UUID: 8e9b57b4-92cd-466f-9c02-c081a768bf40
Checksum: crc32c
Features: extref, skinny-metadata, no-holes, free-space-tree (default)
Data csum: yes
Inline data: yes
Copy xattr: yes
Reported stats:
Total space: 25769803776
Free space: 8862961664 (34.39%)
Inode count: 1572864
Free inodes: 1326275
Block count: 6291456
Create initial btrfs filesystem
Create ext2 image file
Create btrfs metadata
Copy inodes [.] [ 386128/ 246589]
Free space cache cleared
Conversion complete
mount -o ro sda3.img k/
cd k
md5sum -c sda3_files.md5 | grep -v OK
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite:
FAILED
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1657114595AmcateirvtiSty.sqlite:
FAILED
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2823318777ntouromlalnodry--naod.sqlite:
FAILED
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/2918063365piupsah.sqlite:
FAILED
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/1451318868ntouromlalnodry--epcr.sqlite:
FAILED
./root/.mozilla/firefox/s554srh9.default-release/favicons.sqlite: FAILED
./root/.mozilla/firefox/s554srh9.default-release/places.sqlite: FAILED
md5sum: WARNING: 7 computed checksums did NOT match
cd ..
umount k
btrfs-convert -r sda3.img
btrfs-convert from btrfs-progs v6.7.1
Open filesystem for rollback:
Label:
UUID: 8e9b57b4-92cd-466f-9c02-c081a768bf40
Restoring from: ext2_saved/image
Rollback succeeded
mount -o ro sda3.img k/
cd k
md5sum -c sda3_files.md5 | grep -v OK
*******
Few times did the same with exactly the same corruption on exactly the same
7 files.
Then transferred sda3.img to another machine, and did the same with exactly
the same result, to be sure it is not hardware related.
Corrupted files on both machines are exactly the same, with exactly the
same contents - their md5sums match.
Corrupt part of the filles is at the end of all 7 files - piece from
another file each of them - linux kernel source particularly, except the
last - some binary data.
The size of sda3.img file is 24G, 8G used, ext4 , md5sum of sda3.img and
/dev/sda3 matched until the conversion, but not after the rollback.
*******
On the first machine:
Linux livecd 6.6.21-gentoo-x86_64 #1 SMP PREEMPT_DYNAMIC Sun Apr 28
18:15:06 UTC 2024 x86_64 Intel(R) Xeon(R) CPU X5670 @ 2.93GHz GenuineIntel
GNU/Linux
btrfs-progs v6.7.1 (tried with 6.8.1 - all the same)
btrfs fi show
Label: none uuid: 62c086a7-e96e-414d-9c21-e5d1e310fae1
Total devices 1 FS bytes used 8.81GiB
devid 1 size 24.00GiB used 16.25GiB path /dev/loop1
btrfs fi df k
Data, single: total=15.74GiB, used=8.47GiB
System, single: total=4.00MiB, used=16.00KiB
Metadata, single: total=512.00MiB, used=352.00MiB
GlobalReserve, single: total=18.88MiB, used=0.00B
second machine:
Linux le 6.8.8-gentoo #2 SMP PREEMPT_DYNAMIC Sat May 4 19:39:02 EEST 2024
x86_64 Intel(R) Core(TM) i5 CPU M 520 @ 2.40GHz GenuineIntel GNU/Linux
btrfs-progs v6.7.1
btrfs fi show
Label: none uuid: ac30b8a1-a671-4743-aa98-b9dd7b0fd9fc
Total devices 1 FS bytes used 9.17GiB
devid 1 size 24.00GiB used 16.44GiB path /dev/loop0
btrfs fi df m
Data, single: total=15.94GiB, used=8.82GiB
System, single: total=4.00MiB, used=16.00KiB
Metadata, single: total=512.00MiB, used=354.08MiB
GlobalReserve, single: total=19.30MiB, used=0.00B
*******
no errors in dmesgs
*******
on ext4:
ilefrag -v
m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
Filesystem type is: ef53
File size of
m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
is 49152 (12 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 1: 2217003.. 2217004: 2:
1: 2.. 10: 2217019.. 2217027: 9: 2217005:
2: 11.. 11: 2217028.. 2217028: 1: last,unwritten,eof
m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite:
2 extents found
on btrfs:
filefrag -v
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
Filesystem type is: 9123683e
File size of
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
is 49152 (12 blocks of 4096 bytes)
ext: logical_offset: physical_offset: length: expected: flags:
0: 0.. 1: 2217003.. 2217004: 2: shared
1: 2.. 11: 2217019.. 2217028: 10: 2217005: last,shared,eof
k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite:
2 extents found
*********
Tried the four patches from:
https://lore.kernel.org/linux-btrfs/6d2a19ced4551bfcf2a5d841921af7f84c4ea950.1714722726.git.anand.jain@oracle.com/
on the https://github.com/kdave/btrfs-progs.git tree - no change.
**********
Attached file 1st is the original file from the ext4 image, 2nd is the
corrupted file(with some linux source code "injected"), from btrfs.
**********
How looks a corrupted file on the boundary between the non-corrupted(1st
part) and corrupted(2nd part) of the file, seen in less:
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@
^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@^@*vdev,
struct virtio_shm_region *region, u8 id);
};
/* If driver didn't advertise the feature, it will never appear. */
void virtio_check_driver_offered_feature(const struct virtio_device *vdev,
unsigned int fbit);
/**
* __virtio_test_bit - helper to test feature bits. For use by transports.
* Devices should normally use virtio_has_feature,
* which includes more checks.
* @vdev: the device
* @fbit: the feature bit
*/
static inline bool __virtio_test_bit(const struct virtio_device *vdev,
unsigned int fbit)
{
/* Did you forget to fix assumptions on max features? */
if (__builtin_constant_p(fbit))
BUILD_BUG_ON(fbit >= 64);
else
BUG_ON(fbit >= 64);
return vdev->features
}
/**
* __virtio_set_bit - helper to set feature bits. For use by transports.
./root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
lines 10-38/162 93%
Regards.
Jordan.
[-- Attachment #1.2: Type: text/html, Size: 16624 bytes --]
[-- Attachment #2: 3561288849sdhlie.sqlite --]
[-- Type: application/octet-stream, Size: 49152 bytes --]
[-- Attachment #3: 3561288849sdhlie.sqlite --]
[-- Type: application/octet-stream, Size: 49152 bytes --]
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: btrfs-convert on 24Gb image corrupts files.
2024-05-05 0:34 jordan.j
@ 2024-05-05 0:50 ` Qu Wenruo
2024-05-05 0:56 ` Anand Jain
1 sibling, 0 replies; 13+ messages in thread
From: Qu Wenruo @ 2024-05-05 0:50 UTC (permalink / raw)
To: jordan.j, linux-btrfs
在 2024/5/5 10:04, jordan.j@mail.bg 写道:
[...]
> *******
>
> on ext4:
>
> ilefrag -v
> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
> Filesystem type is: ef53
> File size of
> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite is 49152 (12 blocks of 4096 bytes)
> ext: logical_offset: physical_offset: length: expected:
> flags:
> 0: 0.. 1: 2217003.. 2217004: 2:
> 1: 2.. 10: 2217019.. 2217027: 9: 2217005:
> 2: 11.. 11: 2217028.. 2217028: 1:
> last,unwritten,eof
> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite: 2 extents found
>
> on btrfs:
>
> filefrag -v
> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
> Filesystem type is: 9123683e
> File size of
> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite is 49152 (12 blocks of 4096 bytes)
> ext: logical_offset: physical_offset: length: expected:
> flags:
> 0: 0.. 1: 2217003.. 2217004: 2:
> shared
> 1: 2.. 11: 2217019.. 2217028: 10: 2217005:
> last,shared,eof
> k/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite: 2 extents found
>
This is exactly the problem of unwritten (preallocated) extents.
Considering all corrupted files are from sqlite, it's highly possible
related to that.
> *********
>
> Tried the four patches from:
> https://lore.kernel.org/linux-btrfs/6d2a19ced4551bfcf2a5d841921af7f84c4ea950.1714722726.git.anand.jain@oracle.com/
>
> on the https://github.com/kdave/btrfs-progs.git tree - no change.
Unfortunately this means the fix is not working.
Mind to provide the "filefrag -v" output after conversion using Anand's
fixes?
I'm afraid there is something wrong for the fix.
For the worst case, I can go a simpler solution by just converting
preallocated extents to holes as a hot fix.
To me, there is no chance we can utilize the preallocated space anyway,
since the extent is shared with the ext4 image, thus we have to do COW.
So focusing on creating the same preallocated data extents in conversion
makes no sense.
Thanks,
Qu
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: btrfs-convert on 24Gb image corrupts files.
2024-05-05 0:34 jordan.j
2024-05-05 0:50 ` Qu Wenruo
@ 2024-05-05 0:56 ` Anand Jain
2024-05-05 5:54 ` Qu Wenruo
1 sibling, 1 reply; 13+ messages in thread
From: Anand Jain @ 2024-05-05 0:56 UTC (permalink / raw)
To: jordan.j, linux-btrfs
> on ext4:
>
> ilefrag -v
> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
> Filesystem type is: ef53
> File size of
> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite is 49152 (12 blocks of 4096 bytes)
> ext: logical_offset: physical_offset: length: expected:
> flags:
> 0: 0.. 1: 2217003.. 2217004: 2:
> 1: 2.. 10: 2217019.. 2217027: 9: 2217005:
> 2: 11.. 11: 2217028.. 2217028: 1:
> last,unwritten,eof
> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite: 2 extents found
>
btrfs-convert doesn't support ext4's "unwritten" extents. V2 of the fix
is in progress.
[PATCH 0/4] btrfs-progs: add support ext4 unwritten file extent
Thanks, Anand
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: btrfs-convert on 24Gb image corrupts files.
2024-05-05 0:56 ` Anand Jain
@ 2024-05-05 5:54 ` Qu Wenruo
2024-05-06 0:33 ` Anand Jain
0 siblings, 1 reply; 13+ messages in thread
From: Qu Wenruo @ 2024-05-05 5:54 UTC (permalink / raw)
To: Anand Jain, jordan.j, linux-btrfs
在 2024/5/5 10:26, Anand Jain 写道:
>
>> on ext4:
>>
>> ilefrag -v
>> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
>> Filesystem type is: ef53
>> File size of
>> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite is 49152 (12 blocks of 4096 bytes)
>> ext: logical_offset: physical_offset: length: expected:
>> flags:
>> 0: 0.. 1: 2217003.. 2217004: 2:
>> 1: 2.. 10: 2217019.. 2217027: 9: 2217005:
>> 2: 11.. 11: 2217028.. 2217028: 1:
>> last,unwritten,eof
>> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite: 2 extents found
>>
>
>
> btrfs-convert doesn't support ext4's "unwritten" extents. V2 of the fix
> is in progress.
>
> [PATCH 0/4] btrfs-progs: add support ext4 unwritten file extent
But the reporter also tried your patchset already:
> *********
>
> Tried the four patches from:
https://lore.kernel.org/linux-btrfs/6d2a19ced4551bfcf2a5d841921af7f84c4ea950.1714722726.git.anand.jain@oracle.com/
> on the https://github.com/kdave/btrfs-progs.git tree - no change.
>
> **********
Any clue why it doesn't work?
Anyway, for now I'd prefer to change those unwritten extents just to holes.
Remember every file extent is shared between the inode and the ext4
image, meaning even if we created the preallocated extents correctly,
any newer write would be COWed anyway.
Thus for now, I believe changing the unwritten extents to holes would be
much easier.
Thanks,
Qu
>
>
> Thanks, Anand
>
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: btrfs-convert on 24Gb image corrupts files.
2024-05-05 5:54 ` Qu Wenruo
@ 2024-05-06 0:33 ` Anand Jain
0 siblings, 0 replies; 13+ messages in thread
From: Anand Jain @ 2024-05-06 0:33 UTC (permalink / raw)
To: Qu Wenruo, jordan.j, linux-btrfs
On 5/5/24 13:54, Qu Wenruo wrote:
>
>
> 在 2024/5/5 10:26, Anand Jain 写道:
>>
>>> on ext4:
>>>
>>> ilefrag -v
>>> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite
>>> Filesystem type is: ef53
>>> File size of
>>> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite is 49152 (12 blocks of 4096 bytes)
>>> ext: logical_offset: physical_offset: length: expected:
>>> flags:
>>> 0: 0.. 1: 2217003.. 2217004: 2:
>>> 1: 2.. 10: 2217019.. 2217027: 9: 2217005:
>>> 2: 11.. 11: 2217028.. 2217028: 1:
>>> last,unwritten,eof
>>> m/root/.mozilla/firefox/s554srh9.default-release/storage/permanent/chrome/idb/3561288849sdhlie.sqlite: 2 extents found
>>>
>>
>>
>> btrfs-convert doesn't support ext4's "unwritten" extents. V2 of the fix
>> is in progress.
>>
>> [PATCH 0/4] btrfs-progs: add support ext4 unwritten file extent
>
> But the reporter also tried your patchset already:
>
> > *********
> >
> > Tried the four patches from:
> https://lore.kernel.org/linux-btrfs/6d2a19ced4551bfcf2a5d841921af7f84c4ea950.1714722726.git.anand.jain@oracle.com/
>
> > on the https://github.com/kdave/btrfs-progs.git tree - no change.
> >
> > **********
>
> Any clue why it doesn't work?
>
> Anyway, for now I'd prefer to change those unwritten extents just to holes.
>
> Remember every file extent is shared between the inode and the ext4
> image, meaning even if we created the preallocated extents correctly,
> any newer write would be COWed anyway.
>
> Thus for now, I believe changing the unwritten extents to holes would be
> much easier.
>
Ah. I missed that part. Thanks for highlighting.
The data size and extent size might not align. In v2, I'll detect
this and handle it. I'm still figuring out how to split merged
blocks into separate extents, so that prealloc flag/as a hole
can be set on only appropriate extent size.
Thanks, Anand
> Thanks,
> Qu
>>
>>
>> Thanks, Anand
>>
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2024-05-06 14:58 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz follow: Atom feed
-- links below jump to the message on this page --
2024-05-05 22:30 You can reproduce it yourself now Yordan
2024-05-06 10:53 ` btrfs-convert on 24Gb image corrupts files Yordan
2024-05-06 13:17 ` Anand Jain
2024-05-06 13:35 ` Yordan
2024-05-06 13:43 ` Anand Jain
2024-05-06 14:57 ` Yordan
-- strict thread matches above, loose matches on Subject: below --
2024-05-05 15:05 Yordan
2024-05-05 22:31 ` Qu Wenruo
2024-05-05 0:34 jordan.j
2024-05-05 0:50 ` Qu Wenruo
2024-05-05 0:56 ` Anand Jain
2024-05-05 5:54 ` Qu Wenruo
2024-05-06 0:33 ` Anand Jain
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox