Re: AW: Tree-checker Issue / Corrupt FS after upgrade ?

[Qu Wenruo <quwenruo.btrfs@gmx.com>]

[-- Attachment #1.1: Type: text/plain, Size: 8884 bytes --]



On 2020/8/15 下午3:39, benjamin.haendel@gmx.net wrote:
> Hi Qu,
> 
> i did not know how to log this.
> 
> I have copy pasted what i could from the shell attached as a txt file from the latest check.

The fs is kinda OK.

It has some bad extent flags, but that shouldn't affect the fs read and
write.
> 
> Its not windows related, i meant that when i access the mapped/mounted share via my clients (Windows)
> Its showing still the old numbers although a few TB of data are missing.

That's not missing, just btrfs extent booking.
It's pretty common in btrfs, as btrfs do COW for its data extents.

Thanks,
Qu

> 
> Best Regards,
> Ben
> 
> -----Ursprüngliche Nachricht-----
> Von: Qu Wenruo <quwenruo.btrfs@gmx.com> 
> Gesendet: Samstag, 15. August 2020 01:06
> An: benjamin.haendel@gmx.net
> Cc: linux-btrfs@vger.kernel.org
> Betreff: Re: AW: AW: AW: Tree-checker Issue / Corrupt FS after upgrade ?
> 
> 
> 
> On 2020/8/15 上午3:05, benjamin.haendel@gmx.net wrote:
>> Hi Qu,
>>
>> i did all that and i have now again access to the Storage.
> 
> Btrfs check --repair log of that run please.
> I guess you haven't keep it...
> 
> And "btrfs check" of current fs please.
> 
>> Sadly i am now encountering some errors and bugs and i thought You 
>> might want to know about that.
>>
>> 1. I am missing some folders and files 2. Some folders are there but 
>> no files in them 3. i can only access the drive via the samba share - 
>> not on the server directly 4. In Windows it shows "28TB of usage" but 
>> when i mark all data and hit alt+enter it counts to 21.1 TB only
> 
> Windows? Why it's related to Windows then?
> We're talking about btrfs, right?
> 
> For the total usage, it's completely sane as btrfs use extent booking, which takes extra space.
> 
> Thanks,
> Qu
> 
> 
>>
>> I am unsure if that data is now lost forever or if ist just a bug or what exactly is the problem.
>> Do you think i could still "btrfs send" the Data and it will all be back or are those files lost due to new enumeration by btrfs-check Repair ?
>>
>> Best Regards,
>> Ben
>>
>>
>> -----Ursprüngliche Nachricht-----
>> Von: Qu Wenruo <quwenruo.btrfs@gmx.com>
>> Gesendet: Freitag, 14. August 2020 02:57
>> An: benjamin.haendel@gmx.net
>> Cc: linux-btrfs@vger.kernel.org
>> Betreff: Re: AW: AW: Tree-checker Issue / Corrupt FS after upgrade ?
>>
>>
>>
>> On 2020/8/14 上午5:01, benjamin.haendel@gmx.net wrote:
>>> Hi Qu,
>>>
>>> i downloaded the code, compiled and installed it.
>>
>> You still need to run "btrfs check --repair".
>>
>> Considering you haven't post that output, I guess you didn't run that.
>>
>>> Still could not mount the FS with error:
>>> mount: /media/Storage: wrong fs type, bad option, bad superblock on /dev/mapper/Crypto, missing codepage or helper program, or other error.
>>>
>>> Dmesg is telling me:
>>> [  174.061205] Btrfs loaded, crc32c=crc32c-intel [  174.081002] BTRFS: 
>>> device label Storage devid 1 transid 207602 /dev/dm-0 [  192.175157] 
>>> BTRFS info (device dm-0): disk space caching is enabled [  
>>> 204.025699] BTRFS critical (device dm-0): corrupt leaf: 
>>> block=22751711027200
>>> slot=1 extent bytenr=6754755866624 len=4096 invalid generation, have
>>> 22795412619264 expect (0, 207603] [  204.025703] BTRFS error (device
>>> dm-0): block=22751711027200 read time tree block corruption detected 
>>> [ 204.025731] BTRFS error (device dm-0): failed to read block groups: 
>>> -5 [  204.076709] BTRFS error (device dm-0): open_ctree failed
>>>
>>> I am unsure of what to do now. Did i have to run a btrfs check --repair with the compiled btrfs branch ?
>>
>> Yes.
>>
>>> I also removed all other versions of btrfs now to see if an old version could read it, it now says:
>>> root@Userv:~# btrfs --version
>>> btrfs-progs v4.1
>>
>> Nope, you're not using the correct version.
>>
>>>
>>> Still i can not mount the drive. How can i at least get read access to my data ?
>>
>> Just go using your older kernel, and wait for your distro to provide newer enough btrfs-progs.
>>
>> Thanks,
>> Qu
>>
>>>
>>> Best Regards,
>>> Ben
>>>
>>> -----Ursprüngliche Nachricht-----
>>> Von: Qu Wenruo <quwenruo.btrfs@gmx.com>
>>> Gesendet: Donnerstag, 13. August 2020 01:30
>>> An: benjamin.haendel@gmx.net; linux-btrfs@vger.kernel.org
>>> Betreff: Re: AW: Tree-checker Issue / Corrupt FS after upgrade ?
>>>
>>>
>>>
>>> On 2020/8/13 上午7:10, benjamin.haendel@gmx.net wrote:
>>>> Hi Qu,
>>>>
>>>> thanks for your reply, i am not sure what to do from here on.
>>>> What do i have to download from here or compile/make/install etc. ?
>>>
>>> You need to compile that branch.
>>>
>>> For how to compile, please check the README.md.
>>>
>>>
>>>>
>>>> I'm no total idiot but i still don't understand what i have to get 
>>>> And how to apply it...sorry :(
>>>
>>> Or you can use btrfs-send to send out the content of your fs with older kernel, and create a new fs using newer kernel, then receive the stream.
>>>
>>> The uninitialized data is only in extent tree, which won't be sent with the stream, by receiving it with newer kernel, you won't lose anything.
>>>
>>> Thanks,
>>> Qu
>>>
>>>>
>>>> Best Regards,
>>>> Ben
>>>>
>>>> -----Ursprüngliche Nachricht-----
>>>> Von: Qu Wenruo <quwenruo.btrfs@gmx.com>
>>>> Gesendet: Donnerstag, 13. August 2020 00:51
>>>> An: benjamin.haendel@gmx.net; linux-btrfs@vger.kernel.org
>>>> Betreff: Re: Tree-checker Issue / Corrupt FS after upgrade ?
>>>>
>>>>
>>>>
>>>> On 2020/8/13 上午12:58, benjamin.haendel@gmx.net wrote:
>>>>> Hi,
>>>>>
>>>>> i have been running my little Storage (32TB) on a Ubuntu 18.04 LTS 
>>>>> Machine with btrfs-progs 4.17. I then did my monthly upgrade (apt
>>>>> dist-upgrade) and after a reboot my Partition could not mount with the error message:
>>>>> "root@Userv:/home/benjamin# mount -r ro btrfs /dev/mapper/Crypto 
>>>>> /media/Storage
>>>>> mount: bad usage"
>>>>>
>>>>> I then proceeded to run a btrfs check which gave thousands of 
>>>>> errors and then also a super-recover:
>>>>> root@Userv:/home/benjamin# btrfs rescue super-recover -v 
>>>>> /dev/mapper/Crypto All Devices:
>>>>> Device: id = 1, name = /dev/mapper/Crypto
>>>>>
>>>>> Before Recovering:
>>>>> [All good supers]:
>>>>> device name = /dev/mapper/Crypto
>>>>> superblock bytenr = 65536
>>>>>
>>>>> device name = /dev/mapper/Crypto
>>>>> superblock bytenr = 67108864
>>>>>
>>>>> device name = /dev/mapper/Crypto
>>>>> superblock bytenr = 274877906944
>>>>>
>>>>> [All bad supers]:
>>>>>
>>>>> All supers are valid, no need to recover
>>>>>
>>>>> I now checked my dmesg log:
>>>>> [45907.451840] BTRFS critical (device dm-0): corrupt leaf:
>>>>> block=22751711027200 slot=1 extent bytenr=6754755866624 len=4096 
>>>>> invalid generation, have 22795412619264 expect (0, 207589]
>>>>
>>>> This is caused by older kernel, which writes some uninitialized value onto disk.
>>>>
>>>> You can try to fix it with this branch:
>>>> https://github.com/adam900710/btrfs-progs/tree/extent_gen_repair
>>>>
>>>> Or you can use older kernel to delete the offending extents.
>>>>
>>>> Thanks,
>>>> Qu
>>>>
>>>>> [45907.451848] BTRFS error (device dm-0): block=22751711027200 read 
>>>>> time tree block corruption detected [45907.451892] BTRFS error 
>>>>> (device
>>>>> dm-0): failed to read block groups: -5 [45907.510712] BTRFS error 
>>>>> (device dm-0): open_ctree failed
>>>>>
>>>>> Google inquiries to this topic led me to this link:
>>>>> https://btrfs.wiki.kernel.org/index.php/Tree-checker
>>>>> It tells me to mail here first so thats what i am doing. I kind of 
>>>>> suspect since everything worked perfect (Memtest also no errors) 
>>>>> before the update, it has to do something with that update. I am 
>>>>> wondering if it would help if i deleted my OS Disk and reinstalled 
>>>>> an older Version of Ubuntu, like
>>>>> 16.04.6 LTS ?
>>>>>
>>>>> Since then i upgraded to 20.04 LTS with BTRFS-PROGS 5.7 as a lot of 
>>>>> forum entries said it would be wise to use the newer versions as older were buggy.
>>>>> That brought no help as well.
>>>>>
>>>>> Since i am no Linux/Unix Expert i thought it might be better to ask 
>>>>> now first as advised in the link above before proceeding with any other plans.
>>>>>
>>>>> I find it hard to believe that all data is gone, i feel its buggy 
>>>>> behavior as the partition and everything is still there:
>>>>> root@Userv:/home/benjamin# btrfs fi show
>>>>> Label: 'Storage'  uuid: 46c7d04a-d6ac-45be-94cc-724919faca2b
>>>>> Total devices 1 FS bytes used 28.23TiB
>>>>> devid    1 size 29.10TiB used 29.04TiB path /dev/mapper/Crypto
>>>>>
>>>>> Best Regards,
>>>>> Benjamin Händel
>>>>>
>>>>
>>>>
>>>
>>>
>>
>>
> 


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]