Re: mount btrfs takes 30 minutes, btrfs check runs out of memory

[Josef Bacik <jbacik@fb.com>]

On 07/16/2016 07:53 PM, Qu Wenruo wrote:
>
>
> On 07/15/2016 07:29 PM, Christian Rohmann wrote:
>> Hey Qu, all
>>
>> On 07/15/2016 05:56 AM, Qu Wenruo wrote:
>>>
>>> The good news is, we have patch to slightly speedup the mount, by
>>> avoiding reading out unrelated tree blocks.
>>>
>>> In our test environment, it takes 15% less time to mount a fs filled
>>> with 16K files(2T used space).
>>>
>>> https://patchwork.kernel.org/patch/9021421/
>>
>> I have a 30TB RAID6 filesystem with compression on and I've seen mount
>> times of up to 20 minutes (!).
>>
>> I don't want to sound unfair, but 15% improvement is good, but not in
>> the league where BTRFS needs to be.
>> Do I understand you comments correctly that further improvement would
>> result in a change of the on-disk format?
>
> Yes, that's the case.
>
> The problem is, we put BLOCK_GROUP_ITEM into extent tree, along with tons of
> EXTENT_ITEM/METADATA_ITEM.
>
> This makes search for BLOCK_GROUP_ITEM very very very slow if extent tree is
> really big.
>
> On the handle, we search CHUNK_ITEM very very fast, because CHUNK_ITEM are in
> their own tree.
> (CHUNK_ITEM and BLOCK_GROUP_ITEM are 1:1 mapped)
>
> So to completely fix it, btrfs needs on-disk format change to put
> BLOCK_GROUP_ITEM into their own tree.
>
> IMHO there maybe be some objection from other devs though.
>
> Anyway, I add the three maintainers to Cc, and hopes we can get a better idea to
> fix it.

Yeah I'm going to fix this when I do the per-block group extent tree thing.  Thanks,

Josef