Re: Status of FST and mount times

["Ellis H. Wilson III" <ellisw@panasas.com>]

On 02/16/2018 09:20 AM, Hans van Kranenburg wrote:
> Well, imagine you have a big tree (an actual real life tree outside) and
> you need to pick things (e.g. apples) which are hanging everywhere.
> 
> So, what you need to to is climb the tree, climb on a branch all the way
> to the end where the first apple is... climb back, climb up a bit, go
> onto the next branch to the end for the next apple... etc etc....
> 
> The bigger the tree is, the longer it keeps you busy, because the apples
> will be semi-evenly distributed around the full tree, and they're always
> hanging at the end of the branch. The speed with which you can climb
> around (random read disk access IO speed for btrfs, because your disk
> cache is empty when first mounting) determines how quickly you're done.
> 
> So, yes.

Thanks Hans.  I will say multiple minutes (by the looks of things, I'll 
end up near to an hour for 60TB if this non-linear scaling continues) to 
mount a filesystem is undesirable, but I won't offer that criticism 
without thinking constructively for a moment:

Help me out by referencing the tree in question if you don't mind, so I 
can better understand the point of picking all these "apples" (I would 
guess for capacity reporting via df, but maybe there's more).

Typical disclaimer that I haven't yet grokked the various inner-workings 
of BTRFS, so this is quite possibly a terrible or unapproachable idea:

On umount, you must already have whatever metadata you were doing the 
tree walk on mount for in-memory (otherwise you would have been able to 
lazily do the treewalk after a quick mount).  Therefore, could we not 
stash this metadata at or associated with, say, the root of the 
subvolumes?  This way you can always determine on mount quickly if the 
cache is still valid (i.e., no situation like: remount with old btrfs, 
change stuff, umount with old btrfs, remount with new btrfs, pain).  I 
would guess generation would be sufficient to determine if the cached 
metadata is valid for the given root block.

This would scale with number of subvolumes (but not snapshots), and 
would be reasonably quick I think.

Thoughts?

ellis